Least Squares Method

Statistics · Axiom Academy

Understanding how we find the best-fit line by minimizing squared residuals A residual is the vertical distance between an observed data point and the predicted value on the regression line. For each point (x i , y i ), the residual is: The least squares method finds the line that minimizes the sum of all squared residuals. Watch the animation to see how residuals appear as vertical distances from points to the line. 2. Why Do We Square the Residuals? We square the residuals for several important reasons: The sum of squared residuals (SSR) is: Watch how different lines produce different sums of squared residuals. To minimize the sum of squared residuals, we use calculus to derive the optimal slope b 1 . There are two equivalent formulas: where r is the correlation coefficient, s y is the standard deviation of y, s x is the standard deviation of x, and the bars denote means. Once we have the slope b 1 , we can find the y-intercept b 0 using a beautiful property: the regression line always passes through the point of means (x̄, ȳ). This ensures that the regression line goes through the center of the data cloud, providing the best overall fit. 5. The Complete Least Squares Method The regression line is given by: where the coefficients minimize the sum of squared residuals: Watch the complete process visualized with real data.

This is the written version of the interactive lesson above. See the full Statistics course.