Linear Regression

Mathematical Modeling · Axiom Academy

Unit 7: Data-Driven Modeling - Finding the best-fit line through data points Imagine you have collected data points that seem to follow a linear trend, but they do not lie perfectly on a line. Real-world data always has some noise or variability. Our goal is to find the line y = mx + b that best represents this relationship. The key question is: what does "best" mean? We need a mathematical criterion to measure how well a line fits the data. 2. Residuals: Measuring the Error For each data point, the residual is the vertical distance between the observed value and the predicted value on the line. If our data point is (x_i, y_i) and our line predicts y = mx_i + b, then the residual is: We want to minimize these residuals across all data points. But simply adding them up would not work since positive and negative values would cancel out. 3. The Least Squares Criterion The solution is to minimize the sum of squared residuals (SSR). By squaring each residual, we ensure all terms are positive and penalize larger errors more heavily. Using calculus, we take partial derivatives of SSR with respect to m and b, set them to zero, and solve the resulting system of equations. Solving the normal equations yields explicit formulas for the slope m and intercept b. These formulas depend only on the means and sums of the data: The regression line always passes through the point of means (x-bar, y-bar), ensuring it is centered on the data.

This is the written version of the interactive lesson above. See the full Mathematical Modeling course.