Residuals and Error Analysis

Linear Algebra for Machine Learning · Axiom Academy

Computing and interpreting model errors For each data point, the residual is the difference between the observed value and the predicted value: Using our linear regression example: y = 1.1x + 0.8 The R² statistic measures what fraction of variance is explained: R² = 1: Perfect fit (all residuals are zero) R² = 0.9: Excellent fit (90% of variance explained) R² = 0.5: Moderate fit (model explains half the variation) R² = 0: Poor fit (model explains no variance) Normality: For valid inference, residuals should be approximately normally distributed Homoscedasticity: Residuals should have constant variance (spread shouldn't increase with x) Independence: Residuals should be uncorrelated (no patterns in order) Outliers: Large residuals indicate unusual data points worth investigating Model selection: Systematic patterns in residuals suggest the linear model is inadequate

This is the written version of the interactive lesson above. See the full Linear Algebra for Machine Learning course.