Real World: Polynomial & Nonlinear Fitting

Linear Algebra for Machine Learning · Axiom Academy

Linear algebra handles curves too: The Vandermonde matrix Least squares isn't limited to linear relationships. We can fit polynomial models using the same framework: For polynomial fitting, the design matrix A has a special structure called the Vandermonde matrix: For data points and polynomial of degree : This is a system solvable by Gaussian elimination Higher degrees: Same method works for cubic (d=3), quartic (d=4), and higher polynomials Overfitting risk: As degree increases, you fit training data better but may generalize worse. Use cross-validation to choose d Numerical stability: High-degree polynomials on Vandermonde matrices can be numerically unstable; use Chebyshev or Legendre bases instead Generalization: Same idea extends to exponential ( ), log ( ), and other nonlinear models via transformation Splines: For complex curves, piecewise polynomials (splines) offer flexibility without high-degree polynomials

This is the written version of the interactive lesson above. See the full Linear Algebra for Machine Learning course.