Solving a Least Squares Problem

Linear Algebra (Matrices) · Axiom Academy

EXAMPLE Solving a Least Squares Problem Solve an inconsistent system with the normal equations, then measure the least-squares error. The system has three equations but only two unknowns, with and . It is overdetermined, so no solves it exactly. Find the least-squares solution via the normal equations , then compute the least-squares error . Nice work — you solved a full least-squares problem and measured how good the fit is. Overdetermined systems: with more equations than unknowns, usually has no exact solution, so we look for the best approximation instead. Normal equations: the least-squares solution satisfies , turning the tall problem into a small square system. Here . The error: is the smallest possible distance from to the column space of A . Orthogonality: the residual is perpendicular to every column of A , i.e. — the signature of a true least-squares fit. This is exactly the engine behind line-of-best-fit, regression, and data fitting across science and engineering.

This is the written version of the interactive lesson above. See the full Linear Algebra (Matrices) course.