Unit 6 Summary

Linear Algebra for Machine Learning · Axiom Academy

Recap of Singular Value Decomposition Singular Value Decomposition (SVD) is the most powerful tool in applied linear algebra. You've learned how to compute it, interpret it geometrically, and apply it to real-world problems. Any m × n matrix A can be decomposed as A = UΣV T where U and V are orthogonal matrices and Σ contains singular values. This decomposition always exists and is unique (up to sign and ties). U (left singular vectors) form an orthonormal basis for rows; V T (right singular vectors) form an orthonormal basis for columns; Σ contains non-negative singular values in decreasing order. Any linear transformation is a composition of three simpler operations: rotate (V T ), scale by singular values (Σ), then rotate (U). This provides deep insight into how matrices transform space. SVD formula, component properties, and computational algorithms using eigendecomposition Non-negative, ordered, connection to matrix rank, and interpretation as "importance weights" How SVD decomposes transformations into rotations and scaling; visualization of this decomposition Low-rank approximation using truncated SVD; Eckart-Young theorem proves optimality Frobenius norm computation; approximation error analysis; relative error measures Moore-Penrose inverse via SVD; least-squares solutions; handling singular matrices Condition number κ = σ_max/σ_min; numerical stability; why SVD beats normal equations

This is the written version of the interactive lesson above. See the full Linear Algebra for Machine Learning course.