SVD Definition & Form

Linear Algebra for Machine Learning · Axiom Academy

Understanding the Components of Singular Value Decomposition Singular Value Decomposition decomposes any m × n matrix A into three matrices: where U is m × m, Σ is m × n, and V T is n × n SVD takes ANY rectangular matrix A (m rows × n columns) and breaks it into a product of three matrices. The key insight is that this decomposition exists for EVERY matrix—no exceptions. Unlike eigendecomposition which requires square matrices, SVD works universally: Singular or non-square matrices ✓ This universality is one of the greatest strengths of SVD. Every matrix has an SVD decomposition. Step 2: The U Matrix (Left Singular Vectors) U is an m × m orthogonal matrix whose columns are called the left singular vectors . U T U = I (columns are orthonormal) Columns of U are eigenvectors of AA T Step 3: The Σ Matrix (Singular Values) Σ (Sigma) is an m × n diagonal matrix containing the singular values on its diagonal: σᵢ measures the "strength" of the i-th component Large σᵢ means that component is important Small σᵢ (close to 0) means that component contributes little The number of non-zero singular values equals the rank of A Step 4: The V T Matrix (Right Singular Vectors) V T is an n × n orthogonal matrix. Its rows are called the right singular vectors (the columns of V are the right singular vectors). V T V = I (rows are orthonormal) (V T ) T = V -1 (V is orthogonal) Columns of V are eigenvectors of A T A Uniqueness: The SVD is unique up to sign changes and ties in singular values

This is the written version of the interactive lesson above. See the full Linear Algebra for Machine Learning course.