Geometric Interpretation of SVD

Linear Algebra for Machine Learning · Axiom Academy

Understanding How SVD Decomposes Linear Transformations One of the most powerful insights about SVD is that any linear transformation represented by a matrix A can be decomposed into three simple geometric operations: rotation, scaling, and rotation. How SVD Decomposes a Transformation The formula A = UΣV T can be rewritten to show the sequence of operations: This reads as: "Apply V T , then Σ, then U" (reading right to left as function composition). Visualizing the Geometric Decomposition Below is an animation showing how a unit circle is transformed through each stage of SVD: The blue circle shows the original unit circle. Watch how it rotates with V T , stretches with Σ, and rotates again with U. V T is an orthogonal matrix, so multiplication by V T is a pure rotation (and possibly reflection). This rotates the input space so that the data aligns with the coordinate axes. Σ: Scaling Along Principal Axes Σ is a diagonal matrix containing the singular values. After the rotation by V T , Σ scales each coordinate independently by the corresponding singular value. U is another orthogonal matrix. Multiplication by U applies another rotation (and possibly reflection) to orient the output space properly. Why This Decomposition is Powerful Rank-1 approximation: We can approximate A ≈ σ₁u₁v₁ T using just the largest singular value. This gives the best rank-1 approximation of A.

This is the written version of the interactive lesson above. See the full Linear Algebra for Machine Learning course.