Unit 5 Summary

Linear Algebra for Machine Learning · Axiom Academy

Unit 5: Eigenvalues & Eigenvectors Summary The Core Concept: Eigenvectors are special vectors that maintain their direction under a matrix transformation. They only get scaled by an eigenvalue λ. This simple but powerful idea underlies much of applied mathematics and machine learning. Eigendecomposition is the pinnacle of linear algebra. It connects determinants, traces, matrix powers, and spectral properties into one unified framework. PCA, SVD, kernel methods, Markov chains—all rely on eigenvalues/eigenvectors. Understanding this unit is crucial for ML theory. Hessian eigenvalues determine convexity, convergence rates, and stability. Positive definite matrices = local minima. Vibration modes (eigenvectors), natural frequencies (eigenvalues), stress tensors (principal stresses from eigendecomposition). Covariance/correlation matrices (symmetric, real eigenvalues), spectral clustering, graph analysis all use eigendecomposition. Power iteration, QR algorithm, Krylov subspaces—all build on eigenvalue theory for efficient computation. Av = λv is the defining equation. Memorize and understand this. Eigenvalues solve det(A - λI) = 0. Eigenvectors solve (A - λI)v = 0. Symmetric matrices are special: real eigenvalues, orthogonal eigenvectors, always diagonalizable. Diagonalization A = PDP⁻¹ reveals the "natural basis" where the matrix is simple. Positive definite matrices (all eigenvalues > 0) guarantee x^T Ax > 0 and have a unique minimum.

This is the written version of the interactive lesson above. See the full Linear Algebra for Machine Learning course.