Diagonalization

Linear Algebra for Machine Learning · Axiom Academy

If a matrix A has n linearly independent eigenvectors, we can "diagonalize" it. This means expressing A in the form: P is the matrix whose columns are the eigenvectors of A D is a diagonal matrix with eigenvalues on the diagonal Not all matrices can be diagonalized. A matrix A is diagonalizable if and only if: It has n linearly independent eigenvectors (for an n×n matrix) For symmetric matrices: Always diagonalizable (and the eigenvectors are orthogonal!) For defective matrices: The algebraic multiplicity of at least one eigenvalue exceeds its geometric multiplicity. These cannot be diagonalized. Find all eigenvalues of A (solve det(A - λI) = 0) For each eigenvalue, find a basis for its eigenspace by solving (A - λI)v = 0 Arrange the eigenvectors as columns of P Arrange the corresponding eigenvalues on the diagonal of D Compute P⁻¹ (or use a numerical method) Computing Matrix Powers Using Diagonalization One of the biggest applications of diagonalization is computing high powers of matrices efficiently: Markov chains: Computing transition probabilities after many steps Graph analysis: PageRank iterations converge to the eigenvector of the transition matrix Covariance matrices: PCA uses eigendecomposition for dimensional reduction

This is the written version of the interactive lesson above. See the full Linear Algebra for Machine Learning course.