Principal Component Analysis Preview

Linear Algebra (Matrices) · Axiom Academy

LESSON Eigenvectors Point Where Data Spreads A preview of PCA: the eigenvectors of the covariance matrix C are the principal directions of variance, and each eigenvalue is the variance along its direction. 1. A Cloud Has a Direction of Greatest Spread Picture data points scattered in the plane. Rotate a candidate line through the cloud and measure the variance — how spread out the points are — along it. That number rises and falls with the angle, and it is largest along one special direction: the first principal component , PC_1 . 2. Those Axes Are Eigenvectors of C Build the covariance matrix C from the centered data. For this cloud it comes out to a clean symmetric matrix. Its eigenvectors are the principal axes you just saw, and its eigenvalues are the variances along them — the axis lengths grow as . Symmetric covariance matrix of the centered data Each principal component solves the eigenvalue equation Eigenvector . Largest eigenvalue, so it is the long axis — the most variance. Eigenvector . Smaller eigenvalue, so it is the short axis — less variance. Because C is symmetric, the eigenvectors are perpendicular: . The half-length of each axis is , so is the variance the data has along that component. 3. Project onto PC_1 to Reduce Dimension

This is the written version of the interactive lesson above. See the full Linear Algebra (Matrices) course.