Explained Variance Ratio

Linear Algebra for Machine Learning · Axiom Academy

Determining how much variance each component captures Let's compute explained variance from a complete PCA example with real eigenvalues. These represent variance along each principal component of a 4D dataset. Step 1: Calculate Total Variance Sum all eigenvalues (total variance in the data): Total variance = 45 + 30 + 18 + 7 = 100 This makes our calculations easy since everything will be percentages. Step 2: Calculate Individual EVR For each component, divide its eigenvalue by total variance: Interpretation: PC1 captures 45% of the data's variance, PC2 captures 30%, etc. Step 3: Calculate Cumulative EVR Sum the EVRs progressively to see total variance retained: Step 4: Decide on Number of Components For 95% variance: We need 3 components (93% is close, 4 would be overkill) For 90% variance: We need 3 components (93% ≥ 90%) For 75% variance: We need 2 components For 50% variance: We need just 1 component! In this example, k = 3 is the most reasonable choice , preserving 93% of variance while reducing from 4D to 3D. Blue bars show individual EVR, orange line shows cumulative EVR Interpretation and Decision Making What Explains the Drop from PC1 to PC2? Notice the sharp drop from λ₁ = 45 to λ₂ = 30. This is typical: the first component is much more important than subsequent ones. This is why PCA is so effective for dimensionality reduction. The elbow appears between PC3 and PC4: we drop from 18 to 7 variance units. Beyond PC3, additional components don't contribute much.

This is the written version of the interactive lesson above. See the full Linear Algebra for Machine Learning course.