Scree Plot & Elbow Method

Linear Algebra for Machine Learning · Axiom Academy

Creating and Reading Scree Plots Visual approach to choosing the number of components A scree plot visualizes the eigenvalues (or explained variance) of each principal component, plotted in descending order. It helps you spot the "elbow"—the point where additional components stop contributing significantly. Consider a 10-dimensional dataset with these eigenvalues: Total variance = 150 + 120 + 80 + 30 + 10 + ... = 400 Notice the sharp drop after PC4, then a flattening tail. This is typical of scree plots. Look for the point where the curve starts to flatten Components before the elbow capture significant variance Components after the elbow contribute little and can be dropped Interpretation of Different Scree Plots The curve drops steeply then flattens abruptly. This makes choosing k easy—use the components before the elbow. The curve slopes gradually with no clear elbow. This suggests data complexity or high dimensionality. Use the 95% variance threshold instead of visual inspection. Variance decreases uniformly with no obvious break point. The data may be inherently high-dimensional. Consider domain knowledge to guide your choice. Scree Plot vs. Cumulative Variance Plot Easy to spot the elbow visually Shows total variance explained Easy to read off a percentage threshold Emphasizes total information retained Better for precision-critical applications

This is the written version of the interactive lesson above. See the full Linear Algebra for Machine Learning course.