Understanding Covariance

Linear Algebra for Machine Learning · Axiom Academy

Interpreting the Covariance Matrix Understanding relationships through computed examples Worked Example: Computing Covariance Let's work through a complete example with real numbers to understand what covariance values tell us. We have 5 students with scores on two tests (Math and English): Math mean = (85 + 90 + 75 + 95 + 80) / 5 = 85 English mean = (88 + 92 + 70 + 98 + 82) / 5 = 86 Step 3: Calculate Variances and Covariance Var(Math): (0² + 5² + (-10)² + 10² + (-5)²) / 5 = (0 + 25 + 100 + 100 + 25) / 5 = 50 Var(English): (2² + 6² + (-16)² + 12² + (-4)²) / 5 = (4 + 36 + 256 + 144 + 16) / 5 = 91.2 Cov(Math, English): (0·2 + 5·6 + (-10)·(-16) + 10·12 + (-5)·(-4)) / 5 = (0 + 30 + 160 + 120 + 20) / 5 = 330 / 5 = 66 Interpretation: The covariance is positive (66.0). This means Math and English scores tend to move together. Students who score high in Math also tend to score high in English, and vice versa. Comparison: Other Covariance Values If students who score high in Math tend to score low in English, we'd get a negative covariance. This would suggest an inverse relationship. In this case, Cov(x, y) ≈ -50. When one increases, the other tends to decrease. If test scores in Math and English are unrelated, the covariance would be close to zero. One person's Math score tells you nothing about their English score. In this case, Cov(x, y) ≈ 0. The variables are uncorrelated. Interpreting the Full Covariance Matrix Math variance = 50.0: Math scores spread with moderate variability

This is the written version of the interactive lesson above. See the full Linear Algebra for Machine Learning course.