Eigenvalue Definition & Properties

Linear Algebra for Machine Learning · Axiom Academy

Understanding the fundamental concept Geometric Meaning of Eigenvectors When we apply a linear transformation (multiply by a matrix), most vectors change both their direction and magnitude. But eigenvectors are special: Geometrically, if you start with an eigenvector and apply the matrix transformation, you get a vector on the same line through the origin. This is what makes eigenvectors so useful—they're the "axes" along which the transformation acts as simple scaling. Let A be an n×n matrix. A non-zero vector v is an eigenvector of A if there exists a scalar λ such that: The scalar λ is called the eigenvalue corresponding to that eigenvector. The eigenvector v must be non-zero The equation holds for a specific scalar λ Different eigenvectors can have different eigenvalues An eigenvector can be multiplied by any non-zero scalar and still be an eigenvector with the same eigenvalue The eigenvalue λ tells us how much the eigenvector gets stretched or compressed: λ > 1: The eigenvector stretches (gets larger) λ = 1: The eigenvector stays unchanged 0 The eigenvector shrinks (gets smaller) λ = 0: The eigenvector maps to the zero vector λ The eigenvector reverses direction and scales Why Eigenvectors Matter for ML In Machine Learning and Data Science: Why is this powerful? If you have 1000 features but they're correlated, eigenvectors help you find a small set of new, uncorrelated features that capture most of the information. This saves computation and reduces overfitting.

This is the written version of the interactive lesson above. See the full Linear Algebra for Machine Learning course.