Linear Independence & Dependence

Linear Algebra for Machine Learning · Axiom Academy

When vectors are truly independent A set of vectors is linearly independent if the only way to write 0 as a linear combination is to use zero coefficients. Vectors v₁, v₂, ..., vₙ are linearly independent if: c₁v₁ + c₂v₂ + ... + cₙvₙ = 0 implies c₁ = c₂ = ... = cₙ = 0 In other words: no vector can be expressed as a combination of the others. In 2D: Two vectors are independent if they don't lie on the same line. In 3D: Three vectors are independent if they don't lie in the same plane. General: Vectors are independent if you can't express one as a combination of the others. Example: [1, 0] and [0, 1] are independent. Neither is a scalar multiple of the other. Dependent: [1, 0] and [2, 0] are dependent. The second is 2 times the first. Independence tells us which vectors carry new information. Independent vectors form a basis—they can generate the entire space. In machine learning, we look for independent features because dependent features are redundant.

This is the written version of the interactive lesson above. See the full Linear Algebra for Machine Learning course.