Vector Norms: L1, L2, Lp, L∞

Linear Algebra for Machine Learning · Axiom Academy

Measuring vector magnitude in different ways A norm measures the "size" or "magnitude" of a vector. It's always non-negative and tells us how far a vector extends from the origin. A norm is a function ||·|| that assigns to each vector a non-negative number, measuring its magnitude. There are many ways to define a norm, each with different properties and applications. Most common. The straight-line distance. Sum of absolute values. Like city blocks. Maximum absolute component value. L1, L2, and L∞ are all special cases of a single family — the Lp norm , defined for any real number p ≥ 1: The exponent p controls how strongly large components dominate the magnitude: p = 1 — the L1 (Manhattan) norm, sum of absolute values. p = 2 — the L2 (Euclidean) norm, ordinary length. p → ∞ — the L∞ (max) norm, dominated by the largest component. Fractional p (0 < p < 1) — sometimes used in robust statistics, technically a quasi-norm. Example: For v = [3, 4] with p = 3: All Lp norms agree on the basic axioms (non-negativity, scaling, triangle inequality), but they shape the geometry of "unit balls" differently — which matters for regularization, distance metrics, and clustering. L2 Norm: Most natural for measuring distance. Used in Euclidean distance, neural networks, and regularization. L1 Norm: Sparse solutions. Used in LASSO regression to encourage feature selection. L∞ Norm: Robustness. Ensures no single component dominates.

This is the written version of the interactive lesson above. See the full Linear Algebra for Machine Learning course.