Real World: Face Recognition & Eigenfaces

Linear Algebra for Machine Learning · Axiom Academy

Eigenfaces: PCA for Face Recognition How PCA revolutionized facial recognition in the 1990s The Problem: Recognizing Faces A grayscale face image of 100×100 pixels has 10,000 dimensions. Direct comparison of raw pixels is inefficient and noisy. How can we reduce this to a manageable representation while preserving identity information? Pioneered by Turk and Pentland (1991), this method uses PCA to extract "eigenfaces"—the principal components of a face database. Step 1: Collect Training Faces Gather a large set of aligned face images of different people. Align them to the same position/scale. Treat each image as a vector (flattening the 2D image to 1D). Center all faces by subtracting the mean face. Apply PCA to find principal components. The top 50-100 eigenvectors are the "eigenfaces"—basis vectors spanning face space. These aren't human faces—they're more like ghostly, abstract patterns. Each represents a mode of variation (lighting, expression, facial structure, etc.). Represent each face as a weighted combination of eigenfaces: face ≈ mean_face + w₁*eigenface₁ + w₂*eigenface₂ + ... The weights (w₁, w₂, ..., w_k) form a k-dimensional "face code" or "faceprint". To recognize a test face, project it to eigenface space and compare its face code to known individuals using nearest neighbor or other classifiers. On the Yale Face Database (165 faces, 15 subjects, various lighting/expressions): Using 30 eigenfaces: ~90% recognition accuracy

This is the written version of the interactive lesson above. See the full Linear Algebra for Machine Learning course.