Real World: Image Compression

Linear Algebra for Machine Learning · Axiom Academy

Compressing Images Using Low-Rank Approximation A high-resolution image can be represented as a matrix where each entry is a pixel intensity value (0-255). A 1000×1000 image requires 1 million entries. Storage, transmission, and processing are expensive. Question: Can we represent the image with fewer numbers while preserving visual quality? SVD Solution: Low-Rank Approximation For an image matrix I (grayscale), compute its SVD and keep only the top k components: The approximation I_k uses only k(m + n + 1) numbers instead of mn! Example: For a 1000×1000 image with k=50: Compressed: 50 × (1000 + 1000 + 1) = 100,050 entries Visual Quality at Different Ranks Below shows how image quality degradeswith lower rank approximations: Quality: Nearly identical to original Storage: 100,050 entries (10%) Quality: Very good, minor artifacts Quality: Significant blur and banding For images, singular values typically decay rapidly: A few large singular values capture most of the image information The tail of small singular values represents fine details and noise Truncating at k ≈ 50-100 often retains 95%+ of visual information Simple: Just keep top k singular values Optimal: Provably best rank-k approximation Transparent: Error is minimized in Frobenius norm Drawback: Requires full matrix in memory Advanced: Uses DCT, quantization, Huffman coding Perceptually optimized: Exploits human vision properties Industry standard: Widely supported

This is the written version of the interactive lesson above. See the full Linear Algebra for Machine Learning course.