Real World: Latent Semantic Analysis

Linear Algebra for Machine Learning · Axiom Academy

Finding Topics in Document Collections Using SVD Given a collection of documents, we want to: Find the main topics/themes discussed Improve search and information retrieval Organize documents by hidden semantic structure Challenge: Documents with similar meaning may use different words. We need to find the underlying semantic structure. Building the Term-Document Matrix A[i,j] = frequency of term i in document j (or TF-IDF score) Terms: ["computer", "algorithm", "neural", "network", "learning"] Documents: [Doc1: AI article, Doc2: Hardware article, Doc3: Machine Learning article] A 5×3 term-document matrix captures how each term appears in each document. Compute SVD of the term-document matrix: Interpreting the SVD Components Each column of U represents a topic. Values in that column show how relevant each term is to that topic. High values = strong connection between the term and topic. Singular value σᵢ represents the importance/strength of topic i. Larger values = more dominant topics. This helps identify which topics are central to the corpus. Each column of V represents a document. Values show how relevant each document is to each topic. High values = document strongly covers that topic. Discovered Topics (hypothetical): Topic 1 (σ₁=8.5, strong): High loadings on "neural", "network", "learning", "algorithm" → Topic = "Machine Learning" Topic 2 (σ₂=5.3, moderate): High loadings on "computer", "processor", "hardware", "memory" → Topic = "Computer Hardware"

This is the written version of the interactive lesson above. See the full Linear Algebra for Machine Learning course.