Real World: Word Embeddings & Semantic Space

Linear Algebra for Machine Learning · Axiom Academy

Word2Vec & Semantic Embeddings How can we represent words mathematically? How can we measure "similarity" between words? The breakthrough: Train neural networks to place words as vectors in space such that semantically similar words are close together. Word2Vec Insight: "You shall know a word by the company it keeps." — A word's meaning is defined by surrounding context. The algorithm learns vectors where words that appear in similar contexts have similar vectors. Famous Example: King - Man + Woman = Queen This is the canonical example showing that vector arithmetic captures semantic relationships: king_vec - man_vec + woman_vec ≈ queen_vec This works because Word2Vec learns relationships: "king" has a "male authority" component "man" captures generic maleness Removing male and adding female (woman) shifts to female authority (queen) Real Vector Space Geometry: These relationships are literally geometric. Vector arithmetic preserves semantic relationships! To measure word similarity, we use the angle between vectors: similarity(u, v) = (u · v) / (||u|| ||v||) = cos(θ) Words with small angles (cos close to 1) are semantically similar. The dot product does the work! Word embeddings revolutionized NLP because they let computers understand semantic meaning. They're used in: • Machine translation • Sentiment analysis • Search engines • Chatbots • Language models

This is the written version of the interactive lesson above. See the full Linear Algebra for Machine Learning course.