Real World: Spam Filters & Machine Learning

Math for CS · Axiom Academy

Bayes' theorem in action: real-world CS application Your email inbox receives hundreds of messages daily. Some are legitimate ("ham"), some are unwanted ("spam"). How does a computer automatically classify each incoming email? The answer is one of the most elegant applications of probability in all of computer science: Bayesian spam filtering . Applied to spam classification: : prior -- what fraction of all emails are spam (e.g., 0.4) : likelihood -- how likely these words appear in spam : posterior -- probability the email is spam given its content Real emails have many words . Computing exactly would require a probability for every possible word combination -- astronomically many. The naive assumption: words are conditionally independent given the class. This is "naive" because words are obviously not independent ("Nigerian" and "prince" are correlated). But it works remarkably well in practice. In practice, use log probabilities to avoid numerical underflow: Gather a corpus of emails labeled as spam or ham. Typical training sets have thousands of each. Step 2: Estimate Word Probabilities Same for ham. To avoid zero probabilities for unseen words, apply Laplace smoothing : where |V| is the vocabulary size. For each incoming email, compute the spam score and ham score using the trained probabilities. Classify according to whichever is higher (or use a threshold for tuning). Email contains: "free", "click" 96.2% spam probability. Classified as spam.

This is the written version of the interactive lesson above. See the full Math for CS course.