Bayes' Theorem

Discrete Math · Axiom Academy

Learning to update our beliefs rationally when we receive new evidence— the mathematical foundation of rational inference. 1. Conditional Probability Foundation Before we derive Bayes' Theorem, we need to understand conditional probability. The probability of event A given that event B has occurred is: This tells us: "What fraction of the times B occurs does A also occur?" We're restricting our view to only the cases where B happened. From the definition of conditional probability, we can write two equations: Since both equal P(A ∩ B), we can set them equal and solve for P(A|B): 3. Understanding the Components Each part of Bayes' Theorem has a special meaning when we're updating beliefs with evidence: A disease affects 1% of the population. A test for the disease is 95% accurate (correctly identifies both sick and healthy people 95% of the time). If you test positive, what's the probability you actually have the disease? Using Bayes' Theorem with P(B) = P(Positive | Disease) × P(Disease) + P(Positive | No Disease) × P(No Disease): Email spam filters use Bayes' Theorem to classify messages. Suppose 40% of your emails are spam, and the word "FREE" appears in 70% of spam emails but only 5% of legitimate emails. If an email contains "FREE", what's the probability it's spam? The Power of Bayes: By starting with a prior belief (40% spam rate) and updating it with evidence (the word "FREE"), we get a posterior belief (90.3% chance of spam). This is rational inference in action.

This is the written version of the interactive lesson above. See the full Discrete Math course.