Advanced Topics Summary

Probability · Axiom Academy

SUMMARY Advanced Topics in Probability Let's review generating functions, probability inequalities, order statistics, and simulation methods. Probability Generating Function (PGF): for discrete non-negative integer-valued random variables. Encodes the entire probability distribution. Moment Generating Function (MGF): when it exists. Used to find moments via derivatives: Characteristic Function: always exists for any distribution. The Fourier transform of the probability measure. Key Property: All three functions uniquely determine the distribution and convert convolution into multiplication for sums of independent random variables. Markov's Inequality: For non-negative and , we have . Requires only the mean to exist. Chebyshev's Inequality: For any random variable with finite variance: . Bounds probability of deviation from the mean. Chernoff Bound: For any , . Optimized over all to get exponentially tight bounds. Hierarchy: Chernoff provides the tightest bounds (exponential decay), followed by Chebyshev (polynomial), then Markov (weakest). Example Recap: Using Chebyshev's Inequality Step 1 - Setup: Suppose is the sample mean of 100 independent coin flips with . We want . Step 2 - Compute Parameters: Calculate and . Step 3 - Apply Chebyshev: Use with . Step 4 - Conclude: We get , so the probability is at least 0.75 that the sample mean is within 0.1 of the true mean. Definition: Given , the order statistics are the sorted values: Minimum: has CDF (all must exceed )

This is the written version of the interactive lesson above. See the full Probability course.