Characteristic Functions

Probability · Axiom Academy

LESSON Characteristic Functions Universal transform that always exists and connects probability to Fourier analysis 1. Definition of Characteristic Function For any random variable X (discrete or continuous), the characteristic function is defined as: Unlike MGFs which may not exist for heavy-tailed distributions, the characteristic function φ(t) exists for every random variable. Characteristic functions have several crucial advantages: The boundedness property |φ(t)| ≤ 1 comes from |e^ itX | = 1, making characteristic functions well-behaved even when MGFs fail to exist. 3. Connection to Fourier Transform The characteristic function is essentially the Fourier transform of the probability density (or PMF): This connection allows us to use Fourier inversion to recover the density from the characteristic function, and to apply powerful techniques from harmonic analysis to probability problems. When moments exist, they can be extracted from derivatives at t = 0: The factor i^k appears because each derivative of e^ itX brings down a factor of iX. Note: if E[X^k] doesn't exist, φ^(k)(0) won't exist either, but φ(t) still exists as a function. The standard normal distribution N(0,1) has characteristic function: Notice this is purely real (no imaginary part) because the normal distribution is symmetric. We can verify E[X] = -i · φ'(0) = 0 and E[X²] = -φ''(0) = 1.

This is the written version of the interactive lesson above. See the full Probability course.