Delta Method

Probability · Axiom Academy

Approximating the distribution of transformations of random variables 1. The Motivation: Transforming Estimators Suppose we know from the CLT that: Now suppose we want the distribution of g(X̄ₙ), where g is some smooth function. Examples: Estimating a standard deviation: g(x) = √x Estimating odds: g(x) = x/(1-x) Direct application of CLT doesn't work because g(X̄ₙ) is not a linear function. We need a new tool! If √n(X̄ₙ - μ) →ᵈ N(0, σ²) and g is differentiable at μ with g'(μ) ≠ 0, then: In other words, g(X̄ₙ) is approximately normal with mean g(μ) and variance [g'(μ)]²σ²/n. 3. Example: Estimating Standard Deviation Problem: Let X₁, ..., Xₙ be i.i.d. with variance σ². The sample variance S² estimates σ². What's the distribution of S = √S² (the sample standard deviation)? Let g(x) = √x. Then g'(x) = 1/(2√x), so g'(σ²) = 1/(2σ). If √n(S² - σ²) →ᵈ N(0, τ²) for some τ², then by the delta method: So S is approximately normal with mean σ and variance τ²/(4nσ²).

This is the written version of the interactive lesson above. See the full Probability course.