Challenges of Numerical Differentiation

Numerical Analysis · Axiom Academy

INTRO Challenges of Numerical Differentiation Why smaller h isn't always better, and other surprises Logic says: smaller h means better approximation. But watch what really happens as we make h smaller and smaller... Click each error type to understand where it comes from: The error from approximating a derivative with a finite difference Source: We're cutting off the Taylor series Behavior: Decreases as h gets smaller (O(h) or O(h²)) Fix: Use smaller h or higher-order formulas The error from finite precision arithmetic on computers Source: Subtracting nearly equal numbers Behavior: Increases as h gets smaller (≈ ε/h) Real measurements have noise. Watch what happens when we try to differentiate noisy data: See what happens when we subtract two nearly equal numbers: Computing f'(1) for f(x) = sin(x) Numerical differentiation is inherently unstable. Unlike integration, which smooths out errors, differentiation amplifies them. Optimal h exists: Not too big, not too small Central differences: More accurate than one-sided Noise amplification: Requires smoothing or regularization Higher-order formulas: Can improve accuracy For forward differences: h ≈ √ε ≈ 10⁻⁸ For central differences: h ≈ ε^(1/3) ≈ 10⁻⁵ For noisy data: use larger h or apply smoothing first

This is the written version of the interactive lesson above. See the full Numerical Analysis course.