Error in Numerical Differentiation

Numerical Analysis · Axiom Academy

LESSON Error in Numerical Differentiation Understanding and controlling approximation errors Total error in numerical differentiation comes from two sources: From dropping Taylor series terms From finite precision arithmetic These two errors behave oppositely as h changes! Truncation error decreases with smaller h, but round-off error increases. Truncation error comes from the terms we drop in the Taylor series. For forward difference: The leading error term is (h/2)f''(ξ). For central difference: The leading error is (h²/6)f'''(ξ) - much smaller for small h! 3 Round-off Error: The Hidden Danger Computers represent numbers with finite precision. When we compute (f(x+h) - f(x))/h with very small h: The numerator subtracts two nearly equal numbers - a classic source of catastrophic cancellation . 4 Finding the Optimal Step Size To find optimal h, minimize total error = truncation + round-off: Take derivative with respect to h and set to zero: Start with h ≈ √ε × |x| (scale h with x to handle different magnitudes) Prefer central difference over forward/backward when possible For noisy data, use larger h (noise amplification is worse with small h) Consider Richardson extrapolation for better accuracy

This is the written version of the interactive lesson above. See the full Numerical Analysis course.