Floating Point Pitfalls

Numerical Analysis · Axiom Academy

EXAMPLE Floating Point Pitfalls Classic examples of floating-point surprises and how to avoid them Great work! Here's what to remember: Never compare floats with == . Use |a - b| < tolerance instead. 0.1, 0.2, 0.3 are all inexact in binary floating-point. Subtracting nearly equal numbers causes catastrophic cancellation. Reformulate expressions to avoid subtracting similar values. Use special functions like expm1(), log1p() when available.

This is the written version of the interactive lesson above. See the full Numerical Analysis course.