Round-off Error

Numerical Analysis · Axiom Academy

How rounding affects every floating-point operation With only 52 mantissa bits, most real numbers fall "between" representable values. The computer must choose the closest one—this choice is the rounding process. IEEE 754 defines four rounding modes. The default is "round to nearest, ties to even" (banker's rounding). Round to Nearest (default) : Choose closest; if tie, choose even Round Toward Zero : Truncate toward zero Round Up (ceiling) : Round toward +∞ Round Down (floor) : Round toward −∞ The maximum relative round-off error in a single operation is ε/2, where ε is machine epsilon. 4. Accumulation of Round-off Errors Multiple operations compound errors. After n operations, errors may grow like √n (random walk) or n (worst case).

This is the written version of the interactive lesson above. See the full Numerical Analysis course.