Rounding errors, accuracy, range and precision

Overview

Recall questions on why 0.1 cannot be held exactly, absolute against relative error, the trade-off between range and precision, and why floats are not compared for equality.

What you need to know

Answer each one before you reveal it.

Questions from this subtopic

Each one with the model answer underneath.

1. 1

   Explain why both fixed point and floating point representations of decimal numbers may be inaccurate.

   Model answer

   Both use a finite number of bits, and the fractional part can only be built from the binary fractions 1/2, 1/4, 1/8 and so on. A real number can only be stored exactly if it can be expressed as a sum of those binary fractions within the bits available. Any other value has to be rounded or truncated to the nearest value that can be represented, so a rounding error is introduced as soon as it is stored.

2. 2

   Why can the decimal value 0.1 never be represented exactly in binary, however many bits are used?

   Model answer

   0.1 is one tenth, and 10 is not a power of 2. In binary it becomes the recurring fraction 0.0001100110011… which repeats for ever, so no finite number of bits can hold it. Only fractions whose denominator is a power of two terminate in binary.

3. 3

   Which decimal fractions can be stored exactly in binary?

   Model answer

   Those whose fractional part is a sum of negative powers of two — in other words, fractions whose denominator is a power of 2 — and which are short enough to fit the bits available. So 0.5, 0.25, 0.75, 0.125 and 0.625 are exact, while 0.1, 0.2 and 0.3 are not.

4. 4

   Store 0.1 in a fixed point format with 8 bits after the binary point and calculate the value actually held.

   Model answer

   The nearest available pattern is .00011010, which is 1/16 + 1/32 + 1/128 = 0.0625 + 0.03125 + 0.0078125 = 0.1015625. The stored value is therefore slightly more than 0.1, and the difference is 0.0015625. Truncating instead of rounding would give .00011001 = 0.09765625, which is further away.

5. 5

   What practical advice follows for programmers from the fact that values such as 0.1 cannot be stored exactly?

   Model answer

   Never test two real values for exact equality, because two calculations that should agree mathematically may differ in the last bits; instead test whether the difference between them is smaller than a small tolerance. Be aware too that rounding errors accumulate over repeated operations, so a long loop of additions can drift noticeably, and use integers or fixed point for money and other quantities that must be exact.

6. 6

   Define absolute error.

   Model answer

   The absolute error is the size of the difference between the value actually stored or computed and the true value: absolute error = |stored value − true value|. It is measured in the same units as the quantity itself.

Other ways to work this

The same content, four other ways.

What is inside Levo

Six ways to work the same material.