Numbers

Floating point numbers fall neatly into a parallelogram. A bullet • emphasizes the binary point in this 8-bit number system. Each value may have positive or negative algebraic sign. The bs are bits, taking the value 0 or 1.

  |11111111           •                 | largest number
  |1bbbbbbb           •                 |
  | 1bbbbbbb          •                 |
  |  1bbbbbbb         •                 |
  |   1bbbbbbb        •                 |
  
       . . . many lines omitted . . .
  
  |         1bbbbbbb00•                 |
  |          1bbbbbbb0•                 |
  |           1bbbbbbb•                 |
  |            1bbbbbb•b                |
  |             1bbbbb•bb               |
  |              1bbbb•bbb              |
  |               1bbb•bbbb             |
  |                1bb•bbbbb            |
  |                 1b•bbbbbb           |
  |                  1•bbbbbbb          |
  |                   •1bbbbbbb         |
  |                   •01bbbbbbb        |
  |                   •001bbbbbbb       |
  |                   •0001bbbbbbb      |
  
       . . . many lines omitted . . .
  
  |                   •      1bbbbbbb   |
  |                   •       1bbbbbbb  |
  |                   •        1bbbbbbb |
  |                   •         1bbbbbbb|
  |                   •         10000000| smallest normal number
  |                   --------------------------------------------
  |                   •         01111111| largest subnormal number
  |                   •         01bbbbbb|
  |                   •         001bbbbb|
  |                   •         0001bbbb|
  |                   •         00001bbb|
  |                   •         000001bb|
  |                   •         0000001b|
  |                   •         00000001| smallest subnormal number

Normal numbers

This is a normalized number system. The leading bit of each number is 1, except perhaps for the tiniest values.

Of course, not all numbers between the largest and smallest are representable with just 8 bits. For example, \( 1 + 2^{-20} \) has the binary form:

1.00000000000000000001

which does not lie in the parallelogram of numbers.

Normalization ensures that if a value is representable, then there is exactly one representation for it. For example, two appears as 10.000000 in the table, and nowhere else. When it's time to encode numbers as bit strings in memory, normalization gives us a free bit. Because we know the leading significant bit is 1, we don't have to store it.

We will call these normal numbers.

The floating point

The diagram illustrates the sense in which the binary point floats left and right across 8-bit strings. Usually, there is a limit to how far rightward the binary point can float – a limit to how big numbers can get. That largest representable number is called the overflow threshold.

In the same way, there is a limit to how far leftward the binary point can float – a limit to how tiny numbers can get. The diagram shows a tiniest normalized number. This may be called the underflow threshold.

Denormalization

The diagram suggests that we may squeeze out a bit more range by allowing the leading bit to go to 0 for just these tiniest numbers. The IEEE 754 Floating Point Standard famously adopted such denormalization as a means to mitigate the effects of underflow. The resulting values have come to be called subnormal.

Not a toy

The 8-bit numbers fit conveniently in the diagram. They're easier to talk about than 24-bit or 53-bit numbers common in current microprocessors. But the ideas are all the same. The narrow values help convey the dynamic nature of floating point numbers. Please don't think it's a toy example. Every day, billions of such numbers play into artificial intelligence calculations carried out by specialized processors.