Much ado about almost nothing is one way to characterize the historical treatment of tiny values in floating point arithmetic. It has been far the most hotly debated topic in the IEEE 754 committees. This section of Paranoia investigates the behavior of arithmetic on tiny values. But this is a brief history. It tells the story of underflow through Paranoia’s exploration. It leaves the more complete discussion to pages dedicated to the topic. Background As values get tinier and tinier, perhaps by successive division by 2,
Unless \(e^{i\pi} = -1\) strange things happen.
1/2 1/4 1/8 1/16 1/32 1/64 . . . 1/1024 1/2048 . . .
they eventually become too small to represent in the form
FORMULA
where is the radix and the are digits in that radix. The exponent is bounded by the limitations of the representation of the number format. The exponent usually falls in a simple range, such as [–126, 127] for IEEE 754 single format. Some arithmetics will sacrifice significant digits to extend the exponent range as values get huge or small. IEEE 754 supports gradual underflow, allowing the leading digits in the expression above to become 0, thereby supporting smaller and smaller values all with the same exponent . There are more extreme designs, extending the representation as needed to support an exponent of any conceivable size.
Paranoia features many loops that terminate when a magnitude shrinks to zero or to a tiny epsilon value that remains the same when doubled. Faced with arithmetic that will not underflow, automatically extending the range to within the limits of the host computer, Paranoia will loop without end. The code The underflow investigation opens with a deceptively simple message about the objects of the search:
100 * z may result in zero
even though z != 0.0
The diagram below illustrates the three values as will be discovered in arithmetic compliant with IEEE 754. The two nonzero values have the minimum exponent for the number format.
Paranoia starts with powers of the radix. See the discussion of
tiny_powers_of_B() for details about the triple of
returned values. Typically,
too_tiny_B has underflowed to zero.
A description of H and C can be found in the
discussion of the fundamental constants of the arithmetic.
milestone = 110 # ==============================
# UNDERFLOW AND THE BEHAVIOR OF TINY VALUES
# Starts at Basic 4330
print("Seeking three values related to Underflow:")
print(" underflow_threshold = smallest well-behaved value")
print(" min_positive = smallest nonzero value discovered")
print(" too_tiny_x = a value underflowed to zero or a pseudo-zero")
# First step: compute tiny powers of the radix. H and C are two useful
# constants.
# H = min(1/B, 1/2); ONE_OVER_H = max(B, 2)
# C = 1/B**k, not too close to the underflow thereshold; ONE_OVER_C = B**k
less_tiny_B, tiny_B, too_tiny_B = tiny_powers_of_B(C)