ParaPy is an example of Semiliterate Programming, to borrow from the powerful Literate Programing paradigm of Donald Knuth. In both cases, the goal is to present code in a form suited to a human reader, not a software language compiler. Knuth's methodology supports the top-down creation of a program, the way we often think of programming, with the benefit of handsome presentation of the code using Knuth's TeX.
With ParaPy, we start with a few thousand lines of Python that have evolved from the ParaC translation of ParaBas. This is programming from the inside out. Our presentation is a series of web pages documenting code snippets, not a formatted document suitable for publication. It shares the granularity of the literate programming style.
The semiliterate approach differs in the treatment of the pure Python
code. ParaPy is offered
here
to be copied and pasted locally and run as is.
It has sufficient comments to stand on its own, but not to explain
all the inner workings. By contrast, the code output of the
literate programming tangle tool is not intended for
human consumption and may even be obfuscated to remove the temptation
to modify that version instead the literate Web form.
By design, Python discourages such obfuscation. For example, Python enforces
indentation to reveal block structure, inhibiting any attempt to jam
all the code together into a large blob peppered with punctuation.
The semiliterate approach of
ParaPy is expedient.
The source
code presented in the site is what's carved up into the
snippets presented in the discussion. All the pieces
remaain up to date.
While the semiliterate approach lacks the handy indexing
and cross-referencing tools of literate programming,
there is always the full source paranoia.py
as the final arbiter.
When in doubt, think Star Wars: Use the source, Luke!
Paranoia employs devious sequences of algebraic operations to tease out isolated digits of numbers. Most often, the formulas work whether the radix is 2, 8, 10, or 16. To help visualize the flow, we will use digit diagrams like the one just below. This is an example showing three constants in ParaPy:
Digit diagrams have no connection to the internal representation of nummbers. You, good reader, are asked to supply the scale factor – the exponent in floating point jargon – from the surrounding discussion.
Almost all current arithmetic
uses a normalized representation, in which results are aligned
so that the leading digit
is nonzero.
Besides guaranteeing a unique representation for any value, it means
the leading significant bit in a binary format need not be stored,
because it's
known to be 1.
The digit diagrams will leave the radix point aligned across multiple values. Skipping the normalization and encoding that the underlying hardware performs allows us to visualize the calculations more easily. There are no digits lost off the right-hand side due our skipping the normalization in the discussion.
One key to successful use of digit diagrams is to honor the precision (number of significant digits), which Paranoia determines early on. That's what the vertical guidelines indicate. Digits may lie outside out the guidelines in order that the radix points align, but there may never by any more digits than would fit between them.
Consider what happens when subtracting an ULP
(unit in the lsat place)
of 1.0 in any radix:
The lowest-order significant digit is now one place to the right, because the one's place is now zero.
We'll see many of these diagrams.
Sometimes, it's easier to visualize values on a number line.
In floating point number systems, the powers of the radix
B divide the number system into
B-ades, each B-ade enclosing the same number
of values representable in the number system.
We all know decimal decades,
but we will encounter binary binades
frequently. Occasionally, we'll see
an ocatade or hexade.
Here is a span of a decimal system, with the tick marks greatly
magnified so we can see them. The value \(4/3\) is not representable
exactly, so it lies between representable numbers. On the other
hand, \(3/2\) is simply 1.5. The tick marks shown are
at intervals of \(0.5\), so there isn't room to label the values
of interest.
Note how the relative spacing changes by a factor of \(10\) across decades. The tick marks beyond \(10\) are too widely separated to appear on this graph. Base 8 and base 16 have similar graphs.
The binary number line is perhaps easier to visualize because the relative spacing changes by just a facctor of \(2\) across each binade (sounds like limeade). It's easier to see \(4/3\) and \(3/2\).
The gentle factor of two across binades is what makes binary floating point best for most numerical computation.