37. Asking the era

The manuals do not settle every question about MDL. Some behavior is unspecified; some archived programs belong to a different revision; and some printed claims disagree with the surviving interpreter. The project uses MDL 55 under apsim to investigate those cases.

This chapter explains how to ask a reproducible question, compare the answer with pymdl, and preserve the result as a test. It also describes limitations of the instruments: a simulator can return a plausible answer without reproducing the operating-system behavior being investigated.

When a claim concerns MDL 55, prefer a controlled observation to an assumption. Record the release, inputs, result, and relevant test. A measurement on release 55 does not by itself establish behavior on release 105 or on every archived revision.

37.1 What "the era" is, here

MDL 55. MIT's interpreter, release 55, is the ITS TS MUDDLE on the August 1981 tape (7005458/mudsav/ts.mud55), and it boots under pdp10-apsim, a PDP-10 and ITS simulation from the pdp10-xdev toolchain outside this repository (PYMDL_APSIM). It runs on a writable copy of an ITS root, because MDL writes into its disk -- RESTORE, PRINTB, GC-DUMP -- and the copy is of the 1983 tape 9006255, 37 megabytes, kept under ~/scratch/pymdl/era-root (PYMDL_ERA_ROOT). Use a persistent working directory: these sessions write files that later measurements may need. The same directory holds the saved compiler images ecomp.55save, pcomp.55save and npcomp.55save, which a session can RESTORE to become the compiler of chapter 34.

cp -r ~/mdl-1971-1987/mdl/9006255 ~/scratch/pymdl/era-root
chmod -R u+w ~/scratch/pymdl/era-root

MDL 55 from source. A second MDL 55, assembled from MIT's own sources by the user in a provenance-controlled tree (~/mdl/src/mdl55, PYMDL_MDL55), is what tests/test_mdl55_oracle.py asks. Its driver, ask.py, lives with that tree and has moved twice; the test walks up from the tree to find it rather than encode a path that rots.

The simulator is not the oracle. apsim is itself an unverified program where MDL's behaviour depends on the operating system, and 36.5 has the case that cost three sessions: its terminal calls are stubs. An answer that depends on ITS is confirmed against the ITS and MDL sources in the archive before anything is changed to match it.

37.2 Three instruments

They do not overlap, and each is shaped by what it can and cannot see.

Values, in batch. tests/test_mdl55_oracle.py writes a list of expressions into MUDSYS;MUDDLE INIT, boots MDL 55 with no terminal, and reads back what each printed; pymdl is asked the same expressions and the two must never disagree. About 122 expressions in under a second, part of the quick gate, and usable by hand for one:

PYTHONPATH=src python3 tests/test_mdl55_oracle.py -e '<FIX -3.7>'

It never compares the wording of an error, and it cannot ask anything stateful, because the driver it uses abandons the rest of the file on an error and starts a fresh interpreter (37.4). It skips, and says which piece is missing, when apsim or the tree is not there; it used to be run by hand, which is how it rotted three ways at once without anything noticing.

Banners, one boot per form. tests/era_battery/ is one instrument in four parts. Five generators write forms -- 720 unique -- of five kinds: illegal calls at the documented arity (259), legal calls and what the 55 answers (134), two mechanisms meeting (112, the cases the other two are blind to by construction), one argument too many from a legal baseline (66), and the same without a baseline (151), using the finding that the 55 checks the argument count before the types, so a row of zeros one past the documented maximum asks the arity question by itself. The sweep boots the era interpreter per form, about six seconds each, so the whole battery is over an hour; the post-processors print a table and emit entries for subrs/argcheck.py with the error atom taken from the oracle, because adding an argument check by hand means guessing which atom the 55 names, and this project was wrong about that often enough to build a machine to stop. This is the only instrument that can see that the 55 says FIRST-ARG-WRONG-TYPE where pymdl said ARG-WRONG-TYPE. A battery row is a lead, not a finding: some of what it flags is the era's own quirk, and four rounds were spent learning that.

A session, form by form. tools/era_session.py is the era terminal: boot the ITS image, wait for the listener, perhaps RESTORE a compiler, then type forms and read each answer back, in one interpreter that keeps its state. Four tools carried their own copy of that driver until it was made the one copy; era_oracle's collect (34.8), era_saves (the same case compiled with every archived compiler image, to tell an engine property from a revision property), mdl55_errhandler (the error handler's behaviour: which level you land at, whether a nested error stacks, what a bare ERRET does), and the one-off probes that measured FLOAD's INCHAN (chapter 2), REMOVE's new atom (chapter 18) and, today, ALLTYPES (chapter 35) all run through it.

$ PYTHONPATH=src python tools/era_session.py --save ECOMP '<1 <ALLTYPES>>' '<CHTYPE <TYPE-C LOSE> FIX>'
  <1 <ALLTYPES>>                               -> LOSE
  <CHTYPE <TYPE-C LOSE> FIX>                   -> 0

Found along the way, driving the terminal. Each fact below cost a wrong measurement first. MDL 55 activates a form on ESC, not newline: a newline-fed session sits at LISTENING-AT-LEVEL 1 having read nothing, which a harness once reported as "the era evaluated nothing". Every form is followed by a second form that PRINCs a unique marker, built as <STRING "@@K" "n@@"> so its text never appears in an echoed input line, and the answer is everything up to the marker. <RESTORE> flushes read-ahead console input, so its marker would be eaten; the driver waits for the compiler's own NOW READY instead. <RUNTIMER 30000> goes first, because the default is "about 8 cpu minutes" and a corpus run is longer. A second DEFINE of a name errors unless <SET REDEFINE T>. An error drops the listener to LISTENING-AT-LEVEL 2 and every later answer is suspect, so the level is checked after each form and unwound with <ERRET>. Long values come back folded at 80 columns unless the init file widens the channel, <PUT ,OUTCHAN 13 10000>. And apsim simply goes away partway through a large corpus -- at case 216 of 369, once -- so the collector restarts a session on what is left, which is why a 369-case run still completes in under a minute.

37.3 Three ways to measure nothing

An instrument that can report agreement without having looked is worse than no instrument, and this project has built several. They are kept in the record because the shape recurs.

• Fresh interpreters that look like one. Asked through the batch driver, <+ 1 FOO> <+ 2 BAR> <+ 3 BAZ> answers LEVEL 2 three times, which reads exactly like "MDL 55 never stacks levels" and is three fresh interpreters each erring once. Two versions of the error-handler comparison were built on that reading and measured nothing; the second found eleven differences that were all the oracle restarting. The third feeds forms to the terminal, ESC-terminated, in one session, and found that levels are counted per process and the banner names the process.
• A comparison of one field. The first apsim instruction differential (36.1) reported "354 opcodes swept, 0 differing". Its key normaliser matched 'ac ' where the parser produced 'ac0', so every case compared the program counter alone. It now refuses to run unless at least seventeen fields compare.
• A configuration that could not run. EC_RAW=1 was read only inside a function the driver never called, so the authentic path passed its fourteen samples for weeks while running the default build under another name (34.7.3). The gate now demands evidence that the staging happened, not only the score; and a configuration whose sources are absent skips and says so, because a skip is a statement that something could not run, not that it would pass.
• A stub that looked like a fact. apsim's TTYGET returns zeros, which happen to describe a particular terminal, and three sessions reasoned about a case-folding difference from them before the cause turned up in apsim's own input path (36.5).

The countermeasures are ordinary and are used: controls that must fail (era_refusals aborts unless ADD1 compiles the same and DIVG compiles differently, 34.11), mutation testing of a probe before believing its verdict (the error-handler probes were mutation-tested twice), asserting the answer of a compiled case and not only its word count (34.8), and naming the machine a floating-point result was measured on, because a KS10 and a KL10 answer differently (36.1).

37.4 From a measurement to a line in this book

The chain has four links and every Measured box in this book has walked them.

1. Ask. A question the manual does not answer, or a divergence a test or a program surfaced, is put to MDL 55 as forms, through whichever instrument fits: a value in batch, a banner, a stateful session. The 35th chapter's ALLTYPES question was one form and one answer.
2. Pin. The answer becomes a test in tests/, with the date and the image it was measured on, so the engine cannot drift from it silently. Many tests' docstrings begin with the measurement; the harvest (below) reads them.
3. Change, and record. The engine is made to agree. Where the change is to an era source pymdl carries -- a bug in the compiler's own MDL -- the fix goes into the vendored file and its diff into compil/CHANGES.md (34.10), and any oracle row it moves is marked as differing by design. Where the answer is an era program's own property rather than the interpreter's (the CELEST declaration of chapter 2), it is documented and left.
4. Write. The box says what was measured, when, on which image, and what pymdl does about it. A box begins Measured when it pins a fact the manual states or implies, Found along the way when the manual is silent and the fact was discovered doing something else, pymdl's own when there was no era behaviour to measure. Appendix E gathers them.

tools/book_harvest.py exists so step 4 is selection rather than rediscovery: it collects every "MDL 55, measured" comment in the source, every test docstring that states a measurement, the tools' docstrings, the CHANGES.md sections and the commits that record a measurement into one working file (docs/pymdl/harvest.md), each tagged with a first guess at the chapter it belongs to.

37.5 What cannot be asked

• The Tenex personality. MDL 105's manual describes it (chapter 3), and a TOPS-20 MDL 104 exists outside this repository, but no harness here drives it; the Tenex behaviours in this book are the manual's, and are marked so.
• What the image does not have. A bare MDL 55 has no DSKDATE -- 76 of pymdl's 426 default globals are absent from it, of which only four are ever probed for by era code -- and no ACTIVATE-CHARS; a question about those is a question about the library or the game, not the interpreter.
• The operating system. Where the answer depends on ITS, apsim's answer is checked against the ITS and MDL sources before it counts (37.1).
• Time. A full oracle collection, a full self-host tree run and a full tape diff are hours each and disturb the machine they run on; they are run when someone decides to, and the checked-in answers (tools/era_oracle.tsv, the numbers in tools/era_selfhost.py) carry their dates so a reader knows what they were measured against. A targeted case -- python tools/era_oracle.py collect W1 ZO -- is cheap and is the normal way.

37.6 Running it yourself

piece	where	knob
the tape archive	~/mdl-1971-1987/mdl	PYMDL_ERA_CORPUS
the second archive (TOPS-20 compiler sources, pcomp.load.13)	~/muddle/<mdl.comp>	PYMDL_ERA_MUDDLE
apsim	~/pdp10-xdev/bin/pdp10-apsim	PYMDL_APSIM
a writable ITS root with mudsav/ts.mud55 and the .55save images	~/scratch/pymdl/era-root	PYMDL_ERA_ROOT
MDL 55 built from source, with ask.py	~/mdl/src/mdl55	PYMDL_MDL55

Every test that needs one of these skips without it and names what is missing. The tools that ask the era are listed in tools/README.md; the one to start with is era_session.py, and the one question to ask first is the one you were about to reason about.