34. The compilers

pymdl provides two compilers through the COMPILE package. The native compiler translates MDL to Python and is the default. MIT's archived compiler runs as interpreted MDL and produces PDP-10 code, which pymdl's emulator can execute. The two share an interface, but their output, configuration, and limitations differ.

For practical use, begin with Two compilers. Continue to the native compiler for Python output or MIT's compiler for PDP-10 output. The later sections describe comparisons with MDL 55, self-compilation, and changes to the archived source.

Sections 5.1–5.3 explain the historical compiler system: the compiler interface, COMBAT, and compiler internals. These explain the historical system behind the interface. In particular, PCOMP, NPCOMP, and the COMBAT batch service are historical programs, not commands to start pymdl's compiler. Notes identify the facilities provided here.

Chapter 5. The Compiler

The purpose of the MDL compiler is to transform interpreted MDL code into assembly language. The compiler comes in several incarnations for various purposes.

PCOMP is a program which runs the 'installed' compiler -- that is, the one which is most debugged, supported, and otherwise official. The "P" stands for 'purified,' incidentally.

NPCOMP is a program which runs a newer, less well-debugged compiler, if there is one. NPCOMP is often where development work of one sort or another is being debugged.

The 'Batch Compiler', often called COMBAT, though strictly speaking the name refers to a different program (see section 5.2) is a program that compiles, at night, those compilations that have been queued for it.

The remainder of this chapter describes the specifics of interaction with the compiler, including a section on its internals.

pymdl's own. There is one incarnation here, and it is a package: <USE "COMPILE"> brings in COMPILE, COMPILE-GROUP, FILE-COMPILE, FCOMP and STATUS, and nothing else of the compiler exists before that, as the real compiler was a separately loaded MDL and not part of the interpreter. The package does not autoload; a program that says <COMPILE ...> without the USE gets the unbound atom it would have got on an era MDL that was not a PCOMP. The package's entries live on its own OBLIST, so <LOOKUP "COMPILE" <ROOT>> stays #FALSE () after the USE, which matters to programs that test that very thing (34.5.5). There is no demon and no queue: a compilation runs when it is asked for, in the asking process.

5.1. Interfacing to the Compiler

The operation of the MDL compiler is controlled by a few very high-level functions and a sometimes bewildering array of ATOMs whose values are switches and data. This section will describe each such ATOM and its use. The reader should bear in mind that in the normal case he will be using COMBAT to set up his compilations and thus will not have to deal directly with these ATOMs and calls.

5.1.1. Compiler Functions

<COMPILE source:function-or-list output:channel>

is the lowest level call to the compiler. It compiles exactly one FUNCTION (or a LIST of them) and prints the generated code on the CHANNEL given as the second argument. COMPILE is used primarily for compiler debugging.

Found along the way. The COMPILE in the compiler source pymdl carries, cdrive.mud from the April 1981 tape, does not take a channel. Its argument list is

(FCNS "OPTIONAL" (SRC-FLG <>) (BIN-FLG T) (CAREFUL T) (GLOSP <>)
      (REASONABLE T) (GLUE T) (ANALY-OK T) (VERBOSE <>) ...)

so the second argument is a source flag: true means print the assembler source instead of assembling it. The output goes to ,OUTCHAN. What COMPILE returns is the STRING "DONE", after PRINCing a message of the form Job done in: ... for each function; the compiled RSUBR is not returned but stored, by SETLOC into the GLOC of the function's atom, so that ,NAME is the compiled code afterwards. pymdl's COMPILE keeps the manual's two positional arguments -- a source, then a channel or <> for ,OUTCHAN -- prints the RSUBR there, returns it, and stores it under the atom as cdrive does, on both compilers (measured on both, 2026-09-10, tests/test_compile_package_native.py).

<FILE-COMPILE input:string output:string>

FILE-COMPILE attempts to provide a convenient interface between the user and the compiler. The user simply gives FILE-COMPILE the name of an input file, and it can do all the rest. The user may specify other information about output files, compiler modes, etc., but if he doesn't, reasonable assumptions are made.

FILE-COMPILE works in the following way. First it reads in the input file and collects into a LIST the names of all of the defined FUNCTIONs that it finds. It sorts this LIST based on which FUNCTIONs call which other FUNCTIONs. The FUNCTIONs which call no other FUNCTIONs are at the beginning of the LIST, followed by those that only call FUNCTIONs that call no other FUNCTIONs, and so on. Groups of FUNCTIONs that are mutually recursive are collected in LISTs subordinate to the main LIST.

Each FUNCTION will produce a separate RSUBR. COMPILE is called successively on each member of the LIST of FUNCTIONs. LISTs of mutually recursive FUNCTIONs are also passed to COMPILE.

After each FUNCTION or LIST of FUNCTIONs is compiled, the resulting RSUBR is written into a temporary file to enable more convenient crash recovery. This file is written in such a way that, no matter when the system crashes, the contents of the temporary file are guaranteed to be in a consistent state.

When all is compiled, FILE-COMPILE writes out an output file which is identical to the input file except that all FUNCTIONs have been replaced with their compiled counterparts. If any of the FUNCTIONs did not compile due to programmer errors or compiler bugs, those FUNCTIONs are left unchanged in the output file.

During its operation, FILE-COMPILE maintains a "RECORD" file which contains all of the messages, warnings and error messages produced by the compiler. It may optionally produce a listing of the object code produced, in MDL assembler format. This is primarily useful for compiler debugging. (Note that a somewhat less complete listing may be made at a later time. See section 7.3.)

On ITS, FILE-COMPILE usually runs as a demon called COMBAT ZONE. In this case another interface called FCOMP resides above FILE-COMPILE. This interface reads files that are compilation specifications and passes them to FILE-COMPILE.

<FCOMP %.INCHAN input-file output-file>

As most compiler usage is based on COMBAT plan files, FCOMP is the most-seen driver of the compiler. (Note that the % in front of .INCHAN causes the CHANNEL the PLAN file is being read from to be passed as one argument to FCOMP.)

<STATUS>

is an informational function; it tells how far the compilation of a given group has progressed, which FUNCTION is being worked on, and how many FUNCTIONs remain to be compiled. It also prints the accumulated real time and cpu time since the beginning of the compilation. Obviously, you must ^G the compilation to use it, but see section 8.3.

pymdl's own. All four exist and follow this flow on either compiler: the input is read, its DEFINEd names collected and ordered leaves first with the mutually recursive remainder grouped at the end; each is compiled in turn; a RECORD file, name.record beside the output, receives one line per function, compiled NAME or failed NAME: reason, and a final done: n of m; the output file is the input with the compiled functions replaced and the failures left as they were. What the output file is differs: the native compiler writes a Python object file (34.5.2), MIT's compiler a real NBIN (34.7). There is no temporary file, because nothing here crashes in a way a temporary file would help. FCOMP takes the channel the manual passes it and ignores it, so a plan file written for COMBAT is still a file pymdl can FLOAD. STATUS prints No compilation. before any, and afterwards the group, the function in hand with its ordinal, or Compiled n of m, and the real and cpu seconds; since a compilation here runs to completion in the calling process there is no ^G to type, and STATUS is what you call after.

5.1.2. Compiler Switches

The calls to the various compiler drivers are rather short, for the simple reason that the controlling information is passed to the compiler as the LVALs of a set of ATOMs.

<SET DEBUG-COMPILE!- boolean>

(by default FALSE) causes the compiler to generate extra information about what it's doing. This information is in the form of 'warnings' produced when the compiler was forced to generate less than optimal code. For example, invocations of the arithmetic SUBRs can be open-compiled if the variables used can be determined to be exclusively FIXes. The debugging compiler will warn you if it is forced to resort to less efficient arithmetic calls.

<SET PRECOMPILED!- file:string>

Often, a file of FUNCTIONs has been compiled before, and now only a few FUNCTIONs have been updated and need to be compiled again. Most of the file is already correctly compiled: it is quite wasteful to recompile the entire thing. If a PRECOMPILED is given, the file is loaded before compilation: any functions which have corresponding RSUBRs in the precompilation, and which are not on the REDO list, are not recompiled. It is appropriate to specify the temporary file as a precompilation if your previous compilation was interrupted by a system crash.

<SET REDO!- list-of-atoms>

REDO is a LIST of FUNCTION names to be recompiled, regardless of whether or not they are compiled in the precompilation. In conjunction with PRECOMPILED and PACKAGE-MODE, REDO allows compilation of precisely those FUNCTIONs which have been changed since the last compilation. Note that COMBAT will set up these values more-or-less automatically in most situations.

<SET PACKAGE-MODE!- string>

This should be the name of a PACKAGE, which is assumed to be the PACKAGE being compiled. FUNCTION names in the REDO LIST will be looked up in the appropriate PACKAGE OBLISTs if this flag is set, thereby saving some typing of trailers.

<SET TEMPNAME!- file:string>

The compiler writes intermediate results to the temporary file, which is normally the file "sname;fnm >" on ITS, where fnm is the first name of the input file. It is rarely (if ever) necessary to change that default.

<SET SOURCE!- file:string>

Setting this switch causes the compiler to write out the assembler input it generates. This is sometimes useful for compiler debugging. On ITS, such output normally goes to "sname;fnm SOURCE", where fnm is the first name of the input file.

<SET SPECIAL!- boolean>

The compiler normally assumes that variables which aren't declared SPECIAL aren't SPECIAL. This means that they will be available only to the RSUBR in which they are declared: SPECIAL variables are bound on the control stack, just as all variables are in interpreted code. If this flag is T (by default FALSE), all variables will be assumed to be SPECIAL unless declared otherwise. This is analogous to SPECIAL-MODE being SPECIAL, and it is not recommended that any code be written using this convention.

<SET EXPFLOAD!- boolean>

If true, FLOADs in the file being compiled will be expanded at load time: what was FLOADed before will be treated as part of the file. EXPFLOAD is examined by GROUP-LOAD, and not the compiler itself. The default is FALSE.

<SET EXPSPLICE!- boolean>

If true, objects of type SPLICE (PRIMTYPE LIST) which are encountered in the course of EVALing the forms processed by GROUP-LOAD will be spliced directly into the group; it is therefore a lot like EXPFLOAD. EXPSPLICE is examined by GROUP-LOAD, and not the compiler itself. The default is therefore FALSE. Its only known use has been to make functions at load time and have them compiled.

<SET CAREFUL!- boolean>

Defaults to T. If FALSE, the compiler will omit most of the bounds-checking code it normally generates for NTHs, PUTs, and so on. This obviously will make the compiled code run faster, but also makes debugging the compiled code nearly impossible.

<SET REASONABLE!- boolean>

Defaults to T. If FALSE, the compiler will generate reasonable code only if everything ever called from the functions being compiled is loaded into the compiler. A call to a function not loaded produces an EVAL of a FORM, thereby ensuring that such constructs as "CALL" in the called function will work correctly. This is admittedly pretty unreasonable (if not paranoid), whence the name of the switch.

<SET GLUE!- boolean>

Defaults to T. If FALSE, the compiler will not generate GLUE bits. As you always want GLUE bits, there is no reason to ever change this.

<SET MACRO-COMPILE!- boolean>

Defaults to FALSE. If non-FALSE, the compiler will compile MACROs into RSUBRs. This doesn't change anything produced by macro expansions, but does cause the expansion to speed up. Since the compiler expands the macro and then compiles the expansion, this is rarely useful.

<SET MACRO-FLUSH!- boolean>

Defaults to FALSE. If non-FALSE, MACROs which appear in the file being compiled will not appear in the resulting NBIN. This saves space, at the expense of making debugging harder.

<SET MAX-SPACE!- boolean>

Defaults to FALSE. If non-FALSE, the compiler flushes from core most of each RSUBR once it has been compiled; only the DECL is needed to help compile other functions. Since the entire RSUBR is written out in the temporary file, no information is lost. This can, for compilations which are too large, result in considerable improvements in speed, primarily because more space is available in the MDL and less time is spent in the garbage collector.

<SET HAIRY-ANALYSIS!- boolean>

Defaults to T. If this is not set, the compiler will not perform the complex type checking it usually does. If HAIRY-ANALYSIS is FALSE, the code will be generated faster, as type-analysis is expensive, but will not execute as fast.

pymdl's own. These switches belong to MIT's compiler, and when it is the compiler running (34.7) they are its own LVALs inside the companion session that hosts it -- CAREFUL, REASONABLE, GLUE and the rest are the very "OPTIONAL"s of cdrive's COMPILE quoted above. pymdl's FILE-COMPILE does not forward the caller's settings of them into that session, so a compilation from the listener runs with the compiler's own defaults: CAREFUL and GLUE on, DEBUG-COMPILE off, no precompilation, no SOURCE file. The native compiler has no use for most of them: there is no temporary file, no assembler input to write out, no GLUE bits and no macro RSUBRs. Two have counterparts in its behaviour. SPECIAL off is the only mode it has -- an undeclared local is a Python local, invisible to callees -- with the difference described in 34.5.3 that the compiler declares SPECIAL for itself the locals it can see need it. CAREFUL is on and cannot be turned off: a #DECL is enforced in compiled code because the interpreter enforces it, and turning that off would be the era's CAREFUL <> as a default rather than a setting. EXPFLOAD and EXPSPLICE are GROUP-LOAD's (chapter 29) and behave as described when the era compiler's GROUP-LOAD reads the input.

5.2. COMBAT

The usual method of dealing with the compiler is through the program COMBAT, whose specialty is the preparation of 'plan files' to be loaded by the compiler. COMBAT is a program which knows about each of the previously described compiler switches and the interactions among them. It has an easy-to-use interface, an ability to store commonly used 'plan files' as compilation types, and in general is designed to make using the MDL compiler a less-cumbersome task.

pymdl's own. COMBAT is not in pymdl. It was an ITS program of its own whose product, the plan file, is a file of MDL forms ending in a call to FCOMP; pymdl reads such a file with FLOAD, and that is the whole of COMBAT's contract that survives. The section is reproduced because the plan files it describes are what the era's compilations were made of, and the questions it asked are the switches of 5.1.2 in a second guise.

5.2.1. User Interface

COMBAT's user interface is patterned after, though not identical to, a CALICO interface [1]. In particular, it expects in response to any given prompt a particular type of input from the user, which may be a file name, a 'symbol', or text. Ordinarily, the type of input expected is indicated by the 'syntactic prompt' which follows the normal prompt; this is one of '(FILESPEC)', '(SYM)', and '(TEXT)'. The 'Toggle verbosity' compilation type turns the printing of the syntactic prompt on and off, and causes a tailor file to be written out when used.

A number of special characters are defined for any of these types of input:

• ^@: Clears the input buffer, as in MDL.
• ^D: Redisplays the input buffer, as in MDL.
• ^L: Clears the screen and redisplays the input buffer, as in MDL.
• ^G: When given as the first character of an answer, allows one to get the answer from a user-defined type. See the section on tailoring.
• ^Q: Has special effects when a compilation plan is being made (see below). See also the section on file name input.
• ^R: Causes COMBAT to 'back up'. Typically this means go to the previous question asked, but in certain modes it may have a slightly different effect. When a MUDCOM is running, this kills it and backs up to the last question asked.
• ^S: Abnormally ends whatever is being done, and returns to the 'Type of compilation' question. If a MUDCOM is running, it will be killed. When a long compilation plan ('How to run' is 'Many') is being made, the portions already made will be saved. See the 'Flush many' compilation type.
• ?: When given as the first character of an answer, this causes a more detailed description of what is expected to be printed, along with the current default and how to obtain it.
• \: This quotes whatever character follows it, including DEL, ESC, etc. It does not have the effect of quoting strange characters in file names: see the section on file name input. \, used as a quote character, never echoes, and cannot be rubbed out.

In addition, when the syntactic prompt is (SYM), ^F is useful (see below).

5.2.1.1. Symbolic Input

If you are familiar with CALICO, this section can probably be skipped. When entering symbolic input, one need only type the characters required to uniquely specify the desired choice: the interface will complete the response, and in addition can display the available choices at any point.

SPACE completes the response as far as it can. If the response is uniquely specified, it will be displayed in its entirety, followed by '!'; if more than one choice is still possible, then the portion of those choices which is unambiguously specified will be displayed, followed by '?'. For instance, if 'Expand floads' and 'Expand splices' are among the choices, and "Ex SPACE" has been typed, 'Expand ?' will be displayed if the "Ex" reduces the choices to those two.

In some cases, if SPACE is the first character typed, it will select the default (first) choice and terminate.

When ^F is typed, all remaining choices will be displayed.

To terminate responses in this mode, either ESC or CRLF may be used. In either case, the current response is completed as far as it can be. If only one choice then remains, the answer is terminated and the single choice will be used. If more than one choice is possible, it is just as if SPACE had been typed.

Typing ESC or CRLF before any other characters have been entered causes the default answer to be used.

5.2.1.2. Filenames

File names are expected in the standard dev:sname;fname1 fname2 format on ITS; on Tenex/TOPS-20, standard file name recognition is used. Typically, typing simply ESC or CRLF answers 'no' to the question, while SPACE ESC says 'use the default'. In certain special cases ('Input file' and 'Output file'), when some answer to the question is imperative, the default will be used in either case. File names should not be surrounded by quotes in this mode; they are not MDL STRINGs!

It is rather painful to get funny characters (such as SPACE) into file names. When the file-name parser sees ^Q, it uses the following character in the name being generated regardless. Unfortunately, the ^Q must be quoted to get it past the reader, since it has special effects in the normal case. Thus, the file name given to MDL as "AA; FOO>" has to be typed to COMBAT as AA: \^Q FOO>.

5.2.1.3. Text

Text is just that: relatively arbitrary characters, terminated by ESC. Since CRLF is allowed in text, it does not terminate input. Text type input is used in a number of cases where it isn't quite appropriate, such as the 'Redo list' and 'Package mode' questions. If it is known that the expected response is a LIST of STRING, as in those cases, the appropriate brackets or quotes should never be typed.

5.2.2. Combat Questions

This section discusses the questions that can be asked of the user during the preparation of a COMBAT plan file, which is FLOADed by the COMBAT demon or by PCOMP to effect a compilation. The perceptive reader will notice a strong resemblance to section 5.1.2, in which the switches relevant to the compiler are listed. Questions asked by the pre-existing compilation types ('Verbose' and 'Short') are so indicated. All questions are available in user-defined compilation types (see section 5.2.5).

'Sname': sets the default directory for questions that want a file name as an answer; also causes the FORM <SNAME sname>, where sname is the answer given, to be included in the plan. This sets the default directory for files referenced by the compiler; it also causes the temporary file (see below) to go to the sname directory.

'Use new compiler?' (Verbose and Short): specifies whether the 'new' compiler or the 'old' compiler should be used. Often, when there is only one compiler, this question will not be asked. If answered affirmatively, it causes the FORM <OR <GASSIGNED? EXPERIMENTAL!-> <NEWCOMP!->> to be included in the plan. This FORM will load a new compiler on top of the old if necessary.

'Debugging compiler?' (Verbose): causes DEBUG-COMPILE!- to be set to T, which causes the new compiler to generate extra information about what it's doing. This currently is asked only if the new-compiler question is answered affirmatively.

'Input from' (Verbose and Short): the file to be compiled. This appears in two places in the plan: as <SETG COMBAT!- input-file> and in the call to FCOMP described below.

'Output to' (Verbose): the file name to be used for the NBIN. The default is the input file name, with NBIN as the second file name instead of whatever it was for the input. This completes the call to FCOMP that ends every plan:

<FCOMP %.INCHAN input-file output-file>

This call is what actually invokes the compiler.

'Precompilation from' (Verbose): specifies a file containing a previously compiled version of the input file. Any FUNCTIONs which have corresponding RSUBRs in the precompilation, and which are not on the 'Redo' list, are not recompiled. It is appropriate to specify the temporary file as a precompilation if your previous compilation was interrupted by a system crash. Sets PRECOMPILED!-.

'Compare with' (Verbose): This question is asked only if a precompilation file is specified. If answered affirmatively (user types either SPACE ESC or a file name) MUDCOM (see section 8.1) will be run with jcl of the input file name, and the file name provided here (the default is as for precompilation), plus some extra stuff specified below. If 'FOO NBIN' is given here, then MUDCOM will look for the newest revision of FOO which was created before the NBIN. MUDCOM determines which FUNCTIONs in the file have changed and therefore need to be recompiled. It also determines whether the file is a PACKAGE, and answers the 'Package mode' question appropriately. It is therefore not usually necessary for the user to answer the 'Redo' and 'Package mode' questions directly.

'Check macros?' (Verbose): asked only if 'Compare with' is answered affirmatively. This adds '/M' to the jcl passed to MUDCOM, which causes it to check for MACROs and MANIFESTs which have changed: if a FUNCTION uses a MACRO or MANIFEST which has changed, the FUNCTION will be listed as changed. MUDCOM does not normally check for this.

'Extra JCL' (Verbose): asked only if 'Compare with' is answered affirmatively. Whatever is supplied here will be passed to MUDCOM as jcl, before the files to compare. This can be used to load macro files: see section 8.1.

'Redo' (Verbose): asked only if a precompilation file was given. Takes a bunch of FUNCTION names, which will be recompiled. Note that the names supplied here will be appended to the list returned by MUDCOM, if any, and that duplications in the list are ignored. Sets REDO!-.

'Package mode' (Verbose): asked if a precompilation file was given and MUDCOM was not run (MUDCOM will set this if run). This should be the name of a PACKAGE, which is assumed to be the PACKAGE being compiled. FUNCTION names in the 'Redo' list will be looked up in the appropriate PACKAGE OBLISTs if this flag is set, thereby saving some typing of trailers. Sets PACKAGE-MODE!-.

'Temporary file to': The compiler writes intermediate results to the temporary file, which is normally

"sname;fname1 >"        (on ITS)
"<sname>fname.TEMP"     (on Tenex/TOPS-20)

You may change that by answering this question; there is rarely a good reason to do so. Sets TEMPNAME!-.

'Source file to': The compiler can be caused to write out the assembler input it generates by answering this question. Assembler output normally goes to

"sname;fname SOURCE"     (on ITS)
"<sname>fname.SOURCE"    (on Tenex/TOPS-20)

which is the default for this question; another name may be provided if desired. Sets SOURCE!-.

'Special?': The compiler normally assumes that variables which aren't DECLed SPECIAL aren't SPECIAL. If this flag is T (defaults to FALSE), all variables will be assumed to be SPECIAL unless declared otherwise. Sets SPECIAL!-.

'Expand floads?' (Verbose): If true, FLOADs in the file being compiled will be expanded at load time. Sets EXPFLOAD!-.

'Expand splices': If true, objects of type SPLICE (PRIMTYPE LIST) will be expanded and inserted into the group. Sets EXPSPLICE!-.

'Careful?' (Verbose): By default T, but if FALSE, the compiler will omit most of the bounds-checking code it normally generates for NTHs, PUTs, and so on. This obviously will make the compiled code run faster; it also makes debugging the compiled code nearly impossible. Sets CAREFUL!-.

'Reasonable': By default T, but if FALSE, the compiler will generate reasonable code only if everything you call from the functions being compiled is loaded into the compiler. Sets REASONABLE!-.

'Glue?': By default T, but if FALSE, the compiler will not generate GLUE bits. There is no good reason to ever answer this. Sets GLUE!-.

'Macro compile': By default FALSE, but if true, the compiler will compile MACROs. Sets MACRO-COMPILE!-.

'Macro flush': By default FALSE, but if true, MACROs which appear in the file being compiled will not appear in the NBIN. Sets MACRO-FLUSH!-.

'Max space?': By default FALSE, but if true, the compiler flushes from core most of each RSUBR once it has been compiled; only the DECL is needed to help compile other functions. This can, for compilations which are very large, result in considerable improvements in speed. Sets MAX-SPACE!-.

'First things to do', 'Things to do' (Verbose), and 'Last things to do': It frequently is necessary to perform some actions before a compilation can be run: definitions files must be loaded, special environment setup might have to be performed, and so on. All three of these questions are designed to allow that: whatever you supply is put out after everything else in the plan but before the call to FCOMP. There are three questions, instead of one, to allow some things to be specified in a tailored compilation type, while others are provided at compile time, or possibly from another tailored type. The three questions do not depend on each other, they are asked in the order given here, and the answers appear in the plan in the same order.

5.2.3. Requesting Compilations

The first question asked by COMBAT is 'Type of compilation'. In addition to a number of special possibilities described later, there are two answers to this question (in addition to any provided by the user through the tailoring facility) which request pre-defined tailored compilation types. These are 'Verbose' and 'Short'.

'Verbose' causes all the normal questions to be asked: 'New compiler?', 'Input file', 'Precompilation', switches, 'Things to do', and so on. 'Short', on the other hand, defaults the answers to all questions except 'New compiler', 'Input file', and 'How to run'.

When requesting a compilation, one may type ^Q at any time. This has the same immediate effect as an ESC, but in addition causes all questions between the one just answered and the 'Things to do' question to be defaulted. This is particularly useful in the 'Verbose' sequence of questions.

If 'Many' was given as 'How to run' for a previous compilation request, and the resulting plan has not yet been written out, subsequent plans will be appended to it. Using 'Many' will sometimes effect a major savings of time if several compilations wish to perform the same environmental setup; if they USE many of the same PACKAGEs, for example. When using 'Many' in combination with predefined compilation types, it is useful to remember that whatever is specified under 'Things to do' may end up being performed for each plan. You might modify your compilation types to reflect this, or alternatively, edit the plan file produced by COMBAT to remove redundant operations.

The only way to get rid of the 'Many' plan is to answer 'Many flush' to the 'Type' question. Typing ^S or answering 'Abort' to the 'How to run' question will abort the current portion of the 'Many' compilation, but not the whole thing.

If 'Many' was mistakenly given as 'How to run', and you don't wish to destroy the plan you have already generated, it is possible to (in essence) go back to the 'How to run' question by answering 'Many print' for the compilation type. In this case, you are not back in the plan-making loop; ^R acts just like ^S.

^R, here, backs up to the last question asked. There are two qualifications. First, if ^Q has been typed, then it backs up to the last question that would have been asked if ^Q had not been typed. Second, the four questions 'Precompilation', 'Compare', 'Redo', and 'Package mode' are treated as a group: if the 'Package mode' question has not yet been answered, it is possible to back up normally; but once that question has been answered, backing up to it will go to the first member of the group, 'Precompilation'.

^G allows one to obtain the answer to the current question from any user-defined compilation type. It requests a type name, and uses the answer or default supplied therein, printing the information so obtained. The ^G must be typed as the first character of the answer for this to occur. This allows one to use parts of a defined type without either using the type itself or altering it for the occasion. For 'Text' type input (such as 'Things to do'), the string is placed in the input buffer but not completed, so it may be edited before an ESC is typed. See also the 'Xerox type' command.

Note that there is a distinction made between 'Compare' and 'Redo': the former causes a MUDCOM to be run, and the latter asks for the names of FUNCTIONs to be recompiled. It is possible to do both, in which case the two groups of FUNCTIONs are appended to form the 'Redo' list for the compilation. Note also that if a MUDCOM has been run, the 'Package mode' question will not be asked, since the answer is supplied by the MUDCOM. Either ^R or ^S may be used to kill a running MUDCOM.

One of the responses to the 'How to run' question is 'Abort', which returns directly to the 'Type of compilation' question without writing out a plan, starting up a PCOMP, or anything else. Its effect is exactly that of a ^S. In particular, if you are making a long plan, only the portion just completed, not the entire compilation, will be aborted.

It is also possible at the 'How to run' question to supply an answer to any of the compilation questions (Input file, etc.). The 'Question' response asks for the name of a question, then asks that question. Any number of questions can be asked in this manner, one at a time. This is particularly useful for filling in the blanks left by a 'Short' type compilation, or by user-defined compilation types.

When a compilation request has been finished, COMBAT normally loops back to the 'Type of compilation' question, but changes the default from 'Verbose' to 'None' (meaning 'Quit'), unless another compilation may reasonably be expected. Thus, one may leave by typing a single ESC.

It is possible to modify COMBAT's behavior such that it either kills itself after finishing the compilation plan, or loops back with 'Verbose' as the default for the 'Type of compilation' question.

COMBAT first decides whether a long compilation plan is being made; if so, the default remains 'Verbose'. If not, it then examines the current compilation type: if 'Another compilation?' has been set to 'Yes', the question will be asked with default 'Verbose'; if it has been set to 'No', COMBAT will kill itself; if to 'Ask', further consideration is required.

If the user is in 'Multiple' mode (the 'Multiple' compilation type), the type of compilation will be asked with the 'Verbose' default. Otherwise, COMBAT examines the state of two tailorable switches, set by the 'Another compilation?' compilation type. If 'Another compilation?' has been set to 'No', COMBAT will die; if to 'Yes', the type question will be asked with default 'Verbose'; if to 'Ask', the type question will be asked with default 'None'. Normally this is 'Ask'.

Note that 'Another compilation?' is like 'Toggle verbosity' in that it will have no effect unless user-defined compilation types exist.

5.2.4. 'How to Run' Options

There are four options available when answering the 'How to Run' question which determine where your plan file will be written and when the compilation it specifies will be run.

'Pcomp' places the plan file on the <SNAME> directory, and names it "<PCOMP>". Additionally, COMBAT will start a PCOMP (or NPCOMP, as appropriate) process if it is exited after writing the <PCOMP file. 'Pcomp' is the standard method for running a compilation in one's own process.

'COMBAT' writes the plan file to "COMBAT;PLAN>". The COMBAT demon successively compiles all such plans at night, informing the persons who submitted them of the result.

'Waste' is like 'COMBAT', except that the plan is written to "COMBAT;WASTE>". The 'waste' queue is only run after midnight, which is usually sufficient for those who are doing 'overnight' compilations. 'Waste' is the answer used by default for 'How to Run'.

'File' places the plan file on the <SNAME> directory, and names it "<PLAN>". This means that it will not be run until you explicitly load it into a compiler process.

5.2.5. User Tailoring

It is often the case that a particular file is compiled quite often, or that some sequence of actions must be performed as the 'Things to do' before many compilations. COMBAT allows the user to define his own 'Compilation types', each of which specifies exactly those questions which should be asked and the answers for those which should not. For example, one could have a type named 'Esign', which says that the input file is always "SEND;ESIGN>" and in addition provides for the FLOADing of two files in 'Things to do'.

Further, since most questions are defaulted, one might choose to answer only those questions which are interesting, such as 'Precompilation'. It is also possible to supply a default answer for a question which will be asked.

In addition, there are some questions which are not asked by the 'Verbose' compilation type, but which nevertheless are available to user-defined types. These are: 'Macro compile', 'Macro flush', 'Max space', 'Expand splices', 'Special mode', 'Glue', and others.

One can select any of one's own defined compilation types as an answer to the 'Type of compilation' question, just like 'Verbose' and 'Short'. Except that the questions asked may differ, user-defined types are identical to the predefined types.

5.2.5.1. Tailor Files

User-defined types are saved (and loaded) from the file "sname;%COMBT TAILOR". It is possible to load other tailor files, but the "%COMBT" file in sname is loaded during startup. Tailor files are quite similar to MDL GC-DUMPed files and thus cannot be edited other than with COMBAT.

5.2.5.2. Create Type

This special compilation type requests a name for the type being made, then enters a loop with the prompt 'Question'. One may choose any of the available questions, and either supply an answer or (by default) request that the question be asked when a compilation of this type is being submitted. Note that only the 'How to run' and the following 'Type of compilation?' questions will be asked unless others are explicitly supplied; but one may supply answers to 'How to run' when creating a type.

In this mode, ^R will return to the 'Question' loop if one is about to supply an answer; otherwise, it returns to the 'Type of compilation' loop, aborting the type creation.

^G behaves exactly as it does in the normal loop. To indicate that one is finished, one should answer 'Finis' to the 'Question' prompt. It is possible to supply several different versions of the answer to a particular question: the last one given will be used. One may wish to default a particular question, after specifying that it was to be asked or after supplying some different default. This may be done by answering 'Delete question' to the 'Question' prompt, whereupon one will be asked for a particular question to ignore. This question will then be completely ignored. Note that all interesting questions are initially in this state.

There is also a 'Set question default' 'Question'. This requests a question name, then asks the user to supply an answer. The question will be asked, with the default supplied. Thus default settings of switches can be changed, and one can supply a file name for the precompilation while still being asked whether precompilation is desired. Unfortunately, user-supplied defaults for 'Text'-type questions are used if ESC is answered; to get rid of the default, type SPACE ESC. Note that this is exactly the inverse of the convention for defaulting file names.

When 'Finis' has been typed, a new copy of one's tailor file is written out. This may, in combination with 'Load tailor' and 'Replace tailor', have undesirable side effects.

5.2.5.3. Print Type

This requests the name of one of the types currently loaded, and prints out for it all questions which either will be asked when a compilation is being submitted or which have user-supplied defaults. If a particular question has been globally 'turned off' (such as the 'New compiler?' question, when there is no new compiler), an asterisk will be printed on the appropriate line to indicate that the information there is currently not used.

5.2.5.4. Delete Type

This requests the name of one of the currently-loaded types, and deletes it. A new copy of the tailor file is written out so all trace of the type will vanish when this command is used.

5.2.5.5. Alter Type

This requests a type name, then becomes identical to 'Create type', except that some questions already have answers. Again, 'Finis' must be typed to leave the loop and cause the modifications to be filed; typing ^R or ^S will leave the loop, but the modifications will be forgotten.

5.2.5.6. Load Tailor, Replace Tailor

Both of these request a file name, defaulting to the last one used for either a 'Load tailor' or 'Replace tailor' command. Initially this is "sname;%COMBT TAILOR". 'Load tailor' appends the types defined in the specified file to those already loaded, while 'Replace tailor' first throws away those already loaded. The types defined in this way are not distinguished from those loaded from one's own COMBAT tailor file; in particular, using 'Toggle verbosity' or any of 'Create', 'Alter', and 'Delete type' will cause all the types currently loaded to be written out to the COMBAT tailor file. If, therefore, one has done a 'Replace tailor', one can easily lose all of one's own types in this manner. I.e., it is very easy to destroy yourself.

5.2.5.7. Xerox Tailor

This requests the name of an existing user-defined type, and a new type name. The new type becomes an exact copy of the previously-existing type. This is particularly useful when one has several different types which do almost the same thing.

5.3. The Compiler (Internals)

The compiler's job is to take an MDL FUNCTION or group of FUNCTIONs and produce an operationally equivalent machine-language subroutine (RSUBR) using whatever information can be extracted from the source code and whatever additional information the user wishes to supply. The efficiency of the output code produced is directly proportional to the amount of information supplied by the programmer and inversely proportional to the generality of the source program.

The information supplied by the programmer is usually in the form of optional data-type declarations (DECLs) and the use of programmer-defined data types (NEWTYPEs) that have built-in declarations. Unlike many programming languages, however, declarations are never required. The compiler will compile programs with no declarations at all, but the resulting output will not run as fast as with well-declared code.

The current compiler can achieve speed-up factors of anywhere from about 4 to 100. The factor of 4 represents the speed-up for a very general program with very poor declarations. On the other hand, the factor of 100 represents a program with a very narrow range of application that has very good (that is, restrictive) declarations. Typical programs can expect to achieve factors of 20-40.

pymdl's own. Those factors are the era's, machine code against the interpreter on the same PDP-10. Here they invert for MIT's compiler: its output runs on an emulated PDP-10, every instruction a Python step, so era-compiled code is slower than pymdl interpreting the same FUNCTION. That path exists for fidelity -- it is how the era's binaries run and how the compiler is measured (34.8) -- not for speed. Speed is the native compiler's business, and even there the factor is modest, because pymdl's interpreter and the code the native compiler emits run on the same Python.

5.3.1. How It Works

The compiler as it currently exists is really two distinct programs. GETORDER is basically an interface between files of MDL functions and the compiler. It is a relatively small program that reads in the file, sets up the various compiler switches, calls the compiler one or more times and writes out the final file of RSUBRs.

COMPILE itself is basically a compiler with three major and three minor passes. Pass 1 builds a model of the program, pass 2 analyzes each node of the tree and does data type analysis, pass 2.5 (minor) allocates stack space for variables and temporaries, pass 3 generates output code, and two minor passes do final stack allocation and peep-hole optimization.

pymdl's own. The passes are files. The fifty-six modules under src/pymdl/mdl/compil/ are the compiler's PACKAGEs, one per file, and by their entries they fall onto this description:

manual	modules
GETORDER and the drivers	cdrive (COMPILE, COMPILE-GROUP, COMP2, COMPILE-FUNCTION), ncomfi
Pass 1, the model	pass1, mapps1 (the MAPF/MAPR special case), compdec (the NODE type and the 320 shared ENTRYs)
Pass 2, analysis	symana (ANA), with strana, carana, bitana, notana, mapana, varana, backan, chkdcl, subrty (the SUBR tables the analyzer consults), infcmp (the OBLIST and ERRORS SUBRs, analyzers and generators both)
Pass 2.5, the stack and the AC pool	cup (STORE:VAR, CREATE-TMP, END-FRAME: the temporaries and the stack state), cacs (GETREG, SGETREG, the residue and preference slots of 5.3.7)
Pass 3, code generation	codgen (GEN), with strgen, cargen, bitsgen, notgen, mapgen, lnqgen, mmqgen, spcgen, allr, istruc, buildl, case, newrep (PROG/REPEAT/RETURN/AGAIN, the saved control states), comcod (EMIT), comtem, cback, confor
the minor passes	peeph (the peephole optimiser)
messages and printing	advmes, nprint, cprint, prntyp
opcode tables, hacks and helpers	assem (the MIDAS build script), newop, nt, mudref, bophac, mudhak, primhk, sbrnam, eupdat, connect-dir
assembly, in MDL assembler source	atosq, wofch, popwr2 (chapter 35)

FILE-COMPILE and FCOMP are not among them: they belonged to the batch driver, and pymdl supplies them in the package of 34.5.

5.3.1.1. COMPILE and COMPILE-GROUP

There are two distinct modes of compilation available. They are simple and multiple. Simple compilation occurs when COMPILE is called with one FUNCTION. It simply compiles that FUNCTION and returns. Multiple compilation occurs when COMPILE is called with a list of FUNCTIONs. It compiles each FUNCTION into a separate RSUBR. It differs from multiple calls to COMPILE in that it sometimes partially compiles a FUNCTION out of order to determine its calling sequence and do argument type-checking. This behavior is necessary when compiling mutually recursive FUNCTIONs.

In all modes of compilation, COMPILE-FUNCTION is called to actually compile the individual FUNCTIONs. It calls the various compiler passes.

5.3.2. Modeling Pass

The first pass of the compiler takes the input FUNCTION and builds an expanded model of it. In the process of doing this, it produces a symbol table entry for every local variable bound and/or declared in the FUNCTION, any of its PROGs/REPEATs or MAPF/MAPR FUNCTIONs. It also produces the RSUBR DECL for the final output. Pass 1 also tries to decide if an internal entry (that is, an entry which can be called efficiently (see section 6.1)) can be used with this FUNCTION. If an internal entry turns out to be possible, Pass 1 generates an appropriate calling sequence for internal calls to use.

The model built by Pass 1 looks like the original FUNCTION with all of the nodes in the FUNCTION's structure replaced with objects of type NODE (a new type defined for the compiler). A node in the model may have anywhere from 5 to 30 elements; the 5 element node is for simple quoted objects like fixed-point numbers, ATOMs etc. The 30 element nodes are for major elements of the program such as the node for the FUNCTION itself and nodes for PROGs and REPEATs. The majority of the nodes are general SUBR nodes, which have 10 elements.

The Pass 1 structure is built in the following way. The top level program in Pass 1 generates a node for the entire FUNCTION. This node gets the following information put into it:

1. A code specifying that this is a FUNCTION node.
2. The data type that this FUNCTION is declared to return (or ANY).
3. A LIST that will eventually contain the nodes comprising the body of the FUNCTION.
4. A UVECTOR of internal names for internal calls to this FUNCTION.
5. A symbol table for the variables declared and/or bound in this FUNCTION.
6. A list of entries in the symbol table specifying how the arguments are to be set up (whether they are optional, QUOTEd, TUPLE etc.).
7. The final RSUBR DECLs.
8. A specification of how to pass arguments to this FUNCTION when it is compiled (whether the arguments should be in registers or on the stack).
9. The number of required arguments and the total number of possible arguments.

In addition to the above information, slots exist in the node for additional information to be supplied by later compiler passes.

After the main node for the FUNCTION is built, the sub-nodes for the FORMs comprising the body of the FUNCTION are built. This is done by first dispatching to special Pass 1 code for the first element of the FORM. If no special code exists for this first element, a dispatch is made on the TYPE of the first element of the FORM (that is, ATOM, FIX, FUNCTION etc.). If no special code exists for either the first element or its TYPE, a general FORM node is built. In the case of an ATOM as the first element of the FORM, the normal lookup rules are invoked on the ATOM and it is dispatched again based on its value. ATOMs with no values either cause compilation warnings or are assumed to be RSUBRs (depending on compiler switch REASONABLE).

All FSUBRs (COND, AND, OR, FUNCTION, PROG, REPEAT, UNWIND, etc.) have special Pass 1 code and produce very specific nodes. Most SUBRs don't dispatch to specific code during this pass. The exceptions are things like MAPF, ILIST, GET etc., which have somewhat non-standard treatment of their arguments. (Actually, MAPF and MAPR don't treat their arguments non-standardly, but they are treated specially in Pass 1 so that the inner FUNCTION may be open compiled.)

As mentioned previously, all nodes have at least 5 elements. These are as follows:

1. A node type code.
2. A pointer to the parent node (if one exists).
3. A specification of the data type the node will generate.
4. A list of sub-nodes referred to as Kids.
5. A name for the node, which may have different meanings for different nodes.

In addition, nodes other than nodes for QUOTEd objects have additional elements that are filled in during later passes of the compiler.

After Pass 1 all additional passes work on the model built during Pass 1. The original FUNCTION is no longer even considered.

Found along the way. Pass 1 walks the FUNCTION as a structure, and it expects the structure MDL 55 builds. A FUNCTION that names its own activation, <DEFINE F ACT (A) ...>, is three elements long on the era machine: the ATOM, then the argument list, then the body (<1 ,AR10> is ACT and <PRIN1 ,AR10> is #FUNCTION (ACT (A) ...), measured on MDL 55). pymdl once carried that atom in a Python attribute instead, which made the FUNCTION two elements long and the activation invisible to any MDL code that walked it; the compiler is exactly such code, and compiled <RETURN .A .ACT> as a RETURN through a free variable, 21 words against MDL 55's 16. The layout is the era's now (tools/era_oracle.py case AR10; chapter 12).

5.3.3. Analysis Pass

During Pass 1, very little information is determined regarding the resulting data types of various nodes. Indeed, with the exception of nodes produced by quoted objects, structured objects which will produce code to build copies of themselves, and FUNCTIONs, PROGs and REPEATs with declared values, no type information is produced. Even in the cases where type information is produced during Pass 1, it is usually not as detailed as other passes would like. The Analysis Pass has the job of refining the result type of each individual node based on various criteria:

1. The declared types of the variables used in the program including GDECLs and MANIFESTs.

2. The known type transformations produced by various SUBRs. (For example, it is known that LENGTH always produces a FIX result.)

3. Some analysis of the context of the nodes within the program. (For example, in the following code:

    <COND (<AND <TYPE? .X LIST> <NOT <EMPTY? .X>>>
           <1 .X>)>
   

   regardless of how X is declared, it is obviously a LIST when the EMPTY? is run, and it is obviously not empty when the <1 .X> is run.)

The Analysis Pass performs a standard depth-first left-to-right tree walk on the Pass 1 model. The main dispatch function during this pass is called ANA. It does an initial dispatch based on the node type of each node. Since most nodes are still considered 'SUBR nodes', most of the dispatches end up at the SUBR call analyzer. The SUBR call analyzer has two types of further dispatch available. First it looks in a table for SUBRs that are capable of being completely open-coded: if it finds an entry in the table, the analyzer for that SUBR is invoked. If this SUBR is incapable of being open-coded, ANA checks another table to see if this SUBR has an internal entry available. If it does, the node is changed from a SUBR node to an internal SUBR node. If both dispatches fail, another table is checked to see if the object type returned by this SUBR is known, and if it is the result is put into the SUBR node.

Most of the work done by the Analysis Pass happens when the first dispatch occurs and special SUBR analyzers are invoked. Generally speaking, these analyzers check to see if they know enough about their arguments to transform their nodes to an open-code specification. For example, an invocation of the SUBR REST only transforms to an open-code node if both the PRIMTYPE of the first argument is known at compile time and there are no SEGMENTs in the call to REST. If a special SUBR analyzer decides that it can't open-compile in this case, it either leaves the node as a SUBR node or transforms it to an internal SUBR node.

Found along the way. Two analyzers in symana.mud carried bugs that every surviving revision shares, and both were found by this pass disagreeing with the code generator about a shape. BOOL-AN, the analyzer for AND and OR, typed an empty <AND> as FALSE, where the language (chapter 11) and MDL 55's own compiled output say T; and GVAL-ANA set one temporary and tested another. Both are fixed in the vendored source and recorded with diffs in compil/CHANGES.md (34.10); the oracle cases FA5 and FA6 differ from MDL 55 by exactly those repairs (34.8).

5.3.4. The Type Analysis Model

In addition to the model of the FUNCTION built in Pass 1, the Analysis Pass adds additional information to the model concerning the current states of local variables. As the analyzer plunges down into the tree, it tries to keep track of the current DECL of each variable. Specifically, there's a slot in each symbol table entry called CURRENT-TYPE. The analyzer updates that slot based on its current knowledge. A call to SET causes the CURRENT-TYPE slot to be changed to the analyzed type of SET's second argument. When multiple control paths meet, the CURRENT-TYPE slots of a variable are OR'd together at the joining point.

Conditional control structure nodes for COND, AND and OR also maintain two lists of transient information. These are called TRUTH and UNTRUTH. They specify what information will be valid if the true or false branches are taken respectively. For instance, a COND clause compilation can assume that any TRUTH information generated in the predicate of the COND will be valid for the rest of the clause.

Some of the analyzers for the more widely used predicates have special code in them to add information to the current TRUTH and UNTRUTH values. These predicates include TYPE?, EMPTY?, LENGTH? and NOT.

Looping control structures pose additional problems for the type analysis model. The approach taken by the type analyzer is to build a copy of the current types of all variables before analyzing the loop structure. This copy of the local type information constitutes the assumptions currently in effect. After the loop analysis is complete, the assumptions are checked against the current state of the variables. If any of the assumptions have been violated, the assumptions are updated and the loop is re-analyzed.

5.3.5. Life-and-Death Analysis

The Analysis Pass also performs a life-and-death analysis on the local variables. This is done by assuming that the variable's value is dead at each LVAL node for that variable. If another LVAL node for this variable is discovered that is reachable from this one before any intervening SET nodes for this variable, the original node is updated to be alive. This life-and-death information is used during the Code Generation Pass.

Found along the way. The variable that carries this analysis is LIFE, DEFINEd in symana and read as a caller-bound SPECIAL by mapana, a different PACKAGE. In the revision pymdl's source descends from, LIFE is missing from symana's ENTRY list, so under the package system the two files' LIFEs are different atoms and the MAPF analyzer read an unbound one. The era never met this: its compiler was built from flattened FBINs with bare atoms, one LIFE by default. Every other lineage of the file has LIFE in the list, and it is restored here (compil/CHANGES.md, symana.mud).

5.3.6. The Variable Allocation Pass

The Variable Allocation Pass (VAP) is a relatively simple one. Its purpose is to allocate stack space for all of the variables bound in the FUNCTION, its PROGs and REPEATs and its MAPF/MAPR FUNCTIONs. There are various switches that control the manner in which this allocation is performed.

The most important switch specifies whether or not this FUNCTION needs a FRAME or not. The VAP always starts out assuming it does not need to build a FRAME. This assumption will be changed if it is discovered that externally accessible named ACTIVATIONs exist in the FUNCTION or any of its inner blocks (PROGs or REPEATs or FUNCTIONs) or if at any time it is discovered that the address of a variable cannot be specified as a fixed offset from the top of the stack. Whenever this assumption is changed, the VAP starts over again with the new assumption in effect.

Another switch that controls the behavior of the VAP specifies whether or not the stack slots for inner blocks will be pre-allocated because the stack will be in a 'fuzzy' state when these blocks are running. The stack is said to be in a 'fuzzy' state when the number of slots currently being used cannot be determined at compile time. This usually occurs when a TUPLE is being constructed for a MAPF. For instance, in

<DEFINE F (X Z)
   <MAPF ,VECTOR <FUNCTION (Y) <G? .Y .Z>> ...>>

the elements of the VECTOR will be between the top of the stack and the location of variable Z. Even if F has a FRAME, the location of Y will not be known relative to the FRAME pointer at compile time. Therefore, the initialization code for F will pre-allocate the stack space for Y.

During the VAP, each symbol table entry gets its address field set based on where that variable will be on the stack. Also nodes for PROGs, REPEATs and MAPF/MAPR FUNCTIONs that have bound variables get additional information inserted in themselves. This information includes where the SPECIAL variables start and where the UNSPECIAL variables start.

Found along the way. A MAPF with no structure arguments -- a loop that runs until MAPSTOP, which mat.mud's GET-FILENAMES-FROM-FILE writes -- hands the allocator a sentinel slot count of -1. MDL 55 refuses some of these shapes outright at compile time (BAD-SYNTAX, for a MAPSTOP with an argument in a zero-structure MAPF) and compiles the rest; pymdl's compiler died NEGATIVE-ARGUMENT allocating -1 frame slots until mapgen was given the guard the 55-era tape's revision of it carries. With the guard every instruction word is the era's, and the constant pool is one dead literal shorter, which is why oracle cases MF4, MF5 and MF8 differ from MDL 55 by one word each (34.8).

5.3.7. The Code Generation Pass

The Code Generation Pass (CGP) is probably the most complicated of all the passes. Fortunately, the Analysis Pass has already refined the model so that the CGP can dispatch immediately to the special-purpose code generators. Besides building a list of assembly-language instructions as output, the CGP keeps track of the current state of the stack, the contents of the registers, the current state of variables (whether they are in registers or on the stack or both) and the contents of the temporaries.

The general dispatch routine during the CGP is called GEN. It takes two arguments: a NODE and a specification of where to leave the result. The second argument can be any of the following:

1. The ATOM FLUSHED, meaning that the code will be executed for effect rather than value.
2. The ATOM DONT-CARE, meaning that the caller of GEN is leaving the decision up to the specific generator as to where to leave the result.
3. An object of type DATUM which specifies a place for the type and value of the result to be left.

Type DATUM is of PRIMTYPE LIST and contains two elements, one for the type and the other for the value. The elements of a DATUM may take on a variety of values in different circumstances. These include:

1. A TYPE name. This can only occur in the type slot and it means that the type of the object is known at compile time and this is it. It indicates that the code generator need not put the type-code anywhere.
2. The ATOM DONT-CARE. This means that the caller doesn't care where the result for this field is left.
3. The ATOM ANY-AC. This tells the generator to leave the result in any available AC.
4. An object of type AC. This tells the generator to force the result into a specific AC.
5. An object of type ADDRESS:C or ADDRESS:PAIR. Both of these specify addresses on the stack or in the interpreter.
6. An object of type OFFPTR. An OFFPTR has three fields: a DATUM, an offset (a FIX), and a PRIMTYPE. An OFFPTR tells the generator to leave the result in the word pointed to by the inner DATUM and offset by the offset.

If an element of a DATUM is ANY-AC or DONT-CARE, the generator is required to update the DATUM to reflect the actual location of the result. If the element is a TYPE, the generator may change it to an AC which means that it happened to end up with the TYPE in that AC.

The generators always return a DATUM specifying where the result was actually left, unless the caller wanted the result FLUSHED. There is one special DATUM that can be returned. It is the GVAL of the ATOM NO-DATUM and it means that the specified node will not return a value (that is, it is a RETURN or an AGAIN or something).

There are six objects of type AC in the compiler, corresponding to ACs 0, A, B, C, D and E. AC 0 is special since it can't be used as a pointer, and it always contains very transient information. It is never used to fill in an ANY-AC slot in a DATUM. The other five ACs are in the pool of available ACs. Objects of type AC have about ten different slots associated with them. They are used for finding available ACs and generating output code that uses them. The slots used in AC allocation are as follows:

1. ACLINK. If this is FALSE, the AC contains no temporary value for the current computation. Otherwise, it is a list of active DATUMs that contain it.
2. ACAGE. This is only used when the ACLINK is non-FALSE. It is updated to a higher number at each use of the AC and is used in an LRU algorithm when an active AC must be flushed.
3. ACRESIDUE. If this AC is currently equivalent to some local variables, this slot contains a list of the symbol-table entries for these variables. The symbol-table entries themselves have a slot called INACS that points back to the ACs that contain its type and/or value. They also contain a slot called STORED that specifies whether the only copy of the variable is in the ACs or it is also in memory.
4. ACPROT. This slot is a boolean saying whether this AC is protected or not. If the AC is protected, it can't be allocated for any reason. Protection is only invoked for very short stretches of code.
5. ACPREF. This slot says that this AC deserves slightly preferential treatment. It means, all other things equal, don't choose this AC.

The AC allocation algorithm consists primarily of trying to find the best possible candidate when an AC is needed. The routine GETREG is used to find an available AC. First it rejects all ACs that are protected (if they all are protected, the compiler generates an internal error since this should never happen). If there are one or more ACs with their ACLINKs FALSE, GETREG will choose from among them. It will prefer ACs with no ACRESIDUE, that are numerically adjacent to another free AC (because some PDP-10 instructions destroy the next AC), and which do not have their ACPREFs on. If the AC chosen has an ACRESIDUE, code is generated if necessary to store any of the variables that are only in ACs.

If no AC exists with an ACLINK that is FALSE, GETREG finds the AC with the smallest ACAGE. Code is generated to store the contents of the AC in a temporary so that it is available. The DATUMs that were in the ACLINK are updated to indicate that they are now pointing to temporaries as opposed to ACs. Thus it is possible that a generator could need sub-results in ACs, and after causing one to be generated in an AC, find that while generating the second one the first slipped back into a temporary. The generator would then have to generate code to reload an AC from the temporary.

The CGP invokes various special-case optimizations by passing information up and down the tree as code is generated. The generators for conditional branching FSUBRs like OR, AND and COND employ a predicate generator whenever possible. This generator is like GEN except that it takes three additional arguments: a label to branch to, a flag saying whether to branch on truth or falseness, and a flag saying whether this predicate is being NOTed. The general predicate generator then looks at the predicate node to see if it can take the additional arguments for predicate generation. If it can, the general predicate generator just passes all the arguments down; otherwise it calls GEN and generates the additional testing and branching code itself. Currently AND, OR, COND, ==?, N==?, G?, G=?, L?, L=?, 0?, 1?, TYPE?, NOT, ASSIGNED?, MEMQ, LENGTH? and EMPTY? have special predicate code associated with their generators. Others may be added as the need develops.

Other optimizations are invoked by simply recognizing common patterns of MDL code. For instance, the compiler recognizes <SET X <+ .A 1>> as a PDP-10 AOS instruction and it generates very efficient code for <REST .X <- <LENGTH .X> 1>> by recognizing the pattern of code.

The compiler always takes advantage of as much knowledge as it has about the types generated by particular nodes to generate good code. This is especially the case when it is handling the code for NTH, REST and PUT in structures. It uses type information concerning the length of the structure and the amount being RESTed for the NTH, REST or PUT, to figure out whether or not to generate bounds checks in the compiled code. It also uses information about the current type of the slot being read or written to decide whether or not to read or write the type word. Obviously, a lot of this type information was the same information obtained during the Analysis Pass of the compilation.

Some code generation routines are capable of changing the order of generation of the sub-nodes. This is done to try to get the node requiring the most ACs compiled first so that it won't interfere with any AC requirements of the current node. This obviously requires that the commuted nodes have no interacting side effects.

Found along the way. The generator for loops stores a variable back to its stack slot at each point a loop can leave an AC, and decides where by replaying saved control states (newrep.mud: LOOP-REPEAT, LOOP-RETURN, MERGE-RETURNS). Get that wrong by one store and the code still assembles, still runs, and quietly returns the wrong LIST, which is how one oracle case (W5) spent a whole arc described as "one extra word" when it was a miscompilation. Since then the oracle checks the answer of every case as well as its words (34.8), and the loop-store shapes are a test of their own (tests/test_compiled_loop_stores.py).

34.4 Two compilers

	native	MIT's, interpreted
where	pycomp.py, in the interpreter	src/pymdl/mdl/compil/, 56 MDL modules, loaded by tools/era_compiler.py
produces	a Python callable in an RSUBR's code slot (#PYCODE)	PDP-10 words in an RSUBR's code slot (#CODE), and a real NBIN
runs on	Python	the emulator of chapter 36
needs	nothing beyond pymdl	the compiler's helpers from the tape (tests/data/era or the archive), and PYMDL_ERA_COMPILER set
when it cannot compile	refuses, leaves the FUNCTION interpreted, writes the reason down	compiles nearly anything and warns; where the scoping rules of 5.1.2 bite, the code answers differently and the programmer was expected to have written <SPECIAL X>
measured against	the interpreter: the whole suite run with every FUNCTION compiled, zero differing answers	MDL 55 itself: 640 shapes compiled on both, word for word (34.8)
chosen by	the default	an ERA argument, <SETG COMPILER ERA>, or PYMDL_ERA_COMPILER in the environment

The switch is the era's own idiom, an optional argument defaulted from a GVAL, the way GROUP-DUMP's printer is (PRINTER ,PPRINT):

<FILE-COMPILE "prog.mud">                   the default: native
<FILE-COMPILE "prog.mud" "prog.nbin" ERA>   this call, MIT's compiler
<COMPILE FOO <> ERA>                        likewise, for one FUNCTION
<SETG COMPILER ERA>                         every call after it
<SETG COMPILER NATIVE>                      and back

With nothing said, the compiler is MIT's when PYMDL_ERA_COMPILER names the directory holding era_compiler.py and native otherwise. Asking for ERA without that variable set is an error, not a silent fallback: COMPILER is ERA but the era compiler is not available. <PYCOMPILE FOO> is always native and never consults the switch.

34.5 The native compiler

34.5.1 COMPILE

<USE "COMPILE">
<DEFINE SQUARE (N) #DECL ((N) FIX) <* .N .N>>   ⇒ SQUARE
<COMPILE SQUARE <>>        ⇒ #RSUBR [#PYCODE SQUARE SQUARE #FALSE ()]
<TYPE ,SQUARE>             ⇒ RSUBR
<SQUARE 7>                 ⇒ 49
<DEFINE QU ('A) .A>        ⇒ QU
<COMPILE QU <>>            ⇒ #FALSE ("a QUOTEd parameter")
<TYPE ,QU>                 ⇒ FUNCTION
<LOOKUP "COMPILE" <ROOT>>  ⇒ #FALSE ()

The result is a chapter 33 RSUBR: APPLICABLE, TYPE? RSUBR, applied through its code slot, which holds a PYCODE rather than a CODE because the code is Python and an era RSUBR's code slot answers CODE rather than the type a built-in answers. The third element is #FALSE () where the era compiler writes the DECL: the native compiler enforces the #DECL inside the generated code instead (34.5.3). A refusal prints failed NAME: reason on the channel, answers #FALSE with the reason, and changes nothing. The same RSUBR can be had without the package and without the switch:

<DEFINE CUBE (N) <* .N .N .N>>      ⇒ CUBE
<PYCOMPILE CUBE>                    ⇒ CUBE
<TYPE ,CUBE>                        ⇒ RSUBR
<CUBE 3>                            ⇒ 27
<PYCOMPILE <FUNCTION ('X) .X>>      ⇒ #FALSE ("a QUOTEd parameter")

PYCOMPILE of an ATOM compiles its GVAL and rebinds it, answering the atom; of a FUNCTION, answers the RSUBR and binds nothing. Two companions serve the manual's listing and RECORD. <PYCOMPILE-SOURCE name> is the Python a compile would emit, as a STRING, without compiling or binding anything -- the era's assembler listing, and the way a mis-compile is read; an already-compiled RSUBR answers the source it was compiled from. <PYCOMPILE-RECORD> is what the native compiler has done in this interpreter, oldest first, each entry a name, whether it compiled, and the reason if not; <PYCOMPILE-RECORD T> reads and clears it.

34.5.2 FILE-COMPILE and the object file

<USE "COMPILE">$
<FILE-COMPILE "sq.mud">$
"sq.mud.py"
<STATUS>$
Group: SQ
Compiled 3 of 3
Real time: 0.0  CPU time: 0.0
T

For a source sq.mud holding SQUARE, SUMSQ (which maps SQUARE) and the QU above, the RECORD file sq.record reads

FILE-COMPILE sq.mud
compiled SQUARE
compiled SUMSQ
failed QU: a QUOTEd parameter
done: 2 of 3

and the output file is sq.mud.py: the source's full name with .py appended, because that is the name of a sidecar, the per-file Python that FLOAD runs automatically after reading the source it sits beside. The object file is therefore loaded by loading the source. <FLOAD "sq.mud"> in a fresh interpreter reads the three DEFINEs, then the sidecar's install rebinds SQUARE and SUMSQ to their compiled RSUBRs, and QU stays the FUNCTION the source defined: the manual's "identical to the input file except that all FUNCTIONs have been replaced with their compiled counterparts", with the failures unchanged, done by the loader rather than by rewriting the file (tests/test_compile_package_native.py).

The object file is text. Each compiled function is the generated Python as a string, its constant table and its #DECL patterns as MDL text that is re-read at load, because an MDL object is not a Python literal; the header says what wrote it and that reading the source runs it. A second STRING argument names the output file instead; the RECORD goes beside it either way.

The source is loaded into a scratch interpreter to be compiled -- the era compiled in a separate PCOMP job for the same reason, that compiling must not disturb the session asking for it -- and that scratch load is the compiler reading the file.

Found along the way. In an era compiler image FILE-COMPILE is a ROOT global, and a file that starts something when loaded tests for it. DOCTOR's last form is <OR <GASSIGNED? FILE-COMPILE!-> <DOCTOR>>, commented "START IT UP": run the session unless we are being compiled. pymdl's scratch load sets FILE-COMPILE on ROOT for exactly that test, and the alternative -- a session of DOCTOR reading the terminal in the middle of a compile -- is what it did before it did (demo/doctor.mud, tests/test_demos.py).

34.5.3 What the code means

Two choices in the generated code are semantics and not accidents, and both are the era's.

Calls stay late bound. A call to a named function looks the callee up through its GVAL when the call runs, so redefining it takes effect exactly as in interpreted code. This is the era's unlinked mode of chapter 33; RSUBR-LINK is where a linked mode would cache the callee.

Arguments and locals become Python locals. This is the compiler's documented scoping change (5.1.2, SPECIAL): an undeclared local is invisible to callees. A FUNCTION whose callee reads its caller's .X behaves differently compiled, on the era too, and the compiler does not pretend otherwise. What it does, and the era compiler did not, is declare SPECIAL for itself: a local that a FUNCTION literal, a QUOTEd form, a retyped structure literal or a binding SUBR (ASSIGNED? and its kin) names is put on the control stack where the interpreter resolves it -- the remedy an era programmer wrote <SPECIAL X> for, applied because the compiler can see which locals need it. OUTCHAN and INCHAN are promoted wherever a function binds them, because the I/O SUBRs read them implicitly and no scan of the source can find the use. When one binding of a FUNCTION's parameters is SPECIAL they are bound in sequence, in the order of chapter 12, so a later initialiser reads an earlier SPECIAL through the interpreter.

Measured, 2026-08-31. Over the library's 439 top-level DEFINEs, the automatic SPECIAL took the native compiler from 299 compiled (68.1%) to 372 (84.7%) in one round; the largest refusal, a FUNCTION literal reading a local of the function it sits in, went from 54 to zero. The era compiler was asked about that shape first, and compiled it and kept the answer -- <MAPF <> <FUNCTION (X) <SET ANY .X>> (1 2 3)> .ANY> is 3 interpreted and 3 era-compiled -- so the refusal had been a limitation here, not a semantic the era shared. Later rounds took it to 413 (94.1%).

Declarations are enforced. A #DECL is checked in compiled code as the interpreter checks it, because DECL-CHECK is on by default and a violating argument is a TYPE-MISMATCH there; ignoring it would be the era's CAREFUL <> as a default rather than a setting.

34.5.4 What it refuses, and why

The compiler is deliberately partial, and that is what makes a partial compiler safe: everything it cannot compile without changing the answer it refuses, the FUNCTION stays interpreted, and the RECORD says so. The refusals that remain over the library are a floor, each defensible on its own line:

refused	why
an "ARGS" parameter	needs the argument FORMs at the callee, which the RSUBR calling convention does not carry; the interpreter itself calls evaluated application of one MEANINGLESS-PARAMETER-DECLARATION (chapter 12)
a QUOTEd parameter	the same forms; MIT's compiler compiles it, and pymdl's call path honours the "QUOTE" marker in the RSUBR's DECL (33.10), so the era-compiled QU above answers the FORM -- this is a limitation here
<READ chan '<RETURN>>	READ's end-of-file form reaches for the enclosing PROG through a SUBR call; interpreted PROGs bind it and compiled ones deliberately do not, so compiling the caller would change where the end of file lands
a TUPLE/ITUPLE initialiser in an "AUX" list	the dummy of chapter 12
"CALL", "BIND" and the other argument-list markers, a second "NAME" activation, a required parameter with an initialiser	outside the compiled subset; each is rare in the library corpus and stays interpreted

Measured, 2026-09-09. The whole test suite run with every FUNCTION compiled as it is DEFINEd -- PYMDL_PYCOMPILE_ALL=1, the bar in tests/test_comprehensive_bars.py -- stays at zero differing answers, banners and printed output included. PYMDL_PYCOMPILE_ONLY restricts the hook to named functions for bisecting, and PYMDL_PYCOMPILE_LOG writes what a run compiled (Appendix D).

34.5.5 The compile-vs-interpret gate

Era source that behaves differently under the compiler tests for the compiler's presence, and it tests two ways. A file that starts something looks for FILE-COMPILE!- on ROOT (34.5.2). The library's own sources look for COMPILE there -- <LOOKUP "COMPILE" <ROOT>> -- because pcomp.load moved COMPILE and COMPILE-GROUP to ROOT in a compiler image. pymdl's package keeps its entries on the package OBLIST, so that gate stays #FALSE () in an ordinary session even after <USE "COMPILE">, and the library loads as it does on an interpreter that is not a PCOMP.

34.6 Embedding Python

On the PDP-10 an RSUBR's code vector was machine code. In pymdl the host language is Python, so the faithful analog of hand-written machine code is a Python callable, and two SUBRs let new MDL source embed one inline:

<TYPE <SETG SQ2 <PYCODE "lambda interp, args: T.Fix(args[0].value ** 2)">>>  ⇒ RSUBR
<SQ2 7>                                                                      ⇒ 49
<PYDEFINE CUBE2 (N) "return T.Fix(args[0].value ** 3)">                      ⇒ CUBE2
<CUBE2 3>                                                                    ⇒ 27
<TYPE ,CUBE2>                                                                ⇒ RSUBR

The result is a genuine RSUBR, applied via its code slot, and nothing else in the interpreter can tell it from a compiled one. The embedded code takes (interp, args), the arguments already evaluated, and returns an MDL object; in scope are interp, args, T (the types module) and MDLError. PYDEFINE takes a parameter list for the RSUBR's DECL and a body whose return is the value. No era file uses either, so every authentic load is unaffected; a host that loads untrusted source sets the interpreter's allow_pycode false and both refuse, as do the PYCOMPILE family, which use the same door. The pymac peers of chapter 33 are the same mechanism at the module level: a Python file per era RSUBR name, installed at the apply boundary.

34.7 MIT's compiler, running interpreted

The compiler is an MDL program, so pymdl can run it. The difficulty was never the running; it was the sources. They are scattered across tape snapshots that disagree with one another, a few helpers survive only as compiled binaries, and the load file assumes an ITS disk. tools/era_compiler.py stages a workable tree, loads it the way pcomp.load does, and hands it a function; pymac/compile.py drives the same thing from the listener.

34.7.1 From the listener

With PYMDL_ERA_COMPILER=tools in the environment, or an ERA argument, COMPILE and FILE-COMPILE boot a companion session -- a second interpreter holding the compiler image, one per process, booted on first use and kept, accumulating code-region and heap use across compilations as one PCOMP job did -- and talk to it in MDL. COMPILE DEFINEs the function there and applies the compiler's COMPILE to it; FILE-COMPILE writes the input under a WORK; directory the session can see, GROUP-LOADs it, compiles the functions leaves first, and GROUP-DUMPs the group, which is a real NBIN (33.11) with each <DEFINE ...> become <SETG name '#RSUBR ...> over a binary portion.

Measured, 2026-09-10, on the sq.mud of 34.5.2.

<COMPILE SQUARE <> ERA>$
#RSUBR [#CODE ![23851171840 23851171841 23751557124 ... 262146!] SQUARE #DECL ("VALUE" FIX FIX)]
<TYPE ,SQUARE>$          ; stored, as cdrive's COMP2 stores it
RSUBR
<SQUARE 7>$
49
<FILE-COMPILE "sq.mud" "sq.nbin" ERA>$
"sq.nbin"

SQUARE is 17 words of PDP-10 code, printed in decimal as a CODE prints (33.11), with the DECL the compiler derived, ("VALUE" FIX FIX). Three seconds for the one function and two for the file of three: the compiler is being interpreted. The RECORD reads compiled for all three, QU included, and in a fresh interpreter <FLOAD "sq.nbin"> answers "DONE", ,SUMSQ is an RSUBR whose first element is a CODE, <SUMSQ (1 2 3)> is 14, and <QU <+ 1 2>> is <+ 1 2>: the compiled QU receives the FORM because the RSUBR's DECL says "QUOTE" and pymdl's caller honours it (33.10). The compiler's own switches, 5.1.2, are LVALs in the companion session and are not forwarded from the caller.

34.7.2 What the staging reconciles

Each of these was learned the hard way, and each is a fact about the era's sources rather than about pymdl:

• Prefer a source that declares <PACKAGE ...>: only a package answers USE, and the snapshots disagree (one tree's CONFOR is a bare file of DEFINEs where another's is a package).
• A module is staged under both its declared name and the one the load file asks for: symana USEs "ADVMESS" while the load file says "ADVMES", and a USE that misses aborts the whole form, taking its siblings with it.
• A file is staged under every path a snapshot FLOADs it by -- CLR;MUDREF NBIN and PS:<COMPIL>MUDREF.NBIN -- and at the root as well, because CONNECT-DIR never runs here and bare-name FLOADs resolve at <SNAME>.
• COMPDEC declares 320 ENTRYs, and any module that interns one of those names first takes it, so COMPDEC precedes every other module; SYMANA sets its analyzer table only if ARITH-ANA is assigned, and that is a CARANA entry, so the analyzers precede SYMANA.
• SBRNAM is an assembler source the load file never names, because the era shipped a compiled SBRNAM.NBIN; it supplies HACK-NAME, and pymdl's assembler (chapter 35) builds it at the end of the load, along with the three routines of 33.13 that survive only as assembly.
• The image's top-level .OBLIST is reset to (INITIAL ROOT PP EDIT), four long as the era's is, at the end of the load. An FLOAD that errors out of a file leaves that file's <PACKAGE> binding in place -- on MDL 55 as here, both measured going 4 to 3 -- and the compiler's own re-FLOADs of SYMANA and CODGEN had left the image thirteen deep, which is how <TITLE ...> instructions inside a compile of the assembler came to resolve to a redefinition of TITLE with the wrong .NAMED.

34.7.3 Four configurations

The compiler can be built four ways, and the gate test (tests/test_era_compiler_gate.py) runs each on fourteen sample functions, compiles them, runs the result and checks the answer:

configuration	what it is	14 samples
default	the tree as it ships: the modules from MDL source, the helpers and the library as era FBINs where the era had them	14/14
EC_RAW=1	MIT's own compiled FBINs of the compiler modules, staged and executed on the emulator -- the authentic path, 35 era binaries	14/14
EC_SOURCE=all EC_NO_MUDSAV=1	every module from MDL source, the pure-code database left out	14/14; also with the emulator off (PYMDL_NO_PDP10, tests/test_compiler_without_emulator.py)
EC_ITS=all	every module from its ITS-side revision	0/14, a known failure: every sample dies LVAL of unbound atom: TAG:COUNT!-; the test is a strict expected failure where those sources are present and skips where they are not, and says which

Two of the four were once the default build wearing another name, and both passed 14/14 while doing nothing, which is why the gate now requires evidence that the staging happened -- the count of wrapped FBINs -- and not only the score. EC_RAW_EXCEPT names modules to build from source inside a raw build, one at a time, so a difference can be laid at a module or, if none carries it, at the interpreter. EC_PEEP matters for the oracle: MDL 55's compiler prints Peephole 0% on every compile, so the image compared against has the peephole pass on, and a word-level comparison has to match. EC_PYMAC_PEERS and EC_NO_PYMAC_PEERS register only the named Python peers of 33.14, or none, to measure what each changes. Appendix D has the whole list.

Measured, 2026-09-10. The default build had, until that day, loaded the era's pureq.nbin and asked it whether each emitted instruction was a pure constant. cup.mud's EXP-MAC walks an instruction that is not pure and evaluates its operand sub-forms in place -- a <- ...> or a <GVAL ...> becomes its value -- and the era binary, run on the emulator, answered T for everything, so no instruction was ever walked. The native PUREQ answers honestly, the walk ran for the first time, and it exposed a real interpreter defect: REST of a FORM was copying where the era shares cells (chapter 10), so an operand rewritten through a REST did not change the instruction it was part of. Fixed in struc.py, and the oracle rows that had moved came back identical.

34.8 Word for word: the oracle

A compiler's output can be compared to the era's two ways, and only one of them means anything. The tape binaries are not a code-generation reference: a compiled module on the tape is the output of the compiler generation before the one it contains, so a byte difference against it can be version drift. ecomp.55save, the compiler image on both MDL 55 tapes, is a compiler that ran, byte-identical on both, and it compiles our source -- which holds the source fixed and leaves only the engine varying. That is the arrangement in which a disagreement is a finding.

tools/era_oracle.py holds a corpus of 640 shapes, deliberately systematic rather than interesting: argument forms, control shapes, structure building, arithmetic with the DECL as an axis, mapping, mutation and access, type operations, strings, vectors, globals and recursion, SPECIAL, nested and anonymous functions, UNWIND, associations, bits, atoms and oblists, predicates, conversions, FORM/EVAL/APPLY, deep nesting, TUPLE access, FALSE as a type, many arguments, activations across a PROG boundary -- one axis at a time, so a divergence lands on a named feature. collect compiles them on MDL 55 under apsim and writes the words to tools/era_oracle.tsv; compare compiles them here and diffs, and needs neither apsim nor the archive because the era's answers are checked in. Each case carries a call, and compare checks the answer as well as the words, because an answer is the cheaper and the sharper test (5.3.7's box).

	cases
in the corpus	640
MDL 55 produced code	629
MDL 55 refused at compile time (kept as NOCODE rows: a refusal is data about MDL 55)	5
MDL 55 errored	6
ours identical, word for word	624
ours differ	5

The five that differ, each by design and each explained where it happens: FA5 and FA6 are the empty <AND> and <OR> shapes, 12 words on both sides but different words, because symana's and codgen's era bugs are fixed here and not in MDL 55 (5.3.3's box, 34.10); MF4, MF5 and MF8 are the zero-structure MAPF shapes, one word shorter here because the guarded mapgen leaves one dead literal out of the constant pool (5.3.6's box). Every instruction word in all five is the era's.

Found along the way, driving MDL 55. Forms are activated with ESC, not newline. <RESTORE> flushes read-ahead console input, so a sentinel sent after it is eaten; the driver waits for the compiler's own NOW READY instead. Every other form gets a <PRINC "@@Kn@@"> sentinel so the reader knows where the answer stops. <RUNTIMER 30000> comes first, because the default is "about 8 cpu minutes" and a corpus run is longer. The second DEFINE of a name errors unless <SET REDEFINE T>. An error drops MDL to LISTENING-AT-LEVEL 2 and every later answer is suspect, so the level is checked after each case and unwound with <ERRET>. And apsim simply goes away partway through a big corpus -- at case 216 of 369 once -- so the collector restarts a session on whatever is left. A CODE prints as #CODE ![n n n ...!] in decimal (33.11), which parses directly: 23851171840 is 261551000000 octal, PUSH TP,0(AB), the first word of nearly every RSUBR.

Every one of these is in tools/era_session.py, the one driver the other tools share (chapter 37).

34.9 Compiling itself

The surviving FBINs are proof by artifact that MIT's compiler compiled its own fifty-six modules. tools/era_selfhost.py asks the same question of the compiler running on pymdl, and of the other trees pymdl carries, and the answer is a number that moves when the engine gets better.

tree	functions compiled	note
compil (the compiler)	891 / 894	the three are era binaries in the image, not FUNCTIONs: atosq, popwr2, wofch
mdllib (the library, chapter 30)	442 / 442
mudbug (the debugging tools, chapter 29)	468 / 468
mprog (the small programs)	20 / 20
the loose files (mat, matxgp, inquire)	34 / 34

Measured, 2026-09-09, the five trees in parallel with a 600-second budget per function: 159, 71 and 75 minutes for the compiler, the library and the debugger; no timeouts, every file loads, every FUNCTION compiles. The compiler's modules are the image; any other tree's file is FLOADed into a fresh image first, its DEFINEs taken, and everything the image had under those names put back, because the compiler must keep running on its own runtime -- datime.mud redefines the RTIME that formats Job done in:, and with the interpreted one installed every compile after the load died UNBOUND DSKDATE inside the printout, on MDL 55 exactly as here.

The two slowest functions are worth knowing. glue's PASS-1 compiles in 247 seconds: some 2.3 million COND evaluations of interpreted compiler, nothing wrong. coding's DO-ASSEMB, the assembler compiling itself, took 420 seconds and two engine fixes -- the .OBLIST reset of 34.7.2 and REMOVE making a new atom (chapter 18) -- because coding.mud is the era's recipe for compiling the assembler with itself: it REMOVEs TITLE from ROOT and re-DEFINEs it at top level, and the compiler's own <TITLE> instructions kept finding the redefinition.

34.10 CHANGES.md: the record

src/pymdl/mdl/compil/CHANGES.md is the record of every way the vendored compiler differs from the tapes, and it is exact because the directory is no longer a transcript. Six repairs were once applied at stage time, by a tool that copied the directory and edited the copy; that kept the files pristine and meant the file you opened was not the file the compiler ran. They are in the files now, with the diffs.

	count
.mud files byte-identical to an archived original	39
modified, with the diff recorded	11 (symana, cacs, newrep, cdrive, codgen, pass1, strgen, bitsgen, comsub, backan, mapgen)
era source plus a package wrapper	1 (spcgen: the 1977 body under the 55-era header, because the one revision is damaged tape and the other predates the package system)
no era MDL original at all: reconstructions	5 (atosq, popwr2, wofch, chapter 35; nt and mudref, one NEWTYPE each)

The modifications are of three kinds, and the file says which each is:

• Restorations from sibling revisions of the same file, where pymdl's lineage (the xcompi tree, the worst match to MIT's own binaries) lacks something the others have, each with the roll call of the tapes that agree: symana's LIFE entry (5.3.5's box), cacs's UNPREFER entry, codgen's SETZ on the optional-argument entry words, strgen's list-counting loop in LNTH-GEN, bitsgen's ZERQ answer for a two-argument PUTBITS, comsub's SUBSTRUC-GEN, the newrep clause present in ten of the eleven archived revisions, and mapgen's guard from the 55-era tape's revision.
• Era bugs fixed, present in every revision, measured against MDL 55 and against the language: symana's BOOL-AN and GVAL-ANA, codgen's BOOL-GEN (5.3.3's box). With them, the repairs no archived revision has, each recorded with its evidence and the limit of that evidence: cacs's RET-TMP-AC reading an unsupplied optional, cdrive's FCN declared SPECIAL in COMPILE-FUNCTION, codgen's marking of a SPECIAL argument as it is bound, and a second newrep clause whose live risk the file states. The rule is that an era bug in vendored source is fixed and recorded, and the oracle rows it changes are marked as differing by design.
• Package-system adjustments, because the era built flattened FBINs with bare atoms and pymdl builds a package per module: "PASS1" in symana's USE list, SPECIALIZE on pass1's entry list, and the RCHK and GET-SUB-DATA exports of strgen and comsub that backan calls across the package boundary.

Each entry gives the archived original, the diff, the measurement that found it and the oracle rows it moved. tools/mdl_manifest.py records the provenance of every file under src/pymdl/mdl/ and --check fails when it is stale; tools/era_source_outliers.py lists where our revision is a minority against the archive and tools/era_source_audit.py checks our signatures against the #DECLs in MIT's own FBINs.

34.11 How the two compilers refuse

A bare percentage of "how much we compile" compares apples to oranges, because the two compilers refuse differently: the native one refuses what it cannot compile without changing the answer, and MIT's compiles nearly anything and warns. tools/era_refusals.py asks, for every shape the native compiler still refuses, the pair that is comparable: did the era compiler produce an RSUBR at all, and does that RSUBR, run on the emulator, give the answer the interpreted function gives. A shape the era compiled and kept the answer on is a limitation here, ordinary work; a shape the era compiled with a changed answer is one the refusal is right about. Two controls run first and abort the run if either misbehaves: ADD1 must come back compiled and same, or the era compiler did not load and every "the era refused it too" would be a lie of omission; and DIVG, the documented divergence of 5.1.2 (a callee reading its caller's undeclared local), must come back compiled and different, or a fault that made the answer column always say "same" would be invisible.