pymdl

The pymdl book

31. Making it run faster

The environment manual's optimization tools reorganize compiled MDL for the PDP-10: they combine routines, replace mediated calls with direct calls, and move data into pure storage. pymdl runs the archived MDL tools, including GLUE, PDUMP, SUBRFY, PURITY, and CLEAN, against the code and storage represented by its emulator.

Read this chapter to understand those transformations and the archived files they produced. Their historical speed benefits are not host performance guarantees; a change must be measured on pymdl itself. For compilation to Python, start with the native compiler in chapter 34.

Chapter 6. Making It Run Faster

Once you have a working program, you will probably want it to run fast. The most obvious way of doing this is to compile it. MDL provides other ways to speed up code, chiefly by eliminating mediated subroutine calls, and by reducing the size of garbage-collected space.

Mediated subroutine calls (or 'MCALLs') are the standard method of function calling in MDL. They provide a great deal of information and control during program development and debugging, but the overhead of an MCALL is superfluous in debugged production programs. Consequently, several methods exist for removing this overhead.

A subtle impediment to increased speed in a production program is the amount of time devoted to garbage collection. As this is proportional to the size of the garbage collected space, it is advantageous to make that space as small as possible. One way to do this is to purify as many of the static data structures in the MDL as possible.

One by-product of the procedures mentioned above is that much of the resulting code and structure becomes pure and therefore shareable between many MDL processes.

6.1. GLUE

A facility exists to allow separately compiled and assembled RSUBRs to be 'glued' together. This makes calls between RSUBRs in the group much faster, as MCALLs are replaced by PUSHJs. The many instructions of an MCALL are replaced by the single PUSHJ, but the mediation provided by MCALL is lost: no FRAME is produced. GLUEing is accomplished by the concatenation of the code and reference VECTORs of the RSUBRs being GLUEd, which gives them a common 'frame of reference.'

Additionally, GLUE is interfaced with the compiler such that:

  1. The RSUBRs can be run unGLUEd for convenient tracing and debugging. After debugging, they can be GLUEd together and run much faster.
  2. An individual FUNCTION can be recompiled without the overhead of recompiling everything GLUEd to its RSUBR. After the recompilation, the entire set can be reGLUEd.

6.1.1. How to Glue

"GLUE" is a PACKAGE and it may be obtained by doing

<USE "GLUE">

The call to glue a group of RSUBRs and/or RSUBR-ENTRYs is:

<GROUP-GLUE group-name:atom
            substitute:boolean
            script:channel
            package:string-or-list
            survivors:list
            victims:list>

where:

group-name is an ATOM as returned by GROUP-LOAD, and it is the only required argument.

substitute is a flag: if it is true, the current RSUBRs and RSUBR-ENTRYs will be fixed so that they may still run in the current MDL. This is expensive but necessary if PRINTTYPEs or interrupt handlers are among the RSUBRs in the group. If the flag is FALSE or not supplied, the group must be GROUP-DUMPed and reloaded before use.

script, if supplied and a CHANNEL, is used by GROUP-GLUE to print out its progress through its task. Otherwise, GROUP-GLUE works silently.

package, if provided and non-FALSE, implies PACKAGE mode will be used. This argument should be a STRING specifying the PACKAGE that is being glued. In PACKAGE mode only the ENTRYs of that PACKAGE will be preserved and all RSUBR-ENTRYs associated with internal functions will be removed. This option can also be used by setting the ATOM PKG to the name of the PACKAGE. package may also be a LIST of PACKAGE names, in which case the ENTRYs of all the PACKAGEs listed will be preserved.

survivors, if provided, indicates that SURVIVOR mode will be used. This argument should be a list of those RSUBR-ENTRYs to be preserved. All other RSUBR-ENTRYs will be flushed. This option overrides PACKAGE mode. This option can also be used by setting the ATOM SURV to the LIST of RSUBR-ENTRYs being preserved.

victims allows 'survivors' to be specified by default: that is, it is a LIST of those functions which should not survive after GLUE has run. This is sometimes more convenient to specify than explicit survivors.

There are two advantages to removing unneeded RSUBR-ENTRYs. The group is made smaller by the absence of the RSUBR-ENTRYs. Also the code for the group is reduced, as the code for handling MCALLs to those RSUBR-ENTRYs is removed. In general only the ENTRYs need to be kept for a PACKAGE. This can be done by specifying the PACKAGE using PACKAGE mode. SURVIVOR mode should be used if the user wishes to explicitly state which RSUBR-ENTRYs are to be kept.

Measured (tests/test_mudbug.py, the round trip of 6.1 through 6.3 and 4.2 with the era's programs). GLUE here is MIT's ELMER (muddle/elmer.ubt149, 1981). A real 1981 NBIN, MUDBUG;FINDAT, is GROUP-LOADed with .GLUE set so the compiler's glue bits attach (6.2); <GROUP-GLUE FINDAT> answers "DONE" and the group's FINDATOM is one RSUBR of more than a thousand words, the whole package concatenated; <PDUMP FINDAT> answers "DUMPED" and writes findat.fbin, which begins '<PCODE "FINDAT">, with findat.sav55 and findat.fix55 on MUDTMP; the maintainer demon adds them to the pure-mapping databases -- ADDING FILE FINDAT SAV55 -- and <SAV-BLOCK "FINDAT" "MUDSAV;SAV FILE"> in a fresh interpreter hands back the glued code word for word. A second PDUMP into the same library walks pdump.mud's uniquifier to its next candidate, 1FINDA -- the digit the manual's 1MAIL SAV53 carries, and the six-character truncation of ITS names -- which is the only way that loop has ever been put under a test deliberately. The databases the test dumps into are built empty rather than copied from an era library, because what is being checked is the round trip and not a mutation of an era artifact.

6.1.2. GLUE as a Program

In addition to the "GLUE" PACKAGE, there is a program in which GLUE and PDUMP (see section 6.3) are preloaded. It will prompt for each of the usual arguments to GROUP-GLUE, permitting the user to conveniently GLUE (and PDUMP) several PACKAGEs in one session.

pymdl's own. That program's driver survives as muddle/gluer.18 and is here as the GLUER tool; the GLUE and PDUMP it preloads are the same packages.

6.2. Glue Bits

GLUE is able to perform its transformations on compiled or assembled code with the aid of a data structure produced during assembly. This structure is called the 'GLUE Bits'. It is an association placed on the RSUBR by this FORM:

<AND <ASSIGNED? GLUE>
     .GLUE
     <PUT rsubr GLUE glue-bits:uvector>>

Thus if .GLUE is non-FALSE the association will be available to programs wishing to use it.

Internally, the GLUE bits consist of two bits for each word of code in the CODE element of the RSUBR, followed by words specifying calling information. For each INTERNAL-ENTRY in the code, there is a word giving the number of arguments it takes and the offset of the INTERNAL-ENTRY in the CODE UVECTOR. The two bits for individual instructions are interpreted with the index field of the instruction as follows:

  • bits 0 implies the instruction is uninteresting;
  • Index field (M) and bits 1 implies the instruction is a reference to the code itself (a jump, perhaps);
  • Index field (R) and bits 1 implies a reference to an impure slot of the RVECTOR (the compiler does not generate such references);
  • Index field (R) and bits 2 implies the instruction is an MCALL;
  • Index field (R) and bits 3 implies the instruction is a reference to a pure slot of the RVECTOR.

See section 7 for more details on the format of MDL Assembly code.

pymdl's own. The FORM above is what an NBIN carries after each RSUBR (33.11), and GLUE the atom is a ROOT atom interned before the glue package loads, so that the package's reads, the loader's <PUT rsubr GLUE bits> and a user's <SET GLUE T> all resolve to one atom. The assembler of chapter 35 writes glue bits for what it assembles, and the compiler of chapter 34 for what it compiles; tools/era_tape_diff.py reads them to tell a glued sibling call from a standalone MCALL when it compares our words with the tape's (38.3).

6.3. PDUMP

MDL provides a mechanism for sharing compiled programs among several MDL processes, and for dynamically moving the compiled code in and out of the virtual address space as space is needed in the interpreter. This mechanism is described in detail in section 4.2. This section describes how to convert a compiled program into a sharable version, known as an FBIN (fast-binary) version of the program.

First load the group-purifier. Next, GROUP-LOAD your group (or groups).

<GROUP-LOAD binary-file:string>

which returns the group-name of the group. This (and any other groups to be dumped together) is then passed to the pure-dumper:

<PDUMP group-name1:atom group-name2:atom ...>

This creates several files, only one of which you need be concerned with:

sname;group-name1 FBIN

If given more than one group-name, PDUMP will create one FBIN file for each group, but only a single FIXUP and a single SAV file containing the fixups and code for all of the groups named. The FIXUP and SAV files are put on the "MUDTMP" directory and eventually are inserted in the pure code library, as described in section 4.2.

Alternative methods of PDUMPing are to specify that as an option to the program GLUE (see section 6.1.2), or to use its preloaded PDUMP directly after exiting its READER with ^S.

A warning about combining GLUE and PDUMP: if you attempt to PDUMP several groups that have been GLUEd together, you will lose. This is because the references to the 'group-RSUBR' will fall on the wrong OBLISTs.

PDUMP also produces a structure analogous to the GLUE bits (see section 6.2) produced by the compiler, but containing only information about the RVECTOR of the RSUBR, for the use of PURIFY (see section 6.5).

pymdl's own. PDUMP is muddle/pdump.22, and the demon of 4.2.1 that takes its SAVnn and FIXnn files off MUDTMP into the databases is ndem.py: the era demon survives only as MIDAS assembly (mudsys/ndem.46, "MUDDLE SAV-FIXUP MAINTAINER"), and this is its cycle reimplemented from that source against the database formats -- the delete pass with its free-area coalescing, the planning pass placing 'files' first-fit in 1024-word or 256-word blocks, the update pass, the salvage pass -- validated by reading the result back with the era's own DBMAIN (chapter 30). PDUMP checks its names against the site SAV database the way the manual's 4.2 implies and pdump.mud does: FINDAT taken means 1FINDA. The era's SAV and FIXUP files pymdl reads are the 1983 tape's; the ones it writes are its own, in the same format.

6.4. SUBRFY

SUBRification is a way of getting rid of many of the MCALLs which could not be practically removed using GLUE. If a FUNCTION is called by many separate groups, it is difficult to GLUE it to all the groups or to GLUE all the groups together.

What is really needed is to be able to allow something to be called with PUSHJ from separate groups without forcing it to be part of those groups. This is indeed the case with PUSHJ entries to MDL SUBRs (in the interpreter). A user can make his RSUBRs look like SUBRs in this respect.

SUBRFY takes a group, which must be in NBIN format. It purifies the RSUBRs and RSUBR-ENTRYs in the group and changes them so that they can be called with PUSHJ. It also produces a file, known as the 'preload' file, which can be used by the compiler to generate PUSHJs to the functions in the SUBRified group.

SUBRFY should be loaded before loading the group to be processed. The reason for this is that it guarantees that GLUE bits stay around. To load SUBRFY:

<USE "SUBRFY">

You should then GROUP-LOAD the group. Your group should be GLUEd already, since SUBRFY does not GLUE the group together.

SUBRFY can then be called in the following manner:

<SUBRFY group:atom
        file-name:string
        output:channel>

where:

group is the name of the group.

file-name is the name of the file in which SUBRFY should put the information for the compiler. This defaults to the name of the input file with second name "PRELOD".

output is an optional argument which specifies a CHANNEL on which to print information about SUBRFY's progress. The default is not to print anything.

The file produced by SUBRFY should be FLOADed for compilations where functions in the SUBRified group are called. This can be done by FLOADing it in the 'Things to do' part of a COMBAT plan.

Like purification, SUBRification changes the MDL. The only way to preserve the SUBRified group is to SAVE the MDL. Before SAVEing the MDL the "SUBRFY" PACKAGE should be removed. This can be done by doing a

<KILL-SUBRFY>

followed by a

<GC 0 T>

SUBRFYing a group implies that the group is not going to change at all frequently, if ever. A new SUBRFYed SAVE file may be created at any time, and elements of the group may be recompiled. However, if the calling sequences of any of the functions in that group change, you invalidate any functions compiled using the 'preload' file for that group. In short, think twice before tying yourself down with SUBRFY.

Found along the way. The surviving SUBRFY source is not the program MIT ran. Run end to end here over a 1983 library NBIN -- GROUP-LOAD, SUBRFY, the group rewired to the SUBR calling tier and purified, the preload file GROUP-DUMPed -- the source reports DONE and the transformed code then fails to execute, because its glue pass "skips one glue code per internal entry", compensating for a code-removal pass the era itself had retired, and the desynchronised glue relocates the wrong words. Remove that one <SET GLUE-BITS <REST .GLUE-BITS>> from a copy and the identical flow is behaviour-preserving: the rewired code executes on the emulator and answers exactly what it answered before, for every case. The era's own compiled SUBRFY (muddle/subrfy.nbin) does not desynchronise -- its UNRELATE, fed six words and alternating bits, answers (101 2 103 4 105 6) for every internal-entry list, where the source's answer shifts with the list -- so MIT's binary was built from a revision that did not survive. This is not the FRAMES case of chapter 29, where the binary carried the same fault and compilation made it harmless; here a REST compiles to a REST. pymdl runs the era source as it is; the test that demonstrates the one step patches a copy in a temporary directory and says what it would take, not what is done.

6.5. Purification

A facility exists to permit the purification of MDL objects. Purified objects can be shared between MDL processes and also are not examined by the garbage collector. What follows is a description of how this facility can be used.

The purification facility in MDL is most useful in the creation of subsystems. Non-purified RVECTORs of RSUBRs and tables used by subsystems are kept in garbage collected space. This means that these objects, which will never become garbage, are examined at each garbage collection, slowing down the garbage collection process. Also, if two people are using the same subsystem, they cannot share the tables and RSUBRs kept in garbage collected space. By using purification these two problems can be alleviated.

To purify most objects the user can call the PURIFY SUBR. The object will be purified, and all references to that object in the MDL core image will be changed to point to the new pure object. This simple method cannot be used in the case of RSUBRs. Purification of RSUBRs is a several-step process beginning with compilation.

6.5.1. Purifying RSUBRs

Once your FBIN or NBIN is ready you can actually do purification. To do this first

<USE "PURITY">

This PACKAGE contains the routines needed to purify RSUBRs. Then GROUP-LOAD the files you wish to have purified. Once this is done type

<GROUP-PURIFY group:atom output:channel>

This will purify and link all RSUBRs and RSUBR-ENTRYs in the group and will also attempt to purify any RSUBRs or RSUBR-ENTRYs called by the group. Giving the optional channel will cause GROUP-PURIFY to print information concerning the progress of the purification.

GROUP-PURIFY will only purify RSUBRs and RSUBR-ENTRYs. In order to purify tables, etc. use the PURIFY SUBR directly. Since purification is an extremely expensive operation, it is recommended that you collect together the things you wish to purify into a LIST, VECTOR, etc. and purify that structured object.

Once purification has occurred, several things may be done to recover wasted garbage collected space. The user can get rid of the "PURITY" PACKAGE by doing a

~~<KILL-PURITY>~~ <KILL:PURITY>

The user can also remove much of the overhead of keeping a group around by UNASSIGNing the group-name. Removals of this type should be followed by an explicit call to the garbage collector invoking the 'hairy' GC feature, as much of the storage to be regained is pointed to by associations. This can be done by

<GC 0 T>

In order to save a file with purified MDL objects you must SAVE. Restoring a SAVEd file with purified MDL objects will cause those objects to share with any other MDL RESTOREd from the same SAVE file.

Found along the way. The surviving purity.mud (muddle/purity.28, 1976) spells its entry KILL:PURITY, with a colon -- <ENTRY GROUP-PURIFY KILL:PURITY> -- and it answers "KILLED"; there is no KILL-PURITY in the file, so the struck name above is the manual's. SUBRFY's is KILL-SUBRFY, as printed. Purity itself is chapter 24's: PURIFY is a SUBR here as there, and what pymdl keeps of purification is the observable -- PURE? answers, UNPURIFY makes an object writable (29, 3.10.4) -- with no pages to move, since there is no address space for them to be pure in.

6.5.2. Purifying an Environment

Many subsystems maintain a list containing pointers to all the static data structures built by that subsystem: dispatch tables, data bases, and so on. The list can be given to PURIFY to move all its components into the pure area. However, there are other structures in garbage collected space that may be purified: e.g., the RVECTORs of RSUBRs, RSUBR DECLs, and so on.

The "CLEAN" PACKAGE examines these structures, looking for those which may be purified. It may also be used for informational purposes. To get it

<USE "CLEAN">

"CLEAN" has one major ENTRY, CLEANUP, which examines every ATOM of every OBLIST in the MDL. It may perform a variety of functions, but it is most often used to make DECLs share storage and to accumulate a LIST of purifiable structures. All of its arguments are optional.

<CLEANUP print?:boolean
         reset?:boolean
         decl?:boolean
         gdecl?:boolean
         pure?:boolean
         check?:boolean
         avoid:list-of-oblists>

print? is by default FALSE. If non-FALSE, information about each ATOM examined will be printed as CLEANUP runs. This is a lot of information.

reset? is by default T. If non-FALSE, the LISTs of objects previously collected will be reset before CLEANUP runs.

decl? is by default T. If non-FALSE, each DECL element will be made to exist exactly once in the entire core image. E.g., there will be only one copy of the DECL <LIST [REST FIX]> in the core image.

gdecl? is by default T. It is similar to decl?, but refers to GDECLs.

pure? tells whether to make a LIST of all the purifiable objects in the core image. It is by default T.

check? tells whether to make LISTs of all the TYPEs, RSUBRs, RSUBR-ENTRYs, etc. in the core image. It is by default T.

avoid is a LIST of OBLISTs not to look in; it is by default the OBLISTs associated with "CLEAN" and "PURITY".

CLEANUP returns (if pure? is non-FALSE) a structure (also stored as the GVAL of PURELST) which may be given to PURIFY.

The results of running CLEANUP may be examined by

<PRINT-CLEANUP>

As the object in running CLEANUP is to shrink the size of one's MDL and its garbage-collected space, it is useful to be able to remove CLEAN after it has done its work.

<FLUSH-CLEANUP>

removes everything associated with the PACKAGE from the MDL.

6.5.3. Purification Summary

In a simple case, one can purify a 'subsystem' of one group maximally by

<USE "PURITY" "CLEAN">
<GROUP-LOAD "foo">
<CLEANUP>
<GROUP-PURIFY foo>
<KILL:PURITY>
<FLUSH-CLEANUP>
<GC 0 T>
<SAVE "foo">

Measured (tests/test_mudbug.py), the recipe run on a small group: CLEANUP answers its census summary, a VECTOR, and leaves ,PURELST a LIST; <GROUP-PURIFY MYGRP ,OUTCHAN> prints BEGINNING PURIFICATION and answers "DONE"; <KILL:PURITY> answers "KILLED" and <FLUSH-CLEANUP> "PACKAGE REMOVED". The compiled-only primitives the two packages FLOAD from NBINs on the era, PUREQ and HASHER, are pymdl built-ins here (33.14).

6.6. TEMPLATEs

The PRIMTYPE TEMPLATE cuts down on the need for storage by allowing the user to specify exactly what he wants a structured object to contain, similar to 'structures' in PL/I or C.

To use this feature one must create a new TYPE of PRIMTYPE TEMPLATE. This can be accomplished by using the RSUBR TEMPLATE. The procedure for doing so is:

<USE "TEMPLATE">

<TEMPLATE name:atom ... specs ...>

where name is the name of the new TYPE and specs are specifications for each element of the TEMPLATE. This returns the TYPE name of the TEMPLATE and creates a creator of TEMPLATEs of TYPE name, called name itself, which can be applied to arguments to create objects of that TYPE of TEMPLATE.

The specification for the elements can be of several forms. It can be one of:

  • a TYPE: type:atom
  • a 2-element LIST: (type:atom length:fix)
  • a 3-element LIST: (type:atom length:fix count:fix)

Below are some examples along with explanations:

LIST is an 18 bit LIST pointer.

(FIX 18) is a halfword FIX (can be both positive and negative and is checked for overflow).

(FLOAT 18) is an 18 bit FLOAT (which is the left halfword of a 'normal' FLOAT and therefore somewhat restricts the precision).

(FIX n) (where n is less than 18) is a positive FIX of length n bits (is not checked for overflow).

BOOLEAN is not a MDL TYPE, but a one bit FALSE or non-FALSE depending on whether the bit is 0 or 1.

(UVECTOR 18 n) is an 18 bit UVECTOR pointer. The UVECTOR is of length n. The same can be done for VECTORs.

(STRING 36 n) is a 36 bit string byte pointer. The STRING is of length n.

ANY is not a MDL TYPE, rather anything can go here. This is relatively inefficient to use in TEMPLATEs as it takes up 2 words.

In order to provide more flexibility in using TEMPLATEs, two other fields are allowed, an optional field and a rest field. The optional field allows the user to create TEMPLATE TYPEs which will have the same basic structure but which can have optional elements determined when the actual TEMPLATE is created. The rest field, like the optional field, allows elements to be optional but specifies a pattern for any elements that are added on. It is analogous to REST in DECLs. Separation of fields is accomplished by the use of the strings "REST" and "OPTIONAL". For example:

<TEMPLATE FOO FIX "OPTIONAL" LIST BOOLEAN "REST" FLOAT>

This creates a TYPE FOO of PRIMTYPE TEMPLATE which always has a FIX as the first element, can have a LIST as a second element, can have a one bit T or #FALSE () as the third element and can have any number of FLOATs from the fourth element on.

6.6.1. Use of TEMPLATEs

TEMPLATE TYPEs may be thought of as primitive TYPEs, in that they each have a unique storage representation. On the other hand, the TYPEPRIM of any TEMPLATE TYPE is TEMPLATE. A primitive TEMPLATE (which cannot truly exist in the language) would look like

{ element-1 element-2 ... element-n }

All TEMPLATE TYPEs are represented as NEWTYPEs of this primitive TEMPLATE TYPE:

#type-name { ... elements ... }

This method is similar to the usual method in MDL for representing any new TYPE, in that a RESTed TEMPLATE will be printed 'CHTYPEd to its PRIMTYPE.' Note that a TEMPLATE so printed cannot be read by READ: a 'primitive TEMPLATE' cannot exist. It is best to avoid printing RESTed TEMPLATEs.

Below are some examples of the use of TEMPLATEs.

<USE "TEMPLATE">
<TEMPLATE BAR FIX "OPTIONAL" BOOLEAN "REST" (FIX 18) (FLOAT 18)>   ⇒ BAR
<BAR 1>                                     ⇒ #BAR {1}
<BAR 1 T>                                   ⇒ #BAR {1 T}
<BAR 1 <> 1 1.0>                            ⇒ #BAR {1 #FALSE () 1 1.0}
<SET A <BAR 1 <> 1 1.9 2>>                  ⇒ #BAR {1 #FALSE () 1 1.8984375 2}
<PUT .A 1 6>                                ⇒ #BAR {6 #FALSE () 1 1.8984375 2}
<PUT .A 4 1.999>                            ⇒ #BAR {6 #FALSE () 1 1.9960937 2}
<TEMPLATE BAR (STRING 36 4) "REST" ANY>     ⇒ #FALSE ("ALREADY A TEMPLATE")
<TEMPLATE BAR1 (STRING 36 4) "REST" ANY>    ⇒ BAR1
<SET A <BAR1 "HELP" 2 () <>>>               ⇒ #BAR1 {"HELP" 2 () #FALSE ()}
<PUT .A 1 "GOOD">                           ⇒ #BAR1 {"GOOD" 2 () #FALSE ()}
<PUT .A 1 "GOOD-BYE">                       ⇒ *ERROR* TEMPLATE-TYPE-VIOLATION
<TYPEPRIM BAR>                              ⇒ TEMPLATE
<TEMPLATE-DATA BAR>                         ⇒ (FIX "OPTIONAL" BOOLEAN "REST" (FIX 18) (FLOAT 18))

pymdl's own. These are the manual's own examples, and the answers are the manual's to the digit: an 18-bit FLOAT field holds 1.9 as 1.8984375 and 1.999 as 1.9960937, because pymdl's TEMPLATE stores a halfword float as the left half of a PDP-10 float (chapter 21), and a four-character (STRING 36 4) refuses "GOOD-BYE" with the era's TEMPLATE-TYPE-VIOLATION. The manual's printed transcript of the last line shows *ERROR* TEMPLATE-TYPE-VIOLATION PUT LISTENING-AT-LEVEL 2 PROCESS 1, the 55's shape. The TEMPLATE package itself was compiled code on the era (muddle/templt.fbin, an RPACKAGE), so here it is a pymac, this machine's compiled form of the same package (33.14), over the interpreter's own TEMPLATE machinery; TEMPLATE-DATA, from TEMHLP, gives back the raw element specification a type was made from.

6.6.2. Assembly of TEMPLATEs

Once a set of TEMPLATE TYPEs is created, as for the TYPE definitions of a subsystem, it saves time to store away the 'compiled' TEMPLATE generators and not recreate them each time the definitions are to be used.

The "TEMHAK" PACKAGE modifies files which define TEMPLATE TYPEs to contain the TEMPLATE descriptions and RSUBRs rather than the calls to TEMPLATE. It is only useful, of course, when the TEMPLATEs are defined in a file which will not normally be edited, since the new files are in 'NBIN' format. To load this PACKAGE,

<USE "TEMHAK">

The PACKAGE has two entries:

<TEMPLATE-DUMP group:atom file:string>

takes the group and modifies it such that <USE "TEMPLATE"> becomes <USE "TEMHLP">, and all top-level invocations of TEMPLATE are replaced by calls to BUILD-TEMPLATE (for the TEMPLATE descriptions), SETGs of the TEMPLATE-generating RSUBRs, and the GLUE bits for the RSUBRs.

<FILE-TEMPLATE input:string output:string>

takes an input file and performs the same service, GROUP-DUMPing the result to the optional output file (by default the same file with second name "NBIN"). This is useful for files which contain nothing but TYPE definitions, a common practice in large subsystems.

If the TEMPLATE TYPEs are defined in a file which will be edited frequently, a different set of routines is used after creating the TEMPLATE TYPEs:

<DUMP-TEMPLATES descriptions:string>

places the TEMPLATE descriptions (not the RSUBRs) in the specified descriptions file. It does so for all TEMPLATE TYPEs currently defined.

<DUMP-RSUBRS rsubrs:string template-type:atom ... >

will perform the same service for the TEMPLATE-generating RSUBRs of the TYPEs given as the second and later arguments to DUMP-RSUBRS.

There will now be two files, one containing the TEMPLATE descriptions and the other the RSUBRs. These may now be used to create the TEMPLATE TYPEs without USEing "TEMPLATE". To do so:

<USE "TEMHLP">

This defines the RSUBRs needed to take the TEMPLATE descriptions and make them useful to MDL.

<FLOAD descriptions:string>

the file of descriptions (the file created with DUMP-TEMPLATES): this must be loaded before the RSUBRs file. Then load the RSUBRs file (the file created by DUMP-RSUBRS):

<FLOAD rsubrs:string>

For maximum convenience, it may be necessary to put a FORM in files that create TEMPLATEs: if the TEMPLATE files described here exist, FLOAD them; otherwise, <USE "TEMPLATE"> and create the TEMPLATEs from scratch. It is of course possible to manually merge the two TEMPLATE definition files (preferably by using GROUP-LOAD and GROUP-DUMP), so long as the TEMPLATE descriptions precede the TEMPLATE RSUBRs.

TEMPLATE RSUBRs are created with GLUE bits, so it is possible to glue them into groups and to purify them.

Found along the way. The vendored transcription of this chapter ended at "a different set of routines is used after creating the TEMPLATE TYPEs:" -- its page range stopped one scanned page short, and everything from DUMP-TEMPLATES on was missing. The text above is restored from the page image (printed page 114) and the transcription's last page has been put back.

pymdl's own. TEMHAK's two entries are here, and FILE-TEMPLATE round-trips: a file of TEMPLATE calls comes back as <USE "TEMHLP"> and BUILD-TEMPLATE forms that a fresh interpreter loads to the same types (tests/test_template.py). TEMHLP's BUILD-TEMPLATE rebuilds a type from its description unconditionally, since a dumped file must be re-loadable; the era file also carried the generator RSUBRs as machine code, and here the generator is rebuilt from the description, which is this machine's compiled form of the same thing. DUMP-TEMPLATES and DUMP-RSUBRS -- the routines of the restored page -- are not defined here: no surviving source or binary carries them, and the file-based TEMHAK covers the case they served.