pymdl

The pymdl book

32. Informational aids

The environment manual ends with tools for comparing source files, producing listings, and querying another MDL session. Their availability in pymdl differs. The original assembly-language MUDCOM is replaced by a Python implementation that uses pymdl's reader. MAT runs from its archived 1982 MDL source. The MDL side of MUDINQ, the INQUIRE package, also runs here and exchanges messages over pymdl's Unix-socket IPC implementation (chapter 25).

The historical descriptions follow with notes on the versions and facilities available in pymdl.

Chapter 8. Informational Aids

This chapter discusses a few programs, most written in assembly language rather than MDL, which are nonetheless of use to MDL programmers. Most are informational aids of one sort or another. They include:

MUDCOM, a program for comparing versions of a MDL program. It is used by COMBAT (see section 5.2) to aid in the preparation of compiler plan files. It has several useful aliases.

MAT, the MDL 'atsign' program, produces listings, indexes and cross-reference files for MDL programs. @, a similar program which is not MDL-specific, will perform approximately the same tasks.

MUDINQ is an interface to the ITS IPC device and is therefore a means of interacting with any MDL that has the IPC device enabled. It has an alias, STATUS, which is particularly useful for determining the progress of compilations.

8.1. File Comparison and Checking with MUDCOM

MUDCOM is an assembly language program (not written in MDL), which nonetheless understands the syntax of MDL programs. It is used for comparing two versions of the same program, and also (under the name MUDCHK) for checking the syntax of MDL source files more rapidly than they can be loaded into a MDL. MUDCOM is not interactive; all instructions must be passed on the jcl line.

MUDCOM understands the following MDL structures at top level:

  • FUNCTIONs -- <DEFINE FOO ...>
  • MACROs -- <DEFMAC BAR .....>
  • GVALs -- <SETG MUMBLE ....>
  • LVALs -- <SET MUMBLE ....>
  • MANIFEST
  • ENDPACKAGE
  • MSETG -- <MSETG FOO 1> is <SETG FOO 1> <MANIFEST FOO>

The jcl for MUDCOM in the simplest case is filename1, filename2. MUDCOM will compare the two files and print out information concerning those structures it understands which have been removed, changed, or inserted.

MUDCOM has a number of switches which can be set. They are given as /switch, where switch is the name of the switch. Currently the following switches are useful:

  • T prints totals at the end of the comparison.
  • L prints all FUNCTIONs and GVALs in the file.
  • C checks the file given for syntax (only one file name at a time).
  • M checks the files for changed MACROs and MANIFESTs. In this mode, MUDCOM will make a second pass through the first file given in the jcl, looking for all occurrences of calls to changed MACROs and MANIFESTs. MUDCOM will consider FUNCTIONs making such calls as having been 'changed' and will tell which MACRO or MANIFEST caused the 'change'.

The following other jcl is understood by MUDCOM:

  • (atom ... ) appearing before the file names in the jcl will cause MUDCOM to think that those FUNCTIONs have been changed and will print them as such.
  • "filename" appearing anywhere in the jcl causes commands to be read from that file until the end-of-file is reached.
  • {filename ... } is used to specify files to search in calls to MUDFND (see below).

Aliases of MUDCOM:

  1. MUDCHK. MUDCHK filename checks a file for MDL syntax errors. This is the same as MUDCOM /C filename.

  2. MUDLST. MUDLST filename lists all FUNCTIONs and GVALs found in the file. This is the same as MUDCOM /L filename.

  3. MUDFND. MUDFND atom... {file file} searches files for FUNCTIONs/GVALs called atoms. It can be used for finding a FUNCTION in a haystack. This is the same as MUDCOM (atom atom) {file file}. Since typing this can be tedious, it is easier to use the "filename" convention and have a disk file containing the files to be searched (surrounded by {}s). Thus,

    MUDFND FOO BAR BLETCH "MARC;ZORK FILES"

    will look for the typical FUNCTION names in the files specified in MARC;ZORK FILES.

pymdl's own. The command PYTHONPATH=src python -m pymdl.mudcom provides source comparison and checking through pymdl's reader. It is a Python implementation of the interface described here, not the original assembly program. src/pymdl/mudcom.py implements comparison, listing, syntax checking, and dependency checking; tests/test_mudcom.py checks these operations.

From the repository root, compare two source files or check one without loading and evaluating it:

PYTHONPATH=src python -m pymdl.mudcom old.mud new.mud
PYTHONPATH=src python -m pymdl.mudcom /C prog.mud

8.2. The MDL Listing Program MAT

MAT is a program for producing listings of MDL programs on the Xerox Graphics Printer (XGP) or a lineprinter. (MAT is short for "MDL Atsign", after the general listing program named @).

Besides a listing of the program itself, MAT includes a symbol table -- a list of defined objects (arguments to DEFINE, SETG, etc.) and optionally a cross-reference listing -- a list of every place in the program each ATOM is used. MAT can also produce a record file, so that the next time MAT is run on the same program, only pages that have changed will be printed.

MAT is invoked with a jcl line in the following format:

MAT /rec=output input-files ... /switches ...

More specifically, it takes any number of input files (separated on the jcl line by commas) and produces a listing of them in the output file, with options specified by the switches (each preceded by a /), and optionally a record file /rec (see section 8.2.4).

The output file name defaults on ITS to sname;input @ or @XGP depending on whether the X switch is used, and on Tenex/TOPS-20 to input.@ or input.XGP in the connected directory.

<USE "MAT">
<MAT "fact.mud/C">        ⇒ "DONE"

pymdl's own. MAT was MDL -- "A portable cross reference listing program for the MDL programming language", K. Shane Hartman, version 1.4, April 1982 -- and it is here as the tape holds it, the one mat source in the whole archive (5609/mdllib/mat.mud.1). Its jcl line is the argument STRING. Run over the book's fact.mud, it prints its big-letter banner for the file name to the terminal and writes the listing to foo.mat, named from its default title, FOO/MAT: the banner "Muddle Atsign Listing for", a page header rebuilt from the channel's name slots (DSK:<...>fact.mud), the numbered source with each line's cross-references in the left margin -- line 4 of the example group references LIMIT, defined on line 1 -- and the symbol table with each symbol's definition and kind. It is a one-run program: a second <MAT ...> in the same interpreter signals OUT-OF-BOUNDS, which is the era program's own state and is left so; start a fresh one.

8.2.1. MAT Switches

The specific sorts of options available in MAT are controlled by a variety of switches which determine such things as whether to produce a cross-reference listing, whether to use the XGP as the output device, and so on. The following switches are implemented:

  • /C causes a cross-reference listing to be produced. This is a table showing each reference to each ATOM (other than SUBRs, FSUBRs, and locals) in the input files.
  • /D[file-name] specifies file-name as the file containing the user's definitions. Definitions are discussed in detail below.
  • /F[text-font, header-font, comment-font] specifies the XGP fonts to use in the output file. They are respectively the font to use for the program itself, the font for subtitles and other headers, and the font for MDL COMMENTs and top-level STRINGs. The default directory is FONTS and the default second file name is KST. The default font is 20FG. /F also causes a /X to be performed.
  • /I[file-name] specifies a file which contains the names of input files. This is in lieu of typing them all in each time MAT is run, useful for large subsystems incorporating many files. The input files listed should be separated by commas or carriage-returns.
  • /W causes output of only the symbol tables and cross-reference listing (if specified). No heading or title pages are produced.
  • /P On ITS, VALRETs a :PROCED to DDT and continues. Useful for long MAT runs.
  • /Q[message] prints message at the bottom of each page. The default is a copyright message.
  • /R creates a record file (this is automatic if /rec= is used). See below for details about record files.
  • /S outputs each file in a multiple file listing separately.
  • /T[name1 name2] specifies names to use on the title page (in lieu of the file names of the first input file).
  • /U prints a separate symbol table for each type of defined item in the input file(s) (e.g. FUNCTION, GVAL, etc.).
  • /X declares that output is to be for the XGP. This changes the default output file second name to @XGP. If /F is used, /X is done automatically.

Found along the way. The manual describes a later MAT than any tape preserved. The 1982 source has no /X and no /F -- the string XGP does not occur in it -- nor /D, /R or /W, and it has a /N the manual does not mention; given /X it stops with Unrecognized Switch. pymdl once added the two XGP switches to the vendored file, which was the wrong direction: it edited the corpus to implement something the corpus never had, and hid the fact. The file is the tape's again, and the /X and /F of this section are supplied by a separate package of pymdl's own, matxgp.mud, loaded over the era MAT when wanted, whose header says exactly what it adds and why.

8.2.2. Subtitles

Subtitles can be used by including STRINGs in an input file which begin with the word SUBTITLE. The remainder of the STRING will be used as part of the header of each output page until another subtitle is found. The STRING need not be a COMMENT. Subtitles may have a maximum of 79 characters.

Any file containing subtitles will have a table of contents at the beginning of the listing.

8.2.3. MAT Definition

The facility exists in MAT to cause user specified actions to occur at the time a specific ATOM is about to be cross-referenced. The most important use of this is for functions which define things which the user would like MAT to recognize, for example, a function one of whose side-effects is to SETG one of its arguments.

When MAT encounters an invocation of the function FOO, where FOO has been defined to MAT, it runs code generated by the user's MAT definition for FOO, which causes various actions to be performed.

MAT definitions are always located in a disk file which is specified by the /D switch. Each definition must be of the form:

[name arg1 arg2 arg3 ...]

where name is the name of the item which is being defined and the args are action specifications as described below.

The syntax of a MAT definition is somewhat complex. Basically, there are two types of actions which can take place: 'setting' an ATOM to be equivalent to a specified type (i.e., FUNCTION, MACRO, etc.) or 'cross-referencing' the ATOM (i.e., making it appear in the cross-reference listing).

The actual definition for an ATOM is a string of MAT action specifications, one for each argument in a call to that ATOM. For example, defining FOO to be

[FOO SETG SKIP SETG]

implies at least three arguments to FOO, the first and third of which should be treated as if they were SETGed. Thus, if <FOO FROB 1 MUMBLE> were encountered in an input file, it would be treated as though

<SETG FROB any>
<SETG MUMBLE any>

had been encountered. The symbol table would then point to the line on which the application of FOO appeared as the location of the definitions of FROB and MUMBLE.

The following tokens are meaningful action specifications:

  • CREF means to cross-reference this ATOM.
  • SKIP means to do nothing with this argument (a place holder).
  • name (where name is the name of a MDL SUBR which causes some action to be routinely performed) means to act as though the ATOM had had that SUBR applied to it. For example, SETG will cause MAT to treat the item as if a SETG had been performed on it; similarly, MANIFEST will cause MAT to believe it MANIFESTed.
  • ALSO means to do another thing to this ATOM. Thus, [SETG ALSO MANIFEST] specifies that the argument should be treated as though it were both SETGed and MANIFESTed.
  • =xy where xy are two characters, causes a user defined symbol type to be created. In the cross-reference, this will appear as xy in front of the name of the ATOM.

Any of the preceding tokens may have !-OBLIST added. This means that instead of the ATOM being set to the specified type, atom!-OBLIST will be set. Thus, for example,

REST SETG!-FLAGS

might specify a function which takes a LIST of ATOMs and performs

<SETG <INSERT atom <GET FLAGS OBLIST>> any>

on each of them.

[SPEC xy name] specifies name to be the expansion of xy for purposes of the symbol table. name cannot have spaces in it.

Since not all items to be recognized within a function call are at top level, there is a facility for telling MAT to recognize structures. This is done by inserting the correct bracket (which MAT will encounter) around the part of the action specification referring to a structure. For example, a definition for GDECL (which is handled internally, however) might be

REST (REST GDECL) SKIP

which specifies that the arguments are alternately a LIST of things to GDECL and an argument which is unimportant.

A special case of bracketing is when the location of the structure is not known. In this case, bracket! means 'find the next object that starts with this bracket'. An example later demonstrates this.

What follows are some examples from a real definition file.

[NEWSTRUC NEWTYPE SKIP REST SETG SKIP]

NEWSTRUC takes an ATOM which becomes the name of a NEWTYPE, the DECL for that TYPE (which is not interesting to MAT) and an arbitrary number of pairs of ATOMs (names of offsets in the structure) and their DECLs (again, not interesting).

[FLAGWORD REST SETG]

FLAGWORD takes an arbitrary number of ATOMs and SETGs them something.

[SPEC PG Pure-Gval]
[SPEC OB Object]
[SPEC AC Action]
[SPEC VB Verb]
[SPEC OS Object-Synonym]
[SPEC AD Adjective]

These define the long descriptions for the newly defined symbol types created in the examples.

[PSETG =PG]

PSETG takes an ATOM and a value and SETGs the ATOM (also putting it in a LIST of ATOMs to purify).

[GET-OBJ "CREF"]

GET-OBJ takes a STRING PNAME of an object and returns the object. This definition allows "object" to be cross-referenced here. Note that CREF is in quotes because the element being dealt with is a STRING.

[OBJECT ["=OB" REST "=OS"] [REST "=AD"]]

OBJECT creates objects which are referenced by GET-OBJ. OBJECT first takes a VECTOR of STRINGs, the first of which is the true object specifier (OB) and the rest of which are synonyms (OS). The second argument is a VECTOR of STRINGs, which are PNAMEs of adjectives referring to the object (AD).

[ADD-ACTION "=AC!-ACTIONS" SKIP REST [[!"=VB!-WORDS" SKIP]]]

ADD-ACTION creates 'verbs'. The name of the verb is the first argument, which is a STRING. ADD-ACTION SETGs string!-ACTIONS to an item of type ACTION (AC). The second argument is not interesting. The rest of the arguments are VECTORs, somewhere in which is a VECTOR of a STRING and an uninteresting object. ADD-ACTION SETGs this latter STRING (the PNAME of an ATOM in the WORDS OBLIST) to something of type verb (VB). This is about as complicated as a MAT type specification is likely to get.

[ADD-ACTION "=AC!-ACTIONS ALSO =VB!-WORDS"]

ADD-ACTION takes as its first argument a STRING which is SETGed both in the ACTIONS OBLIST and in the WORDS OBLIST, to an ACTION (AC) and a verb (VB), respectively.

Found along the way. The "real definition file" the manual quotes from is Zork's. NEWSTRUC, FLAGWORD, PSETG, GET-OBJ, OBJECT and ADD-ACTION are the 1977-1979 game's own definers, and the symbol kinds -- Object, Action, Verb, Object-Synonym, Adjective, Pure-Gval -- are its vocabulary; NEWSTRUC, FLAGWORD, PSETG and MSETG are the four game-prelude names pymdl's interpreter still knows as shims for that reason (Appendix D). The game itself lives in its own repository now (chapter 38).

8.2.4. MAT Record Files

Listing Record (or LREC) files, akin to @ LREC files, can be produced in MAT by including files in the jcl line. Use of an LREC file has the advantage that future invocations of MAT using it need only output the changed pages of the listing. The LREC file produced will be placed in file and contains all relevant jcl information, so that future calls to MAT for comparison listings need only have files in the jcl line. Additional jcl may then be appended. There is, however, no way to turn off flags once set up. Wherefore, if a cross-reference file is to be used only occasionally, leaving the cross-reference (/C) flag off for the initial listing and appending it at other times is preferable.

An alternate way of creating a Listing Record file is to use /R which is equivalent to

input-file-first-file-name LREC=

in the jcl. Obviously, /R is not sufficient for future comparisons.

8.3. The MDL-IPC Device Interface MUDINQ

MUDINQ is a small program that formulates, sends, and receives messages to and from MDLs over the ITS IPC ('Inter-process Communication') device. The user specifies a target MDL process by its uname and jname, either on the jcl line or to MUDINQ directly. He then inputs the message to be sent to that MDL. The message sent is enclosed in an invisible protective shield (an ERROR handler and so forth) to prevent it from interfering in the operation of the target. The message is PARSEd and EVALed by the target, and the result put in a file which is printed by MUDINQ when it appears.

The most common use of this program is to answer the question "What could my compilation (or whatever) be doing after all this time?" The answer may be obtained by MUDINQing a <FR&> or <FRAMES> at it.

Inquiring after the state of a compilation is such a common use of MUDINQ that there is an alias of it, STATUS, which MUDINQs a <STATUS> (see section 5.1.1) at a compiler process and waits for a response.

Finally, an alias of MUDINQ called WHOM lists those MDL jobs listening on the IPC device.

For more details on the operation of the MDL IPC interface, see [3].

pymdl's own. The MDL end of this is in the tree: inquire.mud (9006255/mprog2/inquir.1, 1976) is the INQUIRE package, and it is the "invisible protective shield" in source. <INQUIRE uname jname 'form> SENDs the target a PROG that opens an output file, binds OUTCHAN to it, installs an ERROR handler that prints the banner and returns, PARSEs and EVALs the form, closes the file, and SENDs the file name back with a message type the sender's own IPC handler recognises, at which point that handler announces ACKNOWLEDGEMENT FROM the target and FILECOPYs the file to the terminal. <USE "INQUIRE"> loads it. In pymdl, the messages travel over Unix domain sockets between listeners established with <IPC-ON> (chapter 25). tests/test_mudinq.py checks the inquiry and acknowledgement exchange. pymdl also provides a command-line driver, PYTHONPATH=src python -m pymdl.mudinq; --whom lists listeners and --status uname jname asks a target for its compiler status. Client and target need the same IPC directory and access to the shared file root used for replies (PYMDL_FILE_ROOT for the command-line driver). For a compilation in the current session, <STATUS> reports the compiler's recorded status (chapter 34, section 5.1.1).

References

[1] Edward H. Black. Using MDL's Calico User Interface. Technical Report SYS.11.21, MIT LCS Programming Technology Division, 1976.

[2] Richard M. Stallman. EMACS. Technical Report 519, MIT AI Laboratory, August, 1979.

[3] S. W. Galley and Greg Pfister. The MDL Programming Language. M.I.T. Laboratory for Computer Science, 1979.

[4] Peter Samson. MIDAS. Technical Report 90, MIT AI Laboratory, October, 1965.

[5] P. David Lebling, R. V. Baron and Bruce K. Daniels. RMODE: A Real-time Edit Facility. Technical Report SYS.04.07-1, MIT LCS Programming Technology Division, October, 1977.

These are the environment manual's own references; the book's are Appendix G.