Handoff

For whoever picks this up next, including me in six months.

What this is

A read-only toolkit for the ITS file system, third in a family with s5fs (UNIX V7/2.xBSD) and t10fs (TOPS-10). Same architecture, same discipline, different file system — and ITS is genuinely different, not TOPS-10 with other field names. See PLAN.md §5 for the four places that matter.

Phases 0–8 (in part) are done: word layer, geometry, cited constants, a reader, an independent checker, a manifest/verify pair, and a writer whose output ITS's own salvager accepts. put and del exist; mkdir and mkfs do not.

Read these, in this order

1. PLAN.md — written before any code, and still accurate.
2. docs/geometry.md — the one layer that is not in either sibling. If you read nothing else, read this: block numbers are not offsets, and being wrong about it is not subtle.
3. src/its.h — the transcription. Every constant, its ITS symbol, its base, and whether it has been seen to work.
4. docs/filesystem.md — the format in prose, with the gap register at the end.
5. docs/validation.md — what is actually established, and what is not.

The shape of the tree

src/itspack.[ch]    36-bit words in 8-bit bytes.  Four packings.
src/itsgeom.[ch]    where a block is.  The drive table.
src/itstext.[ch]    SIXBIT, seven-bit ASCII, six-bit bytes.
src/image.[ch]      an image, addressed in words and blocks.
src/its.h           every on-disk offset, cited to SYSTEM;FSDEFS 43.
src/structure.[ch]  MFD, UFD, name blocks, descriptors, links, TUT.
src/cmd_*.c         the front ends.  cmd_dump.c depends on nothing above image.c.
src/cmd_check.c     the checker.  Shares NO code with structure.c, and writes the
                    block-to-sector conversion out a second time on purpose.
src/cmd_manifest.c  manifest/verify.  The checksum is over WORDS -- see its header.
src/cmd_shell.c     the explorer.  Reads stdin, which is why it is testable.
src/write.[ch]      THE ONE MUTATION PATH.  Nothing else writes to a pack.
src/cmd_write.c     put/del.  Front ends: the host-file conversion and no more.
tests/run.sh        108 checks.  Builds its own ITS file system with dd, and
                    damages copies of it on purpose.
tests/accounting.sh does the space add up?  The same three questions `check`
                    asks, asked with the READER, so the two can be compared.
tests/crosscheck.sh extracted files against the host files they were built from
                    -- the only evidence here that is not the pack agreeing
                    with itself.
tests/fuzz.py       one random word damaged per iteration, every command run --
                    INCLUDING put/del/mkdir, each on its own copy and each
                    rechecked, and tape/saveset over a damaged .tap.
tests/nsalv.sh      `make nsalv`: hand a pack to ITS's own salvager and compare.
tests/nsalv.exp     ...the emulator half of that.  THE SLEEPS ARE LOAD-BEARING.
tests/interop.sh    `make interop`: boot ITS on a pack we wrote, and have
tests/interop.exp   DSKDMP list the directory.  Two emulator runs, on purpose.
tests/mkfs.sh       `make mkfs-test`: a pack built from nothing, graded by
                    NSALV and DSKDMP, both booted from tape.
tests/tape.sh       `make tape-test`: a real ITS tape, decoded and checked
                    against the host files it was made from.  No emulator.
tests/klh10.sh      `make klh10`: dbd9 against KLH10's OWN converter, vdkfmt.
                    No emulator run -- the KLH10 build ships the converter.
tests/interop-klh10.sh  `make interop-klh10`: the interop questions again on a
tests/klh10.exp     SECOND emulator and a second packing.  KLH10 execs its
                    devices as separate processes (dprpxx, dpchaos, dptm03) --
                    without them it runs with NO DISK and DSKDMP says MFDCLB,
                    which reads as a corrupt pack and is not one.  KEEP EVERY
                    expect PATTERN LIST MULTI-LINE; the one-line form does not
                    match on expect 5.45.4, reproducibly, cause unknown.
tests/itsdump.sh    `make itsdump`: ITS's OWN DUMP writes a tape, and cmp says
                    ours is byte-identical.  The only level-1 evidence here.
tests/itsload.sh    `make itsload`: and ITS's own LOAD reads one we wrote,
                    link included -- the only grader for the link form, since
                    ITS's DUMP never writes one.  The tape carries a FILE as
                    well, on purpose: it is the control.
tests/version-diff.sh  two versions of FSDEFS, and the citation column checked.

Things not to do

Each of these has already cost somebody time, here or in a sibling.

Do not treat a block number as an offset. blk * 1024 words is wrong from the second cylinder onward. There is exactly one conversion, its_blk_sector().

Do not read UNDSCP from word 0 of the directory. It counts six-bit bytes from UDDESC, word 11. Getting this wrong is silent: it lands on zero, zero means "end of description", and every file decodes to zero blocks with no error. The only thing that catches it is UDBLKS.

Do not believe FSDEFS about the base of a character. It says the link separator is ";" (73); the disk holds 033. It gives ASCII codes for two characters and the SIXBIT value for a third, in one sentence.

Do not add a constant to its.h without a citation and a marker. [v] means you have watched it decode correctly on a real pack. [s] means you have only read it. Writing [v] because you are confident is how the markers stop meaning anything.

Do not case-fold or truncate a name. SIXBIT has no lower case. Refusing is correct; HELLO and hello are not the same file, and silently making them so produced duplicate directory entries in s5fs on an image its checker called clean.

Do not let the test suite build its fixtures with a future writer. The suite pokes words in with dd on purpose. A fixture built by the writer asks the reader to agree with the writer rather than with ITS.

Do not write one structure per tape record either. itstar appends words to a buffer and flushes when it fills; the flush after a file header is commented out ON PURPOSE. A tidier writer would produce a tape unlike any ITS made.

Do not assume one structure per tape record. A DUMP save set puts the first file's header in the volume record, a link's target in its header's record, and a file's data in whatever is left of its own. Assuming otherwise listed 3,734 of a tape's 3,795 files and looked entirely plausible; itstar disagreeing about the count is what noticed.

Do not assume a second reference to a block is corruption. The TUT is a reference count. On the reference pack nothing is referenced twice, which means nothing here knows what a legitimate second reference looks like.

Do not append to a directory's name area. It is SORTED -- 6,056 entries on the reference pack, none out of order -- and QRELOC in disk.1228 is what keeps it so. put inserts and shifts. A writer that appended would produce a directory that reads back fine here and that ITS's own lookup walks wrongly.

Do not write to a pack an emulator has attached. There is no lock to take. itsfs checks /proc for another process holding the file, which works on Linux and cannot tell anywhere else -- so it is a seat belt, not a guarantee.

Known gaps, stated plainly

• No writer, therefore no level-1 or level-2 evidence of any kind.
• A pack mkfs builds does not BOOT (see above), so ITS has never come up on one; its graders are booted from tape.
• NSALV has been shown only one kind of damage (a cleared TUT word).
• A pack mkfs builds does not BOOT: ITS starts from the front end's blocks at the bottom of the disk, which mkfs does not write (and NSALV's own ZAP refuses to touch). Both graders for such a pack come off tape.
• One pack, one drive, one era: an RP06 built from source in 2026. No multi-pack file system, no artifact recovered from MIT. mkfs DOES build an rm03 and make mkfs-test ITS_DRIVE=rm03 checks it, but ITS cannot grade one: NSALV picks its drive at ASSEMBLY time (IFCE MCHN,PM,[ ... RM03P==1), so the salv.tape here grades rp06 and nothing else. An rm03 salvager needs its own ITS build. The stage skips loudly rather than reporting the refusal as a fault in the pack.
• dbd9 is still corroborated -- settled. KLH10's own vdkfmt writes a dbd9 pack byte-identical to ours (make klh10), and ITS boots on one under KLH10 and prints a file itsfs wrote (make interop-klh10). Building the whole emulator turned out to be unnecessary for the first half and sufficient for the second.
• A tape this project writes has not been compared RECORD FOR RECORD with one ITS wrote -- settled, and it was the most productive thing in the project. make itsdump boots ITS, runs its own DUMP to tape, and cmp's the result with itsfs save: EQUAL over all 2,667,188 bytes. Three fixes to get there, none of them visible to any existing test -- an 8-word file header where this wrote 7, UNREF copied whole where this zeroed the author out of it, and an unknown date written as all ones rather than zero. What is STILL unmeasured is a LINK: ITS's DUMP omits links ENTIRELY -- KSHACK 37 files/3 links dumped as 37/0, KMP 3/1 dumped as 3/0 -- so what save writes for a link is an copy too, once DUMP is asked for one: DMPLNK/LINKS is a SWITCH, default off, listed in DUMP's own help and not in .INFO.;DUMP INFO. With it, a link header is 8 words like a file's, and ours matches byte for byte.
• UNTIM's unit, UNBYTE's other three encodings, UNDUMP's position and the flag field's width are all unknown. See the gap register.
• The version span has a floor and no ceiling: make version-diff shows FSDEFS 40 and 43 define all 71 symbols identically, but both postdate the 1979 TUT change, and a file version is not a release number.

How to get a pack to test against

$ git clone https://github.com/PDP-10/its
$ cd its && make EMULATOR=simh      # or klh10, or pdp10-ka

which leaves an RP06 at out/simh/rp0.dsk. Then:

$ make oracle IMAGE=~/its/out/simh/rp0.dsk

Work on a copy. make oracle makes one itself; repack opens its input read-only either way; the rule stands regardless, because the first thing a writer will do is break it.

Getting a console out of a running ITS

Three things, and all three are needed. They are in tests/interop.exp too, but they cost enough to be worth repeating:

1. Delete SYS;ATSIGN DRAGON first. The finished system auto-starts the job that link names and that job owns the CTY. While it is there, ^Z does nothing at all, forever. itsfs del removes it.
2. One ^Z, after a long settle, retried with long gaps. A ^Z that arrives while ITS is still starting its jobs is lost.
3. Type blind, about a third of a second per character. ITS echoes when it PROCESSES, not when it receives, so waiting for each character's echo deadlocks; sending the line in one burst loses it.

Running NSALV

make nsalv IMAGE=... needs three things that are not in this repo:

• expect;
• an emulator with working ITS support. The pdp10 most distributions package is SIMH 3.8-1, which accepts set cpu its and then does not work. Use the Open SIMH build the PDP-10/its Makefile produces at tools/simh/BIN/pdp10; that is the default;
• a salvager tape, out/simh/salv.tape in the same tree.

Budget ten minutes: two full salvage runs over a 300 MB pack.

The sleeps in nsalv.exp are load-bearing. Answering the salvager's first question the instant it appears makes the RH11 report the drive offline (Error 1 = (40001) ILL-FUNC UNSAFE) and the run is lost. The ITS build's own scripts sleep in the same places; this was rediscovered the hard way.

If you are picking this up

The roadmap is finished, and so is the round of work after it. make itsdump compares a save set with one ITS's own DUMP wrote and they are equal to the byte; make itsload hands ITS one of ours and its own loader reads it back. Read docs/validation.md — it opens with a table of every claim and the command behind it.

What is cheap and still open. Both reuse harnesses that already exist:

• Show NSALV still other kinds of damage. It has now seen a cleared TUT word at two sites, a miscount after an abrupt halt, a zeroed directory (stage 5) and a garbaged MFD check word (stage 6), agreeing on all of them. and a broken descriptor putting two files on one block (stage 7). All four directions the checker distinguishes now have a second opinion. Anything further is a new kind of structure rather than a new direction — a damaged TUT header, say, or a name area out of order.
• A FSDEFS from before September 1979 would put a ceiling on the version span. make version-diff has the harness; it needs an artifact, and none is in the ITS tree.

What is blocked, and by what. Two of the remaining gaps are the same problem wearing different clothes: ITS's configuration is compiled in, not read at run time. The salvager's drive is fixed by IFCE MCHN,…,[ … RM03P==1, and the pack count by FIRSPK/LASTPK/NQS. So grading an RM03 pack, or a multi-pack file system, means building a different ITS — which changes the reference environment every other test here depends on. Neither is hard; neither is a file-system problem.

The rest need artifacts nobody here can make: a pack recovered from MIT, a pre-1979 FSDEFS, and the KS10 front-end file system that stands between mkfs and a bootable pack.

A habit worth keeping, because it earned its place repeatedly: when a check passes, make it fail on purpose before believing it. Three documentation guards in tests/run.sh exist because prose that was true when written stopped being true, and every one of them was verified by breaking it. And when something fails, find the cause rather than the most plausible story — four mechanisms written down here as fact this year turned out to be invented, each one comfortable enough that nothing but an experiment would have dislodged it.