Roadmap | itsfs

Where each phase ends, and what has to be true before it can be called done. PLAN.md has the same table; this is the detail.

All ten phases are done. What follows them is written here so that the next person does not have to re-derive the order — and the order matters, because each phase is validated by something the previous one built.

Done

Phase 0 — the repository

.clang-format, CI, the docs site, a make test skeleton. Lifted from t10fs and stripped, which is the correct amount of originality for a build system.

Ended by: CI green.

Phase 1 — the word layer

Four packings behind one group-based interface, and SIXBIT one layer above it.

Ended by: every word of a real 300 MB pack round-tripping byte-for-byte, including through a packing with a different stride and one that shares a byte between two words.

Phase 2 — geometry

The layer neither sibling project needs: block numbers are cylinder-major and the blocks-per-cylinder division truncates.

Ended by: the MFD being found where the formula says it is, and identifying itself with its own check word. Also by itsfs dump -s, which addresses raw sectors and lets the two be compared.

Phase 3 — the constants

its.h, transcribed from SYSTEM;FSDEFS 43 with a citation and an evidence marker per field.

Ended by: the MFD, a UFD and the TUT all decoding into values that are sane in more than one way at once — dates in two plausible eras, names that match between the MFD and each UFD's own header.

Phase 4 — the reader

Directories, name blocks, the descriptor bytecode, links, file extraction, and the TUT.

Ended by: tests/accounting.sh — the space on a real pack accounting for exactly, three ways, against numbers this project did not compute. See validation.

Phase 5 — `itsfs check`

An independent checker: it shares no code with the reader. cmd_check.c includes neither structure.h nor itsgeom.c's conversion, and re-derives the block-to-sector arithmetic, the MFD-slot formula, the UFD layout, the descriptor bytecode and the TUT from its.h.

It counts references per block, because the TUT is a reference count rather than a bitmap, and reports the four kinds of disagreement separately; it checks each UFD's UDBLKS against its own descriptors, that the two areas in a directory have not overrun each other, and that the locked-out set is exactly the directories, the MFD and the TUT.

Ended by: clean on a real pack, with the same numbers the reader gets; naming the right blocks and the right files on a pack damaged on purpose; and tests/accounting.sh's three checks now being things it does in C rather than things a shell script does with the reader. Both still run in make oracle, because two implementations agreeing is the point and either alone is one opinion.

One thing it deliberately does not call damage: a link pointing at a file that is not there. Seven on the reference pack do, and a live system is like that. They are notes, they are listed under -v, and they do not change the exit status.

Phase 5a — `NSALV`, out of order and on purpose

This needed no writer, which is why it ran before phase 6 rather than as part of phase 8. NSALV is ITS's own salvager: it boots standalone from a tape, walks every directory on a pack, rebuilds the allocation table from scratch and reports what disagrees. Pointing it at a pack this project has only read asks the one question itsfs check cannot answer about itself.

Two runs, because the clean one proves less than it looks: NSALV finishes silently on a good pack, which is indistinguishable from its never having run. The damaged run is what makes the clean one evidence.

Ended by: both checkers naming exactly the same blocks and the same files on a pack with one TUT word cleared — 11 at one damage site, 8 at another, sorted and compared as pairs rather than as counts. make nsalv does it; see validation.

It also settled a format question as a side effect. NSALV's link parser compares against ': and ';, and MIDAS assembles 'X as SIXBIT — so ITS's own code confirms the encoding that FSDEFS's comment gets wrong. See sources.

Phase 6 — manifest, verify, and a shell

manifest fingerprints a pack — one line per directory, file and link, with a checksum over words rather than bytes. verify diffs a pack against one. The shell keeps the image path and the current directory so that exploring is three short lines rather than three long ones.

Ended by: a manifest taken from the reference pack in le64 verifying clean against the same file system in dbd9, over all 6,303 entries — two images sharing no byte boundary and no differing word. And by the inverse: one flipped bit in 39.6 million words reported by file name. See validation.

The shell is read-only and has no -w, because there is nothing to guard yet. Its prompt says (ro) so that the day there is a writer, the difference is visible rather than assumed.

One thing worth recording from building it: blocks prints runs rather than block numbers, and that turned out to matter. 5,431 of the reference pack's 5,650 files are a single run — a fresh build writes sequentially — but 219 are not, and one is 59 runs. A display that printed 1,904 numbers would have hidden that.

Phase 7 — the writer

One file, one mutation path, every front end calling it. put and del exist; mkdir and mkfs do not yet.

Ended by: NSALV accepting a file system this project wrote — level 2 in the oracle's own terms, and the first time anything here has been graded by ITS rather than compared with it. See validation.

The two ITS-specific things it had to get right, neither of which has an analogue in t10fs:

A directory is one block, so it can be full, and full is a refusal rather than an allocation problem — there is no way to grow a UFD. The message names the two numbers that met.
The TUT is a reference count, so a block at TUTMNY ("many or more") cannot be decremented correctly by anybody. del refuses it by name rather than guessing.

And one that was not on the list: the name area is sorted, and a writer that appended would produce a directory ITS's own lookup walks wrongly. That was found by measuring the reference pack rather than by reading the source, and it is why put inserts and shifts.

Phase 8 — native-tool interop

Everything here exists:

NSALV accepts a pack we wrote (phase 7, make nsalv).
DSKDMP lists our file, with its own reader — a third implementation, sharing nothing with the monitor or with NSALV — and the listing is still in sorted order, which is what catches a writer that appended (make interop).
mkdir exists, and DSKDMP reads what it makes. The MFD entry's position is the address of the directory's block, with no pointer to check it against, so a third implementation resolving it is the only way to know it is right.
ITS boots on a pack we wrote, running its startup salvage over every directory on the way (make interop).
The monitor opens the file and prints it, which was the last thing here and took the longest. The obstacle was never the file system: the finished system auto-starts the job SYS;ATSIGN DRAGON names, and that job owns the console — while it is there ^Z produces nothing at all. itsfs del removes it, which is a pleasing shape for the last step: the project's own writer clearing the way for its own reader to be graded. See validation.
mkfs exists, and both NSALV and DSKDMP accept a pack built from nothing (make mkfs-test). It writes a file system rather than a bootable pack, so both are booted from tape; see validation.

Ended by: make interop — five stages, from itsfs put to :print on an ITS console.

Phase 9 — tapes

itsfs tape reads the SIMH .tap container: records with their lengths on both sides, tape marks, end of medium. It checks the two lengths against each other, which is the one thing the format itself lets a reader notice, and -x extracts each file as 36-bit words.

Ended by: make tape-test — the salvager tape's two files extracted and re-encoded to the host originals byte for byte, and core promoted from corroborated to confirmed on the strength of finding three strings in the image that this project has watched NSALV print. See word packing.

And the archive layer over it. itsfs saveset reads a DUMP save set: volume header, a header per file, data to a tape mark. Graded against itstar, which reads these for the PDP-10/its project — 3,795 entries on a 91 MB tape, name for name and in the same order. It caught a real bug doing it; see validation.

And writing one. itsfs save writes a save set from files on a pack, and itstar reads it, extracts it, and returns the host file ITS was originally given — byte-identical, with its date and its link intact. See validation.

Writing follows itstar's own shape, which is not the obvious one: words are appended to a record buffer and flushed when it fills, so a header shares a record with the data after it. itstar's save has the flush after the header commented out on purpose. Writing one structure per record would be tidier and would produce a tape unlike any ITS made.

Not on the roadmap

Repair. A checker that can also fix things is two programs, and the second is written after the first is trusted.
FUSE, or any kind of mount. tools/mldev in the ITS tree mounts an ITS file system over the network protocol; that is a different problem and it is already solved.
DECtape.
Anything that is not ITS.

Open questions that could reorder this

A pack recovered from MIT would be worth more than any two phases here, and would immediately exercise UNBYTE encodings that nothing has written since the 1980s. If one turns up, read it before writing anything.
A KLH10-built ITS pack would promote dbd9 — settled, and not the way this bullet expected. It said a full make EMULATOR=klh10 was required because "the shortcut of repacking an existing pack and pointing KLH10's DSKDMP at it does not work". The shortcut works fine; what did not work was a KLH10 missing its dprpxx device process, which had left it running with no disk at all. Building KLH10 also turned out to ship vdkfmt, its own format converter, so the first half needed no emulator run at all. make klh10 compares the bytes; make interop-klh10 boots ITS on one. The story is in word packing.
A FSDEFS from before September 1979. make version-diff already compares versions 40 and 43 and finds all 71 symbols identical, but both postdate the TUT format change both of them mention, so the span this reader covers has a floor and no ceiling. A pre-1979 one would be the interesting artifact.
A multi-pack file system. UNPKN and QTRSRV are read and ignored today. t10fs found two real bugs the first time it met a genuinely multi-unit structure. ITS's own note on it, doc/sysdoc/adding.packs, is short enough to quote in full effect: "Change the configuration in SALV or NSALV. FIRSPK and LASTPK defines the range of packs… Change the definition of NQS in ITS… Run MARK in SALV to mark the packs. Use UCOP in SALV to update the MFD and UFDs on the fresh pack with information from a pack already in use." So the pack count is fixed when ITS and the salvager are assembled, and a multi-pack file system needs its own build of both — and UCOP says the MFD and the UFDs are propagated to the new pack rather than shared, which is a structural fact worth having before writing any of it.

The shape of what is left

Two of these are blocked by the same thing, and it is worth naming: ITS's configuration is compiled in, not read at run time. The salvager's drive (IFCE MCHN,…,[ … RM03P==1) and the pack count (FIRSPK, LASTPK, NQS) are both assembly-time, so exercising either against ITS means building a different ITS — which changes the reference environment every other test here depends on. Neither is a file-system problem, and neither is hard; both are a day of build plumbing rather than an afternoon of reading.

The other three need artifacts nobody here can produce: a pack recovered from MIT, a pre-1979 FSDEFS, and the KS10 front-end file system. MIT's Tapes of Tech Square collection holds real ITS backup tapes, but its published output is extracted files — the raw images stay in the archive.

An RM03-assembled salvager. mkfs builds an RM03 and make mkfs-test ITS_DRIVE=rm03 checks its arithmetic, but ITS cannot grade one: NSALV picks its drive when it is assembled (IFCE MCHN,PM,[ … RM03P==1), so the tape here grades RP06 and nothing else. A second geometry graded by ITS would be a real strengthening of the one layer with no check word — it is the only drive where blocks-per-cylinder truncates differently (18 from 18.75, against the RP06's 47 from 47.5).
The front-end file system — and it is this format, which is a correction to what this entry used to say. .;.FEFS. PK0000 is an ordinary ITS file, 9 blocks in the directory named ., and get -w/put -w move it already. The emulator supplies the bootstrap, so the console path is not in the way: SIMH's ROM reads the home block, follows word 0103 to a page of the FE file, follows word 4 of that to another, and jumps into it — a chain traced end to end on the reference pack. The home block is quoted from NSALV's FESET. What is not known is the FE file's interior: its pointers are absolute disk addresses, so a copy that lands on different blocks needs them rewritten, and nothing here knows how many there are. See validation.

What closed, and what it took

Worth recording, because the pattern is more useful than the list. Of everything settled after the ten phases, none of it needed a new idea about the format. Every one came from making ITS produce an artifact and comparing:

was open	what closed it
`dbd9` unconfirmed	KLH10's own `vdkfmt`, which the build ships — no emulator run at all
one emulator	KLH10 boots ITS on a `dbd9` pack and prints our file
no tape compared with one ITS wrote	`:dump` under KLH10, then `cmp` — level 1, and it found four faults nothing else could see
ITS never shown a tape of ours	DUMP's `LOAD` reads one, link and all
a link's header unmeasured	DUMP's `LINKS` switch, which its own help documents and the pack's `.INFO.` file does not

And three of those exposed a mistake in something already written down here: the MFDCLB cause was a guess, "DUMP does not write links" was a switch I had not set, and the seven-word header had been wrong since phase 8 with two independent readers agreeing about it.