- archive.py: PROVENANCE.json / archive-index.json / archive-state.json
now written atomically (tmp + os.replace) — a truncated integrity
record is the one thing this tool must never produce (AUDIT §4.4);
manifest entries validated as mappings up front (§4.7); refresh
rejects provenance with a missing/empty artifact key instead of
crashing on IsADirectoryError (§4.7); wayback save URL quotes
unsafe characters (§4.7)
- download-leaflet.sh: existing files are re-verified before being
skipped, and downloads land in a .part temp moved into place only
after checksum verification — a failed verification can no longer
leave a bad file that the next run silently accepts (§4.5)
- download-model.sh, convert-images.sh: same temp-then-move pattern so
interrupted downloads/conversions never persist at final paths (§4.6)
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Preserve external works the site cites against link rot, host them at
permanent /archive/<slug>/ URLs in site chrome, and treat them as
first-class citizens of the backlinks and similar-pages indexes.
Curated, not crawled: the author adds one line to archive/manifest.yaml
and the build fetches, hashes, snapshots, and indexes the work.
* archive/manifest.yaml + tools/archive.py (fetch / refresh / wayback /
check / gc) — PDFs downloaded directly, HTML pages snapshotted with a
vendored monolith (tools/bin/monolith @ 2.10.1) into a single
self-contained file with the archive CSP and a noarchive robots meta
injected. Per-entry PROVENANCE.json committed; gitignored .txt
sidecars regenerated from the artifact's SHA-256.
* build/Archive.hs + build/ArchiveIndex.hs + build/Filters/Archive.hs
— Hakyll rules for /archive/ and /archive/<slug>/, a body Pandoc
filter that appends an archive affordance to live citations and
flips dead ones to the local copy on archive.py check's asymmetric
hysteresis (rotted needs 3 fails over >= 14 days; one ok recovers).
* build/Backlinks.hs — keeps archived external URLs through pass 1 and
canonicalises them to /archive/<slug>/ in pass 2, producing a
"Referenced by" section grouped by the fragment each citation
targets. build/Stats.hs gains a "Link archive" telemetry block on
/build/ (count, total size, median age, by-status / by-quality /
by-visibility, orphans).
* Integrity: archive.py fetch and build/Archive.hs (via sha256sum)
both re-hash every committed artifact, so a tampered file halts the
build even with cabal invoked directly or no .venv present. refresh
refuses to replace an uncommitted prior snapshot and rolls back
atomically on any exit path. removed.yaml is honoured by fetch,
wayback, and check using canonical-form (tracking-stripped,
arXiv-canonicalised) comparison.
* visibility: private keeps an entry in-repo but undeployed.
nginx/archive.conf emits X-Robots-Tag: noindex, noarchive for raw
artifacts that cannot carry meta directives.
The full design, phase plan (1-5), and three refinement passes live
in ARCHIVE.md.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Levi Neuwirth