levineuwirth.org
A Hakyll static site: essays, research preprints, music catalog and score reader, photography pipeline, commonplace book, and a semantic-similarity index over the whole corpus. History restarted at this commit.
This commit is contained in:
commit
22f02a9d32
|
|
@ -0,0 +1,31 @@
|
|||
# Copy this file to .env and fill in the values, then run:
|
||||
# chmod 600 .env
|
||||
# so other local users cannot read your VPS path / token. .env is
|
||||
# gitignored — never commit it. The auto-snapshot in `make build`
|
||||
# uses an explicit pathspec under content/ to keep stray .env files
|
||||
# out of the snapshot, but **/.env is also in .gitignore as a backstop.
|
||||
#
|
||||
# `make deploy` pushes to GitHub first, then rsyncs the built _site/
|
||||
# to the VPS. The Makefile aborts with a clear error if any of
|
||||
# VPS_USER / VPS_HOST / VPS_PATH is unset, if VPS_PATH points at an
|
||||
# obviously dangerous parent directory, or if _site/index.html does
|
||||
# not exist (a sign of a broken build).
|
||||
|
||||
# --- VPS deployment target -------------------------------------------------
|
||||
# SSH user on the deployment VPS.
|
||||
VPS_USER=
|
||||
# Hostname or IP of the deployment VPS.
|
||||
VPS_HOST=
|
||||
# Absolute path to the document root on the VPS (e.g. /var/www/levineuwirth.org).
|
||||
VPS_PATH=
|
||||
|
||||
# --- GitHub mirror push ----------------------------------------------------
|
||||
# A GitHub fine-grained personal access token with Contents: read+write
|
||||
# on the levineuwirth.org repository. Currently optional — `make deploy`
|
||||
# uses your local git credential helper for `git push`, so this is only
|
||||
# needed if you wire token-based push into a credential helper yourself.
|
||||
# Generate at: https://github.com/settings/personal-access-tokens/new
|
||||
GITHUB_TOKEN=
|
||||
|
||||
# The GitHub repository in owner/repo format.
|
||||
GITHUB_REPO=levineuwirth/levineuwirth.org
|
||||
|
|
@ -0,0 +1,173 @@
|
|||
dist-newstyle/
|
||||
_site/
|
||||
_cache/
|
||||
.DS_Store
|
||||
.env
|
||||
# Defense-in-depth: the auto-snapshot in `make build` stages content/
|
||||
# wholesale. These patterns prevent any stray credential-shaped file
|
||||
# (dropped accidentally during writing) from being staged + pushed.
|
||||
# To intentionally commit one of these (rare), use `git add -f`.
|
||||
**/.env
|
||||
**/.env.*
|
||||
**/*.env
|
||||
# .env.example is documentation (tracked), not a credential file — the
|
||||
# patterns above would otherwise shadow it for status/add purposes.
|
||||
!.env.example
|
||||
**/*.key
|
||||
**/*.pem
|
||||
**/*.p12
|
||||
**/*.pfx
|
||||
**/id_rsa*
|
||||
**/id_dsa*
|
||||
**/id_ecdsa*
|
||||
**/id_ed25519*
|
||||
**/.netrc
|
||||
**/.npmrc
|
||||
**/.pypirc
|
||||
**/credentials
|
||||
**/credentials.json
|
||||
**/credentials.yaml
|
||||
**/credentials.yml
|
||||
|
||||
# Editor backup/swap files
|
||||
*~
|
||||
*.swp
|
||||
*.swo
|
||||
|
||||
# Python bytecode caches
|
||||
**/__pycache__/
|
||||
*.pyc
|
||||
*.pyo
|
||||
|
||||
# LaTeX build artifacts (sitewide — covers paper/, any future TeX sources)
|
||||
*.aux
|
||||
*.bbl
|
||||
*.blg
|
||||
*.brf
|
||||
*.fdb_latexmk
|
||||
*.fls
|
||||
*.glo
|
||||
*.gls
|
||||
*.idx
|
||||
*.ilg
|
||||
*.ind
|
||||
*.lof
|
||||
*.lot
|
||||
*.nav
|
||||
*.out
|
||||
*.snm
|
||||
*.synctex.gz
|
||||
*.toc
|
||||
*.vrb
|
||||
# PGF/TikZ scratch outputs
|
||||
pgftest*.pdf
|
||||
pgftest*.log
|
||||
pgftest*.aux
|
||||
# LaTeX run logs (scoped to paper/ — bare *.log would be too broad sitewide)
|
||||
paper/*.log
|
||||
|
||||
# Data files that are generated at build time (not version-controlled)
|
||||
data/embeddings.json
|
||||
data/similar-links.json
|
||||
data/backlinks.json
|
||||
data/build-stats.json
|
||||
data/build-start.txt
|
||||
data/build-stamp.txt
|
||||
data/last-build-seconds.txt
|
||||
data/semantic-index.bin
|
||||
data/semantic-meta.json
|
||||
# Both embed caches (pages + paragraphs); the trailing glob also
|
||||
# catches interrupted-write debris (.tmp / .tmp.npz)
|
||||
data/embed-cache-*
|
||||
|
||||
# Archive: generated text + its staleness stamp (recreated from the
|
||||
# committed artifact on every build — deterministic, so committing them is
|
||||
# churn). archive/**/PROVENANCE.json is deliberately NOT ignored — it is
|
||||
# the committed, immutable record of each archival event.
|
||||
archive/**/*.txt
|
||||
archive/**/*.txt.sha256
|
||||
data/archive-index.json
|
||||
data/archive-state.json
|
||||
|
||||
# IGNORE.txt is for the local build and need not be synced.
|
||||
IGNORE.txt
|
||||
|
||||
# Working notes / planning docs at the repo root (not site content).
|
||||
content/drafts/
|
||||
checklist.md
|
||||
FORGEJO-MIGRATION.md
|
||||
|
||||
# CV/résumé build pipeline (YAML → Jinja → xelatex). The canonical PDFs
|
||||
# live under static/ and ship with the site. The pipeline *source* is
|
||||
# tracked: data/ is the single source of truth for both documents (and is
|
||||
# read by the site build to render the vita page), and templates/ +
|
||||
# build.py are what regenerate them. Only artifacts stay local.
|
||||
yaml-source/build/
|
||||
yaml-source/output/
|
||||
yaml-source/data/*~
|
||||
# Handover bundle (archived locally for reference; not part of the site).
|
||||
levineuwirth_handover.zip
|
||||
|
||||
# Model files for client-side semantic search (~22 MB binary artifacts).
|
||||
# Download with: make download-model
|
||||
static/models/
|
||||
|
||||
# Vendored PDF.js viewer (~18 MB uncompressed, pinned in tools/download-pdfjs.sh).
|
||||
# Download with: make download-pdfjs
|
||||
static/pdfjs/
|
||||
|
||||
# Vendored Leaflet + leaflet.markercluster (~150 KB total, pinned in
|
||||
# tools/download-leaflet.sh). Used by the /photography/map/ page only.
|
||||
# Download with: make download-leaflet (runs as part of `make build`).
|
||||
static/leaflet/
|
||||
|
||||
# Generated WebP companions (produced by tools/convert-images.sh at build time).
|
||||
# To intentionally commit a WebP, use: git add -f path/to/file.webp
|
||||
static/**/*.webp
|
||||
content/**/*.webp
|
||||
|
||||
# Photography sidecars (produced by tools/extract-exif.py and
|
||||
# tools/extract-palette.py at build time; consumed by Hakyll). Recreated
|
||||
# from the photo file on every `make build`, so they don't belong in
|
||||
# version control — committing them would just produce churn.
|
||||
content/photography/**/*.exif.yaml
|
||||
content/photography/**/*.palette.yaml
|
||||
|
||||
# Image-dimension sidecars (produced by tools/extract-dimensions.py at
|
||||
# build time; consumed by build/Filters/Images.hs to emit width / height
|
||||
# attrs on every <img> for CLS prevention). Same churn-avoidance reasons
|
||||
# as the photography sidecars above; recreated on every `make build`.
|
||||
**/*.dims.yaml
|
||||
|
||||
# Photography delivery images.
|
||||
#
|
||||
# PHOTOGRAPHY.md originally kept web-optimized JPEGs in the repo and excluded
|
||||
# only the originals. That reversed in August 2026: 64 frames added 45 MB
|
||||
# against a 28 MB .git, and the trajectory was clear. They are derived
|
||||
# artifacts — tools/import-photo.sh regenerates them from the originals — so
|
||||
# git carries the .md entries and the deployed site carries the pixels, via
|
||||
# `make deploy`'s rsync of _site/.
|
||||
#
|
||||
# Consequence, stated plainly: a fresh clone builds with missing images, and
|
||||
# the originals under ~/Photos are now load-bearing for the whole section.
|
||||
# (.webp companions are already covered by content/**/*.webp above.)
|
||||
content/photography/**/*.jpg
|
||||
content/photography/**/*.jpeg
|
||||
content/photography/**/*.png
|
||||
|
||||
# Defense-in-depth — refuse to commit RAW or oversize originals, which were
|
||||
# never meant to be here even when the delivery JPEGs were. To intentionally
|
||||
# commit one of these formats (rare), use `git add -f path/to/file`.
|
||||
content/photography/**/*.cr2
|
||||
content/photography/**/*.cr3
|
||||
content/photography/**/*.nef
|
||||
content/photography/**/*.arw
|
||||
content/photography/**/*.dng
|
||||
content/photography/**/*.raf
|
||||
content/photography/**/*.orf
|
||||
content/photography/**/*.tif
|
||||
content/photography/**/*.tiff
|
||||
content/photography/**/*.psd
|
||||
# Claude Code local settings — machine-specific, and this repo mirrors
|
||||
# to a public GitHub remote.
|
||||
.claude/settings.local.json
|
||||
|
|
@ -0,0 +1 @@
|
|||
3.14
|
||||
File diff suppressed because it is too large
Load Diff
|
|
@ -0,0 +1,931 @@
|
|||
---
|
||||
title: Repository audit
|
||||
date: 2026-06-09
|
||||
---
|
||||
|
||||
# Repository audit — levineuwirth.org (2026-06-09)
|
||||
|
||||
Comprehensive audit of the repo on `main` at commit `620b974` (working tree
|
||||
modified: branding refresh across `static/` + `templates/partials/`, plus
|
||||
`tools/embed.py` rework; untracked `static/og-image.png`,
|
||||
`templates/partials/logo-mark.svg`, `data/embed-cache-pages.npz.tmp.npz`).
|
||||
|
||||
Severity legend: **HIGH** (likely to break a build, cause data loss, or
|
||||
expose a security weakness) — **MED** (latent bug, brittleness, or
|
||||
documentation drift) — **LOW** (minor robustness gap or fragile assumption) —
|
||||
**NIT** (style, polish, or paranoia).
|
||||
|
||||
Numbers are file:line against the working tree at audit time. Findings
|
||||
marked "verified" were reproduced empirically (solver runs, built `_site/`
|
||||
output inspection, live HTTP checks, binary parsing); the rest were
|
||||
confirmed by reading the code.
|
||||
|
||||
Prior audit: `AUDIT.md` (2026-05-07). Follow-up status in §10.
|
||||
|
||||
---
|
||||
|
||||
## 1. Build & dependency chain
|
||||
|
||||
### 1.1 `cabal.project.freeze` is unsolvable again — next clean build fails — **HIGH**
|
||||
|
||||
`cabal build --dry-run` fails today (verified): the freeze pins
|
||||
`distributive ==0.6.2.1`, but the system (pacman) GHC package db has
|
||||
`comonad-5.0.10` built against `distributive-0.6.3`:
|
||||
|
||||
```
|
||||
rejecting: distributive-0.6.3/installed... (constraint from
|
||||
cabal.project.freeze requires ==0.6.2.1)
|
||||
After searching the rest of the dependency tree exhaustively...
|
||||
```
|
||||
|
||||
The conflict set also names aeson, warp, hakyll, http2, semigroupoids. This
|
||||
is the same failure mode as prior-audit §1.1 — that audit's specific aeson
|
||||
pin was fixed (now 2.2.2.0/hashable 1.4.7.0), but a different package broke
|
||||
the same way after a system update. Recent builds succeed only off the
|
||||
cached `dist-newstyle/cache/plan.json`; the freeze file has since changed,
|
||||
so the next cabal invocation re-solves and fails. Because `make deploy`
|
||||
starts with `make clean`, the next deploy hits this. `levineuwirth.cabal`'s
|
||||
own bounds are compatible with the freeze — the conflict is
|
||||
freeze-vs-installed-db, not freeze-vs-cabal-file.
|
||||
|
||||
Fix: `tools/refreeze.sh` (written for exactly this post-`pacman -Syu`
|
||||
situation). The underlying fragility — freezing against a mutable system
|
||||
package db — remains; consider documenting the refreeze step as part of any
|
||||
system-upgrade ritual. *(In progress at time of writing.)*
|
||||
|
||||
### 1.2 Missing `data/archive-index.json` / `archive-state.json` crashes the build — **HIGH**
|
||||
|
||||
`build/ArchiveIndex.hs:134-146`. The module doc (lines 18-22) promises "An
|
||||
absent or malformed file degrades safely: an empty index makes the link
|
||||
consumers no-op; an absent state file makes every entry @Live@." But
|
||||
`rawIndex = unsafePerformIO $ do decoded <- A.eitherDecodeFileStrict' indexPath`
|
||||
(and identically `rawState`) never checks `doesFileExist`, and aeson's
|
||||
`eitherDecodeFileStrict'` throws an uncaught `IOException` on a missing
|
||||
file (verified: `withBinaryFile: does not exist`). Both files are
|
||||
gitignored (`.gitignore:84-85`), so a fresh clone or a no-`.venv` build —
|
||||
the exact path `build/Archive.hs:20-24` promises to support — throws when
|
||||
the CAF is first forced. Contrast `readUrlSet` (line 109) in the same file,
|
||||
which guards correctly. Currently latent on this machine only because both
|
||||
generated files happen to exist.
|
||||
|
||||
### 1.3 `embed.py` `trust_remote_code=True` executes unpinned third-party code — **HIGH**
|
||||
|
||||
`tools/embed.py:329` (line ~341 in the uncommitted version). The new
|
||||
page-model load is
|
||||
`SentenceTransformer(PAGE_MODEL_NAME, revision=PAGE_MODEL_REVISION, trust_remote_code=True)`.
|
||||
The `revision` arg pins only the `nomic-ai/nomic-embed-text-v1.5` repo; the
|
||||
actual modeling code is pulled via `auto_map` from a *different* repo —
|
||||
verified in the local HF cache: the executed code lives under
|
||||
`transformers_modules/nomic_hyphen_ai/nomic_hyphen_bert_hyphen_2048/...`,
|
||||
i.e. `nomic-ai/nomic-bert-2048` at its current head, which nothing pins. A
|
||||
compromise of that second repo runs arbitrary Python at build time, in a
|
||||
repo whose every other download path (download-model.sh, pdfjs, leaflet) is
|
||||
sha256-pinned. The comment "Both pins are deliberate" is therefore
|
||||
misleading. Fix: pin via `code_revision`, or run with `HF_HUB_OFFLINE=1`
|
||||
after first fetch, or document the accepted risk.
|
||||
|
||||
### 1.4 Working-tree commit hazard: tracked templates reference untracked files — **HIGH (process)**
|
||||
|
||||
`templates/partials/nav.html:5` (tracked, modified) adds
|
||||
`$partial("templates/partials/logo-mark.svg")$` and
|
||||
`templates/partials/head.html` references `/og-image.png` — both target
|
||||
files are **untracked** (no git history). Committing the template diff
|
||||
without `git add`-ing both breaks every page's Hakyll build on a fresh
|
||||
clone (`$partial$` aborts compilation) and 404s the og:image. They must
|
||||
land in the same commit. Conversely, `data/embed-cache-pages.npz.tmp.npz`
|
||||
must **not** be committed (see §4.1). The partial itself is safe as a
|
||||
Hakyll template (verified: zero `$` characters; `match "templates/**"`
|
||||
compiles it).
|
||||
|
||||
### 1.5 `einops` dependency: undocumented, unbounded, imported nowhere — **LOW**
|
||||
|
||||
`pyproject.toml:27` adds `einops>=0.8.2`. No import anywhere in
|
||||
`tools/`/`build/`/`static/js/`; its only consumer is nomic's
|
||||
`trust_remote_code` module (§1.3). Every sibling dependency has an
|
||||
explanatory comment and an upper bound per the file's own stated policy
|
||||
("Upper bounds are intentionally generous (next major) but always
|
||||
present"); einops has neither. `uv lock --check` passes (0.8.2 pinned).
|
||||
|
||||
---
|
||||
|
||||
## 2. Haskell build code — core
|
||||
|
||||
### 2.1 Nav, home grid, and library link `/fiction/` and `/poetry/` — confirmed 404s — **MED**
|
||||
|
||||
`build/Site.hs:50-60` (`homePortals` contains `("Fiction","fiction")`,
|
||||
`("Poetry","poetry")`), `templates/partials/nav.html:56,61`,
|
||||
`templates/library.html:44,58`. No rule generates either index: fiction and
|
||||
poetry are not in `tagIndexable` (`build/Patterns.hs:148-151` = essays +
|
||||
blog + photos) and Site.hs has no landing rule. Verified: `_site/fiction`
|
||||
does not exist; `_site/poetry/` has no `index.html`. nginx has no
|
||||
redirects. Both links 404 in production today.
|
||||
|
||||
### 2.2 Tag/route collisions guarded for `photography` only — **MED**
|
||||
|
||||
`build/Tags.hs:98-99`. `tagIdentifier` maps tag `t` → `t ++ "/index.html"`;
|
||||
`sectionOwnedTopLevelTags = ["photography"]` is the only guard. A
|
||||
tagIndexable item tagged `music` (or `music/x`, which expands to `music`)
|
||||
emits `music/index.html`, already owned by the music index route
|
||||
(`build/Site.hs:486-487`); similarly `essays`, `blog`, `cv`, `archive`,
|
||||
`authors`, `bibliography`. Hakyll does not error on duplicate routes — one
|
||||
silently overwrites the other.
|
||||
|
||||
### 2.3 Sidenotes filter destroys the documented no-JS fallback — **MED**
|
||||
|
||||
`build/Filters/Sidenotes.hs:30-36` vs `static/css/sidenotes.css:125-135`.
|
||||
The module doc claims the Pandoc `<section class="footnotes">` "serves as
|
||||
fallback," but `apply` replaces every `Note`, so the writer never emits the
|
||||
section. CSS depends on it below 1500px. Verified in output:
|
||||
`_site/essays/scaling_outage.html` has 3 `class="sidenote"` and zero
|
||||
`footnotes` occurrences. With JS disabled, footnote content is invisible on
|
||||
narrow viewports. The comment, the CSS, and ozymandias.md's own prose all
|
||||
contradict actual behavior.
|
||||
|
||||
### 2.4 Sidenote bodies rendered without the KaTeX writer — **MED**
|
||||
|
||||
`build/Filters/Sidenotes.hs:103-115`. `inlinesToHtml`/`blocksToHtml` use
|
||||
`writeHtml5String (def :: WriterOptions)` (PlainMath), while the main
|
||||
pipeline uses `KaTeX ""` (`build/Compilers.hs:47`). Math inside a footnote
|
||||
never gets `<span class="math inline">\(...\)</span>`, so KaTeX never
|
||||
renders it — degrades to plain italics, silently inconsistent with body
|
||||
math.
|
||||
|
||||
### 2.5 SourceRefs whitelist vs `/source/` serving whitelist have drifted — **MED**
|
||||
|
||||
`build/Filters/SourceRefs.hs:114-141` vs `build/Site.hs:217-240`. Site.hs:209
|
||||
says "must stay aligned with 'isSourcePath'". Mismatches: SourceRefs wraps
|
||||
`content/` and `yaml-source/` (no Site counterpart); `static/` + any known
|
||||
ext vs Site's `static/js/**`/`static/css/**` only; `tools/` + any ext vs
|
||||
Site's `tools/**.sh`/`tools/**.py`; `data/` at any depth vs Site's
|
||||
top-level `data/*.{json,yaml,md,bib}`. Each mismatch yields a wrapped
|
||||
source-ref whose popup fetch 404s (Forgejo href fallback still works).
|
||||
Inverse: Site serves `data/*.bib` but `.bib` is missing from
|
||||
`hasKnownExt` — dead whitelist entry.
|
||||
|
||||
### 2.6 `epistemicEntry` ignores `confidence: proved` — **MED**
|
||||
|
||||
`build/Site.hs:1014-1024`. Comment: "Compute overall-score the same way
|
||||
Contexts.overallScoreField does," but it uses
|
||||
`readMaybe =<< lookupString "confidence" meta`, which is `Nothing` for
|
||||
`"proved"`/`"proven"`, whereas `Contexts.overallScoreField`
|
||||
(`build/Contexts.hs:574-576`) substitutes 100 via `isProvedConfidence`.
|
||||
Proved pages get no `score` in `data/epistemic-meta.json` and export the
|
||||
raw string under `confidence`, so client-side filtering silently misses
|
||||
them.
|
||||
|
||||
### 2.7 Empty affiliation `<div>` ships on every essay without `affiliation:` — **MED**
|
||||
|
||||
`build/Contexts.hs:84-89` + `templates/partials/metadata-tail.html:12`.
|
||||
`affiliationField` returns an empty list instead of `noResult`; Hakyll's
|
||||
`$if$` is truthy for empty list fields (the codebase knows this —
|
||||
`tagLinksFieldExcludingScope` uses `noResult` for exactly this reason).
|
||||
Verified in output: `_site/essays/asymmetric-forgetting.html` contains
|
||||
`<div class="meta-row meta-affiliation">` with whitespace-only content.
|
||||
|
||||
### 2.8 Library page hard-depends on `content/library.md` — **LOW**
|
||||
|
||||
`build/Site.hs:675`. `_ <- loadSnapshot libraryIntroId "body"` is a
|
||||
top-level compiler statement (not inside a `field`), so it's a hard
|
||||
failure. The block is documented as "optional prose block"; deleting
|
||||
`content/library.md` breaks the whole `library.html` compile. Contrast the
|
||||
existence-guarded sidecars at `build/Tags.hs:277-283` and
|
||||
`build/Site.hs:843-850`.
|
||||
|
||||
### 2.9 Library `primaryPortalOf` reads only list-form `tags:` — **LOW**
|
||||
|
||||
`build/Site.hs:632-638`. `lookupStringList "tags"` returns `Nothing` for
|
||||
scalar comma form (`tags: research, ai`), which Hakyll's `getTags`
|
||||
accepts. Such an item appears on tag pages but is silently dropped from
|
||||
the library. All current content uses list form — latent.
|
||||
|
||||
### 2.10 `allContent` omits me/, memento-mori/, photography from the link graph — **LOW**
|
||||
|
||||
`build/Patterns.hs:124-133`, used by `build/Backlinks.hs:334,345`. Despite
|
||||
"Every content file the backlinks pass should index," `content/me/index.md`
|
||||
and `content/memento-mori/index.md` (full essays, rendered with
|
||||
`backlinksField`) never have their outgoing links extracted; photography
|
||||
likewise. Either deliberate-but-undocumented or the exact silent omission
|
||||
the module header says it exists to prevent.
|
||||
|
||||
### 2.11 Paginated tag pages: split by creation date, sorted by display date — **LOW**
|
||||
|
||||
`build/Tags.hs:371-377`. `buildPaginateWith (sortAndGroupAt tagPageSize)`
|
||||
partitions via `sortRecentFirst` (creation date), then each page re-sorts
|
||||
with `recentFirstByDisplay` (revision-aware). A recently revised old item
|
||||
stays on a late page but jumps to its top — cross-page ordering is not
|
||||
monotone. Only fires above the 150-item threshold.
|
||||
|
||||
### 2.12 `fill:#000` replacement corrupts longer hex colors — **LOW**
|
||||
|
||||
`build/Filters/Score.hs:118-133` (and `Filters/Viz.hs` `processColors`).
|
||||
The 6-digit pass protects only `#000000`; for `fill:#000080` the 3-digit
|
||||
pass produces `fill:currentColor80` — invalid CSS, silently mangled SVG.
|
||||
Quoted attribute forms are safe; only unquoted style-property forms are
|
||||
exposed.
|
||||
|
||||
### 2.13 Source-level preprocessors rewrite inside fenced code blocks — **LOW**
|
||||
|
||||
`build/Filters/Wikilinks.hs:24-31`, `Filters/Transclusion.hs:18-20`,
|
||||
`Filters/EmbedPdf.hs`. All run on the raw source before Pandoc parses
|
||||
fences: `[[anything]]` in a code block becomes a link; a code-block line
|
||||
that is exactly `{{slug}}` or `{{pdf:...}}` becomes raw HTML.
|
||||
Transclusion's comment ("prevents accidental substitution inside prose or
|
||||
code") is false for full-line directives in code blocks. A live foot-gun
|
||||
for a site that documents its own syntax (ozymandias.md does exactly
|
||||
this).
|
||||
|
||||
### 2.14 `domainIcon` matches substrings of the whole URL, not the host — **LOW**
|
||||
|
||||
`build/Filters/Links.hs:120-153`. `"x.com" `T.isInfixOf` url` etc. —
|
||||
`https://example.org/why-x.com-failed` gets the Twitter icon. Contradicts
|
||||
the strict-hostname discipline `isExternal` documents at lines 95-101 of
|
||||
the same file. Cosmetic (icon only).
|
||||
|
||||
### 2.15 `gsubRoute "content/"` strips every occurrence, not just the prefix — **LOW**
|
||||
|
||||
`build/Site.hs:171,357,417` etc. Hakyll's `gsubRoute` is replace-all; a
|
||||
co-located directory literally named `content` would be silently mangled
|
||||
(`content/essays/slug/content/data.csv` → `essays/slug/data.csv`). Same
|
||||
for `gsubRoute "static/"`. Improbable but silent.
|
||||
|
||||
### 2.16 `existsCached` memoizes non-existence for the process lifetime — **LOW**
|
||||
|
||||
`build/Filters/SourceRefs.hs:160-166`. Under `make watch`, a source file
|
||||
created after first reference stays cached as absent until restart.
|
||||
|
||||
### 2.17 Core NITs
|
||||
|
||||
- `build/Site.hs:42-44`: comment says "eight portals"; the list has nine.
|
||||
Echoed at Site.hs:606 ("the eight") vs line 657's "nine times".
|
||||
- `build/Site.hs:866-877`: random-pages.json comment says "essays + blog
|
||||
posts only" but the rule loads fiction and flat poetry too; uses
|
||||
flat-only `content/poetry/*.md` while the epistemic rule uses
|
||||
`allPoetry` — collection poems are epistemic-indexed but never
|
||||
randomizable.
|
||||
- `build/Utils.hs:64-73`: `authorSlugify` comment claims runs of spaces
|
||||
collapse; code maps each space (`"A B"` → `"a--b"`). Consistent
|
||||
everywhere, so links work; comment wrong.
|
||||
- `build/Utils.hs:31-32`: `readingTime` truncates (`div 200`) — 399 words
|
||||
reports "1 min"; comment implies ceiling semantics.
|
||||
- `build/Pagination.hs:42` + `build/Site.hs:77-82`: hardcoded pattern
|
||||
literals duplicate `Patterns.hs`, defeating that module's stated purpose
|
||||
(Patterns.hs:6-10).
|
||||
- `build/Contexts.hs:174-180`: plain `tagLinksField` returns an empty list
|
||||
rather than `noResult` — `$if(item-tags)$` is true and templates emit
|
||||
empty tag wrappers (author-index.html, item-card.html).
|
||||
- `build/Tags.hs:296-304`: `tagItemCtx` composes `defaultContext`, not
|
||||
`siteCtx`, so `$if(has-monogram)$` never fires on tag pages — monograms
|
||||
render on new.html/library but silently never on tag indexes.
|
||||
- `build/Contexts.hs:485-492`: `dotsField` comment says "1–5" but accepts
|
||||
0 (`max 0 (min 5 n)`) — `importance: 0` renders five empty circles.
|
||||
- `build/Contexts.hs:375-381`: `descriptionField` doc says `noResult`;
|
||||
code uses `fail` — behaviorally fine under Hakyll 4.16 `$if$` (verified
|
||||
against Hakyll 4.16.7.1 source) but logs `[ERROR]` debug noise per
|
||||
abstract-less page. Same in `abstractField`, `summaryField`,
|
||||
`bibliographyField`.
|
||||
- `build/Filters/Images.hs:233-234`: `webpSrc` interpolated into `srcset`
|
||||
unescaped while sibling `src` goes through `esc`.
|
||||
- `build/Filters/Links.hs:37-46,63-69`: internal PDF links double-classified
|
||||
(`pdf-link` + `link-internal` chrome) despite the "no overlap" comment.
|
||||
- `build/Filters/Smallcaps.hs:31-34` + `Filters/Archive.hs:42-44`:
|
||||
"headers are skipped" only at top level; a Header nested in a
|
||||
Div/BlockQuote is processed, contradicting the comments.
|
||||
|
||||
Verified clean: no unguarded `head`/`fromJust`/`read`/`!!` hazards in the
|
||||
core modules; filter composition order matches its documenting comments;
|
||||
Hakyll 4.16.7.1 `$if$` treats both `fail` and `noResult` as false.
|
||||
|
||||
---
|
||||
|
||||
## 3. Haskell build code — feature modules
|
||||
|
||||
### 3.1 Stats heatmap day-of-week off-by-one: Sunday clipped out of the SVG — **MED**
|
||||
|
||||
`build/Stats.hs:185,300,317`. `dowOf d = fromEnum (dayOfWeek d) -- Mon=0..Sun=6`
|
||||
— but `time-1.12.2` is ISO-numbered (verified:
|
||||
`map fromEnum [Monday..Sunday] == [1..7]`). So Sunday lands at y=106 while
|
||||
`svgH` = 104 — every Sunday cell is clipped out of the viewBox and grid
|
||||
row 0 is permanently blank. Relatedly, `weekStart` returns the previous
|
||||
*Sunday* (and for a Sunday, 7 days back), not the "first Monday on or
|
||||
before" its comment claims; builds run on a Sunday also clip the newest
|
||||
column horizontally.
|
||||
|
||||
### 3.2 `Commonplace.hs` uses `Char8.pack` — non-ASCII YAML corruption — **MED**
|
||||
|
||||
`build/Commonplace.hs:143`. `Y.decodeEither' (BS.pack raw)` with
|
||||
`Data.ByteString.Char8` truncates each `Char` to 8 bits — the exact hazard
|
||||
`build/Now.hs:249-253` documents and fixes with `TE.encodeUtf8`.
|
||||
`data/commonplace.yaml` is currently pure ASCII, so latent — but a
|
||||
commonplace book of quotations is the likeliest file to acquire an em-dash
|
||||
or curly quote, which will then either fail the YAML parse or publish
|
||||
mojibake.
|
||||
|
||||
### 3.3 Backlinks: links inside tight lists are invisible — **MED**
|
||||
|
||||
`build/Backlinks.hs:220-226`. `extractLinksWithContext`'s `go` handles
|
||||
`Para`, `BlockQuote`, `Div`, `BulletList`, `OrderedList`, then `go _ = []`.
|
||||
Tight list items (the default `- item` form) are `Plain` blocks, not
|
||||
`Para`, so recursion into list children yields nothing. Every internal
|
||||
link written in a tight list never produces a backlink. `Header`, `Table`,
|
||||
and `DefinitionList` blocks are likewise skipped. The doc comment implies
|
||||
coverage it doesn't deliver.
|
||||
|
||||
### 3.4 Stability "age" is the first→last commit span, not time since first commit — **MED**
|
||||
|
||||
`build/Stability.hs:89-93,99-112`. Docs say "age in days since first
|
||||
commit," but `classify (length dates) (daySpan (last dates) newest)`
|
||||
computes the span between first and most recent *commit*, with no
|
||||
reference to today. A piece written in a one-week burst years ago reports
|
||||
"volatile" forever; time passing without commits can never increase
|
||||
stability. Either the comment or the metric is wrong.
|
||||
|
||||
### 3.5 Frontmatter `history:` assumed newest-first; WRITING.md documents oldest-first — **MED**
|
||||
|
||||
`build/Stability.hs:204-217,299-336` vs `WRITING.md:105-109`.
|
||||
`loadVersionHistory` keeps authored order and all range fields treat the
|
||||
head as newest (`es@(newest:_) -> let oldest = last es`). Git history is
|
||||
newest-first, but WRITING.md's `history:` example is oldest-first. With
|
||||
the documented ordering, `version-history-range` renders reversed
|
||||
("14 March 2026 – 1 March 2026"), `range-start` returns the newest date,
|
||||
and `version-history-primary` shows the three *oldest* entries.
|
||||
|
||||
### 3.6 Archive manifest→provenance join is exact-string, rest of system is normalized — **MED**
|
||||
|
||||
`build/Archive.hs:269`. `Map.lookup (meUrl me) provByUrl` joins on the raw
|
||||
URL; everywhere else equivalence is `normalizeUrl` (ArchiveIndex
|
||||
filtering, dup detection, ARCHIVE.md:189-192). Editing a manifest URL to a
|
||||
normalization-equivalent form (`http`→`https`, trailing slash, tracking
|
||||
param) silently unpublishes `/archive/<slug>/` while ArchiveIndex's
|
||||
normalized filter keeps the slug active — links keep pointing at a 404.
|
||||
|
||||
### 3.7 Photography `buildPin` computes wrong slug/thumb/title for flat entries — **MED**
|
||||
|
||||
`build/Photography.hs:354,362`. `slug = takeFileName (takeDirectory fp)` —
|
||||
for a flat `content/photography/foo.md` this yields `"photography"`, so
|
||||
map.json gets `"slug": "photography"`, the title fallback is wrong, and
|
||||
`thumb = "/photography/photography/<p>"` 404s (flat-single assets route to
|
||||
`/photography/<asset>`). PHOTOGRAPHY.md:214 explicitly supports flat
|
||||
singles. Latent — `content/photography/` currently has only `index.md` —
|
||||
but breaks the first geo-tagged flat single.
|
||||
|
||||
### 3.8 `geo-precision` fails open: a typo'd "hidden" publishes coordinates — **MED**
|
||||
|
||||
`build/Photography.hs:347-349,312-320`. Only the exact string matches
|
||||
(`(_, Just "hidden", _) -> return Nothing`); any other value (e.g.
|
||||
`Hidden`, `hiddn`) falls into `roundCoord`, whose catch-all treats unknown
|
||||
values as `city` (~10 km rounding) — publishing coordinates the author
|
||||
meant to suppress. Contradicts the file's own privacy comment (lines
|
||||
287-289) and the fail-closed precedent for `visibility:` in
|
||||
`build/Archive.hs:77-83`.
|
||||
|
||||
### 3.9 Archive state is process-lifetime cached — `watch` goes stale — **LOW**
|
||||
|
||||
`build/ArchiveIndex.hs:123-146` + `build/Archive.hs:304`.
|
||||
`activeUrls`/`rawIndex`/`rawState` are NOINLINE `unsafePerformIO` CAFs read
|
||||
once per process, and `archiveRules` reads the manifest in `preprocess`.
|
||||
Under `site watch`, edits to `manifest.yaml`, `removed.yaml`, or the
|
||||
regenerated state JSONs are never re-read until restart. One-shot builds
|
||||
unaffected.
|
||||
|
||||
### 3.10 Pinned pages render raw ISO in `$last-reviewed$` — **LOW**
|
||||
|
||||
`build/Stability.hs:166-170`. The git branch formats via `fmtIso`
|
||||
("1 May 2026"); the IGNORE.txt-pinned branch returns the frontmatter value
|
||||
verbatim ("2026-05-01") — inconsistent display formatting.
|
||||
|
||||
### 3.11 Empty/all-comments `manifest.yaml` halts the build — **LOW**
|
||||
|
||||
`build/Archive.hs:158-170`. An empty YAML stream decodes as `Null`, which
|
||||
fails to parse as `[ManifestEntry]` and takes the `exitFailure` branch —
|
||||
draining the manifest to zero entries is fatal rather than the empty
|
||||
archive the absent-file branch supports.
|
||||
|
||||
### 3.12 Backlinks `normaliseUrl` misses directory-form canonical URLs — **LOW**
|
||||
|
||||
`build/Backlinks.hs:275-281`. Strips `.html` but not
|
||||
`index.html`/trailing slash: a page routed `essays/foo/index.html` keys as
|
||||
`/essays/foo/index`, but a body link authored `/essays/foo/` doesn't
|
||||
match — backlink silently dropped. `build/SimilarLinks.hs:97-99` handles
|
||||
exactly this case and its comment flags the divergence.
|
||||
|
||||
### 3.13 SimilarLinks PDF viewer URL not percent-encoded — **LOW**
|
||||
|
||||
`build/SimilarLinks.hs:155-164`.
|
||||
`viewerUrl = "/pdfjs/web/viewer.html?file=" ++ escapeHtml raw` —
|
||||
`escapeHtml` handles HTML metachars only; a path containing `&`, `?`, `#`,
|
||||
or spaces breaks the `file=` query value.
|
||||
|
||||
### 3.14 Photography feed thumbnails only for directory-form entries — **LOW**
|
||||
|
||||
`build/Photography.hs:449-453`. `imgTag` requires `isDir`; flat singles
|
||||
and series children (`<series>/<photo>.md`) get text-only feed entries,
|
||||
against PHOTOGRAPHY.md's "thumbnails embedded inline" (lines 36, 445) and
|
||||
the feed's deliberate inclusion of series children.
|
||||
|
||||
### 3.15 Marks: missing confidence/evidence renders a literal "0 TRUST" — **LOW**
|
||||
|
||||
`build/Marks.hs:272-278,565`. `computeTrust _ _ = 0` with a comment
|
||||
claiming the figure "collapses to the bare frame," but
|
||||
`renderEpistemicFigure` unconditionally calls `renderTrustLabel`, so a
|
||||
piece with `status:` but no `confidence`/`evidence` (a case MARKS.md:696
|
||||
says should render) displays a prominent center "0" — indistinguishable
|
||||
from an authored zero-trust score.
|
||||
|
||||
### 3.16 Feature-module NITs
|
||||
|
||||
- `build/Catalog.hs:228-235`: two distinct unknown categories render as
|
||||
adjacent duplicate "Other" sections (equal rank, `groupBy` on raw
|
||||
string).
|
||||
- `build/Stats.hs:754-777`: `pageTOC` comment says "nine h2 sections";
|
||||
lists eleven (matching the eleven rendered).
|
||||
- `build/SimilarLinks.hs:51-54`: comment says "the template caps the
|
||||
display"; the code caps it (`take maxSimilar` at line 80).
|
||||
- `build/Stats.hs:169-171`, `build/Archive.hs:564-569`: "median" is the
|
||||
upper-median for even-length lists.
|
||||
- `build/Backlinks.hs:133-153`: protocol-relative `//host/path` URLs pass
|
||||
`isPageLink` and pollute backlinks.json.
|
||||
- `build/BibExtras.hs:75-98`: `@string`/`@comment`/`@preamble` blocks
|
||||
parsed as citekey entries — only consequential on a citekey/macro-name
|
||||
collision.
|
||||
|
||||
Verified clean: Marks tick positions/axis order/radii match MARKS.md §3;
|
||||
proved-confidence trust substitution matches §4.3; Archive's fail-closed
|
||||
`visibility` validation, removed.yaml conflict rejection, and double-sided
|
||||
SHA-256 verification all match ARCHIVE.md.
|
||||
|
||||
---
|
||||
|
||||
## 4. Python & shell tooling
|
||||
|
||||
### 4.1 `data/embed-cache-pages.npz.tmp.npz` orphan: explained; cleanup + ignore gaps — **MED**
|
||||
|
||||
The orphan (mtime May 26) is the fossil of a fixed bug: an earlier
|
||||
embed.py passed a bare path to `np.savez_compressed`, numpy appended
|
||||
`.npz` (verified in numpy's `_savez` source), and the subsequent
|
||||
`os.replace` raised FileNotFoundError, stranding the file. The current
|
||||
file-handle code (`tools/embed.py:173-183`) is correct, but: (a) nothing
|
||||
deletes the stale orphan — **delete it, don't commit it**; (b) the tmp
|
||||
write has no try/finally, so any mid-write exception strands
|
||||
`embed-cache-pages.npz.tmp`; (c) the new `.gitignore` entry is exact-path
|
||||
(`data/embed-cache-pages.npz`) and covers neither `.tmp` nor `.tmp.npz`
|
||||
variants — widen to `data/embed-cache-pages.npz*`; (d) the fixed tmp name
|
||||
means two concurrent runs interleave writes.
|
||||
|
||||
### 4.2 Corrupt embed cache crashes instead of being discarded — **MED**
|
||||
|
||||
`tools/embed.py:154`. The discard path catches
|
||||
`(OSError, KeyError, ValueError)`, but `np.load` on a truncated `.npz`
|
||||
raises `zipfile.BadZipFile` (verified MRO: `BadZipFile → Exception`), and
|
||||
`EOFError` is also uncaught. A half-written cache (exactly what §4.1(b)
|
||||
can produce) makes every subsequent build print "Warning: embedding
|
||||
failed" and leaves similar-links/semantic index stale until the file is
|
||||
manually deleted — the opposite of the docstring's "unreadable →
|
||||
discarding" contract.
|
||||
|
||||
### 4.3 embed.py staleness check structurally defeated by stamp-build-time — **MED**
|
||||
|
||||
`tools/embed.py:195-200` + `Makefile:68`. `needs_update()` compares
|
||||
`_site/**/*.html` mtimes against embed's outputs — but the build order is
|
||||
`embed.py` → `stamp-build-time.py _site`, and the stamper rewrites the
|
||||
footer timestamp in essentially every HTML file each build. So every page
|
||||
is always newer than embed's outputs and the "skip if fresh" fast path
|
||||
never fires: the full paragraph-embedding pass (and model load) runs on
|
||||
every build. The new page cache papers over half the cost; the paragraph
|
||||
pass pays full price every time. Related (`tools/embed.py:297-299`):
|
||||
model/config changes never invalidate outputs — currently masked by this
|
||||
bug; fixing one exposes the other.
|
||||
|
||||
### 4.4 archive.py writes provenance/index/state non-atomically — **MED**
|
||||
|
||||
`tools/archive.py:718-721,734-737,953-957,1077-1080`. All plain
|
||||
`write_text()`. An interrupt mid-write truncates `PROVENANCE.json`; the
|
||||
next build's `json.loads` (line 642) raises an unhandled
|
||||
`JSONDecodeError` — and a truncated provenance is indistinguishable from
|
||||
corruption in a tool whose whole contract is integrity checking. embed.py
|
||||
got atomic-write helpers; archive.py did not.
|
||||
|
||||
### 4.5 download-leaflet.sh: checksum verification bypassable — **MED**
|
||||
|
||||
`tools/download-leaflet.sh:43-47,90`. The early-exit skip checks file
|
||||
existence only (download-model.sh re-verifies on its skip path), and
|
||||
`curl -o "$target"` writes directly to the final path: a download that
|
||||
*fails* `verify_or_warn` aborts via `set -e` *after* the bad file is in
|
||||
place, and the next run's existence check accepts it permanently. A
|
||||
MITM'd unpkg.com download survives one failed run and is silently
|
||||
vendored on the next.
|
||||
|
||||
### 4.6 Other download/convert scripts leave partial files in final paths — **LOW**
|
||||
|
||||
`tools/download-model.sh:84`: interrupted curl leaves a partial
|
||||
`model_quantized.onnx`; caught today only because model-checksums.sha256
|
||||
pins all five files — any unpinned file would persist forever. Use
|
||||
`-o "$dst.part" && mv`. `tools/convert-images.sh:33`: interrupted cwebp
|
||||
leaves a partial `.webp` that the `-nt` staleness gate then skips forever
|
||||
— a truncated WebP ships until manually deleted.
|
||||
|
||||
### 4.7 archive.py robustness gaps — **LOW**
|
||||
|
||||
- `tools/archive.py:788,795-799`: provenance missing the `artifact` key
|
||||
makes `prev_artifact == slug_dir`, then `sha256_of` raises an uncaught
|
||||
`IsADirectoryError` instead of the structured "prior snapshot
|
||||
incomplete" error.
|
||||
- `tools/archive.py:614-617,938-940,1066-1068`: non-dict manifest entries
|
||||
(`- https://example.com` instead of `- url: ...`) crash with
|
||||
`AttributeError: 'str' object has no attribute 'get'`.
|
||||
- `tools/archive.py:896`: `wayback_save` concatenates the raw URL
|
||||
(contrast `wayback_lookup` at 909, which uses `quote(url, safe="")`).
|
||||
|
||||
### 4.8 add-popup-source.sh: dead CSP reminder + unvalidated nginx interpolation — **LOW**
|
||||
|
||||
`tools/add-popup-source.sh:214`: the connect-src reminder gates on
|
||||
`[[ "$NEEDS_PROXY" -eq 0 && -n "$UPSTREAM_HOST" ]]`, but `UPSTREAM_HOST`
|
||||
is only set in the `NEEDS_PROXY -eq 1` branch (lines 124-131) — the
|
||||
reminder can never print, and the no-proxy case is exactly when it's
|
||||
needed (the provider will be CSP-blocked with no hint). Line 71: `NAME`
|
||||
from a free-text prompt is interpolated into
|
||||
`location /proxy/$NAME/`/`set $upstream_$NAME` with no
|
||||
`^[a-z0-9-]+$` validation (import-photo.sh validates; this doesn't).
|
||||
|
||||
### 4.9 refreeze.sh deletes the freeze before the replacement succeeds — **LOW**
|
||||
|
||||
`tools/refreeze.sh:13-16`. `rm -f "$FREEZE"` then `cabal freeze`; a failed
|
||||
resolve leaves no freeze file (recoverable via git, but write-temp-then-move
|
||||
is safer).
|
||||
|
||||
### 4.10 embed.py / atomic-write NITs — **LOW/NIT**
|
||||
|
||||
`tools/embed.py:109-115`: `atomic_write_bytes` uses a fixed `.tmp` name
|
||||
(concurrent-run collision) and no `fsync` before `os.replace` (power loss
|
||||
can leave an empty target). Same pattern in `_atomic_write_yaml` of
|
||||
extract-exif.py:377, extract-palette.py:65, extract-dimensions.py:65.
|
||||
`tools/embed.py:144`: NpzFile never closed — use
|
||||
`with np.load(...) as npz:`.
|
||||
|
||||
### 4.11 Tooling NITs
|
||||
|
||||
- `tools/import-photo.sh:147-155`: on `mogrify -strip` failure the
|
||||
EXIF-laden JPEG (GPS, serials) remains under `content/`, where
|
||||
`make build`'s `git add content/` could auto-commit it. Delete `$TARGET`
|
||||
on that failure path.
|
||||
- `tools/hooks/pre-commit-marks.sh:28-31`: `awk '{ print $2 }'` truncates
|
||||
paths with spaces; the `status:` probe reads the working tree, not the
|
||||
staged blob. Advisory-only hook.
|
||||
- `tools/preset-signing-passphrase.sh:30`: `echo -n "$PASSPHRASE"` eats a
|
||||
passphrase starting with `-e`/`-n`/`-E`; use `printf '%s'`.
|
||||
- `tools/stamp-build-time.py:52-54`: in-place non-atomic rewrite of
|
||||
`_site/` HTML.
|
||||
- `tools/archive.py:244`: `pdftotext` without `--`; a slug starting with
|
||||
`-` parses as an option. Same in extract-exif.py:159.
|
||||
- `tools/monolith-version.txt` records a sha256 (matches the binary
|
||||
today, verified) but `find_monolith()` never checks it.
|
||||
|
||||
Verified clean: sign-site.sh (atomic sig writes, post-pass manifest
|
||||
verification); compress-assets.sh and download-pdfjs.sh (mktemp + EXIT
|
||||
trap, hash verified before extraction); audit-marks.py, viz_theme.py,
|
||||
extract-dimensions.py, extract-palette.py; embed.py's faiss `-1` padding
|
||||
is safely filtered; `uv lock --check` passes; model-checksums.sha256 pins
|
||||
all five model files.
|
||||
|
||||
---
|
||||
|
||||
## 5. Frontend JavaScript
|
||||
|
||||
### 5.1 Score-reader pages never restore theme/settings — **MED**
|
||||
|
||||
`templates/score-reader-default.html:10` + `static/js/theme.js:12-13`. The
|
||||
template loads `theme.js` without `utils.js` (unlike head.html:66-67), so
|
||||
`window.lnUtils.safeStorage` is undefined and theme/text-size/focus-mode/
|
||||
reduce-motion all silently fail to restore — a dark-theme user gets a
|
||||
light flash-and-stay on every score page. Compounding: settings.js (line
|
||||
15; the template does render the settings toggle) falls back to its no-op
|
||||
store, so theme picks made on score pages never persist either.
|
||||
|
||||
### 5.2 search-filters.js: epistemic filters silently bypass clean-URL pages — **MED**
|
||||
|
||||
`static/js/search-filters.js:117-125`. `normUrl()` returns `u.pathname`
|
||||
verbatim and looks it up in `epistemicMeta[url]`. Verified:
|
||||
`_site/data/epistemic-meta.json` keys include
|
||||
`/essays/beyond-comorbidity-indices/index.html` while rendered result
|
||||
links use `/essays/beyond-comorbidity-indices/`. The lookup misses,
|
||||
`passes(null)` returns true ("no metadata = don't filter"), so every
|
||||
directory-style page bypasses all active epistemic filters. Flat `.html`
|
||||
pages match fine, which hides the bug.
|
||||
|
||||
### 5.3 viz.js ignores the cappuccino theme — **MED**
|
||||
|
||||
`static/js/viz.js:94-99`. `isDark()` knows only
|
||||
`'dark'`/`'light'`/OS-preference, but theme.js/settings.js support
|
||||
`'cappuccino'` — a dark-brown theme (`--bg: #553a28`, base.css:203). With
|
||||
OS-light + cappuccino, charts render the LIGHT config (near-black marks
|
||||
and axis labels) on a dark background.
|
||||
|
||||
### 5.4 collapse.js localStorage keys collide across pages — **MED**
|
||||
|
||||
`static/js/collapse.js:44,83`. Key is
|
||||
`'section-collapsed:' + heading.id` with no pathname namespace (contrast
|
||||
annotations.js). Pandoc auto-slugs (`#introduction`, `#background`) recur
|
||||
across essays, so collapsing "Introduction" on one essay collapses it
|
||||
everywhere. Also uses raw `localStorage` rather than
|
||||
`lnUtils.safeStorage`.
|
||||
|
||||
### 5.5 semantic-search.js: stale-response race + duplicate index fetch — **MED**
|
||||
|
||||
`static/js/semantic-search.js:117-144`. `runSearch` has no generation
|
||||
token; overlapping queries render in promise-resolution order, so an
|
||||
older query's hits can replace a newer one's (with `setStatus('')`
|
||||
masking it). `loadIndex()` (42-59) has no in-flight-promise dedup (unlike
|
||||
`loadModel`'s `loadModelPromise`), so concurrent first searches fetch
|
||||
`semantic-index.bin` + `semantic-meta.json` twice.
|
||||
|
||||
### 5.6 lightbox.js: aria-modal with no focus trap, no keyboard activation — **MED**
|
||||
|
||||
`static/js/lightbox.js`. Overlay sets `role="dialog"` +
|
||||
`aria-modal="true"` but has no Tab handling (gallery.js's `trapTab` at
|
||||
235-257 shows the in-repo pattern) — focus walks into the obscured page.
|
||||
Trigger images get only a `click` listener and no `tabindex`/keydown, so
|
||||
keyboard users can't open it; `close()` focuses a non-focusable `<img>`,
|
||||
which no-ops.
|
||||
|
||||
### 5.7 Frontend LOWs
|
||||
|
||||
- `static/js/gallery.js:122-125,270-275`: math/score overlay is
|
||||
click-only (no role/tabindex/keydown); `closeOverlay()` focus-returns
|
||||
to a non-focusable div — focus drops to `<body>`.
|
||||
- `static/js/popups.js:478,515`: the Wikipedia provider's
|
||||
`decodeURIComponent` runs synchronously before the `.catch` attaches —
|
||||
a malformed percent sequence in a link path throws an uncaught
|
||||
`URIError` per hover.
|
||||
- `static/js/popups.js:359,390`: fetched monogram SVG injected via
|
||||
`innerHTML` unescaped — the single unsanitized path in an otherwise
|
||||
fully escaped pipeline. Build-authored content, so not exploitable
|
||||
today; the comment acknowledges the trust assumption.
|
||||
- `static/js/citations.js`: dead file — no template loads it; popups.js
|
||||
supersedes it. If ever re-added it would double-bind and inject
|
||||
bibliography innerHTML without popups.js's cloned-node hardening.
|
||||
Delete.
|
||||
- `static/js/nav.js:26,30-31`: raw `localStorage` unguarded; if storage
|
||||
access throws, the throw lands before `toggle.addEventListener`,
|
||||
leaving the Portals toggle completely dead (utils.js exists precisely
|
||||
for this).
|
||||
- `static/js/annotations.js:209-215`: marks are mouse-only; the tooltip's
|
||||
Delete button is unreachable by keyboard (only recourse is the
|
||||
all-or-nothing "Clear Annotations").
|
||||
- `static/js/search.js:10`: unguarded `new PagefindUI(...)` — if the
|
||||
pagefind bundle 404s, the ReferenceError aborts the whole handler
|
||||
including the `?q=` pre-fill that the selection-popup "Here" flow
|
||||
depends on.
|
||||
- `static/js/semantic-search.js:55-56,96-107`: no
|
||||
`vectors.length === meta.length * DIM` consistency check — a stale
|
||||
CDN-cached mismatch yields NaN scores and silently garbage ranking.
|
||||
(Current files verified consistent: 1,256,448 bytes = 818 × 384 × 4.)
|
||||
- `static/js/transclude.js:149-151` + `collapse.js:111-114`: nested
|
||||
transcludes render a bare placeholder (no rescan of injected content);
|
||||
`reinitCollapse` is not idempotent (would stack toggle buttons if ever
|
||||
called twice on the same container).
|
||||
- `static/js/popups.js:985-988,1009-1014`: `daysBetween` uses `Math.abs`,
|
||||
so future dates render "N days ago" (now.js:17 handles this correctly).
|
||||
|
||||
### 5.8 Frontend NITs
|
||||
|
||||
- `static/js/copy.js:20-22,39`: code-less `<pre>` fallback copies the
|
||||
"copy" button label along with content.
|
||||
- `static/js/score-reader.js:50`: URL rewritten to `?p=1` on every load
|
||||
even without a `?p=` param.
|
||||
- `static/js/search-filters.js:271`: `parseInt(v,10) || 0` turns junk
|
||||
threshold input into an active ≥0 filter that matches everything.
|
||||
- `static/js/selection-popup.js:90-95`: shift-keyup while typing capitals
|
||||
in the annotation picker re-summons the selection toolbar over it.
|
||||
|
||||
Verified clean: the semantic-search ↔ embed.py contract post-model-split
|
||||
(DIM 384, 818-entry meta, no prefix for MiniLM — the nomic
|
||||
`search_document:` prefix is confined to the build-only page path); XSS
|
||||
escaping across semantic-search, popups providers, map tooltips,
|
||||
annotations (sole exception §5.7 monogram); theme.js ↔ settings.js
|
||||
storage schema identical; all JS selector contracts against templates
|
||||
(including the uncommitted head/nav edits); popups/sidenotes
|
||||
double-init guards; settings.js and gallery.js focus traps.
|
||||
|
||||
---
|
||||
|
||||
## 6. Templates & content
|
||||
|
||||
### 6.1 Draft in undocumented location is never built — **MED**
|
||||
|
||||
`content/drafts/inclusionist-manifesto.md`. WRITING.md:34 says drafts go
|
||||
under `content/drafts/essays/`; `draftEssayPattern`
|
||||
(`build/Patterns.hs:46-49`) matches only that, so this file is invisible
|
||||
even to `make watch`/`make dev` — silently orphaned.
|
||||
|
||||
### 6.2 SIMD/PQC essay `repository:` URL 404s — **MED**
|
||||
|
||||
`content/essays/where-does-simd-help-post-quantum-cryptography/index.md:24`.
|
||||
`https://git.levineuwirth.org/where-simd-helps` is missing the owner
|
||||
segment — verified HTTP 404, while the sibling essay's
|
||||
`.../neuwirth/beyond_comorbidity_indices` returns 200.
|
||||
|
||||
### 6.3 Tracked drafts contradict the gitignore policy — **MED**
|
||||
|
||||
`.gitignore:88` ignores `content/drafts/` as local-only "working notes,"
|
||||
but `git ls-files -i -c` shows four tracked drafts
|
||||
(`digital_progeny.md`, `modern_idolatry.md`, `test-essay.md`,
|
||||
`university_care.md`) — ignore rules don't untrack, so edits are
|
||||
auto-staged by `make build` and pushed publicly by deploy. The over-broad
|
||||
`**/.env.*` pattern also matches the tracked `.env.example`.
|
||||
|
||||
### 6.4 Template/content LOWs and NITs
|
||||
|
||||
- `content/colophon.md:5`: `modified:` is dead frontmatter — nothing
|
||||
reads it; `$date-modified$` (page-footer.html:108) is Hakyll's
|
||||
`dateField` over the `date` key.
|
||||
- Seven files end frontmatter with a valueless `confidence-history:`
|
||||
(YAML null; WRITING.md:97 documents a list of ints) — harmless, but
|
||||
`content/essays/scaling_outage.md` also retains the full WRITING.md
|
||||
scaffold comments in a published essay.
|
||||
- `static/images/canto31.jpg`: still 4.0 MB (prior-audit §6.1 unfixed).
|
||||
- `templates/blog-post.html:25,34`: `id="similar-links"` appears twice in
|
||||
mutually exclusive `$if$` branches — safe, fragile under edit.
|
||||
- `content/drafts/essays/digital_progeny.md`: title duplicates the
|
||||
published "The Specification Dilemma" — stale draft.
|
||||
- Frontmatter flags `home:`/`library:`/`links:`/`search:`/`portal:` are
|
||||
consumed (head.html CSS gates, default.html:6 `data-portal`) but
|
||||
undocumented in WRITING.md.
|
||||
|
||||
Verified clean: all `$partial(...)$` includes resolve; all ~140 distinct
|
||||
template variables have context providers; no missing `alt` attributes,
|
||||
tag-balance failures, or within-page duplicate IDs in composed pages; all
|
||||
26 CSS files referenced by head.html exist; sampled enum values across
|
||||
all sections are legal per WRITING.md and Contexts.hs validation lists.
|
||||
|
||||
---
|
||||
|
||||
## 7. Documentation / spec drift (WRITING.md, README.md)
|
||||
|
||||
### 7.1 `js:` page-script paths documented as content-relative; emitted root-relative — **MED**
|
||||
|
||||
`WRITING.md:773-775` vs `templates/default.html:37`
|
||||
(`<script src="/$script-src$" defer>`). The doc claims a composition's
|
||||
`js: scripts/widget.js` serves at `/music/symphony/scripts/widget.js`; the
|
||||
template emits raw root-relative frontmatter. The only current user
|
||||
(memento-mori) works by coincidence of its root-level route. A
|
||||
composition following the doc would 404.
|
||||
|
||||
### 7.2 "Standalone page `content/my-page/index.md`" has no generic rule — **MED**
|
||||
|
||||
`WRITING.md:20` presents directory-form standalone pages as a general
|
||||
capability; `build/Site.hs` hardcodes only `content/me/index.md` (293) and
|
||||
`content/memento-mori/index.md` (307); the generic rule (351) matches flat
|
||||
`content/*.md` only. A new `content/my-page/index.md` silently doesn't
|
||||
build.
|
||||
|
||||
### 7.3 Portal table lists 8 portals; the build has 9 — **MED**
|
||||
|
||||
`WRITING.md:221-231` omits Photography, which is in `homePortals`
|
||||
(`build/Site.hs:50-60`), the nav, and `content/tag-meta/photography.md`.
|
||||
|
||||
### 7.4 Three implemented frontmatter fields undocumented — **MED**
|
||||
|
||||
WRITING.md:3 claims to cover "all frontmatter fields"; zero hits for:
|
||||
`summary:` (`build/Contexts.hs:415-427`, rendered by essay.html:16 and
|
||||
reading.html:12, in live use), `revised:` (`build/Contexts.hs:815`
|
||||
`getRevisions` — drives `$date-display$`/`$date-original$`/
|
||||
`$revision-note$` and list sort order), `keywords:`
|
||||
(`build/Contexts.hs:283` → `/bibliography/<kw>/` links).
|
||||
|
||||
### 7.5 Documentation LOWs
|
||||
|
||||
- `WRITING.md:268-269,82`: default citation style called "Chicago
|
||||
Author-Date"; the injected CSL (`build/Citations.hs:114,167-168`) is
|
||||
`data/chicago-notes.csl`, titled "Chicago Notes Bibliography".
|
||||
- `README.md:12,19`: `make watch` described as "rebuilds on save without
|
||||
a server"; it runs Hakyll's preview server (WRITING.md:1139 has it
|
||||
right).
|
||||
- `WRITING.md:105-109`: `history:` example ordering contradicts the code
|
||||
(see §3.5).
|
||||
|
||||
---
|
||||
|
||||
## 8. nginx, Makefile & deployment
|
||||
|
||||
### 8.1 Multi-line CSP value embeds literal `\` + LF bytes — **MED**
|
||||
|
||||
`nginx/security-headers.conf:60-71`. The
|
||||
`Content-Security-Policy-Report-Only` value is a single quoted string
|
||||
spanning 12 lines with trailing `\` characters — nginx has no
|
||||
line-continuation inside quoted strings, so the emitted header contains
|
||||
raw backslash, LF, and leading-space bytes between directives. Raw LF in
|
||||
a header value is illegal in HTTP/2 (vhost example enables `http2 on`);
|
||||
strict clients reject the whole response. Sent on every response even as
|
||||
Report-Only. Must be collapsed to one line.
|
||||
|
||||
### 8.2 CSP gaps that will fire under enforcement — **MED**
|
||||
|
||||
`nginx/security-headers.conf:66-67`. (a) `font-src 'self' data:` blocks
|
||||
KaTeX webfonts: head.html:61 loads `katex.min.css` from cdn.jsdelivr.net,
|
||||
whose relative font URLs resolve to the CDN. (b) `connect-src 'self'`
|
||||
blocks the onnxruntime `.wasm` that transformers.js v2 (dynamically
|
||||
imported in `static/js/semantic-search.js:25`) fetches from jsdelivr —
|
||||
the config comment covers the same-origin model files but not the
|
||||
runtime. Both latent while Report-Only.
|
||||
|
||||
### 8.3 Makefile auto-commit sweeps any pre-staged changes — **MED**
|
||||
|
||||
`Makefile:28-29`. `git add content/` followed by
|
||||
`git diff --cached --quiet || git commit -m "auto: ..."` commits the
|
||||
*entire index* — anything previously staged gets folded into an
|
||||
`auto: <timestamp> [skip ci]` commit and pushed publicly on deploy. Use
|
||||
`git commit -- content/` or verify no foreign paths are staged.
|
||||
|
||||
### 8.4 Makefile LOWs
|
||||
|
||||
- pdf-thumbs: the `find | while read` pipeline swallows `pdftoppm`
|
||||
failures (loop exit status is the last iteration's) — a corrupt PDF
|
||||
silently ships without a thumbnail.
|
||||
- deploy: prerequisite order `clean build sign` is guaranteed only under
|
||||
serial make; no `.NOTPARALLEL:` guard for `-j` invocations. (Confirmed:
|
||||
deploy does run `clean` first; `.PHONY` is complete; `.env` export
|
||||
allowlist is sound.)
|
||||
- `tools/hooks/pre-commit-marks.sh` is documented (Makefile:175 comment)
|
||||
but not installed — `.git/hooks/` has only samples and `core.hooksPath`
|
||||
is unset.
|
||||
|
||||
Verified clean: all seven `data/` JSON/YAML files parse;
|
||||
`data/embed-cache-pages.npz` is untracked, so the new gitignore entry is
|
||||
fully effective; nginx archive.conf's add_header-inheritance re-include is
|
||||
correct; no redirect loops; popup-proxy rate-limit/cache zones correctly
|
||||
documented for http{} scope.
|
||||
|
||||
---
|
||||
|
||||
## 9. Working-tree diff review (branding refresh + embed split)
|
||||
|
||||
The model contract is **intact** — the diff splits one MiniLM pipeline
|
||||
into two: pages now use nomic-embed-text-v1.5 (768d, build-only, for
|
||||
similar-links.json); paragraphs stay on all-MiniLM-L6-v2@c9745ed (384d,
|
||||
the browser contract). download-model.sh, model-checksums.sha256,
|
||||
semantic-search.js (`DIM = 384`), and both WRITING.md lines (1108 nomic
|
||||
for Related-pages, 1128 MiniLM for client search) are all consistent.
|
||||
Icon declarations all match real files (verified with `file`: apple-touch
|
||||
180×180, favicon-96 96×96, manifest PNGs 192/512, og-image 1200×630
|
||||
matching declared og:image dimensions; the webp sidecar was regenerated).
|
||||
|
||||
Open items beyond §1.3/§1.4/§4.1:
|
||||
|
||||
### 9.1 32.8 KB traced SVG inlined into every page — **MED**
|
||||
|
||||
`templates/partials/logo-mark.svg` (32,818 bytes, potrace-style single
|
||||
giant `<path>`) is inlined via the nav partial into every HTML page —
|
||||
a ~33 KB per-page weight regression (pre-compression). The two-tone
|
||||
`--logo-ink`/`--logo-bg` cutout (components.css:72-98) genuinely needs
|
||||
inline SVG or `<use>`; an external sprite + `<use href>` restores
|
||||
cacheability. Better still: a hand-drawn or simplified path — a traced
|
||||
bitmap at nav size carries detail that can never resolve.
|
||||
|
||||
### 9.2 Icon asset bloat — **LOW**
|
||||
|
||||
`static/favicon.ico` is now 71,766 bytes; parsed directory shows
|
||||
16/32/48/64/128/256 px entries, the 128+256 pair alone 55.8 KB. The .ico
|
||||
is only the legacy fallback (modern browsers take the SVG); 16+32+48
|
||||
(~8 KB) is conventional. `static/favicon.svg` is a 32,844-byte traced
|
||||
path. `static/images/link-icons/internal.svg` went ~2 KB → 32,818 bytes
|
||||
yet renders at 0.7–1.6 rem via CSS mask in three stylesheets
|
||||
(components.css:853, typography.css:833, popups.css:161).
|
||||
|
||||
### 9.3 Webmanifest regressions — **NIT**
|
||||
|
||||
`static/site.webmanifest`: `purpose` changed maskable→`any` for both
|
||||
icons (Android adaptive launchers will letterbox; convention is separate
|
||||
`any` + `maskable` entries); still no `start_url`/`scope`/`description`
|
||||
(Lighthouse installability warnings). JSON valid; icons verified.
|
||||
|
||||
---
|
||||
|
||||
## 10. Prior audit (AUDIT.md 2026-05-07) follow-up
|
||||
|
||||
| Finding | Status |
|
||||
|---|---|
|
||||
| §1.1 freeze unsolvable | **Effectively still open** — aeson pin fixed, but the freeze broke again via `distributive` after a system update (§1.1 above); the underlying freeze-vs-system-db fragility is unaddressed |
|
||||
| §1.3 Python version mismatch | Fixed (`requires-python = ">=3.14"` matches `.python-version`) |
|
||||
| §1.4 model checksums | Fixed (`tools/model-checksums.sha256`, 5 entries) |
|
||||
| §9.1 nginx headers | Fixed (`nginx/security-headers.conf` + vhost example, README'd) — but see §8.1/§8.2 for new issues in that file |
|
||||
| §6.1 `canto31.jpg` 4 MB | **Unfixed** |
|
||||
| robots.txt / sitemap | Fixed (Site.hs:941/963, present in `_site/`) |
|
||||
| README `paper/`/`spec.md` ghosts | Fixed |
|
||||
| rsync target quoting | Fixed |
|
||||
| date-quoting doc | Fixed (WRITING.md:106) |
|
||||
| tag-meta no-title exception | Fixed (WRITING.md:238-251) |
|
||||
|
||||
---
|
||||
|
||||
## Suggested triage order
|
||||
|
||||
1. ~~`tools/refreeze.sh`~~ (§1.1 — in progress)
|
||||
2. Delete `data/embed-cache-pages.npz.tmp.npz`; widen the gitignore
|
||||
pattern; `git add` `logo-mark.svg` + `og-image.png` before committing
|
||||
the branding diff (§1.4, §4.1)
|
||||
3. Guard `ArchiveIndex.hs` file reads with `doesFileExist` (§1.2)
|
||||
4. Pin or sandbox the nomic remote code (§1.3)
|
||||
5. Fix the `/fiction/`–`/poetry/` 404s (§2.1) and the production-visible
|
||||
frontend MEDs (§5.1, §5.2)
|
||||
6. Collapse the nginx CSP to one line before ever flipping it to
|
||||
enforcing (§8.1, §8.2)
|
||||
7. The rest by severity as time allows
|
||||
|
|
@ -0,0 +1,612 @@
|
|||
---
|
||||
title: Repository audit
|
||||
date: 2026-05-07
|
||||
---
|
||||
|
||||
# Repository audit — levineuwirth.org
|
||||
|
||||
Comprehensive audit of the repo on `main` at commit `670d477` (working tree
|
||||
modified: `data/now.yaml`, `static/cv.pdf`, `static/resume.pdf`; untracked
|
||||
`Fermata_2.pdf`).
|
||||
|
||||
Severity legend: **HIGH** (likely to break a build, cause data loss, or
|
||||
expose a security weakness) — **MED** (latent bug, brittleness, or
|
||||
documentation drift) — **LOW** (minor robustness gap or fragile assumption) —
|
||||
**NIT** (style, polish, or paranoia).
|
||||
|
||||
Numbers are file:line. "Unverified" means I noticed the issue but did not
|
||||
reproduce its consequence; the line still appears load-bearing enough to
|
||||
flag.
|
||||
|
||||
---
|
||||
|
||||
## 1. Build & dependency chain
|
||||
|
||||
### 1.1 `cabal build` from scratch is unsolvable with the current freeze — **HIGH**
|
||||
|
||||
Running `cabal build` resolves the dependency tree freshly because no fresh
|
||||
`.ghc.environment` link exists for the current GHC. The freeze pins
|
||||
`aeson ==2.2.1.0`, but `warp` (pulled in by `hakyll +previewserver`) needs
|
||||
`hashable ==1.4.7.0/installed`, while `aeson 2.2.1.0` needs
|
||||
`hashable >=1.4.2.0 && <1.4.5.0`. Result:
|
||||
|
||||
```
|
||||
[__8] fail (backjumping, conflict set: aeson, levineuwirth, warp)
|
||||
After searching the rest of the dependency tree exhaustively, these were
|
||||
the goals I've had most trouble fulfilling: aeson, warp, hakyll, http2,
|
||||
async, network-control, unliftio, levineuwirth, hakyll:previewserver
|
||||
```
|
||||
|
||||
Day-to-day this is masked because `dist-newstyle/` has cached binaries
|
||||
from an earlier successful resolve. A fresh clone, a `cabal clean`, or a
|
||||
GHC upgrade will make `make build` fail. (`cabal.project.freeze:9` pins
|
||||
aeson; `levineuwirth.cabal:60` allows `>= 2.1 && < 2.3`.)
|
||||
|
||||
Fix: regenerate the freeze (`tools/refreeze.sh`) against the current
|
||||
hackage index. If `tools/refreeze.sh` is what produced the broken freeze,
|
||||
a manual `cabal freeze --constraint='aeson >= 2.2.2'` is needed.
|
||||
|
||||
### 1.2 `levineuwirth.cabal` upper bounds are tight — **MED**
|
||||
|
||||
- `hakyll >= 4.16 && < 4.17` (`levineuwirth.cabal:52`) — pins to a single
|
||||
minor line. 4.17.x is already on Hackage; the freeze is one rebase away
|
||||
from forcing a bound bump.
|
||||
- `pandoc >= 3.1 && < 3.7` (`levineuwirth.cabal:53`) — pandoc historically
|
||||
ships breaking changes on minor bumps, so the caution is fair, but 3.7
|
||||
exists.
|
||||
- `aeson >= 2.1 && < 2.3` (`levineuwirth.cabal:60`) — see 1.1; this bound
|
||||
combined with the freeze conflict is what makes the build unsolvable.
|
||||
|
||||
### 1.3 Python version mismatch — **HIGH**
|
||||
|
||||
`.python-version` says `3.14`. `pyproject.toml:5` says
|
||||
`requires-python = ">=3.12"`. `uv.lock:3` agrees with pyproject. Anyone
|
||||
who clones with pyenv/asdf will install Python 3.14. Anyone whose system
|
||||
ships 3.12/3.13 only will be told the project is fine, then hit
|
||||
`.python-version` later. Either bump `requires-python` to `>=3.14` or
|
||||
downgrade `.python-version` to a release that's actually a baseline.
|
||||
|
||||
### 1.4 No `tools/model-checksums.sha256` despite supply-chain hardening
|
||||
in `download-model.sh` — **HIGH**
|
||||
|
||||
`tools/download-model.sh:75-78` reads the checksum file when present and
|
||||
falls through with a printed note when it's missing. The file is absent
|
||||
from the tree. So today: model weights are pulled from Hugging Face
|
||||
unverified. If the upstream is compromised or MITM'd, the embedding +
|
||||
client-side semantic search ship trojaned weights. The fix path is
|
||||
already documented in the script comments — generate and commit the
|
||||
checksum file.
|
||||
|
||||
### 1.5 Cabal modules vs filesystem — verified consistent
|
||||
|
||||
Every `.hs` under `build/` is listed in `levineuwirth.cabal`'s
|
||||
`other-modules`. No orphan files. No phantom modules.
|
||||
|
||||
---
|
||||
|
||||
## 2. Makefile
|
||||
|
||||
### 2.1 `rsync` line does not quote variables — **MED (security-shaped)**
|
||||
|
||||
`Makefile:147`:
|
||||
|
||||
```make
|
||||
rsync -avz --delete _site/ $(VPS_USER)@$(VPS_HOST):$(VPS_PATH)/
|
||||
```
|
||||
|
||||
If `VPS_PATH` ever contains a space or a shell metacharacter, the
|
||||
expansion splits and rsync is handed extra arguments. The Makefile does
|
||||
guard `VPS_PATH` against `/`, `/srv`, etc., but does not guard against
|
||||
whitespace or against `;` / `&&`. Most variables in this Makefile are
|
||||
already quoted (`@test -s _site/index.html`), so this is the odd one out.
|
||||
Quote with `"$(VPS_USER)@$(VPS_HOST):$(VPS_PATH)/"`.
|
||||
|
||||
### 2.2 `> IGNORE.txt` line — **NIT**
|
||||
|
||||
`Makefile:55`. The recipe truncates `IGNORE.txt` at the repo root. It is
|
||||
gitignored. The purpose is undocumented in this Makefile (its intent
|
||||
seems to be "tell whatever sync tool watches the workspace to ignore the
|
||||
build output"). Either replace with `: > IGNORE.txt` (POSIX no-op) and a
|
||||
one-line comment explaining its consumer, or drop it.
|
||||
|
||||
### 2.3 `notify-send … || true` swallows errors — **NIT**
|
||||
|
||||
`Makefile:141`. Fine for a desktop notification, but the `|| true`
|
||||
silently masks `notify-send` failures. Acceptable.
|
||||
|
||||
### 2.4 Auto-snapshot recipe — **NIT (worth re-reading)**
|
||||
|
||||
`Makefile:12-26` runs `git add content/` and creates an automatic
|
||||
`auto: <ts> [skip ci]` commit before every build. The .gitignore
|
||||
excludes credential-shaped patterns under `content/`, so accidental
|
||||
secrets won't be staged. But:
|
||||
|
||||
- The commit happens **regardless of the build outcome**. A build that
|
||||
starts and crashes mid-way still leaves a snapshot commit. The comment
|
||||
says this is intentional. It does mean the recent commit history is
|
||||
full of `auto:` commits even for failed builds.
|
||||
- The recipe reads `.env` via `-include .env` and exports
|
||||
`VPS_USER VPS_HOST VPS_PATH GITHUB_REPO`. The comment claims this
|
||||
prevents future GITHUB_TOKEN from leaking. That's correct only if
|
||||
`GITHUB_TOKEN` is never added to the explicit export list. Worth a
|
||||
comment in `.env.example` reminding the future author.
|
||||
|
||||
### 2.5 Nested `$(MAKE)` and parallelism — **LOW**
|
||||
|
||||
`Makefile:29` (`@$(MAKE) -s pdf-thumbs`) and `:126` (`@$(MAKE) -C
|
||||
yaml-source all`) — fine in serial mode, but `make -j build` will
|
||||
parallelize sub-makes against the parent's job server only if they
|
||||
inherit `MAKEFLAGS`. The `-s` flag is fine, but if parallelism is ever
|
||||
desired, audit this.
|
||||
|
||||
---
|
||||
|
||||
## 3. Haskell build code (`build/`)
|
||||
|
||||
### 3.1 `unsafePerformIO` with module-global IORef — **MED**
|
||||
|
||||
`build/Filters/SourceRefs.hs:155`:
|
||||
|
||||
```haskell
|
||||
{-# NOINLINE existsCacheRef #-}
|
||||
existsCacheRef :: IORef (Map.Map Text Bool)
|
||||
existsCacheRef = unsafePerformIO (newIORef Map.empty)
|
||||
```
|
||||
|
||||
Standard "global mutable cache" pattern. `NOINLINE` is correct. `cabal
|
||||
run site -- watch` and `cabal run site -- build` are single-threaded
|
||||
today (Hakyll's compile loop is sequential), but the cabal file enables
|
||||
`-threaded`, and the cache is reachable from any compiler thread. Cache
|
||||
entries can also become stale between watches if a referenced source
|
||||
file is deleted: the cache holds `Just True`, but `doesFileExist` would
|
||||
now return `False`. Two practical consequences:
|
||||
|
||||
1. If a file is moved, `watch` may keep treating wikilinks/source-refs
|
||||
to the old path as live until the build server restarts.
|
||||
2. If `existsCacheRef` is ever read concurrently by two threads, the
|
||||
`atomicModifyIORef'` is safe but the underlying check race could let
|
||||
two threads call `doesFileExist` on the same path. Harmless.
|
||||
|
||||
Acceptable as-is; document the staleness caveat.
|
||||
|
||||
### 3.2 Lazy `readFile` in IO — **MED**
|
||||
|
||||
- `build/Stats.hs:857-860`: `readFile "data/last-build-seconds.txt"` is
|
||||
lazy, wrapped in a `catch` that returns `"\x2014"` on any IOException.
|
||||
The em-dash fallback hides "file missing", "permission denied", and
|
||||
"encoding error" alike. Worse, lazy IO means the handle may be open at
|
||||
the time the catch fires. Use `Data.Text.IO.readFile` or
|
||||
`withFile`+`hGetContents'`.
|
||||
- `build/BibExtras.hs:66`: `parseBibExtras path = … <$> readFile path`.
|
||||
Same concern. Failure surfaces only when the result is forced.
|
||||
|
||||
Fix: standardize on strict `Data.Text.IO.readFile` (already used in
|
||||
`build/Stability.hs:56,144` and `build/Now.hs`).
|
||||
|
||||
### 3.3 Defensive but technically partial pattern matches — **LOW**
|
||||
|
||||
These are "this case can't happen because of the guard" patterns. They
|
||||
all carry a comment, so they're not bugs, but `-Wall` may warn (and
|
||||
they reduce confidence under refactor). Cite-and-fix is straightforward.
|
||||
|
||||
- `build/Stats.hs:169-172` — `median` falls through to `0` on
|
||||
unreachable empty after a `length`-based guard.
|
||||
- `build/Stability.hs:109-113` — `stabilityFromDates` falls through.
|
||||
- `build/Catalog.hs:233-235` — `renderGroup []` when `groupBy` cannot
|
||||
produce empty groups.
|
||||
- `build/Tags.hs:181` — `init segs` after a length-> 1 guard.
|
||||
- `build/Stability.hs:297, 311, 324` — `last (newest:more)`.
|
||||
|
||||
Replace each with structural pattern matches (`(x:xs)`, `NonEmpty`) or
|
||||
use `Data.List.NonEmpty`. Or pragma-suppress the warning.
|
||||
|
||||
### 3.4 Magic offsets / hardcoded prefixes — **LOW**
|
||||
|
||||
- `build/Site.hs:388, 392`: `replaceExtension (drop 8 fp) "html"` —
|
||||
`drop 8` is "strip `content/`". `T.stripPrefix` reads better and
|
||||
fails closed.
|
||||
- `build/Filters/Wikilinks.hs:43, 77-78`: assumes destination URLs end
|
||||
with `.html`. Documented in code; brittle if routing changes.
|
||||
|
||||
### 3.5 `fail` for parse errors aborts the entire build — **LOW**
|
||||
|
||||
- `build/Commonplace.hs:144` and `build/Now.hs:258`: a malformed
|
||||
`commonplace.yaml` or `now.yaml` aborts the build. The data is
|
||||
hand-edited and small, so this is fine; a friendly error message
|
||||
would be nicer.
|
||||
- `build/Backlinks.hs:359`: `fail "backlinks: could not parse
|
||||
data/backlinks.json"` aborts every page that uses the backlinks
|
||||
context. The file is generated at build time, so corruption is
|
||||
unlikely, but consider degrading to "no backlinks" instead.
|
||||
|
||||
### 3.6 Silent-drop parsers — **LOW**
|
||||
|
||||
- `build/BibExtras.hs:95`: malformed `.bib` entries become `[]` with no
|
||||
warning. The author edits these by hand; a stderr note for dropped
|
||||
entries would catch typos.
|
||||
- `build/Contexts.hs:198-205`: malformed history entries are silently
|
||||
dropped. Same trade-off.
|
||||
- `build/Stats.hs:464`: `listDirectory dir `catch` …` returns `[]` on
|
||||
any IOException. Acceptable for stats.
|
||||
|
||||
### 3.7 `trim` does double-reverse — **NIT**
|
||||
|
||||
`build/Utils.hs:61`. `dropWhileEnd` (Data.List) avoids the second
|
||||
`reverse`. Cosmetic.
|
||||
|
||||
---
|
||||
|
||||
## 4. Tools (`tools/`)
|
||||
|
||||
### 4.1 `tools/extract-exif.py:292` uses Pillow's deprecated `_getexif()` — **MED**
|
||||
|
||||
```python
|
||||
exif = img._getexif() or {}
|
||||
```
|
||||
|
||||
Pillow has marked `_getexif` private since 9.0. The public API is
|
||||
`img.getexif()`. The bound in `pyproject.toml` allows up to Pillow 12,
|
||||
so a future `uv sync` could break this silently. One-line fix.
|
||||
|
||||
### 4.2 `embed.py` and `import-poetry.py` are not executable — **LOW**
|
||||
|
||||
Both have `#!/usr/bin/env python3` shebangs but bits are `0644`, while
|
||||
their siblings (`extract-*.py`) are `0755`. The Makefile invokes them
|
||||
via `uv run python tools/embed.py`, so this is cosmetic — unless a
|
||||
future contributor tries `./tools/embed.py`. `chmod +x` both.
|
||||
|
||||
### 4.3 `tools/import-photo.sh` does not check `magick` exit codes — **MED**
|
||||
|
||||
- Lines ~115-122: the resize/`-strip` `magick` call has no `|| exit`.
|
||||
- Line ~144: `magick mogrify -strip "$TARGET"` likewise. If mogrify
|
||||
fails, EXIF survives, but the script proceeds to write frontmatter
|
||||
asserting the photo was stripped.
|
||||
|
||||
The shell prelude already runs `set -euo pipefail`, but `magick … |
|
||||
…` can still partial-succeed with the pipefail correctly catching it.
|
||||
A direct `magick … "$TARGET" || exit 1` is clearer.
|
||||
|
||||
### 4.4 `tools/import-photo.sh` does not validate `$SLUG` — **LOW**
|
||||
|
||||
The slug is taken from CLI input and used as `content/photography/$SLUG`.
|
||||
A slug containing `../` traverses out of the photography tree. The
|
||||
Hakyll build would refuse to ingest it later, but the import has
|
||||
already written files. Add a `[[ "$SLUG" =~ ^[a-z0-9-]+$ ]] || exit 1`
|
||||
near the argument parse.
|
||||
|
||||
### 4.5 `subset-fonts.sh` hardcodes Arch font paths — **LOW**
|
||||
|
||||
`SPECTRAL=/usr/share/fonts/ttf-spectral`,
|
||||
`FIRA=/usr/share/fonts/TTF`, etc. macOS / Debian put fonts elsewhere.
|
||||
Doesn't break the site (the script is rarely run), but the README does
|
||||
not mention this constraint.
|
||||
|
||||
### 4.6 `download-pdfjs.sh` checksum scope is narrow — **LOW**
|
||||
|
||||
`tools/pdfjs-checksums.sha256` pins only the archive. After extraction,
|
||||
the unpacked tree is trusted blindly. Compare to
|
||||
`tools/leaflet-checksums.sha256`, which pins individual extracted files.
|
||||
The archive pin is sufficient against tampered downloads but offers
|
||||
nothing against a corrupted unzip on disk.
|
||||
|
||||
### 4.7 `add-popup-source.sh` masks curl failures — **LOW**
|
||||
|
||||
Lines ~67, ~98: `curl -sSI … 2>&1 || true` followed by piping into
|
||||
`grep`. A network failure produces an empty `$HEADERS`, and the
|
||||
downstream "CORS allowed?" detection silently reports OK. The script
|
||||
is interactive, so a user notices, but a stricter `if curl … ; then`
|
||||
guard would be better.
|
||||
|
||||
### 4.8 `embed.py` model staleness window — **LOW**
|
||||
|
||||
`tools/embed.py:39` hardcodes `MODEL_NAME = "all-MiniLM-L6-v2"` and
|
||||
`DIM = 384`. The Hugging Face cache is unpinned, so a model bump would
|
||||
silently change embedding semantics. The script regenerates everything,
|
||||
so the immediate breakage would be benign, but commits referencing the
|
||||
similar-links file would then drift. Pin to a model revision SHA.
|
||||
|
||||
### 4.9 `embed.py` `needs_update` race — **LOW**
|
||||
|
||||
`tools/embed.py:79-84` calls `.stat().st_mtime` while iterating
|
||||
`SITE_DIR.rglob("*.html")`. A file deleted mid-walk raises
|
||||
`FileNotFoundError`. In practice the build runs solo, so this never
|
||||
triggers; mention it.
|
||||
|
||||
### 4.10 `extract-*.py` swallow exceptions without traceback — **LOW**
|
||||
|
||||
`extract-dimensions.py:101`, `extract-exif.py:424`,
|
||||
`extract-palette.py:105`: each prints `f"…: {e}"` and continues. When a
|
||||
file is corrupt, the operator sees the exception type but no stack
|
||||
trace. Adding `traceback.format_exc()` to the stderr line costs
|
||||
nothing.
|
||||
|
||||
### 4.11 Other shell scripts
|
||||
|
||||
- All shell scripts in `tools/` already use `set -euo pipefail`.
|
||||
- `convert-images.sh`, `compress-assets.sh`, `download-leaflet.sh`,
|
||||
`sign-site.sh`, `preset-signing-passphrase.sh` are clean.
|
||||
- `compress-assets.sh:21` — no validation that `MIN_SIZE` is numeric.
|
||||
A misconfigured env var fails with a cryptic arithmetic error. NIT.
|
||||
|
||||
### 4.12 Stray `TODO`s in tooling — **NIT**
|
||||
|
||||
- `tools/add-popup-source.sh:12,128,131,134,137,156,194` — by design;
|
||||
the script is a scaffolder.
|
||||
- `tools/import-photo.sh:185` — emits `caption: TODO — short caption…`
|
||||
into the generated `index.md`. Authors who forget to edit will ship
|
||||
the literal `TODO`. A `make`-time check (`! grep -r "TODO " content/
|
||||
photography`) would catch it.
|
||||
|
||||
---
|
||||
|
||||
## 5. Content & frontmatter (`content/`)
|
||||
|
||||
### 5.1 Every `date:` in frontmatter is unquoted — **MED**
|
||||
|
||||
`WRITING.md:103` shows the canonical form as `date: "2026-03-01"`. Across
|
||||
all of `content/` (sample: 40+ files), every `date:` line is
|
||||
**unquoted**. Examples:
|
||||
|
||||
- `content/index.md`, `content/about.md`, `content/colophon.md`,
|
||||
`content/library.md`, `content/search.md`, `content/current.md`,
|
||||
`content/links.md`, `content/gpg.md`, `content/commonplace.md`
|
||||
- All essays under `content/essays/` and drafts under `content/drafts/`
|
||||
- All tag-meta files
|
||||
|
||||
YAML promotes ISO 8601 to a `Date`, not a `String`. Hakyll's `dateField`
|
||||
historically reads the string back, but as the Pandoc YAML decoder
|
||||
evolves, this can shift. Either the documentation is wrong (and dates
|
||||
are deliberately stored as YAML dates) or the corpus is. Reconcile by
|
||||
either:
|
||||
|
||||
1. Quoting all dates project-wide (sed across `content/`).
|
||||
2. Updating `WRITING.md:103` to show the unquoted form.
|
||||
|
||||
### 5.2 `content/tag-meta/*.md` lack `title:` — **MED (likely intentional, undocumented)**
|
||||
|
||||
Nine files under `content/tag-meta/` have only `tooltip:` in their
|
||||
frontmatter, no `title:`. `WRITING.md` documents `title:` as required
|
||||
on every authored page. Either:
|
||||
|
||||
- The Hakyll rules for tag-meta consume a different schema (likely —
|
||||
the title comes from the tag itself), in which case `WRITING.md`
|
||||
should mention this exception, **or**
|
||||
- Hakyll is silently inserting empty titles into rendered tag pages.
|
||||
|
||||
Files: `ai.md`, `fiction.md`, `miscellany.md`, `music.md`,
|
||||
`nonfiction.md`, `photography.md`, `poetry.md`, `research.md`,
|
||||
`tech.md`.
|
||||
|
||||
### 5.3 `Fermata_2.pdf` at the repo root — **MED**
|
||||
|
||||
48 KB PDF, untracked, not in `.gitignore`, not referenced by any
|
||||
template/CSS/script/Markdown. `git log` shows no history. Likely
|
||||
dropped by accident during writing. Either move it under
|
||||
`static/papers/` (with thumbnail) or delete it. While present at the
|
||||
root, the auto-snapshot `git add content/` will not pick it up — but
|
||||
any future `git add .` typo will.
|
||||
|
||||
### 5.4 `data/now.yaml` shows `last-updated: 2026-05-06`, today is 2026-05-07 — **NIT**
|
||||
|
||||
Working-tree modification, not yet committed. If the page is meant to
|
||||
read "yesterday", it's fine; if it's meant to read "today", refresh.
|
||||
|
||||
### 5.5 Wikilinks — verified
|
||||
|
||||
A spot-grep for `[[...]]` references against the page slugs found
|
||||
nothing pointing outside the corpus. The audit only verified the
|
||||
high-traffic pages (essays, drafts, photography); a complete
|
||||
walk-through would need a Hakyll-aware checker.
|
||||
|
||||
### 5.6 Image references — verified
|
||||
|
||||
All relative image references in essays I sampled
|
||||
(`memento-mori`, `specification-dilemma`, `beyond-comorbidity-indices`,
|
||||
`where-does-simd-help-post-quantum-cryptography`) resolve to existing
|
||||
files.
|
||||
|
||||
---
|
||||
|
||||
## 6. Static assets (`static/`)
|
||||
|
||||
### 6.1 `static/images/canto31.jpg` is 4.0 MB — **MED**
|
||||
|
||||
Single largest static asset. Loads on whichever page references it. A
|
||||
2400px JPEG should be ≤ 800 KB at quality 85. WebP companion will help
|
||||
modern browsers, but the legacy JPEG still ships. Either re-export at
|
||||
quality 80 / 2400px, or move to `content/` so the photography pipeline
|
||||
can manage it.
|
||||
|
||||
### 6.2 No `console.log` survivors — verified
|
||||
|
||||
A grep across `static/js/` finds none.
|
||||
|
||||
### 6.3 No orphaned vendored libraries — verified
|
||||
|
||||
`pdfjs/`, `leaflet/`, `models/` are all `.gitignore`'d and downloaded
|
||||
fresh by the Makefile.
|
||||
|
||||
### 6.4 No `http://` references in CSS / templates — verified
|
||||
|
||||
Only the SVG/XML namespace declarations in vendored `pdfjs/` use
|
||||
`http://`, which is the correct (non-fetched) form for XML.
|
||||
|
||||
---
|
||||
|
||||
## 7. Templates (`templates/`)
|
||||
|
||||
### 7.1 No `robots.txt` and no `sitemap.xml` are emitted — **MED (SEO)**
|
||||
|
||||
`_site/` after a build does not contain either file. `build/Site.hs`
|
||||
has no rule for them. `templates/` has no template for them. For a
|
||||
content-heavy personal site this is meaningful: search engines have no
|
||||
crawl guidance and no canonical URL list. Add a `create "robots.txt"`
|
||||
and a `create "sitemap.xml"` rule (Hakyll supports both via
|
||||
`makeItem`/`renderRss`-style compilers).
|
||||
|
||||
### 7.2 No `<meta name="robots">` — **NIT**
|
||||
|
||||
`templates/partials/head.html` has `og:image`, canonical, og:title /
|
||||
og:description. No `<meta name="robots" content="…">` and no fallback
|
||||
indexing hint. Together with §7.1, this is "search visibility is
|
||||
unconfigured".
|
||||
|
||||
### 7.3 Tag-balance — verified
|
||||
|
||||
A pairing check across `templates/*.html` for `$if$/$endif$` and
|
||||
`$for$/$endfor$` blocks (accounting for partial inheritance) reported
|
||||
no mismatches. The earlier flagged occurrences resolve when the
|
||||
relevant partial is included.
|
||||
|
||||
---
|
||||
|
||||
## 8. Data files (`data/`)
|
||||
|
||||
### 8.1 `data/annotations.json` is `{}` — **NIT**
|
||||
|
||||
Empty object. Either populate or document that it's intentionally a
|
||||
schema slot.
|
||||
|
||||
### 8.2 `data/now.yaml` — see §5.4.
|
||||
|
||||
### 8.3 Generated files (`semantic-index.bin`, `semantic-meta.json`,
|
||||
`similar-links.json`, `build-start.txt`, `last-build-seconds.txt`) —
|
||||
verified gitignored.
|
||||
|
||||
---
|
||||
|
||||
## 9. nginx (`nginx/`)
|
||||
|
||||
### 9.1 No security headers — **HIGH (security)**
|
||||
|
||||
`nginx/static-assets.conf` and `nginx/popup-proxy.conf` set neither of:
|
||||
|
||||
- `server_tokens off;`
|
||||
- `Strict-Transport-Security` (HSTS, with `preload` if HSTS-preload-listed)
|
||||
- `Content-Security-Policy` (or at minimum a CSP report-only)
|
||||
- `X-Content-Type-Options: nosniff`
|
||||
- `X-Frame-Options: SAMEORIGIN` (or `frame-ancestors` in CSP)
|
||||
- `Referrer-Policy: strict-origin-when-cross-origin`
|
||||
- `Permissions-Policy` (camera/microphone/geolocation deny)
|
||||
|
||||
These would normally live in the **vhost** rather than these include
|
||||
snippets, which is presumably where they belong on the VPS. But the
|
||||
repo has no vhost file checked in, which means the configuration in the
|
||||
repo is incomplete. Either commit a `nginx/vhost.conf` with the
|
||||
security headers or document explicitly that the vhost is owned outside
|
||||
the repo.
|
||||
|
||||
### 9.2 `nginx/static-assets.conf:75-78` — CSS/JS `must-revalidate` with
|
||||
`max-age=86400` — **MED**
|
||||
|
||||
CSS/JS filenames are not fingerprinted (no `app.abc123.css`). A 1-day
|
||||
`must-revalidate` means a stylesheet bug ships for up to 24 hours per
|
||||
client. Either drop `max-age` to 3600 or add a build-time content hash
|
||||
to filenames (and switch to `immutable`).
|
||||
|
||||
### 9.3 `popup-proxy.conf:28` — public DNS resolver — **LOW**
|
||||
|
||||
`resolver 1.1.1.1 8.8.8.8 ipv6=off valid=300s;`. Fine on a VPS without
|
||||
local DNS, but if the host runs systemd-resolved, prefer
|
||||
`127.0.0.1:53`. Also leaks "this server proxies to {arxiv,
|
||||
internet-archive, ncbi}" to whichever resolver answers — the public
|
||||
resolvers see the upstream queries.
|
||||
|
||||
### 9.4 popup-proxy caching — verified
|
||||
|
||||
30-day cache on arXiv/PubMed metadata, 7-day on Internet Archive,
|
||||
`proxy_cache_lock on`, `proxy_cache_use_stale`. PubMed has
|
||||
`limit_req zone=pubmed burst=3 nodelay;`, which matches NCBI etiquette.
|
||||
|
||||
---
|
||||
|
||||
## 10. README and ancillary docs
|
||||
|
||||
### 10.1 `README.md` references files that do not exist — **MED**
|
||||
|
||||
- `README.md:70-71`: "`paper/` — LaTeX source for in-progress academic
|
||||
papers." There is no `paper/` directory.
|
||||
- `README.md:71, 82`: "`spec.md` — full architectural notes". There is
|
||||
no `spec.md`.
|
||||
|
||||
`yaml-source/` is mentioned **and** explained as local-only on
|
||||
`README.md:118`. `paper/` and `spec.md` are not. Either create the
|
||||
files (even as stubs) or remove the references.
|
||||
|
||||
### 10.2 README "Repository layout" section is otherwise current —
|
||||
verified
|
||||
|
||||
`build/`, `content/`, `templates/`, `static/`, `tools/`, `data/`, all
|
||||
present and described accurately.
|
||||
|
||||
### 10.3 `checklist.md`, `HOMEPAGE.md`, `PHOTOGRAPHY.md`, `WRITING.md` —
|
||||
not shipped to `_site/` — verified
|
||||
|
||||
`checklist.md` is gitignored. `HOMEPAGE.md`, `PHOTOGRAPHY.md`,
|
||||
`WRITING.md` are tracked but not copied into `_site/` (no Hakyll rule
|
||||
matches them). Acceptable.
|
||||
|
||||
---
|
||||
|
||||
## 11. `.env`, `.env.example`, `.gitignore`
|
||||
|
||||
### 11.1 `.env` is mode `0600` and gitignored — verified.
|
||||
|
||||
### 11.2 `.env.example` documents every variable the Makefile reads —
|
||||
verified.
|
||||
|
||||
### 11.3 `.gitignore` defense-in-depth credential exclusion — verified
|
||||
(`.gitignore:10-27`).
|
||||
|
||||
### 11.4 Redundant entries — **NIT**
|
||||
|
||||
`.gitignore:81-86` lists `README.profile.md`, `README.arcana.md`,
|
||||
`README.simd.md`, `README.icd.md`, `README.neuropose.md`. None exist.
|
||||
These are presumably scratch-pad names; harmless but cluttering.
|
||||
|
||||
---
|
||||
|
||||
## 12. Repo hygiene
|
||||
|
||||
### 12.1 Working-tree dirty on `main` — **NIT**
|
||||
|
||||
`data/now.yaml`, `static/cv.pdf`, `static/resume.pdf` are modified;
|
||||
`Fermata_2.pdf` is untracked. The CV/resume PDFs are produced by
|
||||
`make pdfs`, so the diff is presumably expected. Commit or revert
|
||||
before the next deploy.
|
||||
|
||||
### 12.2 Cache size — **NIT**
|
||||
|
||||
`_cache/` 8.5 MB, `dist-newstyle/` 22 MB. Reasonable.
|
||||
|
||||
### 12.3 Auto-commit pollution — **NIT**
|
||||
|
||||
`git log --oneline -20` shows ~12 of the last 20 commits are `auto:
|
||||
<timestamp> [skip ci]`. This is by design (see §2.4); just note that
|
||||
`git log` for narrative review needs `--invert-grep --grep='^auto:'`.
|
||||
|
||||
---
|
||||
|
||||
## 13. Recommended fix order
|
||||
|
||||
In rough order of cost-to-impact:
|
||||
|
||||
1. **§1.1** — Regenerate `cabal.project.freeze` so a fresh clone can
|
||||
build. (Single command if `tools/refreeze.sh` works; otherwise a
|
||||
manual `cabal freeze` after bumping aeson.)
|
||||
2. **§9.1** — Commit a vhost (or document explicitly that the vhost
|
||||
lives on the VPS) and add the standard security header set.
|
||||
3. **§1.4** — Generate and commit `tools/model-checksums.sha256`.
|
||||
4. **§1.3** — Reconcile `.python-version` (3.14) and `requires-python`
|
||||
(>= 3.12).
|
||||
5. **§5.1** — Decide canonical date form, then sweep `content/` or
|
||||
`WRITING.md:103`.
|
||||
6. **§10.1** — Drop `paper/` + `spec.md` references from `README.md`
|
||||
(or write them).
|
||||
7. **§7.1** — Emit `robots.txt` + `sitemap.xml` from Hakyll.
|
||||
8. **§5.3** — Move or delete `Fermata_2.pdf`.
|
||||
9. **§4.1, §4.3, §4.4, §3.2** — Small Python and Haskell hardening.
|
||||
10. **§3.3** — Replace defensive partial matches with structural ones.
|
||||
|
||||
Everything else in this document is style/polish or low-risk
|
||||
brittleness.
|
||||
|
|
@ -0,0 +1,21 @@
|
|||
MIT License
|
||||
|
||||
Copyright (c) 2026 Levi Neuwirth
|
||||
|
||||
Permission is hereby granted, free of charge, to any person obtaining a copy
|
||||
of this software and associated documentation files (the "Software"), to deal
|
||||
in the Software without restriction, including without limitation the rights
|
||||
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
||||
copies of the Software, and to permit persons to whom the Software is
|
||||
furnished to do so, subject to the following conditions:
|
||||
|
||||
The above copyright notice and this permission notice shall be included in all
|
||||
copies or substantial portions of the Software.
|
||||
|
||||
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
||||
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
||||
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
||||
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
||||
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
||||
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
||||
SOFTWARE.
|
||||
|
|
@ -0,0 +1,959 @@
|
|||
# Frontmatter Marks: Specification
|
||||
|
||||
A two-part visual identity for essay and research frontmatter, designed
|
||||
to extend (not replace) the existing epistemic profile system.
|
||||
|
||||
The mark system has two pieces:
|
||||
|
||||
1. **Monogram** — a hand-authored SVG glyph per piece, abstracted from
|
||||
the work's central concept. The author's statement of *what* the
|
||||
piece is about.
|
||||
2. **Epistemic figure** — a build-time SVG generated deterministically
|
||||
from existing frontmatter fields. The site's statement of *where
|
||||
the piece stands*.
|
||||
|
||||
The two are paired in the frontmatter: monogram on the left, title and
|
||||
abstract in the middle, epistemic figure on the right. Either can be
|
||||
present alone; both can be absent. When a field that drives the
|
||||
epistemic figure is missing, the figure is omitted entirely rather
|
||||
than rendered with empty axes.
|
||||
|
||||
This document specifies the authoring interface, the field schema
|
||||
(extending the existing one in `WRITING.md`), the visual contract,
|
||||
the build-time rendering pipeline, and the migration plan.
|
||||
|
||||
It is written to slot in alongside `WRITING.md` as a sibling reference,
|
||||
and to live as a section in the colophon once shipped.
|
||||
|
||||
---
|
||||
|
||||
## 1. Scope and non-goals
|
||||
|
||||
### In scope
|
||||
|
||||
- A monogram convention (location, dimensions, line-weight, palette)
|
||||
that any author or generator can produce SVGs for.
|
||||
- Two new optional frontmatter fields (`peer-status`, `result-shape`)
|
||||
that surface useful information for both essays and formal research,
|
||||
with a colophon-glossed interpretation that adapts to genre.
|
||||
- One narrow exception (`confidence: proved`) that lets formal proofs
|
||||
honestly opt out of a numeric credence without forcing false precision.
|
||||
- A new Pandoc filter (`Filters/Mark.hs`) that emits the epistemic
|
||||
figure SVG inline in the essay header at build time.
|
||||
- Template changes in `templates/essay.html` and `templates/blog-post.html`
|
||||
to provide three-column frontmatter slots (monogram | title | figure).
|
||||
- A `make audit-marks` build target and an addition to `/build/`
|
||||
surfacing which essays are missing one or both marks.
|
||||
|
||||
### Out of scope
|
||||
|
||||
- A separate "research badge" figure type. The single radial figure
|
||||
handles both essays and formal research; see §3.4.
|
||||
- A unified mode-switched figure with axes that change meaning based on
|
||||
a `claim-mode` flag. Visual grammar should be unambiguous; one figure
|
||||
type, stable axis semantics. See §11 for the rejected alternatives.
|
||||
- Per-portal iconographic systems (the Approach 5 idea from earlier
|
||||
exploration). Not ruled out for the future, but not specified here.
|
||||
- Author UI for generating monograms. Authors may use any tool —
|
||||
hand-drawn, prompt-driven, traced from references — provided the
|
||||
output meets §2.
|
||||
|
||||
---
|
||||
|
||||
## 2. The monogram
|
||||
|
||||
### 2.1 Authoring contract
|
||||
|
||||
A monogram is a single SVG file at:
|
||||
|
||||
content/essays/{slug}/mark.svg ← directory-form essays
|
||||
content/essays/{slug}.mark.svg ← flat-file essays
|
||||
content/blog/{slug}.mark.svg
|
||||
content/poetry/{slug}.mark.svg
|
||||
content/fiction/{slug}.mark.svg
|
||||
content/music/{slug}/mark.svg
|
||||
content/{slug}.mark.svg ← standalone pages
|
||||
content/drafts/essays/{slug}.mark.svg ← drafts
|
||||
|
||||
The build picks up the file by the same slug-resolution rules already
|
||||
used for score fragments and page-local JS. No frontmatter key is
|
||||
required to opt in; the file's presence is the opt-in. To opt out
|
||||
(suppress an inherited or stale mark), delete the file.
|
||||
|
||||
### 2.2 Visual contract
|
||||
|
||||
Monograms must satisfy the following constraints. These are enforced
|
||||
by `tools/audit-marks.py` and by `make audit-marks` (§9), not at
|
||||
build-fail level — violations warn but do not break the build.
|
||||
|
||||
| # | Constraint | Rationale |
|
||||
|---|---|---|
|
||||
| M1 | `viewBox="0 0 280 280"` (or proportional, square) | Renders at 130–280 px equally. |
|
||||
| M2 | All strokes use `stroke="currentColor"`; all fills use `fill="none"` except small filled point-marks which use `fill="currentColor"` | Inverts cleanly under Light, Dark, Cappuccino without per-theme assets. The score-fragment filter already does this substitution; monograms must be authored this way from the start. |
|
||||
| M3 | Outer roundel: `<circle cx="140" cy="140" r="128" stroke-width="0.6"/>` | The unifying frame. Without it, marks read as illustrations, not as a system. |
|
||||
| M4 | Stroke widths within {0.3, 0.5, 0.6, 0.8, 1.0, 1.2, 1.4} | Limits visual rhythm to a small palette. |
|
||||
| M5 | `stroke-linecap="round"` and `stroke-linejoin="round"` everywhere | Spectral-compatible terminals. |
|
||||
| M6 | No `<text>`, no `<image>`, no gradients, no filters, no embedded fonts, no rasters | Letterforms and color belong to the page, not the mark. Note: `<title>` and `<desc>` for accessibility are required (§2.3), not forbidden. |
|
||||
| M7 | No XML prologue, no `<?xml` declaration, no DOCTYPE | The file is inlined; a prologue would land mid-body. |
|
||||
| M8 | File size ≤ 8 KiB | A working corpus of 200 marks at this ceiling is 1.6 MiB total; larger is overkill for a 280-px frontispiece. |
|
||||
| M9 | Validates as well-formed XML (round-trips through `xmllint --noout`) | Inlining a malformed SVG breaks the surrounding page. |
|
||||
|
||||
A reference monogram template lives at `static/templates/mark-template.svg`
|
||||
and is the recommended starting point for hand-authoring.
|
||||
|
||||
### 2.3 Accessibility
|
||||
|
||||
Each monogram must include a `<title>` element as the first child of
|
||||
`<svg>` and may include a `<desc>`:
|
||||
|
||||
<svg viewBox="0 0 280 280" xmlns="http://www.w3.org/2000/svg" role="img" aria-labelledby="mark-title">
|
||||
<title id="mark-title">Half-buried column on a low desert horizon</title>
|
||||
<desc>A frontispiece mark for the essay "Ozymandias: A Static Site Framework".</desc>
|
||||
...
|
||||
</svg>
|
||||
|
||||
The author writes the visible-content description in `<title>`. The
|
||||
`role="img"` and `aria-labelledby` attributes are required by M9.
|
||||
Screen readers announce the title; the desc is supplementary.
|
||||
|
||||
### 2.4 Inlining and theming
|
||||
|
||||
Monograms are inlined into the page HTML at build time by the same
|
||||
mechanism `Filters/Score.hs` uses for score fragments. The build step:
|
||||
|
||||
1. Reads the SVG file.
|
||||
2. Strips any `width=` and `height=` attributes from the `<svg>` root
|
||||
(presentation is controlled by CSS).
|
||||
3. Replaces any `fill="#000000"`, `fill="black"`, `stroke="#000000"`,
|
||||
`stroke="black"` with `currentColor` (defensive: lets authors
|
||||
produce SVGs from generators that hardcode black without breaking
|
||||
the contract).
|
||||
4. Wraps the SVG in `<figure class="frontmatter-mark frontmatter-mark--monogram">…</figure>`.
|
||||
|
||||
The CSS rule `.frontmatter-mark svg { color: var(--text); }` propagates
|
||||
the page color to the SVG strokes. No theme-switching JS is required.
|
||||
|
||||
### 2.5 Print and reduce-motion
|
||||
|
||||
Monograms render in print (they are inert SVG; there is nothing to
|
||||
suppress). They are unaffected by `prefers-reduced-motion` because
|
||||
they do not animate. The Display panel's "Focus Mode" hides them via
|
||||
`.focus-mode .frontmatter-mark { display: none; }`.
|
||||
|
||||
---
|
||||
|
||||
## 3. The epistemic figure
|
||||
|
||||
### 3.1 Visibility rule
|
||||
|
||||
The epistemic figure is rendered for a piece if, and only if, the
|
||||
existing visibility rule for the epistemic block is met:
|
||||
|
||||
> The epistemic figure renders when `status:` is set in frontmatter.
|
||||
|
||||
This matches the existing rule in `WRITING.md` ("The epistemic footer
|
||||
section appears when `status` is set"). No new gating field is
|
||||
introduced. A piece without `status:` gets no figure — the same as it
|
||||
gets no epistemic block today.
|
||||
|
||||
This is deliberate. A figure showing five missing axes and one filled
|
||||
axis would look like a build bug, not a deliberate position. Either
|
||||
the piece has taken a position (and therefore renders the figure), or
|
||||
it has not (and therefore renders only the monogram). The audit job
|
||||
in §9 lists pieces in `research/` or with `peer-status:` that lack
|
||||
`status:`, so the absence is visible without being silently broken.
|
||||
|
||||
### 3.2 Inputs
|
||||
|
||||
The figure consumes only fields already in the schema (per `WRITING.md`),
|
||||
plus the two new optional fields specified in §4:
|
||||
|
||||
| Field | Existing? | Maps to |
|
||||
|---|---|---|
|
||||
| `confidence` | yes (0–100) | confidence axis length |
|
||||
| `importance` | yes (1–5) | importance axis length |
|
||||
| `evidence` | yes (1–5) | evidence axis length |
|
||||
| `scope` | yes (5-step ordinal) | scope axis length |
|
||||
| `novelty` | yes (4-step ordinal) | novelty axis length |
|
||||
| `practicality` | yes (5-step ordinal) | practicality axis length |
|
||||
| *(stability)* | auto from git | outer-ring tick count |
|
||||
| *(trust score)* | auto from formula | center number |
|
||||
| `peer-status` | **new** (§4.1) | outer-ring tick *style* |
|
||||
| `result-shape` | **new** (§4.2) | center glyph beside trust |
|
||||
| `confidence: proved` | **new exception** (§4.3) | confidence axis renders as proof-cap |
|
||||
|
||||
Ordinal-to-numeric mapping is exactly what `Contexts.hs` already does
|
||||
for the dot-rendering of these fields:
|
||||
|
||||
| `scope` value | Numeric |
|
||||
|---|---|
|
||||
| `personal` | 1 |
|
||||
| `local` | 2 |
|
||||
| `average` | 3 |
|
||||
| `broad` | 4 |
|
||||
| `civilizational` | 5 |
|
||||
|
||||
| `novelty` value | Numeric |
|
||||
|---|---|
|
||||
| `conventional` | 1 |
|
||||
| `moderate` | 2 |
|
||||
| `idiosyncratic` | 3 |
|
||||
| `innovative` | 4 |
|
||||
|
||||
(Note: `novelty` is a 4-step scale in the existing schema, not 5. The
|
||||
figure normalizes to a 0–1 axis length the same way the dots do —
|
||||
`(value - 1) / (max - 1)` — so a 4-step scale renders one step shorter
|
||||
at maximum than a 5-step scale. This is honest and matches existing
|
||||
behavior; do not silently widen the scale to 5.)
|
||||
|
||||
| `practicality` value | Numeric |
|
||||
|---|---|
|
||||
| `abstract` | 1 |
|
||||
| `low` | 2 |
|
||||
| `moderate` | 3 |
|
||||
| `high` | 4 |
|
||||
| `exceptional` | 5 |
|
||||
|
||||
The `confidence` axis is 0–100; it normalizes as `confidence / 100`.
|
||||
|
||||
### 3.3 Geometry
|
||||
|
||||
The figure is a 200×200 SVG rendered at frontmatter scale (170 px on
|
||||
desktop, 130 px on mobile). It consists of:
|
||||
|
||||
- An outer roundel (two thin concentric circles, `r=88` and `r=90`,
|
||||
both `stroke-width="0.5"`).
|
||||
- Six radial axes at 60° spacing, in this fixed clockwise order
|
||||
starting from 12 o'clock:
|
||||
1. confidence (top, 0°)
|
||||
2. novelty (60°)
|
||||
3. practicality (120°)
|
||||
4. scope (180°, bottom)
|
||||
5. evidence (240°)
|
||||
6. importance (300°)
|
||||
Axis stroke `0.3`, opacity `0.55`. Visible at all times whether or
|
||||
not the corresponding field is set; a missing field renders the
|
||||
axis without a polygon vertex (see below).
|
||||
- Four inner concentric guide circles at `0.2, 0.4, 0.6, 0.8` of the
|
||||
axis radius. Stroke `0.25`, opacity `0.4`.
|
||||
- A polygon connecting the field values along their axes.
|
||||
Stroke `1.1`, fill `currentColor` at `fill-opacity="0.08"`. The
|
||||
polygon is closed only if all six fields are present; otherwise it
|
||||
is rendered as an open polyline through the present vertices in
|
||||
axis order, and missing axes contribute no vertex (the line jumps
|
||||
the missing axis). This is the only mode where partial fields are
|
||||
rendered; in practice §3.1 ensures all six are present whenever the
|
||||
figure renders at all.
|
||||
- Vertex point marks at each present field's position
|
||||
(`r=2`, `fill="currentColor"`).
|
||||
- The center number: trust score, in Spectral 500 weight, font-size 16,
|
||||
centered on the geometric center.
|
||||
- Below the trust number, in 5 pt Fira Sans with letter-spacing 0.18em,
|
||||
the literal text `TRUST`.
|
||||
- Stability ticks on the outer ring at 12 o'clock (see §3.5).
|
||||
- A result-shape glyph immediately to the right of the trust number
|
||||
(rendered only when `result-shape:` is set; see §4.2).
|
||||
|
||||
The figure deliberately omits the confidence-trend arrow; the trend is
|
||||
rendered inline in the compact epistemic strip instead (see §3.4). The
|
||||
figure carries only the geometry, the trust score, and the result-shape
|
||||
glyph.
|
||||
|
||||
A reference renderer in pure SVG, with annotated coordinates, is
|
||||
checked in at `static/templates/epistemic-figure-reference.svg` for
|
||||
visual regression testing.
|
||||
|
||||
### 3.4 Confidence trend
|
||||
|
||||
The trend arrow is rendered inline in the compact epistemic strip,
|
||||
immediately after the confidence percentage (e.g. `conf 80%↑`). It
|
||||
indicates the direction of the *last* step in `confidence-history`:
|
||||
|
||||
| Last step | Glyph |
|
||||
|---|---|
|
||||
| Increase (∆ > 5) | ↑ |
|
||||
| Decrease (∆ < −5) | ↓ |
|
||||
| Equal (within ±5) | → |
|
||||
|
||||
When `confidence-history` is absent or has fewer than two entries, the
|
||||
arrow is not drawn. The arrow uses the same parsing and ±5 threshold
|
||||
the existing epistemic block uses; no new heuristic is introduced.
|
||||
|
||||
The arrow lives in the strip rather than on the figure for two reasons:
|
||||
the figure stays visually clean, and the arrow sits next to the value
|
||||
it modifies. This is a deliberate departure from earlier drafts that
|
||||
placed the arrow at the confidence vertex.
|
||||
|
||||
### 3.5 Stability ticks
|
||||
|
||||
The existing `Stability.hs` heuristic produces one of five labels:
|
||||
`volatile`, `revising`, `fairly stable`, `stable`, `established`.
|
||||
These map to outer-ring ticks at 12 o'clock:
|
||||
|
||||
| Stability | Tick count | Tick positions (visible / dim) |
|
||||
|---|---|---|
|
||||
| volatile | 1 | center only |
|
||||
| revising | 2 | center + left |
|
||||
| fairly stable | 3 | center + left + right |
|
||||
| stable | 4 | center + left + right + far-left |
|
||||
| established | 5 | all five |
|
||||
|
||||
Ticks are short radial line segments just outside the outer roundel,
|
||||
1–1.5 px in length, `stroke-width="1"`. Inactive ticks are drawn at
|
||||
`opacity="0.4"` so the full set is always visible; this gives the
|
||||
reader a constant five-step scale to anchor against.
|
||||
|
||||
The manual override mechanism (`IGNORE.txt`) documented in WRITING.md
|
||||
applies unchanged: a path listed there pins stability for one build
|
||||
and is cleared by `make build`. The figure honors the override.
|
||||
|
||||
### 3.6 Visibility under reduce-motion and print
|
||||
|
||||
The figure does not animate, so reduce-motion has no effect.
|
||||
|
||||
In print, the figure renders at fixed size and inverts to black-on-white
|
||||
via the existing `@media print` rules in `static/css/print.css`. No
|
||||
new print rules are required.
|
||||
|
||||
### 3.7 Theming
|
||||
|
||||
The figure uses `currentColor` exclusively. The `<text>` elements
|
||||
explicitly set `fill="currentColor" stroke="none"` to prevent text
|
||||
nodes from inheriting the strokes used for geometry.
|
||||
|
||||
### 3.8 Tooltip and link
|
||||
|
||||
The figure is wrapped in `<a href="#epistemic">…</a>`. Hovering it
|
||||
triggers the existing epistemic-jump-link popup (per WRITING.md:
|
||||
"Epistemic jump link (`#epistemic`) — Clone of the full epistemic
|
||||
profile"). Clicking jumps to the epistemic block at the page footer.
|
||||
|
||||
This means the figure does double duty: it is a glance-readable
|
||||
summary at the top, and a clickable handle for the full block at the
|
||||
bottom. The popup logic is already implemented; this is a free pickup.
|
||||
|
||||
---
|
||||
|
||||
## 4. New optional frontmatter fields
|
||||
|
||||
Two new fields and one new value of an existing field. All optional;
|
||||
all backward-compatible; existing essays continue to render
|
||||
identically until the author opts in.
|
||||
|
||||
### 4.1 `peer-status`
|
||||
|
||||
Captures the *external* review state of a piece, distinct from
|
||||
`status` (which captures the author's internal position).
|
||||
|
||||
| Value | Meaning |
|
||||
|---|---|
|
||||
| `unreviewed` | No external review has taken place. Default if omitted. |
|
||||
| `under-review` | Currently in submission or peer review. |
|
||||
| `peer-reviewed` | Has been peer-reviewed (e.g. preprint with referee reports addressed) but not yet formally published. |
|
||||
| `published` | Appeared in a peer-reviewed venue. Treat as canonical. |
|
||||
| `retracted` | Formally retracted. Renders with a strikethrough on the field name in the epistemic block. |
|
||||
|
||||
This is genre-agnostic. An essay can be `under-review` at a magazine;
|
||||
a paper can be `peer-reviewed` at a journal. The vocabulary doesn't
|
||||
change.
|
||||
|
||||
#### Visual encoding
|
||||
|
||||
`peer-status` modulates the *style* of the stability ticks (§3.5),
|
||||
not their count. Stability and peer-status are factored: stability
|
||||
remains git-derived and counts ticks; peer-status changes how those
|
||||
ticks are drawn:
|
||||
|
||||
| `peer-status` | Tick style |
|
||||
|---|---|
|
||||
| `unreviewed` (default) | Plain solid ticks. |
|
||||
| `under-review` | Solid ticks with a small unfilled circle (`r=1`) just outside the outermost tick — "in flight" mark. |
|
||||
| `peer-reviewed` | Solid ticks with a single horizontal bar above the outer roundel arc. |
|
||||
| `published` | Solid ticks bracketed by two short vertical marks at ±15° on the outer roundel — a printer's bracket. |
|
||||
| `retracted` | Solid ticks struck through with a horizontal line `stroke-width="1.5"` across the tick group. |
|
||||
|
||||
The reading order on the figure thus becomes: outer ring (stability +
|
||||
peer-status) communicates *external* standing; inner shape
|
||||
(polygon + trust + result-shape) communicates *internal* claim.
|
||||
Cleanly factored.
|
||||
|
||||
#### Compact-row rendering
|
||||
|
||||
In addition to modulating the figure, `peer-status` adds a compact
|
||||
chip to the existing epistemic-block primary row, alongside `status`
|
||||
and the trust chip:
|
||||
|
||||
88% trust · Durable · under review · 80% confidence · ●●●○○ importance · …
|
||||
|
||||
Rendered for any non-`unreviewed` value. The label uses the
|
||||
hyphen-stripped form (`under review`, `peer-reviewed`, `published`,
|
||||
`retracted`).
|
||||
|
||||
### 4.2 `result-shape`
|
||||
|
||||
Captures the *shape* of the piece's central claim. This is missing
|
||||
from the current vocabulary and surfaces information that's currently
|
||||
buried in the abstract.
|
||||
|
||||
| Value | Meaning | Center glyph |
|
||||
|---|---|---|
|
||||
| `positive` | Argues for or proves something works. | `+` |
|
||||
| `negative` | Argues against or proves a barrier. | `−` |
|
||||
| `mixed` | Both positive and negative results coexist (e.g. *Branch-Based Local Capture*'s "double pincer"). | `±` |
|
||||
| `comparative` | Compares two or more approaches. | `∼` |
|
||||
| `descriptive` | Describes a system, observation, or position without arguing for or against. | `□` |
|
||||
|
||||
The glyph appears immediately to the right of the trust score, in
|
||||
Spectral, font-size 16, vertically centered on the trust number. When
|
||||
omitted, no glyph is drawn (the trust number sits alone).
|
||||
|
||||
#### Compact-row rendering
|
||||
|
||||
`result-shape` adds nothing to the compact row. The character is small
|
||||
enough that the figure carries it without competing with the chip
|
||||
sequence. If omitted, the figure simply renders the trust number alone.
|
||||
|
||||
### 4.3 `confidence: proved` (and `proven`)
|
||||
|
||||
Formal mathematical results don't have credences in the same sense
|
||||
that essays do. A theorem with a complete proof has confidence
|
||||
~100 modulo soundness, but writing `confidence: 95` invites a
|
||||
false-precision reading. The colophon's commitment to honest
|
||||
epistemic accounting requires a way to opt out.
|
||||
|
||||
The exception:
|
||||
|
||||
confidence: proved
|
||||
|
||||
(or the equivalent `proven` — both forms accepted) does three things:
|
||||
|
||||
1. The trust score is computed as `100 × 0.6 + ((evidence-1)/4) × 100 × 0.4`,
|
||||
i.e. as if `confidence` were 100. Evidence still varies; trust is
|
||||
not pinned to 100.
|
||||
2. The confidence axis on the figure is drawn full-length, with a
|
||||
small distinct cap at the vertex: a 3×3-px filled square (instead
|
||||
of the usual 2-px circle). The square is the visual marker that
|
||||
reads "this is not a credence, it is a proof-completeness flag."
|
||||
3. The compact row renders `proved confidence` instead of the
|
||||
`XX% confidence` form.
|
||||
|
||||
`confidence-history` is incompatible with `confidence: proved`. If
|
||||
both are set, the build emits a warning and `confidence-history` is
|
||||
ignored (a proof either is or is not; tracking history of a
|
||||
binary-after-the-fact value is incoherent).
|
||||
|
||||
This is the *only* genre-specific carve-out in the schema. All other
|
||||
fields read across genres without modification, with the colophon
|
||||
gloss in §6 explaining the cross-genre interpretation.
|
||||
|
||||
### 4.4 `subtitle`
|
||||
|
||||
Captures a short secondary title shown below the main `title` in the
|
||||
center column of the new three-column header (§7.2). The field is
|
||||
optional and free-form.
|
||||
|
||||
subtitle: "A Static Site Framework"
|
||||
|
||||
When omitted, no subtitle line is rendered and the byline collapses
|
||||
upward against the title. The subtitle is *not* an abstract; abstracts
|
||||
remain in the existing `abstract:` field and render below the byline.
|
||||
|
||||
Subtitles do not feed the epistemic figure or any audit metric; they
|
||||
are purely a presentation field. They render in print and do not
|
||||
participate in any focus-mode hiding.
|
||||
|
||||
---
|
||||
|
||||
## 5. Frontmatter layout
|
||||
|
||||
The combined frontmatter for an essay using all features:
|
||||
|
||||
---
|
||||
title: "The Title"
|
||||
subtitle: "An Optional Secondary Line"
|
||||
date: 2026-05-07
|
||||
abstract: >
|
||||
One-paragraph description.
|
||||
tags:
|
||||
- research/mathematics
|
||||
|
||||
# existing epistemic
|
||||
status: "Durable"
|
||||
confidence: 80
|
||||
importance: 3
|
||||
evidence: 5
|
||||
scope: average
|
||||
novelty: moderate
|
||||
practicality: moderate
|
||||
confidence-history: [60, 70, 80]
|
||||
|
||||
# new
|
||||
peer-status: under-review
|
||||
result-shape: mixed
|
||||
---
|
||||
|
||||
For a formal-mathematics piece using `confidence: proved`:
|
||||
|
||||
---
|
||||
title: "Branch-Based Local Capture in Tree-Ball Geometry"
|
||||
status: "Durable"
|
||||
confidence: proved
|
||||
importance: 3
|
||||
evidence: 5
|
||||
scope: average
|
||||
novelty: idiosyncratic
|
||||
practicality: low
|
||||
peer-status: under-review
|
||||
result-shape: mixed
|
||||
---
|
||||
|
||||
The monogram lives outside frontmatter, in
|
||||
`content/essays/branch-based-local-capture-in-tree-balls/mark.svg`.
|
||||
|
||||
---
|
||||
|
||||
## 6. Colophon gloss
|
||||
|
||||
The colophon's *Living Documents* section is updated to add a
|
||||
paragraph documenting genre-specific reading of the existing
|
||||
fields. Proposed text (to be inserted before the field list):
|
||||
|
||||
> The epistemic vocabulary above is genre-general but reads
|
||||
> differently across genres. For a personal essay, `confidence`
|
||||
> reflects credence in a thesis — "I might change my mind." For an
|
||||
> empirical research paper, it reflects expected generalization —
|
||||
> "this would replicate." For formal mathematics, it reflects
|
||||
> credence in proof correctness, with a special value `proved`
|
||||
> available for theorems with complete proofs (where any numeric
|
||||
> value would be false precision). `evidence` reads analogously: the
|
||||
> strength of arguments and supporting writing in essays, the
|
||||
> empirical base in research, the structure of the proof in
|
||||
> mathematics. The fields are the same; the interpretive frame
|
||||
> shifts with the work.
|
||||
|
||||
Two new field rows are appended to the existing field list:
|
||||
|
||||
> **Peer status** — the external review state, distinct from
|
||||
> `status` (which is the author's internal position). Values:
|
||||
> *unreviewed* (default), *under review*, *peer reviewed*,
|
||||
> *published*, *retracted*. This information modulates the outer
|
||||
> ring of the epistemic figure; a *retracted* piece is also rendered
|
||||
> with the field name struck through.
|
||||
>
|
||||
> **Result shape** — the shape of the central claim: *positive*
|
||||
> (argues something works), *negative* (argues something does not),
|
||||
> *mixed* (both, as in a double-pincer barrier paper), *comparative*
|
||||
> (compares approaches), or *descriptive* (describes without arguing
|
||||
> for or against). Encoded as a small glyph beside the trust score
|
||||
> on the epistemic figure.
|
||||
|
||||
---
|
||||
|
||||
## 7. Pandoc filter and template integration
|
||||
|
||||
### 7.1 New Haskell module
|
||||
|
||||
A new module `build/Filters/Mark.hs` exports two functions:
|
||||
|
||||
-- | Render the monogram inline. Reads from disk; substitutes
|
||||
-- black/#000000 fills and strokes with currentColor; strips
|
||||
-- width/height attributes from <svg>; wraps in <figure>.
|
||||
-- Returns an empty document fragment if the file is absent.
|
||||
renderMonogram :: FilePath -> Compiler Html
|
||||
|
||||
-- | Build the epistemic figure SVG from a Context.
|
||||
-- Reads exactly the fields listed in §3.2.
|
||||
-- Returns Nothing when `status` is absent.
|
||||
renderEpistemicFigure :: EpistemicFields -> Maybe Html
|
||||
|
||||
`EpistemicFields` is a small record type that reuses the parsing
|
||||
logic already in `Contexts.hs` for the existing block. The figure
|
||||
generator is a pure function from this record to SVG markup.
|
||||
|
||||
The filter is wired into `build/Compilers.hs` as the last step of
|
||||
the AST transformation, so it runs after the existing image,
|
||||
sidenote, and citation passes. It produces no AST mutation; instead,
|
||||
the rendered SVG is added to the page Context as two new fields:
|
||||
|
||||
monogramSvg -- inline SVG or empty
|
||||
epistemicSvg -- inline SVG or empty
|
||||
|
||||
These are referenced in templates as `$monogramSvg$` and
|
||||
`$epistemicSvg$`.
|
||||
|
||||
### 7.2 Template change
|
||||
|
||||
The current `templates/partials/metadata.html` carries everything
|
||||
between the title and the cursive-L divider: tags, keywords, abstract,
|
||||
byline, affiliation, the compact epistemic strip, and the page-nav
|
||||
links. To make room for the three-column header without losing the
|
||||
ordering or the divider, the partial is split in two:
|
||||
|
||||
- `templates/partials/metadata-header.html` — byline, abstract, and
|
||||
the compact epistemic strip. Renders inside the center column of
|
||||
the new header.
|
||||
- `templates/partials/metadata-tail.html` — tags, keywords,
|
||||
affiliation, page-nav, in that exact order (matching the current
|
||||
`metadata.html` rendering order). Renders as a row beneath the
|
||||
three-column header, above the cursive-L divider.
|
||||
|
||||
`templates/essay.html` and `templates/blog-post.html` then become:
|
||||
|
||||
<header class="essay-frontmatter">
|
||||
<div class="frontmatter-mark frontmatter-mark--monogram">$monogramSvg$</div>
|
||||
<div class="frontmatter-title">
|
||||
<h1 class="page-title">$title$</h1>
|
||||
$if(subtitle)$<p class="essay-subtitle">$subtitle$</p>$endif$
|
||||
$partial("templates/partials/metadata-header.html")$
|
||||
</div>
|
||||
<div class="frontmatter-mark frontmatter-mark--epistemic">
|
||||
<a href="#epistemic" aria-label="Jump to epistemic profile">$epistemicSvg$</a>
|
||||
</div>
|
||||
</header>
|
||||
$partial("templates/partials/metadata-tail.html")$
|
||||
<div class="content-divider" aria-hidden="true">
|
||||
<a href="/new.html" class="content-divider-logo" aria-label="New"></a>
|
||||
</div>
|
||||
|
||||
The cursive-L `content-divider-logo` is preserved exactly as it is
|
||||
today; nothing about the frontmatter↔body separator changes. Only the
|
||||
material *above* the divider is reorganized.
|
||||
|
||||
When either SVG is empty, the corresponding column collapses (CSS
|
||||
grid, `auto` sizing). When both are absent, the header degrades to a
|
||||
single-column layout that visually matches the existing one
|
||||
(title + subtitle + metadata-header), so existing pages render
|
||||
identically until they opt in.
|
||||
|
||||
`reading.html` (poetry/fiction) does NOT receive the figure column,
|
||||
since these content types omit the epistemic block by design. They
|
||||
do receive the monogram column when a `mark.svg` is present, plus the
|
||||
`subtitle` field if set.
|
||||
|
||||
`pageCtx` (standalone pages) receives neither column but does honor
|
||||
`subtitle` if set.
|
||||
|
||||
### 7.3 CSS
|
||||
|
||||
A new file `static/css/marks.css` defines the grid layout, the
|
||||
collapse behavior, the print rules, the focus-mode hiding, and the
|
||||
two `.frontmatter-mark` modifiers. It loads with the rest of the
|
||||
stylesheet bundle; no new HTTP request.
|
||||
|
||||
The breakpoint at which the figure column drops below the title
|
||||
(rather than sitting beside it) is the existing narrow-screen
|
||||
breakpoint where sidenotes collapse to footnotes. A reader on
|
||||
mobile sees: monogram → title → figure, stacked.
|
||||
|
||||
---
|
||||
|
||||
## 8. Build behavior
|
||||
|
||||
### 8.1 Determinism
|
||||
|
||||
Both monograms and epistemic figures must be deterministic at build
|
||||
time. The monogram is just a file read; the epistemic figure is a
|
||||
pure function of frontmatter and `git log --follow`. Two consecutive
|
||||
builds of the same content tree must produce byte-identical SVGs.
|
||||
|
||||
This is enforced by:
|
||||
|
||||
- No timestamps in generated SVGs.
|
||||
- No floating-point coordinates beyond two decimal places.
|
||||
- Stable ordering of attributes (alphabetical) and elements
|
||||
(declaration order).
|
||||
- No build-time UUIDs or random IDs (use deterministic IDs derived
|
||||
from slug, e.g. `id="mark-title-{slug}"`).
|
||||
|
||||
This matters for the GPG signing pipeline (`make sign`): a
|
||||
non-deterministic SVG would invalidate page signatures across
|
||||
otherwise-identical builds.
|
||||
|
||||
### 8.2 Performance
|
||||
|
||||
Reading 200 small SVG files at build is negligible. Rendering 200
|
||||
epistemic figures is a few hundred lines of string concatenation each
|
||||
and well within the existing build budget. No new build step is
|
||||
needed; the work happens inside `Compilers.hs` alongside existing
|
||||
filter passes.
|
||||
|
||||
The Hakyll dependency tracking already keys on frontmatter changes
|
||||
via the existing essay context. Adding `monogramSvg` and
|
||||
`epistemicSvg` to the same context propagates dependency tracking
|
||||
for free: editing a frontmatter field invalidates the page; replacing
|
||||
a `mark.svg` invalidates only that page's dependencies (Hakyll's
|
||||
file-watch already tracks `content/**`).
|
||||
|
||||
### 8.3 Failure modes
|
||||
|
||||
| Condition | Build behavior |
|
||||
|---|---|
|
||||
| `mark.svg` absent | Monogram column collapses; no warning. |
|
||||
| `mark.svg` malformed XML | Warn; render the slot empty; do not fail the build. |
|
||||
| `mark.svg` exceeds 8 KiB | Warn; render anyway. |
|
||||
| `mark.svg` violates §2.2 contract | Warn (with specific violation); render anyway. |
|
||||
| `status:` absent | Epistemic column collapses; no warning. |
|
||||
| `status:` set, `confidence` missing | Render figure; confidence axis has no vertex point. |
|
||||
| `peer-status:` invalid value | Warn; treat as `unreviewed`. |
|
||||
| `result-shape:` invalid value | Warn; render figure without center glyph. |
|
||||
| `confidence: proved` and `confidence-history:` both set | Warn; ignore `confidence-history`. |
|
||||
|
||||
Warnings go to stderr during `make build`. They are captured and
|
||||
surfaced on `/build/` (§9).
|
||||
|
||||
### 8.4 Backwards compatibility
|
||||
|
||||
Every existing essay must render identically after this change is
|
||||
deployed, until and unless the author edits the file to add a
|
||||
`mark.svg` or new frontmatter fields. The new template grid must
|
||||
collapse to the existing single-column layout when both
|
||||
`$monogramSvg$` and `$epistemicSvg$` are empty; CSS feature-tests
|
||||
for grid fallback are not needed because the existing template uses
|
||||
flexbox/block already.
|
||||
|
||||
A pre-merge regression test runs `make build` on a snapshot of
|
||||
`content/` from before the change and diffs `_site/` against a
|
||||
known-good snapshot. The only allowed diffs are template-driven
|
||||
whitespace.
|
||||
|
||||
---
|
||||
|
||||
## 9. Audit and telemetry
|
||||
|
||||
### 9.1 `make audit-marks`
|
||||
|
||||
A new build target lists pieces missing one or both marks. Output
|
||||
columns: path, has-monogram?, has-epistemic-figure?, suggested
|
||||
action.
|
||||
|
||||
$ make audit-marks
|
||||
content/essays/ozymandias.md ✓ ✓
|
||||
content/essays/branch-based-...md ✗ ✗ add mark.svg, set status:
|
||||
content/essays/beyond-comorbidity-... ✗ ✓ add mark.svg
|
||||
...
|
||||
|
||||
Implementation: `tools/audit-marks.py` walks `content/**/*.md`,
|
||||
parses YAML frontmatter, checks for the corresponding `mark.svg`,
|
||||
checks whether `status:` is set, and emits the table.
|
||||
|
||||
The script also emits two summary metrics: corpus monogram coverage
|
||||
percentage and corpus epistemic-figure coverage percentage.
|
||||
|
||||
### 9.2 `/build/` integration
|
||||
|
||||
The existing build telemetry page already includes "epistemic
|
||||
coverage" per the WRITING.md auto-generated-pages list. Two new
|
||||
sub-sections are added to that page:
|
||||
|
||||
- **Monogram coverage**: count and percentage of essays/blog/poetry/
|
||||
fiction/music with `mark.svg` present, broken down by portal.
|
||||
- **Epistemic-figure coverage**: count and percentage of pieces
|
||||
with `status:` set and a renderable figure, broken down by portal.
|
||||
|
||||
The same Stats.hs module that produces existing coverage figures
|
||||
extends to compute these. No new external dependencies.
|
||||
|
||||
### 9.3 Linting hook
|
||||
|
||||
A pre-commit hook (`tools/hooks/pre-commit-marks.sh`) runs
|
||||
`make audit-marks` and warns on any new `.md` file under
|
||||
`content/essays/` or `content/research/` (effectively, anything
|
||||
tagged `research/*` or in those directories) added without a
|
||||
`mark.svg` or with `status:` unset. Warning only; does not block
|
||||
the commit. Authors who genuinely want to publish without marks
|
||||
can ignore the warning.
|
||||
|
||||
---
|
||||
|
||||
## 10. Migration
|
||||
|
||||
### Phase 1 — Wire the system, no content (1 build)
|
||||
|
||||
- Land `Filters/Mark.hs`, the template changes, and `static/css/marks.css`.
|
||||
- Land `tools/audit-marks.py`.
|
||||
- Land the two new schema fields (`peer-status`, `result-shape`)
|
||||
and the `confidence: proved` exception in `Contexts.hs` and
|
||||
`Stability.hs`.
|
||||
- Update `WRITING.md` with the new fields and the `mark.svg`
|
||||
convention.
|
||||
- Update the colophon with the §6 gloss.
|
||||
- Build. Every existing page renders identically (§8.4).
|
||||
|
||||
### Phase 2 — Reference monograms (1 week of evenings)
|
||||
|
||||
- Author monograms for the 8–10 most-trafficked pieces (likely:
|
||||
*Colophon*, *Memento Mori*, *Ozymandias*, *Beyond Comorbidity Indices*,
|
||||
*Branch-Based Local Capture*, the *Music* index, the *Library* portal
|
||||
landing, and a poetry collection landing).
|
||||
- Author the reference monogram template at
|
||||
`static/templates/mark-template.svg`.
|
||||
- Validate them against §2.2 with `make audit-marks`.
|
||||
|
||||
### Phase 3 — Backfill epistemic fields (incremental)
|
||||
|
||||
- For each piece in `research/` and `nonfiction/`, decide whether to
|
||||
add `status:`, `peer-status:`, `result-shape:`. The audit script
|
||||
surfaces the candidates.
|
||||
- Specifically: *Branch-Based Local Capture* gets `status: Durable`,
|
||||
`confidence: proved`, `evidence: 5`, `peer-status: unreviewed`,
|
||||
`result-shape: mixed`. *Beyond Comorbidity Indices* gets
|
||||
`peer-status: under-review` and `result-shape: comparative`
|
||||
added.
|
||||
|
||||
### Phase 4 — Iterate
|
||||
|
||||
- Once 30+ marks exist, review the corpus as a system. Tighten
|
||||
§2.2 constraints if cross-mark consistency is weaker than expected.
|
||||
Loosen if the constraints are pinching authorship.
|
||||
- Decide whether portal-level base monograms (the
|
||||
Approach 5 idea) are worth adding as a third tier.
|
||||
|
||||
---
|
||||
|
||||
## 11. Rejected alternatives
|
||||
|
||||
These were considered and not adopted; recording them so future
|
||||
revisions don't relitigate.
|
||||
|
||||
- **Two figure types (essay vs. research badge).** The existing
|
||||
fields handle both genres when read with appropriate gloss
|
||||
(§6). Two figures would create a visual fork that costs more
|
||||
than it pays. *Beyond Comorbidity Indices* is the proof point:
|
||||
formal research already renders cleanly with the existing
|
||||
vocabulary.
|
||||
- **Mode-switched figure with axes that change meaning per genre.**
|
||||
Visual grammar should be unambiguous. A single radial figure
|
||||
where the axes mean different things depending on a frontmatter
|
||||
flag would require footnotes to read. Genre-gloss in the colophon
|
||||
handles the same need without ambiguity.
|
||||
- **Auto-derived monograms from semantic-search embeddings.** Tried
|
||||
in spec drafting. The result is generic and lacks the editorial
|
||||
statement that a hand-authored monogram makes. Authors may use
|
||||
AI-assist tools to generate monograms (against §2.2 contract),
|
||||
but the system does not derive them automatically.
|
||||
- **Ghosted axes for missing fields.** Tested visually. Reads as a
|
||||
bug, not a deliberate position. Better to suppress the figure
|
||||
entirely (§3.1) and surface the absence in `/build/` (§9).
|
||||
- **A separate `claim-mode: formal | empirical | essay` field.**
|
||||
Solved the wrong problem. The fields don't need a mode flag; they
|
||||
need a gloss. The two new fields (`peer-status`, `result-shape`)
|
||||
plus the `confidence: proved` exception cover the genre-specific
|
||||
needs surfaced in audit.
|
||||
- **Folding `peer-status` into `status`.** Tempting but wrong.
|
||||
`status` is the author's position ("I expect this to hold up").
|
||||
`peer-status` is the world's position ("the field has confirmed
|
||||
it"). A piece can be `Durable` and `unreviewed` simultaneously
|
||||
(the author believes it; the world hasn't checked yet). Keeping
|
||||
them factored preserves that distinction.
|
||||
|
||||
---
|
||||
|
||||
## 12. Open questions for review
|
||||
|
||||
1. **Monogram filename convention.** Spec proposes `mark.svg` (in
|
||||
directory-form) and `{slug}.mark.svg` (flat-form). Alternative:
|
||||
always require directory-form for any piece that wants a
|
||||
monogram, simplifying the resolver. Cost: forces directory-form
|
||||
migration on currently-flat essays. Recommend keeping both
|
||||
forms; the resolver is small and the migration cost is real.
|
||||
|
||||
2. **Should `peer-status: retracted` survive `make build`?**
|
||||
Currently spec'd as a normal field with a strikethrough.
|
||||
Alternative: `make build` refuses to publish pieces marked
|
||||
`retracted` and instead generates a tombstone page at the
|
||||
original URL. Probably overkill for the personal-site context;
|
||||
leaving as a normal field with visual indicator. Worth flagging.
|
||||
|
||||
3. **Should the figure's confidence trend arrow distinguish
|
||||
"stable" (∆ ≤ 2) from "unchanged" (∆ = 0)?** Currently treats
|
||||
them the same as `→`. The existing trend arrow in the epistemic
|
||||
block does too. No need to diverge.
|
||||
|
||||
4. **Per-portal monogram defaults.** Should an absent `mark.svg`
|
||||
fall back to a portal-level base monogram (e.g. all `research/`
|
||||
pieces show a default research mark)? Spec says no — absence
|
||||
is meaningful and surfaces in `/build/`. The visual specimen
|
||||
sheets in the earlier exploration suggested portal-level
|
||||
iconography is interesting; defer to a future spec.
|
||||
|
||||
5. **Naming.** The pair of glyphs is currently called "monogram"
|
||||
and "epistemic figure." Considered alternatives: "device"
|
||||
(printer's-mark lineage) and "figure" (Tufte lineage), or
|
||||
"mark" and "badge" (more colloquial). Spec uses
|
||||
"monogram + epistemic figure" because it most accurately
|
||||
describes what each thing *is*. Open to naming bikeshed.
|
||||
|
||||
---
|
||||
|
||||
## 13. Files touched
|
||||
|
||||
A complete list of files this spec creates or modifies, for tracking
|
||||
PR scope:
|
||||
|
||||
**New:**
|
||||
|
||||
- `build/Filters/Mark.hs`
|
||||
- `tools/audit-marks.py`
|
||||
- `tools/hooks/pre-commit-marks.sh`
|
||||
- `static/css/marks.css`
|
||||
- `static/templates/mark-template.svg`
|
||||
- `static/templates/epistemic-figure-reference.svg`
|
||||
- `MARKS.md` (this file, after merge)
|
||||
|
||||
**Modified:**
|
||||
|
||||
- `build/Compilers.hs` (expose new context fields where essay /
|
||||
blog / reading / page contexts are assembled)
|
||||
- `build/Contexts.hs` (parse `subtitle`, `peer-status`,
|
||||
`result-shape`, `confidence: proved`; produce `monogramSvg` and
|
||||
`epistemicSvg` context fields; render the inline trend arrow
|
||||
inside the compact-row `confidence` chip)
|
||||
- `build/Stability.hs` (consume `peer-status` for tick styling
|
||||
if rendering moves out of pure SVG generator)
|
||||
- `build/Stats.hs` (monogram + epistemic-figure coverage on
|
||||
`/build/`)
|
||||
- `templates/essay.html`
|
||||
- `templates/blog-post.html`
|
||||
- `templates/reading.html` (monogram column + `subtitle`; no figure)
|
||||
- `templates/partials/metadata.html` (split into the two new
|
||||
partials below; this file becomes a thin shim or is removed)
|
||||
- `templates/partials/metadata-header.html` (new — center-column
|
||||
metadata: byline, abstract, compact epistemic strip)
|
||||
- `templates/partials/metadata-tail.html` (new — row beneath the
|
||||
three-column header: tags, keywords, affiliation, page-nav, in
|
||||
that order)
|
||||
- `Makefile` (`audit-marks` target)
|
||||
- `WRITING.md` (new fields including `subtitle`; monogram convention)
|
||||
- `content/colophon.md` (genre gloss)
|
||||
|
||||
**Per-essay (Phase 2+):**
|
||||
|
||||
- `content/essays/{slug}/mark.svg` × N (hand-authored monograms)
|
||||
- frontmatter edits to add `peer-status:`, `result-shape:`,
|
||||
`confidence: proved` where applicable.
|
||||
|
||||
---
|
||||
|
||||
## 14. Future work (out of scope for the initial rollout)
|
||||
|
||||
These extensions are explicitly deferred. They are recorded here so
|
||||
that the structural decisions in §§2–9 do not foreclose them.
|
||||
|
||||
- **Monogram in hyperlink popup previews.** The existing on-hover
|
||||
page-preview popup (which already renders title and abstract for
|
||||
internal links) should display the monogram alongside the title
|
||||
when one exists. The popup is the smallest place a reader meets a
|
||||
page; the monogram earns its keep there.
|
||||
- **Monogram in `/library/` and `/new/` feed listings.** Both the
|
||||
library portal and the recent-changes feed render lists of pages.
|
||||
Once monogram coverage is non-trivial (Phase 2 ships ≥10), each
|
||||
list item should render the monogram as a small inline glyph
|
||||
beside the title.
|
||||
- **Portal-level base monograms.** Deferred per §12.4, but the
|
||||
correct natural place to introduce them is once the popup and
|
||||
feed-listing wiring is in place — base monograms compensate for
|
||||
list rows where the per-page monogram is absent.
|
||||
|
||||
These items are scoped as a follow-up PR (informally "PR 4") after
|
||||
the audit-tool PR ships and after at least 10 hand-authored monograms
|
||||
exist to test the popup/listing rendering against real content.
|
||||
|
|
@ -0,0 +1,250 @@
|
|||
.PHONY: build deploy sign download-model download-pdfjs download-leaflet compress-assets convert-images pdf-thumbs pdfs watch clean dev audit-marks archive-gc archive-wayback archive-check archive-suggest
|
||||
|
||||
# deploy's prerequisite order (clean -> build -> sign) is only correct
|
||||
# serially; under `make -j` they could interleave. This build has no
|
||||
# intra-target parallelism worth preserving, so disable it outright.
|
||||
.NOTPARALLEL:
|
||||
|
||||
# Source .env for deploy / GitHub config if it exists.
|
||||
# .env format: KEY=value (one per line, no `export` prefix, no quotes needed).
|
||||
# Only the variables explicitly listed below are exported to recipe
|
||||
# subprocesses — bare `export` would leak every .env key (including any
|
||||
# future GITHUB_TOKEN) into every child process.
|
||||
-include .env
|
||||
export VPS_USER VPS_HOST VPS_PATH GITHUB_REPO
|
||||
|
||||
build:
|
||||
# Auto-snapshot any uncommitted content/ changes BEFORE the build
|
||||
# so the stability heuristic in build/Stability.hs sees a stable
|
||||
# git history. If a subsequent step fails, the snapshot remains in
|
||||
# the history — that's intentional. The next successful build
|
||||
# either reuses it (no new content/ changes) or appends another
|
||||
# snapshot on top, so failures don't disappear from the log.
|
||||
#
|
||||
# `git add content/` respects .gitignore, which excludes credential-
|
||||
# shaped patterns (.env, *.key, *.pem, id_rsa*, credentials*, etc.)
|
||||
# so a stray secret dropped under content/ is NOT auto-staged. To
|
||||
# intentionally commit a normally-ignored file, use `git add -f`
|
||||
# manually before running `make build`.
|
||||
#
|
||||
# The commit and its guard are pathspec-limited to content/ so that
|
||||
# anything the user had previously staged for other reasons is left
|
||||
# staged, not silently swept into the auto-commit.
|
||||
@git add content/
|
||||
@git diff --cached --quiet -- content/ || git commit -m "auto: $$(date -u +%Y-%m-%dT%H:%M:%SZ) [skip ci]" -- content/
|
||||
@mkdir -p data
|
||||
@date +%s > data/build-start.txt
|
||||
@./tools/convert-images.sh
|
||||
@$(MAKE) -s pdf-thumbs
|
||||
@./tools/download-pdfjs.sh
|
||||
@./tools/download-leaflet.sh
|
||||
# Photography pipeline (Phase 3): generate per-photo EXIF + palette
|
||||
# sidecars under content/photography/**/*.{exif,palette}.yaml so the
|
||||
# Hakyll context can merge them with frontmatter at compile time.
|
||||
# Plus per-image dimension sidecars across static/images/ and
|
||||
# content/** so build/Filters/Images.hs can attach width / height
|
||||
# attrs to body images for CLS prevention.
|
||||
# Gated on .venv presence, same as embed.py — failures are non-fatal.
|
||||
@if [ -d .venv ]; then \
|
||||
uv run python tools/extract-exif.py || echo "Warning: EXIF extraction failed (build continues with frontmatter only)"; \
|
||||
uv run python tools/extract-palette.py || echo "Warning: palette extraction failed (build continues with frontmatter only)"; \
|
||||
uv run python tools/extract-dimensions.py || echo "Warning: dimension extraction failed (build continues without width/height attrs)"; \
|
||||
else \
|
||||
echo "Photography sidecars skipped: run 'uv sync' to enable EXIF + palette + dimension extraction (build continues with frontmatter only)"; \
|
||||
fi
|
||||
# Archive pipeline (Phase 1): fetch any manifest URL without a local
|
||||
# artifact, extract text, write archive/<slug>/PROVENANCE.json and
|
||||
# data/archive-index.json. Gated on .venv, same as embed.py. A SHA or
|
||||
# slug-URL integrity error exits non-zero and halts the build; a
|
||||
# transient network failure is non-fatal (the entry retries next build).
|
||||
@if [ -d .venv ]; then \
|
||||
uv run python tools/archive.py fetch; \
|
||||
else \
|
||||
echo "Archive fetch skipped: run 'uv sync' to enable link archiving (build continues)"; \
|
||||
fi
|
||||
cabal run site -- build
|
||||
pagefind --site _site
|
||||
@if [ -d .venv ]; then \
|
||||
HF_HUB_DISABLE_IMPLICIT_TOKEN=1 uv run python tools/embed.py || echo "Warning: embedding failed — data/similar-links.json not updated (build continues)"; \
|
||||
else \
|
||||
echo "Embedding skipped: run 'uv sync' to enable similar-links (build continues)"; \
|
||||
fi
|
||||
# Site-wide footer timestamp: rewrite every <span data-build-time>
|
||||
# in _site/**/*.html so cached (un-recompiled) pages don't show a
|
||||
# stale per-page build time. See tools/stamp-build-time.py for the
|
||||
# full rationale. Must run before compress-assets so the .gz/.br
|
||||
# sidecars include the fresh stamp.
|
||||
@python3 tools/stamp-build-time.py _site
|
||||
@./tools/compress-assets.sh _site
|
||||
> IGNORE.txt
|
||||
@BUILD_END=$$(date +%s); \
|
||||
BUILD_START=$$(cat data/build-start.txt); \
|
||||
echo $$((BUILD_END - BUILD_START)) > data/last-build-seconds.txt.tmp && \
|
||||
mv data/last-build-seconds.txt.tmp data/last-build-seconds.txt
|
||||
|
||||
sign:
|
||||
@./tools/sign-site.sh
|
||||
|
||||
# Download the quantized ONNX model for client-side semantic search.
|
||||
# Run once; files are gitignored. Safe to re-run (skips existing files).
|
||||
download-model:
|
||||
@./tools/download-model.sh
|
||||
|
||||
# Vendor Mozilla's prebuilt PDF.js viewer into static/pdfjs/.
|
||||
# Runs automatically as part of `build` (skips when already present).
|
||||
# Files are gitignored; sha256-verified against tools/pdfjs-checksums.sha256.
|
||||
download-pdfjs:
|
||||
@./tools/download-pdfjs.sh
|
||||
|
||||
# Vendor Leaflet + leaflet.markercluster into static/leaflet/.
|
||||
# Used only by /photography/map/. Runs automatically as part of `build`
|
||||
# (skips when already present). Files are gitignored; sha256-verified
|
||||
# against tools/leaflet-checksums.sha256.
|
||||
download-leaflet:
|
||||
@./tools/download-leaflet.sh
|
||||
|
||||
# Generate .gz and .br sidecars for compressible text assets in _site/.
|
||||
# Runs automatically as part of `build`. Pairs with `gzip_static` /
|
||||
# `brotli_static` in the nginx vhost (see nginx/static-assets.conf).
|
||||
compress-assets:
|
||||
@./tools/compress-assets.sh _site
|
||||
|
||||
# Convert JPEG/PNG images to WebP companions (also runs automatically in build).
|
||||
# Requires cwebp: pacman -S libwebp / apt install webp
|
||||
convert-images:
|
||||
@./tools/convert-images.sh
|
||||
|
||||
# Generate first-page thumbnails for PDFs in static/papers/ (also runs in build).
|
||||
# Requires pdftoppm: pacman -S poppler / apt install poppler-utils
|
||||
# Thumbnails are written as static/papers/foo.thumb.png alongside each PDF.
|
||||
# Skipped silently when pdftoppm is not installed or static/papers/ is empty.
|
||||
pdf-thumbs:
|
||||
# A failing pdftoppm must at least warn: the `find | while` pipeline's
|
||||
# exit status is the last iteration's, so without the `||` a corrupt
|
||||
# PDF would silently ship without a thumbnail.
|
||||
# Walk ALL of static/ (not just papers/): /cv.pdf and /resume.pdf are
|
||||
# the most-linked PDFs on the site and need hover thumbnails too.
|
||||
# pdfjs/ is pruned — the vendored viewer ships sample PDFs.
|
||||
@if command -v pdftoppm >/dev/null 2>&1; then \
|
||||
find static -path static/pdfjs -prune -o -name '*.pdf' -print 2>/dev/null | while read pdf; do \
|
||||
thumb="$${pdf%.pdf}.thumb"; \
|
||||
if [ ! -f "$${thumb}.png" ] || [ "$$pdf" -nt "$${thumb}.png" ]; then \
|
||||
echo " pdf-thumb $$pdf"; \
|
||||
pdftoppm -r 100 -f 1 -l 1 -png -singlefile "$$pdf" "$$thumb" \
|
||||
|| echo "Warning: pdf-thumb failed for $$pdf (page ships without a thumbnail)" >&2; \
|
||||
fi; \
|
||||
done; \
|
||||
else \
|
||||
echo "pdf-thumbs: pdftoppm not found — install poppler (skipping)"; \
|
||||
fi
|
||||
|
||||
# Rebuild the CV + website résumé from yaml-source/ and refresh static/.
|
||||
# Standalone helper — NOT a dependency of `build` or `deploy`. Run manually
|
||||
# after editing a YAML under yaml-source/data/. The site build copies
|
||||
# static/*.pdf through unchanged, so a subsequent `make build` picks them up.
|
||||
#
|
||||
# The ATS variant (yaml-source/output/resume_ats.pdf) is intentionally not
|
||||
# copied to static/ — it's a submission artifact, not a website asset. To
|
||||
# regenerate it too, run `make -C yaml-source ats` directly.
|
||||
#
|
||||
# Silently skipped on hosts without the pipeline (e.g., the VPS): yaml-source/
|
||||
# is gitignored, so it's absent on a fresh clone, and that's the expected
|
||||
# state wherever the LaTeX toolchain isn't installed.
|
||||
pdfs:
|
||||
@if [ ! -d yaml-source ]; then \
|
||||
echo "pdfs: yaml-source/ not present — skipping (pipeline is local-only)"; \
|
||||
exit 0; \
|
||||
fi
|
||||
@$(MAKE) -C yaml-source all
|
||||
@cp yaml-source/output/cv.pdf static/cv.pdf
|
||||
@cp yaml-source/output/resume.pdf static/resume.pdf
|
||||
@echo "pdfs: static/cv.pdf and static/resume.pdf refreshed."
|
||||
|
||||
deploy: clean build sign
|
||||
@test -n "$(VPS_USER)" || (echo "deploy: VPS_USER not set in .env" >&2; exit 1)
|
||||
@test -n "$(VPS_HOST)" || (echo "deploy: VPS_HOST not set in .env" >&2; exit 1)
|
||||
@test -n "$(VPS_PATH)" || (echo "deploy: VPS_PATH not set in .env" >&2; exit 1)
|
||||
# Refuse to deploy a manifestly broken build. _site/index.html must
|
||||
# exist and be non-empty before we run rsync --delete on the VPS.
|
||||
@test -s _site/index.html || { echo "deploy: _site/index.html is missing or empty — refusing to rsync" >&2; exit 1; }
|
||||
# Defense-in-depth: refuse rsync --delete to obviously dangerous
|
||||
# parents in case VPS_PATH was typo'd (e.g. trailing-slash mistake).
|
||||
@case "$(VPS_PATH)" in /|/srv|/srv/http|/var|/var/www|/home|/root|"") echo "deploy: VPS_PATH=$(VPS_PATH) looks unsafe — refusing" >&2; exit 1 ;; esac
|
||||
@command -v notify-send >/dev/null 2>&1 && notify-send "make deploy" "Ready to push & rsync — waiting for auth" || true
|
||||
# Push first: a successful push is cheap to roll back, while a
|
||||
# half-completed rsync is harder to recover from. If the push
|
||||
# fails (auth, branch protection, network), abort before touching
|
||||
# the VPS so the public source repo and the live site stay in sync.
|
||||
git push -u origin main
|
||||
rsync -avz --delete _site/ "$(VPS_USER)@$(VPS_HOST):$(VPS_PATH)/"
|
||||
|
||||
watch: export SITE_ENV = dev
|
||||
watch:
|
||||
cabal run site -- watch
|
||||
|
||||
clean:
|
||||
cabal run site -- clean
|
||||
|
||||
# Report which content pieces are missing a monogram (mark.svg) and / or
|
||||
# the epistemic figure (status: frontmatter). Exits 0 unconditionally;
|
||||
# this is a coverage report, not a build gate. The pre-commit hook at
|
||||
# tools/hooks/pre-commit-marks.sh runs the same script for newly-staged
|
||||
# .md files.
|
||||
audit-marks:
|
||||
@if [ -d .venv ]; then \
|
||||
uv run python tools/audit-marks.py; \
|
||||
else \
|
||||
python3 tools/audit-marks.py; \
|
||||
fi
|
||||
|
||||
# Evict archived works: delete archive/<slug>/ directories whose slug is
|
||||
# recorded in archive/removed.yaml. Opt-in — NEVER run by `make build`.
|
||||
# Orphan directories (not in manifest.yaml, not in removed.yaml) are
|
||||
# reported, never deleted. See ARCHIVE.md - Eviction & removal.
|
||||
archive-gc:
|
||||
@if [ -d .venv ]; then \
|
||||
uv run python tools/archive.py gc; \
|
||||
else \
|
||||
python3 tools/archive.py gc; \
|
||||
fi
|
||||
|
||||
# Submit archived URLs to the Wayback Machine and backfill the capture URL
|
||||
# into each PROVENANCE.json. A slow network job — opt-in, never run by
|
||||
# `make build`. Always exits 0; an entry without a capture retries next run.
|
||||
archive-wayback:
|
||||
@if [ -d .venv ]; then \
|
||||
uv run python tools/archive.py wayback; \
|
||||
else \
|
||||
python3 tools/archive.py wayback; \
|
||||
fi
|
||||
|
||||
# Print works cited in data/*.bib but not yet archived, as manifest-ready
|
||||
# lines the author copies by hand. Read-only — it never edits the manifest
|
||||
# (bibliography auto-seeding is rejected by design; see ARCHIVE.md).
|
||||
# Offline: scans local files only, no network.
|
||||
archive-suggest:
|
||||
@if [ -d .venv ]; then \
|
||||
uv run python tools/archive.py suggest; \
|
||||
else \
|
||||
python3 tools/archive.py suggest; \
|
||||
fi
|
||||
|
||||
# Probe every archived URL for link rot, updating data/archive-state.json.
|
||||
# A slow network job — opt-in, never run by `make build`. Asymmetric
|
||||
# hysteresis: `rotted` needs 3 consecutive failures over >=14 days; a
|
||||
# single success recovers immediately. The next build consumes the state.
|
||||
archive-check:
|
||||
@if [ -d .venv ]; then \
|
||||
uv run python tools/archive.py check; \
|
||||
else \
|
||||
python3 tools/archive.py check; \
|
||||
fi
|
||||
|
||||
# Dev build includes any in-progress drafts under content/drafts/essays/.
|
||||
# SITE_ENV=dev is read by build/Site.hs; drafts are otherwise invisible to
|
||||
# every build (make build / make deploy / cabal run site -- build directly).
|
||||
dev: export SITE_ENV = dev
|
||||
dev:
|
||||
cabal run site -- clean
|
||||
cabal run site -- build
|
||||
python3 -m http.server 8000 --bind 127.0.0.1 --directory _site
|
||||
|
|
@ -0,0 +1,487 @@
|
|||
# Photography
|
||||
|
||||
Design and implementation plan for the photography section of levineuwirth.org. This is the source of truth for the section's architecture, authoring conventions, and build pipeline. It sits alongside `WRITING.md` and `HOMEPAGE.md` as authoritative spec.
|
||||
|
||||
## Status
|
||||
|
||||
Pre-implementation. Decisions locked; phased build to follow.
|
||||
|
||||
---
|
||||
|
||||
## Goals
|
||||
|
||||
- A first-class photography section with the same architectural rigor as essays, music, and poetry.
|
||||
- Custom metadata system tailored to photographs (location, capture date, camera, lens, film, exposure, palette) that is *distinct from* but *consistent with* the essay frontmatter system.
|
||||
- Multiple ways to browse the same corpus: masonry, uniform grid, chronological, map, contact sheet, by tag, by series.
|
||||
- Static-friendly throughout: every page renders at build time; JS is layered on top, never required.
|
||||
- No images in the repo at all. Originals live outside source control, and as of August 2026 so do the
|
||||
web-optimized delivery JPEGs — they are derived artifacts, regenerated by `tools/import-photo.sh`,
|
||||
and `make deploy` ships them by rsyncing `_site/`. See the note under Decisions.
|
||||
|
||||
---
|
||||
|
||||
## Decisions (locked)
|
||||
|
||||
| Topic | Decision |
|
||||
|-------|----------|
|
||||
| Content model | Hybrid: flat singles + collection directories (mirrors `content/poetry/`) |
|
||||
| Portal | Yes — 9th entry in `homePortals` |
|
||||
| EXIF | Auto-extract via build-time tool, written to sidecar; frontmatter overrides |
|
||||
| Map library | Leaflet, vendored locally |
|
||||
| Map tiles | CartoDB Positron (free, attribution-only, monochrome) |
|
||||
| Default geo precision | `city` (~10 km rounding); per-photo override allowed |
|
||||
| Default visual mode | Masonry (native aspect ratios) |
|
||||
| Mode toggle | Built day one: masonry / grid / chronological / map; persisted to localStorage |
|
||||
| Contact sheet | Secondary view at `/photography/contact-sheet/` |
|
||||
| Lightbox | Darkroom mode scoped to photography pages |
|
||||
| Color palette | 5-color k-means strip, auto-extracted at build time |
|
||||
| Feed | Separate `/photography/feed.xml` with thumbnails |
|
||||
| Originals storage | Outside the repo |
|
||||
| Delivery JPEGs | **Revised August 2026** — also outside the repo. Originally committed; 64 frames added 45 MB against a 28 MB `.git`. Git tracks the `.md` entries, the deployed site carries the pixels. A fresh clone therefore builds with missing images. |
|
||||
| `Filters/Images.hs` | Extended with a richer wrapper for photography-page images |
|
||||
| Crop policy | CSS `object-fit: cover` for now; build-time crop later if needed |
|
||||
|
||||
---
|
||||
|
||||
## Content model & directory structure
|
||||
|
||||
```
|
||||
content/photography/
|
||||
├── index.md # /photography/ landing copy & frontmatter
|
||||
├── reykjavik-rooftops.md # flat single (single photo entry)
|
||||
├── reykjavik-rooftops/ # OR directory form for richer entries
|
||||
│ ├── index.md
|
||||
│ ├── photo.jpg # web-optimized; max 2400px long edge
|
||||
│ ├── photo.exif.yaml # generated by build (gitignored)
|
||||
│ └── photo.palette.yaml # generated by build (gitignored)
|
||||
└── copenhagen-2025/ # series (collection)
|
||||
├── index.md # series landing
|
||||
├── 01-canal-morning.md
|
||||
├── 02-tivoli-evening.md
|
||||
└── ...
|
||||
```
|
||||
|
||||
- **Flat single entries** are appropriate when the photo's metadata + a short caption are all you want.
|
||||
- **Directory single entries** are used when the photo has co-located assets or longer prose.
|
||||
- **Collection directories** group multiple photos under a series. Series have their own landing page at `/photography/{series}/`; individual photos live at `/photography/{series}/{photo}/`.
|
||||
|
||||
The hybrid pattern parallels `content/poetry/` and reuses the existing routing patterns in `build/Patterns.hs`.
|
||||
|
||||
---
|
||||
|
||||
## Frontmatter schema
|
||||
|
||||
All fields except `title` and one of (`date`, `captured`) are optional. Author-written values always win over auto-extracted EXIF.
|
||||
|
||||
```yaml
|
||||
title: "Reykjavík Rooftops"
|
||||
date: 2026-04-27 # publication date on this site
|
||||
captured: 2026-03-15 # when the shutter clicked
|
||||
|
||||
# Location
|
||||
location: "Reykjavík, Iceland" # human-readable
|
||||
geo: [64.146, -21.940] # decimal lat/lon — author-set or from EXIF
|
||||
geo-precision: city # exact | km | city | hidden (default: city)
|
||||
|
||||
# Camera & exposure (auto-fillable from EXIF)
|
||||
camera: "Pentax K1000"
|
||||
lens: "50mm f/1.4"
|
||||
film: "Kodak Portra 400" # film stock for analog shots
|
||||
exposure: "1/125 f/8 ISO 400" # combined string OR individual fields:
|
||||
# shutter: "1/125"
|
||||
# aperture: "f/8"
|
||||
# iso: 400
|
||||
# focal-length: "50mm"
|
||||
|
||||
# Process
|
||||
process: "scanned" # scanned | digital-raw | darkroom | lightroom | jpeg-ooc
|
||||
|
||||
# Organization
|
||||
series: copenhagen-2025 # optional — slug of containing series
|
||||
tags: # hierarchical, slash-separated
|
||||
- photography
|
||||
- photography/landscape
|
||||
- photography/film
|
||||
featured: true # appears in curated landing rotation
|
||||
nsfw: false # gate behind a click-through if true
|
||||
|
||||
# Display
|
||||
orientation: portrait # portrait | landscape | square (auto-detected if omitted)
|
||||
photo: photo.jpg # filename relative to entry — required for directory form
|
||||
|
||||
# License & provenance
|
||||
license: "CC BY-SA 4.0" # license name; canonical URL auto-resolved
|
||||
# for known shortcodes (CC variants, CC0,
|
||||
# public domain). Omit for "All Rights Reserved".
|
||||
license-url: "https://..." # explicit override; only needed for
|
||||
# custom licenses or non-canonical wording
|
||||
links: # outbound links — Wikimedia Commons,
|
||||
- "Wikimedia Commons | https://commons.wikimedia.org/wiki/File:Foo.jpg"
|
||||
- "Flickr | https://flickr.com/photos/levi/123"
|
||||
# Flickr, exhibition catalog, print-sale
|
||||
# page, etc. Same "Name | URL" pipe
|
||||
# syntax used by authors/affiliation.
|
||||
|
||||
# Palette (auto-filled by build; override only for artistic reasons)
|
||||
palette:
|
||||
- "#2a3f5f"
|
||||
- "#d4a574"
|
||||
- "#7c8a8a"
|
||||
- "#1a1a1a"
|
||||
- "#e8d8c0"
|
||||
```
|
||||
|
||||
### Field semantics
|
||||
|
||||
- **`date`** — when the photo was published to this site. Used for feed ordering and "recently added."
|
||||
- **`captured`** — when the photograph was made. Used for `/by-year/` indexes and chronological mode.
|
||||
- **`geo` + `geo-precision`** — coordinates and the precision at which they're rendered. The build rounds `geo` according to `geo-precision` before writing it to map data and the rendered page. Original precision is never delivered to the browser.
|
||||
- **`tags`** — uses the existing slash-hierarchy tag system (`build/Tags.hs`). The top-level `photography` tag is implicit for all entries; sub-tags like `photography/landscape`, `photography/portrait`, `photography/film`, `photography/architecture` are freeform.
|
||||
- **`series`** — slug-only. The series's own metadata lives in `content/photography/{series}/index.md`.
|
||||
- **`license`** — license name as displayed (e.g., `"CC BY-SA 4.0"`). Canonical URL auto-resolved at build time for known shortcodes (`CC BY 4.0`, `CC BY-SA 4.0`, `CC BY-NC 4.0`, `CC BY-NC-SA 4.0`, `CC BY-ND 4.0`, `CC BY-NC-ND 4.0`, `CC0`, `Public Domain`). Author-supplied `license-url:` always wins. Omit for "All Rights Reserved" (renders as plain text without a link).
|
||||
- **`links`** — outbound list of named external URLs. Same `"Name | URL"` syntax as `authors:` and `affiliation:`. Entries without a URL are dropped. Used for Wikimedia Commons, Flickr, exhibition catalog, print-sale page, etc.
|
||||
|
||||
### What is *not* in photography frontmatter
|
||||
|
||||
These essay fields do not apply and are not exposed on photography pages:
|
||||
|
||||
- Epistemic profile (`status`, `confidence`, `evidence`, `scope`, `novelty`, `practicality`)
|
||||
- `abstract` (use a brief caption in the body instead)
|
||||
- `further-reading`, `bibliography`, `csl`
|
||||
- Reading time, word count
|
||||
- Backlinks, similar-links (text-embedding signals are noise on visual content)
|
||||
- TOC
|
||||
|
||||
---
|
||||
|
||||
## Routing & generated pages
|
||||
|
||||
| URL | Source | Notes |
|
||||
|-----|--------|-------|
|
||||
| `/photography/` | `content/photography/index.md` | Landing; default masonry view + mode toggle |
|
||||
| `/photography/{slug}/` | `content/photography/{slug}.md` or `{slug}/index.md` | Single-photo page |
|
||||
| `/photography/{series}/` | `content/photography/{series}/index.md` | Series landing |
|
||||
| `/photography/{series}/{photo}/` | `content/photography/{series}/{photo}.md` | Photo within a series |
|
||||
| `/photography/by-year/` | Auto-generated | Year index with photo counts |
|
||||
| `/photography/by-year/{year}/` | Auto-generated | All photos with `captured` in that year |
|
||||
| `/photography/contact-sheet/` | Auto-generated | Frame-numbered grid of all photos |
|
||||
| `/photography/map/` | Auto-generated | Leaflet map of geo-tagged photos |
|
||||
| `/photography/{tag}/` | Auto-generated | Tag pages via existing `Tags.hs` |
|
||||
| `/photography/feed.xml` | Auto-generated | Atom feed with thumbnails |
|
||||
|
||||
Tag pages (`/photography/landscape/`, etc.) come "for free" from the existing tag system — sub-tags of `photography` automatically generate their own index pages.
|
||||
|
||||
---
|
||||
|
||||
## Visual system
|
||||
|
||||
### Modes (toggle from day one)
|
||||
|
||||
The `/photography/` landing offers four browsing modes via a toggle in the page header. Selection persists to localStorage under `photography-mode`, mirroring the existing settings panel pattern.
|
||||
|
||||
1. **Masonry** *(default)* — variable-height cells respecting native aspect ratios. CSS Grid with `grid-auto-rows: 1px` + `grid-row-end: span N` computed from each image's aspect ratio (shipped in HTML to avoid layout shift).
|
||||
2. **Grid** — uniform square cells using `object-fit: cover`. Rhythmic, scannable.
|
||||
3. **Chronological** — single column, large, ordered by `captured` (desc). One photo per row with caption beneath. Closest aesthetic to reading mode.
|
||||
4. **Map** — switches the page to the `/photography/map/` route. (Or: inlines the map in place of the grid; decide during Phase 4.)
|
||||
|
||||
### Contact sheet (separate URL)
|
||||
|
||||
`/photography/contact-sheet/` renders all photos with thin white borders, frame numbers in the corner, and a subtle film-grain texture. Distinct from the toggle modes — a deep-cut alternate view, not a primary mode.
|
||||
|
||||
### Color palette strip
|
||||
|
||||
Beneath each photo on its detail page, a thin row of 5 swatches drawn from `palette` frontmatter. Hover reveals the hex value. CSS-only; no JS needed.
|
||||
|
||||
### Darkroom lightbox
|
||||
|
||||
Scoped to `body[data-page-type="photography"]` so essays' lightbox is unaffected.
|
||||
|
||||
- Page chrome fades to deep black on open (not just an overlay)
|
||||
- Subtle vignette behind the photo
|
||||
- Caption + key metadata appear below in muted Spectral italic
|
||||
- Arrow keys / swipe navigate within the current series (or within the current page's photo list)
|
||||
- `i` key toggles full EXIF reveal
|
||||
- `Escape` closes; click outside the photo closes
|
||||
|
||||
### Visual identity rationale
|
||||
|
||||
Masonry default + clean grid alternate gives the considered/organic tone of essays a visual analog: each photo's geometry is honored, but a uniform mode is one click away for comparison. Contact sheet stays as a distinctive alternate URL — referenced from `/photography/` but not the default — so the analog aesthetic is available without being prescriptive.
|
||||
|
||||
---
|
||||
|
||||
## Image storage & pipeline
|
||||
|
||||
### What lives where
|
||||
|
||||
- **Originals (RAW, full-resolution exports)**: outside the repo. Levi's local archive (external drive, NAS, or backup service). Not Levi's responsibility to define for this plan; only the *contract* matters: originals never enter source control.
|
||||
- **Web-optimized JPEGs**: committed to the repo at `content/photography/{slug}/photo.jpg` (or alongside `.md` for flat singles). Long edge ≤ 2400px, quality 85, sRGB, EXIF stripped before commit.
|
||||
- **WebP companions**: generated at build time by the existing `tools/convert-images.sh`; gitignored (already covered by the existing `content/**/*.webp` rule on line 105 of `.gitignore`).
|
||||
- **EXIF sidecar (`{photo}.exif.yaml`)**: generated at build time; gitignored.
|
||||
- **Palette sidecar (`{photo}.palette.yaml`)**: generated at build time; gitignored.
|
||||
|
||||
### Defense-in-depth gitignore additions
|
||||
|
||||
Append to `.gitignore`:
|
||||
|
||||
```
|
||||
# Photography: generated sidecars (recreated by build pipeline)
|
||||
content/photography/**/*.exif.yaml
|
||||
content/photography/**/*.palette.yaml
|
||||
|
||||
# Photography: defense-in-depth — refuse to commit RAW or oversize originals.
|
||||
# To intentionally commit one (rare), use `git add -f path/to/file`.
|
||||
content/photography/**/*.cr2
|
||||
content/photography/**/*.cr3
|
||||
content/photography/**/*.nef
|
||||
content/photography/**/*.arw
|
||||
content/photography/**/*.dng
|
||||
content/photography/**/*.raf
|
||||
content/photography/**/*.orf
|
||||
content/photography/**/*.tif
|
||||
content/photography/**/*.tiff
|
||||
content/photography/**/*.psd
|
||||
```
|
||||
|
||||
### Import workflow
|
||||
|
||||
`tools/import-photo.sh` (Phase 3): given a path to an original and a target slug, the script:
|
||||
|
||||
1. Resizes to ≤ 2400px long edge as JPEG quality 85, sRGB.
|
||||
2. Strips all EXIF from the delivered JPEG with `exiftool -all=`.
|
||||
3. Writes the EXIF sidecar to `{slug}/photo.exif.yaml` (so the metadata is preserved for display, but not embedded in the file shipped to viewers).
|
||||
4. Computes the 5-color palette and writes `{slug}/photo.palette.yaml`.
|
||||
5. Drops a frontmatter stub at `{slug}/index.md` ready for editing.
|
||||
|
||||
Until that script exists, Phase 1 + 2 work with manually prepared JPEGs.
|
||||
|
||||
### Build-pipeline integration
|
||||
|
||||
New steps slot into the Makefile alongside the existing `convert-images` and `pdf-thumbs` targets, all gated on tool availability (silent skip if missing, matching the `embed.py` pattern):
|
||||
|
||||
```
|
||||
make build:
|
||||
...
|
||||
→ tools/extract-exif.py (gated on `exiftool` or `Pillow`)
|
||||
→ tools/extract-palette.py (gated on Python + colorthief)
|
||||
→ tools/build-map-data.py (always runs; no external deps)
|
||||
→ tools/convert-images.sh (existing)
|
||||
→ hakyll-build (existing)
|
||||
...
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## `Filters/Images.hs` extension
|
||||
|
||||
Currently `Filters/Images.hs` emits `<picture>` with WebP companion + lazy-loading for any image in any document. Photography pages need a richer wrapper that includes:
|
||||
|
||||
- The palette strip beneath the photo
|
||||
- A small EXIF metadata block (camera, lens, exposure, captured-date) toggled by an "ⓘ" button
|
||||
- A figure caption (Pandoc already handles this)
|
||||
- The `data-photography="true"` attribute that scopes the darkroom lightbox
|
||||
|
||||
The cleanest approach: extend `Filters/Images.hs` to detect when the document being processed is a photography page (via document metadata or path pattern) and emit the richer wrapper in that case. Essays continue to get the simple `<picture>` wrapper unchanged.
|
||||
|
||||
Alternative considered: render the richer wrapper from the template instead of the filter. Rejected because the template loses the per-image palette/EXIF lookup; doing it in the filter keeps the data flow Pandoc-native.
|
||||
|
||||
---
|
||||
|
||||
## Map architecture
|
||||
|
||||
### Library & tiles
|
||||
|
||||
- **Leaflet 1.9.x** vendored to `static/leaflet/`. No CDN. `tools/download-leaflet.sh` mirrors the `download-pdfjs.sh` pattern.
|
||||
- **CartoDB Positron** raster tiles. Free for any volume, attribution required ("© OpenStreetMap contributors © CARTO"). Monochrome, doesn't fight the typography.
|
||||
- Fallback: if CartoDB ever rate-limits or disappears, swap to Stadia Maps or self-hosted.
|
||||
|
||||
### Build-time data
|
||||
|
||||
`tools/build-map-data.py` walks `content/photography/`, reads each entry's `geo` + `geo-precision`, applies the precision rounding, and emits `_site/photography/map.json`:
|
||||
|
||||
```json
|
||||
[
|
||||
{
|
||||
"slug": "reykjavik-rooftops",
|
||||
"title": "Reykjavík Rooftops",
|
||||
"url": "/photography/reykjavik-rooftops/",
|
||||
"thumb": "/photography/reykjavik-rooftops/photo.jpg",
|
||||
"lat": 64.15,
|
||||
"lon": -21.94,
|
||||
"captured": "2026-03-15"
|
||||
}
|
||||
]
|
||||
```
|
||||
|
||||
### Geo precision
|
||||
|
||||
Applied at build time before any data leaves the build directory:
|
||||
|
||||
| Precision | Rounding | Approximate |
|
||||
|-----------|----------|-------------|
|
||||
| `exact` | 4 decimals | ~10 m |
|
||||
| `km` | 2 decimals | ~1 km |
|
||||
| `city` *(default)* | 1 decimal | ~10 km |
|
||||
| `hidden` | omit from `map.json` entirely | not pinned |
|
||||
|
||||
### Page-scoped JS/CSS
|
||||
|
||||
`static/leaflet/leaflet.js`, `static/leaflet/leaflet.css`, and `static/js/photography-map.js` are loaded **only** on `/photography/map/` via the per-page `js:` frontmatter mechanism (already supported — see `WRITING.md`). Other photography pages stay lightweight.
|
||||
|
||||
### Marker behavior
|
||||
|
||||
- Click marker → photo page.
|
||||
- Marker thumbnail tooltip on hover (uses Leaflet's tooltip API).
|
||||
- Marker clustering when zoomed out, via `leaflet.markercluster` plugin (also vendored).
|
||||
|
||||
---
|
||||
|
||||
## Templates
|
||||
|
||||
New files under `templates/`:
|
||||
|
||||
| File | Role |
|
||||
|------|------|
|
||||
| `photography.html` | Single photo page chrome — figure, palette strip, metadata, navigation within series |
|
||||
| `photography-index.html` | `/photography/` landing — mode toggle, masonry/grid/chrono modes |
|
||||
| `photography-series.html` | `/photography/{series}/` landing — series intro + photo list |
|
||||
| `photography-map.html` | `/photography/map/` — Leaflet container, vendored JS/CSS |
|
||||
| `photography-contact-sheet.html` | `/photography/contact-sheet/` — frame-numbered grid |
|
||||
| `photography-by-year.html` | `/photography/by-year/{year}/` — chronological year index |
|
||||
|
||||
New partial:
|
||||
|
||||
| File | Role |
|
||||
|------|------|
|
||||
| `partials/photo-card.html` | Reusable photo-card markup for grids and listings |
|
||||
| `partials/photo-meta.html` | Camera/lens/exposure/captured block, toggleable |
|
||||
| `partials/photo-palette.html` | 5-swatch palette strip |
|
||||
|
||||
Existing partials reused unchanged: `nav.html`, `head.html`, `footer.html`.
|
||||
|
||||
---
|
||||
|
||||
## Build module structure
|
||||
|
||||
New Haskell modules under `build/`:
|
||||
|
||||
- **`build/Photography.hs`** — patterns, routing rules, contexts specific to photography. Separated from `Site.hs` for the same reason `Catalog.hs` and `Authors.hs` are separated: scoped concerns, easier to reason about.
|
||||
|
||||
Edits to existing modules:
|
||||
|
||||
- **`build/Patterns.hs`** — add `photographyPattern`, `photographySinglesPattern`, `photographySeriesPattern`, `photographyAssetsPattern`.
|
||||
- **`build/Compilers.hs`** — add `photographyCompiler` (essay-pipeline minus epistemic/reading-time/backlinks/TOC).
|
||||
- **`build/Contexts.hs`** — add `photographyCtx` with photo-specific fields, sidecar merge logic.
|
||||
- **`build/Site.hs`** — add `("Photography", "photography")` to `homePortals`; wire photography rules from `Photography.hs`.
|
||||
- **`build/Filters/Images.hs`** — extend to emit richer wrapper on photography pages.
|
||||
|
||||
---
|
||||
|
||||
## Phased implementation
|
||||
|
||||
Each phase has explicit exit criteria. Don't move to the next phase until the current one passes.
|
||||
|
||||
### Phase 1 — Skeleton end-to-end ✓
|
||||
|
||||
- [x] Add `photographyPattern` family to `Patterns.hs`
|
||||
- [x] Create `build/Photography.hs` with routing rules
|
||||
- [x] Add `photographyCompiler` to `Compilers.hs`
|
||||
- [x] Add `photographyCtx` to `Contexts.hs`
|
||||
- [x] Add `("Photography", "photography")` to `homePortals` in `Site.hs`
|
||||
- [x] Create `templates/photography.html`, `templates/photography-index.html`, `templates/partials/photo-card.html`
|
||||
- [x] Add minimal `static/css/photography.css` (no modes yet — single column is fine)
|
||||
- [x] Manually drop one prepared JPEG into `content/photography/{slug}/` and author its frontmatter by hand
|
||||
- [x] Verify the photography portal appears in nav, the landing page lists the photo, the photo page renders
|
||||
- [x] **Bonus**: license + outbound-links metadata (auto-resolved canonical URLs for known CC variants)
|
||||
|
||||
**Exit criteria**: A photo renders at `/photography/{slug}/` with correct nav portal, basic styling, and is reachable from `/photography/` and `/library.html`. **Met.**
|
||||
|
||||
### Phase 2 — Visual system & toggle ✓
|
||||
|
||||
- [x] Build masonry layout in `photography.css` (CSS grid + computed row spans)
|
||||
- [x] Build uniform-grid mode
|
||||
- [x] Build chronological mode
|
||||
- [x] `static/js/photography-modes.js` — toggle UI, localStorage persistence
|
||||
- [x] Extend `static/js/lightbox.js` with darkroom mode branch (gated on `body[data-page-type="photography"]`)
|
||||
- [x] Body `data-page-type="photography"` attribute (added in Phase 1 as a free hook)
|
||||
|
||||
**Exit criteria**: `/photography/` switches between three modes smoothly; localStorage persists choice; lightbox enters darkroom mode on photography pages only. **Met.** Visual verification pending Levi's run of the dev server.
|
||||
|
||||
### Phase 3 — EXIF, palette, and import pipelines ✓
|
||||
|
||||
- [x] `tools/extract-exif.py` (uses `exiftool` if present, falls back to `Pillow`)
|
||||
- [x] `tools/extract-palette.py` (Python + colorthief)
|
||||
- [x] `tools/import-photo.sh` (resize, strip EXIF from delivered file, write sidecars, scaffold frontmatter)
|
||||
- [x] Wire both into the Makefile, gated on `.venv` (silent-skip pattern matching `embed.py`)
|
||||
- [x] Extend `photographyCtx` to merge sidecar EXIF + palette into the template context (frontmatter wins)
|
||||
- [x] Update `.gitignore` with photography sidecars and RAW patterns
|
||||
- [ ] **Deferred** — Extend `Filters/Images.hs` with the richer photography wrapper
|
||||
- [ ] **Deferred** — Build `partials/photo-meta.html` and `partials/photo-palette.html`
|
||||
|
||||
The two deferred items are now redundant with the Phase 1 template structure: the per-photo metadata block and palette strip live in `templates/photography.html` directly and consume `photographyCtx` fields. Extending `Filters/Images.hs` would only matter if Levi later writes long-form "photo essays" with multiple inline photos that should each carry the rich wrapper — defer until that content type exists.
|
||||
|
||||
**Exit criteria**: A photo with no manually authored camera/lens/exposure/palette frontmatter still displays full metadata + palette strip on its detail page, sourced entirely from sidecars. **Met.** Verified with `canto31.jpg` (no frontmatter `captured:`, `camera:`, etc.; `captured-display` and palette swatches both flowed from sidecar through to rendered HTML).
|
||||
|
||||
### Phase 4 — Map ✓
|
||||
|
||||
- [x] `tools/download-leaflet.sh` — vendors Leaflet 1.9.4 + leaflet.markercluster 1.5.3 to `static/leaflet/`, sha256-pinned
|
||||
- [x] **`build/Photography.hs` map.json rule** (Haskell, not Python — cleaner: Hakyll already has the metadata, no extra dep)
|
||||
- [x] `templates/photography-map.html`
|
||||
- [x] `static/js/photography-map.js`
|
||||
- [x] Add map mode to the toggle on `/photography/`
|
||||
- [x] CartoDB Positron tile attribution wired into the page
|
||||
|
||||
**Exit criteria**: `/photography/map/` shows pins at city precision, marker thumbnails on hover, clicking a pin navigates to the photo. Leaflet JS/CSS load only on the map page. **Met.**
|
||||
|
||||
Implementation notes worth knowing:
|
||||
|
||||
- `map.json` is generated by Hakyll (`photographyMapDataRule` in `build/Photography.hs`), not a Python step. The original spec called for `tools/build-map-data.py`; Haskell turned out cleaner because Hakyll already has every photo's frontmatter loaded and the precision-rounding logic is six lines.
|
||||
- `geo-precision: hidden` photos are dropped from `map.json` entirely.
|
||||
- Pin URLs are stripped of trailing `index.html` so click-through goes directly to the canonical directory URL with no implicit redirect.
|
||||
- UTF-8 in titles is decoded via `Data.Text.Lazy.Encoding.decodeUtf8` rather than `LBS.unpack` to avoid double-encoding bugs (em-dashes, accents, etc.).
|
||||
- Map page is tile-rate-limit-friendly: scroll-zoom is disabled until the user clicks into the map (prevents accidental zoom while scrolling past), tiles cached aggressively (`fetch(..., {cache: 'force-cache'})`).
|
||||
- `leaflet.markercluster` is loaded but degrades gracefully to plain `L.featureGroup` if the plugin failed to load.
|
||||
|
||||
### Phase 5 — Auxiliary surfaces ✓
|
||||
|
||||
- [x] `/photography/by-year/` and `/photography/by-year/{year}/`
|
||||
- [x] `/photography/contact-sheet/`
|
||||
- [x] `/photography/feed.xml` (Atom with thumbnails embedded inline in entry descriptions)
|
||||
- [x] Series landing pages auto-generated from collection directories
|
||||
- [x] Photography shelf on `/library.html`
|
||||
- [x] Tag pages (`/photography/landscape/`, etc.) wired into the existing tag system; the bare `photography` tag is filtered out of the expansion in `Tags.getExpandedTags` to avoid colliding with the section-landing route
|
||||
|
||||
**Exit criteria**: All routing-table URLs from this document resolve and are linked from somewhere reachable. **Met.**
|
||||
|
||||
Implementation notes worth knowing:
|
||||
|
||||
- A new `allPhotoEntries` pattern in `Patterns.hs` enumerates every photographic file (top-level entries + series children); used by surfaces that need every frame (by-year, contact-sheet, feed, map). The original `photographyPattern` (top-level only) feeds the main `/photography/` landing and the library shelf, where a series should appear as a single aggregate card rather than once for the landing plus once per child.
|
||||
- Series detection is purely structural: a directory has siblings ↔ it's a series. No `series: true` flag in frontmatter. `photographyEntryRules` uses a `Set String` of series-slugs computed once at rule-gen time to branch template selection (`photography-series.html` vs `photography.html`).
|
||||
- Sibling photo URLs are canonical directory form: `/photography/<series>/<photo>/`.
|
||||
- The `sectionOwnedTopLevelTags` filter in `Tags.hs` is named generally so other portal tags can be added if their content types ever feed `tagIndexable`.
|
||||
- By-year extraction reads frontmatter `captured:` first, falling back to `date:`. Photos with neither are silently dropped from by-year only; they remain visible everywhere else. (Future improvement: also fall back to the EXIF sidecar's `captured:` so frontmatter-free photos appear automatically.)
|
||||
|
||||
---
|
||||
|
||||
## Open / deferred questions
|
||||
|
||||
These are non-blocking but worth tracking:
|
||||
|
||||
- **Build-time crop** — currently `object-fit: cover`. If the contact-sheet aesthetic feels weak with center-crops, introduce a crop-aware build step (Phase 6 or later).
|
||||
- **`nsfw` gate** — frontmatter field is reserved but no UI is planned in initial phases. Add when first needed.
|
||||
- **Print availability / contact info** — out of scope for v1; revisit if Levi wants to sell prints.
|
||||
- **Diptych / triptych layouts** — frontmatter-driven pairing exists in concept (`pair: other-slug`) but unimplemented; defer until there's actual content that demands it.
|
||||
- **Random photo entry point** — `/photography/random` redirect; trivial JS, defer.
|
||||
- **EXIF reliability for film** — non-concern per design discussion; sidecar/frontmatter merge handles missing EXIF gracefully.
|
||||
- **High-res download links** — out of scope. Originals are not online.
|
||||
|
||||
---
|
||||
|
||||
## References
|
||||
|
||||
- `WRITING.md` — frontmatter conventions for essays (template for the photography schema's structure)
|
||||
- `HOMEPAGE.md` — homepage portal grid
|
||||
- `build/Patterns.hs` — current content pattern definitions
|
||||
- `build/Tags.hs` — slash-hierarchy tag system (reused for photography tags)
|
||||
- `build/Filters/Images.hs` — current image filter (to be extended)
|
||||
- `static/css/gallery.css` — exhibit/overlay system (reference for darkroom lightbox)
|
||||
- `tools/convert-images.sh` — WebP companion generation (reused as-is)
|
||||
|
|
@ -0,0 +1,87 @@
|
|||
# levineuwirth.org
|
||||
|
||||
Personal site of Levi Neuwirth — essays, blog posts, poetry, fiction, and music.
|
||||
Built with [Hakyll](https://jaspervdj.be/hakyll/) and [Pandoc](https://pandoc.org/),
|
||||
with a custom build system in `build/` and a Haskell + JS + Python toolchain.
|
||||
|
||||
## Quickstart
|
||||
|
||||
```sh
|
||||
make build # one-shot production build into _site/
|
||||
make dev # dev build (drafts visible) + local server on :8000
|
||||
make watch # Hakyll live-reload dev server (drafts visible)
|
||||
make clean # cabal run site -- clean
|
||||
make deploy # clean → build → sign → push → rsync to VPS
|
||||
```
|
||||
|
||||
`make build` always runs `make clean` implicitly when invoked from `make deploy`.
|
||||
For day-to-day work, prefer `make dev` (which serves the site on
|
||||
`http://localhost:8000`) or `make watch` (Hakyll's live-reload preview server,
|
||||
which rebuilds on save and serves the site locally).
|
||||
|
||||
**Run `make build` any time you add or replace binary assets** (JPEG/PNG
|
||||
figures, PDFs, music assets). `make dev` and `make watch` skip the
|
||||
`convert-images.sh` / `pdf-thumbs` preprocessing steps, so a fresh JPEG
|
||||
will have no `.webp` companion and a fresh PDF will have no thumbnail
|
||||
until a full `make build` regenerates them. Once the companions exist
|
||||
they survive subsequent `make dev` runs.
|
||||
|
||||
## Optional features
|
||||
|
||||
- **Similar-links and embeddings.** `tools/embed.py` precomputes
|
||||
page-level embeddings for the "Related" block. To enable:
|
||||
|
||||
```sh
|
||||
uv sync # creates .venv with sentence-transformers, faiss-cpu
|
||||
```
|
||||
|
||||
The build silently skips embedding when `.venv` is absent.
|
||||
|
||||
- **Client-side semantic search.** Downloads a quantized ONNX model
|
||||
used by `static/js/semantic-search.js` (run once; files are gitignored):
|
||||
|
||||
```sh
|
||||
make download-model
|
||||
```
|
||||
|
||||
- **Image conversion.** `make build` calls `tools/convert-images.sh` to
|
||||
produce `.webp` companions next to every JPEG/PNG. Requires `cwebp`
|
||||
(`libwebp` on Arch, `webp` on Debian/Ubuntu).
|
||||
|
||||
- **PDF thumbnails.** `make pdf-thumbs` generates first-page thumbnails
|
||||
for PDFs in `static/papers/` using `pdftoppm` (`poppler` on Arch,
|
||||
`poppler-utils` on Debian/Ubuntu). Skipped silently when missing.
|
||||
|
||||
## Configuration
|
||||
|
||||
`.env` (gitignored, copy from `.env.example`) holds the GitHub PAT and
|
||||
the VPS rsync target consumed by `make deploy`. Never commit it.
|
||||
|
||||
## Repository layout
|
||||
|
||||
- `build/` — Haskell build system (Hakyll rules, Pandoc filters, contexts).
|
||||
See `build/Filters/` for the Pandoc AST transforms (sidenotes,
|
||||
wikilinks, transclusion, score embedding, viz, …).
|
||||
- `content/` — authored Markdown (essays, blog, poetry, fiction, music).
|
||||
- `templates/` — Hakyll/Pandoc HTML templates.
|
||||
- `static/` — CSS, JS, fonts, images, vendored PDF.js.
|
||||
- `tools/` — Python tooling (embeddings, importers) and shell scripts.
|
||||
- `data/` — generated and source data (commonplace.yaml, annotations.json,
|
||||
bibliographies, similar-links.json).
|
||||
- `nginx/` — vhost snippets shipped to the VPS (`security-headers.conf`,
|
||||
`static-assets.conf`, `popup-proxy.conf`). The live vhost on the VPS
|
||||
is the source of truth; see `nginx/vhost.conf.example` for the
|
||||
canonical structure and the include order these snippets expect.
|
||||
|
||||
## Architecture pointers
|
||||
|
||||
- `build/Site.hs` is the Hakyll rules entry point.
|
||||
- `build/Patterns.hs` defines canonical content patterns shared by
|
||||
Backlinks, Authors, Tags, and Site.
|
||||
- `build/Compilers.hs` wires the Pandoc filter chain into Hakyll.
|
||||
- `build/Filters/Images.hs` does WebP `<picture>` wrapping; requires
|
||||
the `.webp` companions produced by `tools/convert-images.sh`.
|
||||
|
||||
## License
|
||||
|
||||
See `LICENSE`.
|
||||
File diff suppressed because it is too large
Load Diff
|
|
@ -0,0 +1,14 @@
|
|||
{
|
||||
"url": "https://cr.yp.to/aes-speed.html",
|
||||
"slug": "djb-aes-speed",
|
||||
"title": "Cache-timing attacks on AES (cr.yp.to)",
|
||||
"type": "html",
|
||||
"artifact": "snapshot.html",
|
||||
"sha256": "8da2d5aedeccf9f602e1680631aa77308683803c0cc9b04caad52c7a70c60832",
|
||||
"previous-sha256": "0a50bf6d64b2ec08771d83be5ef47721ecbfc431e3512ff55978e76f452dbd3f",
|
||||
"bytes": 26186,
|
||||
"archived": "2026-05-23",
|
||||
"source-date": null,
|
||||
"snapshot-quality": "ok",
|
||||
"wayback": null
|
||||
}
|
||||
|
|
@ -0,0 +1,470 @@
|
|||
<!-- Saved from https://cr.yp.to/aes-speed.html at 2026-05-23T13:04:33Z using monolith v2.10.1 -->
|
||||
<html><head><meta content="default-src 'none'; img-src data:; style-src 'unsafe-inline'; style-src-elem 'unsafe-inline'; style-src-attr 'unsafe-inline'; font-src data:; script-src 'none'; object-src 'none'; frame-src 'none'" http-equiv="Content-Security-Policy"/><meta content="noindex, noarchive" name="robots"/><link href="data:text/html;base64,PGh0bWw+PGJvZHk+ZmlsZSBkb2VzIG5vdCBleGlzdDwvYm9keT48L2h0bWw+DQo=" rel="icon"/></head><body>
|
||||
<title>AES speed</title>
|
||||
<meta content="aes" name="keywords"/>
|
||||
<a href="https://cr.yp.to/djb.html">D. J. Bernstein</a>
|
||||
<br/><a href="https://cr.yp.to/hash.html">Hash functions and ciphers</a>
|
||||
<h1>AES speed</h1>
|
||||
<b>Update:</b>
|
||||
Peter Schwabe and I now have a paper on this topic:
|
||||
<ul>
|
||||
<li>
|
||||
<a name="aesspeed-paper">[aesspeed]</a>
|
||||
15pp.
|
||||
<a href="https://cr.yp.to/aes-speed/aesspeed-20080926.pdf">(PDF)</a>
|
||||
D. J. Bernstein, Peter Schwabe.
|
||||
New AES software speed records.
|
||||
Document ID: b90c51d2f7eef86b78068511135a231f.
|
||||
URL: https://cr.yp.to/papers.html#aesspeed.
|
||||
Date: 2008.09.26.
|
||||
Supersedes:
|
||||
<a href="https://cr.yp.to/aes-speed/aesspeed-20080908.pdf">(PDF)</a>
|
||||
2008.09.08.
|
||||
</li></ul>
|
||||
The software is now available as part of the
|
||||
<a href="https://cr.yp.to/streamciphers/timings.html#toolkit-estreambench">estreambench</a>
|
||||
toolkit.
|
||||
We have placed the software into the public domain;
|
||||
feel free to integrate it into your own AES applications!
|
||||
<p>
|
||||
Information below this line has not yet been updated.
|
||||
</p><hr/>
|
||||
This document describes various speedups in AES software.
|
||||
This document assumes that
|
||||
the software is going to be used in an application
|
||||
where timing information is <i>not</i> exposed to attackers.
|
||||
<p>
|
||||
The reader is expected to already know the standard structure of AES software:
|
||||
</p><ul>
|
||||
<li>each of the 16 state bytes is used as an index for a table lookup producing a 32-bit word;
|
||||
</li><li>16 xors combine these 16 words and 4 expanded key words into 4 new state words;
|
||||
</li><li>those 4 words are viewed as the starting 16 bytes for the next round.
|
||||
</li></ul>
|
||||
See Section 5.2.1 of "AES Proposal: Rijndael" by Daemen and Rijmen.
|
||||
<h2>Endianness</h2>
|
||||
On a little-endian CPU,
|
||||
extracting the first byte of a 32-bit word
|
||||
is an &0xff arithmetic instruction;
|
||||
on a big-endian CPU,
|
||||
extracting the first byte of a 32-bit word
|
||||
is a >>24 arithmetic instruction.
|
||||
Similar comments apply to the other bytes.
|
||||
<p>
|
||||
One can write AES software
|
||||
that uses arithmetic instructions as if the CPU were little-endian.
|
||||
If the CPU is actually big-endian,
|
||||
the software swaps the bytes of the AES key, input, and output (at run time).
|
||||
The software also swaps the bytes of the table (at compile time),
|
||||
for example by expressing the table as a sequence of 32-bit integers.
|
||||
</p><p>
|
||||
<b>Matched endianness.</b>
|
||||
One can easily eliminate the byte-swapping time for the AES key, input, and output:
|
||||
simply use the appropriate arithmetic instructions
|
||||
for the endianness of the CPU.
|
||||
In this case the table must not be swapped.
|
||||
</p><h2>Table structure</h2>
|
||||
All else being equal, smaller AES tables are faster:
|
||||
they take less time to load into cache and are more likely to stay in cache.
|
||||
Beware that most benchmarking tools preload caches and thus can't see this speedup.
|
||||
<p>
|
||||
Daemen and Rijmen suggest "4 KBytes of tables."
|
||||
There are 4 tables.
|
||||
Each table has 256 words occupying 1024 bytes.
|
||||
The loads are spread evenly across the tables.
|
||||
</p><p>
|
||||
<b>Rotated lookups.</b>
|
||||
Daemen and Rijmen suggest an alternative "with a total table size of 1KByte"
|
||||
but with extra arithmetic.
|
||||
The point is that the tables are rotations of each other:
|
||||
for example,
|
||||
the first word of the first table is (0xc6,0x63,0x63,0xa5),
|
||||
the first word of the second table is (0xa5,0xc6,0x63,0x63),
|
||||
the first word of the third table is (0x63,0xa5,0xc6,0x63),
|
||||
and the first word of the fourth table is (0x63,0x63,0xa5,0xc6).
|
||||
One can store the first table,
|
||||
and simulate a lookup in another table at the cost of an extra rotation.
|
||||
</p><p>
|
||||
<b>Unaligned loads.</b>
|
||||
One can instead use a single 2KB table having 256 8-byte entries
|
||||
such as (0x00,0x63,0xa5,0xc6,0x63,0x63,0xa5,0xc6).
|
||||
There are many reasonable choices of pattern here;
|
||||
what's important is that the pattern includes the desired
|
||||
(0xc6,0x63,0x63,0xa5) and (0xa5,0xc6,0x63,0x63) and so on as substrings.
|
||||
On the Pentium, the PowerPC, et al.,
|
||||
one can load 4-byte words from memory addresses that aren't divisible by 4,
|
||||
and there's no penalty when the word doesn't cross an 8-byte boundary.
|
||||
</p><h2>Masked loads</h2>
|
||||
16 of the 160 table lookups in 10-round AES are masked.
|
||||
The 40 table lookups in 10-round AES key expansion are also masked.
|
||||
The masks are 0x000000ff, 0x0000ff00, 0x00ff0000, and 0xff000000, each used equally often.
|
||||
<p>
|
||||
The simplest way to compute a mask is with an arithmetic instruction: for example, &0xff00.
|
||||
</p><p>
|
||||
<b>Byte loads.</b>
|
||||
One can eliminate 25% of the masks,
|
||||
namely the bottom-byte masks,
|
||||
by combining them with load instructions.
|
||||
All popular CPUs have single-byte-load instructions.
|
||||
</p><p>
|
||||
<b>Two-byte loads.</b>
|
||||
One can eliminate another 25% of the masks
|
||||
on CPUs with two-byte-load instructions.
|
||||
This constrains the table pattern:
|
||||
it's important to have (0x00,0x63) on little-endian CPUs,
|
||||
and (0x63,0x00) on big-endian CPUs.
|
||||
</p><p>
|
||||
<b>Masked tables.</b>
|
||||
One can eliminate all of the masks by precomputing masked tables, using extra table space.
|
||||
The simplest table structure uses a total of 8KB.
|
||||
Two tables, one with entries such as (0x00,0x63,0xa5,0xc6,0x63,0x63,0xa5,0xc6)
|
||||
and another with entries such as (0x00,0x00,0x00,0x00,0x63,0x00,0x00,0x00),
|
||||
use a total of 4KB.
|
||||
In my experience,
|
||||
the cost of larger tables outweighs the benefit of eliminating a few masks.
|
||||
</p><h2>Key expansion</h2>
|
||||
A 4-word (128-bit) key is expanded in 40 steps.
|
||||
Each step produces a new word, totalling 44 words in the expanded key.
|
||||
A step has a byte extraction (see below), a masked load, and two xors.
|
||||
The total work is 40 byte extractions, 40 masked loads, and 80 xors.
|
||||
For comparison, the subsequent work to encrypt a block involves
|
||||
160 byte extractions, 160 loads (of which 16 are masked), and 160 xors.
|
||||
<p>
|
||||
Daemen and Rijmen say (Section 4.3.2)
|
||||
that key expansion involves "almost no computational overhead."
|
||||
Obviously key expansion is less expensive than encrypting a block.
|
||||
On the other hand, the cost of key expansion is still quite noticeable.
|
||||
</p><p>
|
||||
<b>Expanded keys.</b>
|
||||
A typical AES implementation precomputes and stores an expanded key.
|
||||
The 40 byte extractions, 40 masked loads, and 80 xors aren't repeated for every block;
|
||||
they are done only once, along with 44 stores.
|
||||
Each block then involves 44 extra loads for the expanded key.
|
||||
Some stores and loads can be eliminated
|
||||
if many blocks are handled at once
|
||||
and some extra registers are available.
|
||||
</p><p>
|
||||
Long-term storage of an expanded key can slow down applications that handle many keys:
|
||||
the expanded keys take more time to load into cache
|
||||
than the original keys and are less likely to stay in cache.
|
||||
</p><p>
|
||||
<b>Partially expanded keys.</b>
|
||||
An alternative is to precompute and store a partially expanded key,
|
||||
only 14 words instead of 44 words.
|
||||
The partially expanded key consists of words
|
||||
0, 1, 2, 3, 4, 8, 12, 16, 20, 24, 28, 32, 36, 40 from the expanded key.
|
||||
Loading the partially expanded key, and converting it into the fully expanded key,
|
||||
takes only 14 loads and 30 xors.
|
||||
</p><p>
|
||||
One can interpolate between partial expansion and full expansion,
|
||||
using various amounts of storage per key and achieving various balances between load and xor.
|
||||
</p><h2>Index extraction</h2>
|
||||
The 16 xor operations in an AES round
|
||||
produce 4 words in 4 integer registers.
|
||||
The 16 bytes of these words are then extracted and used as indices for the next round.
|
||||
<p>
|
||||
The simplest way to extract 4 bytes is using 6 instructions,
|
||||
namely 3 shifts and 3 bottom-byte extractions:
|
||||
&255;
|
||||
(>>8)&255;
|
||||
(>>16)&255;
|
||||
>>24.
|
||||
</p><p>
|
||||
Using a byte as an index then requires multiplying the byte by a constant
|
||||
that depends on the table structure.
|
||||
Let's assume the 2KB tables described above; then the constant is 8.
|
||||
The multiplications use 4 shifts:
|
||||
<<3;
|
||||
<<3;
|
||||
<<3;
|
||||
<<3.
|
||||
</p><p>
|
||||
<b>Scaled-index loads.</b>
|
||||
Many CPUs can multiply an index register by 8 for free as part of a load.
|
||||
</p><p>
|
||||
<b>Scaled-index extractions.</b>
|
||||
What about CPUs that can't multiply an index register by 8 for free?
|
||||
Two of the multiplications can nevertheless be eliminated,
|
||||
because they can be combined with shifts.
|
||||
The overall extract-and-scale sequence has 8 instructions:
|
||||
(<<3)&2040;
|
||||
(>>5)&2040;
|
||||
(>>13)&2040;
|
||||
(>>21)&2040.
|
||||
The PowerPC has a combined rotate-and-mask instruction,
|
||||
making this sequence take only 4 instructions.
|
||||
</p><p>
|
||||
<b>Scaled tables.</b>
|
||||
One can rotate table entries by 3 bits,
|
||||
reducing the above 8 instructions to 7 instructions.
|
||||
</p><p>
|
||||
<b>Second-byte instructions.</b>
|
||||
The x86 architecture (Pentium, Athlon, etc.)
|
||||
includes a combined (>>8)&255 instruction.
|
||||
This means that extracting 4 bytes takes only 5 instructions:
|
||||
&255;
|
||||
(>>8)&255;
|
||||
>>16;
|
||||
&255;
|
||||
>>8.
|
||||
Alternate 5-instruction sequence:
|
||||
&255;
|
||||
(>>8)&255;
|
||||
>>16;
|
||||
&255;
|
||||
(>>8)&255.
|
||||
</p><p>
|
||||
Of course, the ultimate measure of performance is a cycle count, not an instruction count.
|
||||
Matsui states that the (>>8)&255; instruction is "a bit expensive"
|
||||
on the Pentium 4 Prescott (f33, f34, f41);
|
||||
presumably this means that the instruction takes more cycles than, e.g., a mere &255.
|
||||
But all of the measurements I've seen indicate the opposite.
|
||||
I'm not sure what I'm missing here.
|
||||
</p><p>
|
||||
<b>32-bit shifts on 64-bit architectures.</b>
|
||||
The amd64 architecture (P4E, Athlon 64, Core 2, etc.) can right-shift a 64-bit register,
|
||||
but Matsui comments that this operation is extremely slow on the P4E.
|
||||
It's much better to use the amd64's x86-compatible right-shift instruction;
|
||||
this instruction sets the top 32 bits of its 64-bit input to 0 before shifting.
|
||||
</p><p>
|
||||
<b>Byte extraction via loads.</b>
|
||||
A completely different way to extract 4 bytes is with 1 store and 4 loads.
|
||||
One can mix this with the previous approaches
|
||||
to achieve various balances between load and arithmetic.
|
||||
</p><p>
|
||||
Consider, for example, the UltraSPARC,
|
||||
which has 2 integer units and 1 load/store unit.
|
||||
A traditional sequence of
|
||||
14 partially-expanded-key loads (see below), 30 key-expansion xors,
|
||||
160 scaled-index extractions, 160 table-lookup loads, 160 xors, 16 masks,
|
||||
4 input loads, and 4 output stores
|
||||
occupies a total of 526 integer instructions (at least 263 cycles)
|
||||
and 182 loads (at least 182 cycles).
|
||||
Using loads for some byte extractions,
|
||||
replacing 36 scaled-index extractions with 9 stores and 36 loads,
|
||||
means a total of 454 integer instructions (at least 227 cycles)
|
||||
and 227 loads/stores (at least 227 cycles).
|
||||
</p><h2>Unrolling</h2>
|
||||
A typical 9-iteration AES loop
|
||||
involves 9 increments of a loop index, 9 comparisons, and 9 branches,
|
||||
one of which is mispredicted on most CPUs.
|
||||
The loop index also consumes a register,
|
||||
forcing an extra 9 stores and 9 loads on CPUs that don't have registers to spare.
|
||||
<p>
|
||||
<b>Full unrolling.</b>
|
||||
One can eliminate all of these costs by fully unrolling the loop.
|
||||
Beware, however, that full unrolling costs a few kilobytes of code-cache space.
|
||||
</p><p>
|
||||
<b>Partial unrolling.</b>
|
||||
CPUs are more likely to correctly predict a 4-iteration loop than a 9-iteration loop.
|
||||
</p><h2>Instruction scheduling</h2>
|
||||
The 16 table lookups in an AES round are independent
|
||||
and can be scheduled in many different ways.
|
||||
One can, for example,
|
||||
perform all the table lookups for the first input from bottom byte to top
|
||||
(outputs 0, 3, 2, 1),
|
||||
then perform all the table lookups for the second input from bottom byte to top
|
||||
(outputs 1, 0, 3, 2),
|
||||
then perform all the table lookups for the third input from bottom byte to top
|
||||
(outputs 2, 1, 0, 3),
|
||||
then perform all the table lookups for the fourth input from bottom byte to top
|
||||
(outputs 3, 2, 1, 0).
|
||||
One can, as another example,
|
||||
first perform all the table lookups for the first output in order of the inputs,
|
||||
then perform all the table lookups for the second output in order of the inputs,
|
||||
etc.
|
||||
<p>
|
||||
<b>Maximum parallelism.</b>
|
||||
The overall depth of the AES round is
|
||||
one byte extraction plus one table lookup plus two xors:
|
||||
a mythical CPU offering extensive parallelism
|
||||
could perform all sixteen byte extractions in parallel,
|
||||
then all sixteen table lookups in parallel,
|
||||
then eight xors in parallel,
|
||||
then four xors in parallel.
|
||||
Note that each output is obtained by xor'ing two parallel xor's,
|
||||
rather than by three serial xor's.
|
||||
</p><p>
|
||||
<b>Deferring loads.</b>
|
||||
The amd64 architecture poses several challenges to AES instruction scheduling.
|
||||
First,
|
||||
most integer instructions require the output register to be one of the input registers.
|
||||
Second,
|
||||
typical amd64 CPUs handle a load and xor most efficiently as a unified load-xor,
|
||||
but a unified load-xor gives no opportunity to switch registers.
|
||||
Third,
|
||||
only 4 registers (eax, ebx, ecx, edx) allow second-byte instructions.
|
||||
</p><p>
|
||||
Matsui concludes that, on amd64 (and x86),
|
||||
keeping each round's inputs y0, y1, y2, y3 and outputs z0, z1, z2, z3 in eax, ebx, ecx, edx,
|
||||
to allow second-byte instructions,
|
||||
is "impossible without saving/restoring."
|
||||
But that's incorrect.
|
||||
No extra copies are required.
|
||||
A careful instruction sequence
|
||||
uses the minimal conceivable number of instructions:
|
||||
20 for byte extraction,
|
||||
16 for table lookups,
|
||||
and 4 for handling the expanded key.
|
||||
The idea is to extract all the bytes from an input,
|
||||
freeing the input's register for an output,
|
||||
before doing any table lookups involving that output:
|
||||
</p><ul>
|
||||
<li>Extract the 4 bytes from y0.
|
||||
At this point y1, y2, y3, and the 4 bytes are live.
|
||||
</li><li>Feed 1 byte into z0.
|
||||
At this point y1, y2, y3, z0, and 3 more bytes are live.
|
||||
</li><li>Extract the 4 bytes from y1, immediately feeding 1 into z0.
|
||||
At this point y2, y3, z0, and 6 more bytes are live.
|
||||
</li><li>Feed 2 bytes into z1.
|
||||
At this point y2, y3, z0, z1, and 4 more bytes are live.
|
||||
</li><li>Extract the 4 bytes from y2, immediately feeding 2 into z0 and z1.
|
||||
At this point y3, z0, z1, and 6 more bytes are live.
|
||||
</li><li>Feed 3 bytes into z2.
|
||||
At this point y3, z0, z1, z2, and 3 more bytes are live.
|
||||
</li><li>Extract the 4 bytes from y3, immediately feeding 3 into z0, z1, and z2.
|
||||
At this point z0, z1, z2, and 4 more bytes are live.
|
||||
</li><li>Feed 4 bytes into z3.
|
||||
At this point z0, z1, z2, and z3 are live.
|
||||
</li><li>Handle 4 words of the expanded key.
|
||||
</li></ul>
|
||||
The maximum number of live registers here is 9,
|
||||
fitting easily into the amd64 instruction set.
|
||||
<p>
|
||||
<b>Squeezing inputs and outputs into 7 32-bit registers.</b>
|
||||
The x86 architecture poses an additional challenge to AES instruction scheduling:
|
||||
there are only 7 general-purpose integer registers.
|
||||
</p><p>
|
||||
It's still possible to handle a round with 0 stores, 4 expanded-key loads,
|
||||
and 16 loads for table lookups.
|
||||
The shortest instruction sequence that I know has a total of 46 instructions,
|
||||
6 more than what would be possible with extra registers;
|
||||
1 of the 46 instructions can be eliminated if the key expansion is changed.
|
||||
</p><p>
|
||||
The idea of this instruction sequence
|
||||
is to rotate y0 by 16 bits,
|
||||
use the bottom two bytes of both y0 and y2,
|
||||
and then merge the remaining four bytes of y0 and y2 into a single register
|
||||
(for example, shifting y0 down 16 bits, masking y1, and adding the results),
|
||||
freeing a register at the cost of 3 extra instructions (the rotate, the mask, and the add);
|
||||
splitting 3 load-xor instructions into 3 loads and 3 xors
|
||||
then easily puts all outputs into suitable registers.
|
||||
The rotation can be eliminated if the expanded-key word that corresponds to y0
|
||||
is rotated by 16 bits.
|
||||
</p><h2>Speed reports</h2>
|
||||
Speed reports vary in whether they use CTR, CBC, etc.,
|
||||
and in the exact rules for measuring speeds.
|
||||
The "eSTREAM" cycles/byte counts are
|
||||
for counter-mode AES measured by the eSTREAM benchmarking toolkit;
|
||||
future implementors are encouraged to support the eSTREAM interface for direct comparability.
|
||||
<table border="">
|
||||
<tbody><tr><th>Architecture</th><th>CPU</th><th>eSTREAM cycles/byte</th><th>Ad-hoc cycles/byte</th><th>Software</th></tr>
|
||||
<tr><td>amd64</td><td>Intel Core 2 Duo (6f6)?</td><td></td><td>9.2</td><td>Matsui/Nakajima (CHES 2007)</td></tr>
|
||||
<tr><td>amd64</td><td>AMD Athlon 64 (15,75,2)?</td><td></td><td>10.625 (170/block)</td><td>Matsui (FSE 2006)</td></tr>
|
||||
<tr><td>amd64</td><td>AMD Athlon 64 (15,75,2)?</td><td></td><td>12.4375 (199/block)</td><td>Lipmaa</td></tr>
|
||||
<tr><td>amd64</td><td>Intel Core 2 Duo (6f6); katana</td><td>12.56</td><td></td><td>hongjun/v1/1</td></tr>
|
||||
<tr><td>amd64</td><td>Intel Core 2 Quad Q6600 (6fb); latour</td><td>12.57</td><td></td><td>hongjun/v1/1</td></tr>
|
||||
<tr><td>amd64</td><td>AMD Athlon 64 (15,75,2)?</td><td></td><td>13.125 (210/block)</td><td>Osvik</td></tr>
|
||||
<tr><td>amd64</td><td>AMD Athlon 64 X2 (15,75,2); mace</td><td>13.32</td><td></td><td>hongjun/v1/1</td></tr>
|
||||
<tr><td>amd64</td><td>AMD Opteron 240 (f58); nmisles8amd64</td><td>13.45</td><td></td><td>bernstein/amd64-1/1</td></tr>
|
||||
<tr><td>x86</td><td>Intel Pentium III (68a)?</td><td></td><td>14 (224/block)</td><td>Osvik</td></tr>
|
||||
<tr><td>x86</td><td>AMD Athlon (622)?</td><td></td><td>14.0625 (225/block)</td><td>Osvik</td></tr>
|
||||
<tr><td>x86</td><td>Intel Pentium III (68a)?</td><td></td><td>14.125 (226/block)</td><td>Lipmaa</td></tr>
|
||||
<tr><td>x86</td><td>Intel Pentium 4 (f12)?</td><td></td><td>15 (240/block)</td><td>Osvik</td></tr>
|
||||
<tr><td>x86</td><td>Intel Pentium 4 (f12)?</td><td></td><td>15.875 (254/block)</td><td>Lipmaa</td></tr>
|
||||
<tr><td>x86</td><td>Intel Pentium M (695); whisper</td><td>15.96</td><td></td><td>bernstein/x86-mmx-1/1</td></tr>
|
||||
<tr><td>amd64</td><td>Intel Pentium 4 (f64)?</td><td></td><td>16 (256/block)</td><td>Matsui (FSE 2006)</td></tr>
|
||||
<tr><td>x86</td><td>Intel Pentium III (68a)?</td><td></td><td>16.25 (260/block)</td><td>Gladman</td></tr>
|
||||
<tr><td>amd64</td><td>Intel Pentium D (f64); nmi0161</td><td>16.74</td><td></td><td>bernstein/amd64-2/1</td></tr>
|
||||
<tr><td>amd64</td><td>Intel Pentium D (f64); svlin001</td><td>16.75</td><td></td><td>bernstein/amd64-2/1</td></tr>
|
||||
<tr><td>amd64</td><td>Intel Xeon (f41); nmi0056</td><td>16.75</td><td></td><td>bernstein/amd64-2/1</td></tr>
|
||||
<tr><td>amd64</td><td>Intel Xeon (f4a); nmi0090</td><td>16.77</td><td></td><td>bernstein/amd64-2/1</td></tr>
|
||||
<tr><td>sparc</td><td>Sun UltraSPARC III</td><td></td><td>16.875 (270/block)</td><td>Lipmaa</td></tr>
|
||||
<tr><td>amd64</td><td>Intel Xeon (f41); nmi0057</td><td>16.89</td><td></td><td>bernstein/amd64-2/1</td></tr>
|
||||
<tr><td>amd64</td><td>Intel Pentium D (f64); speed</td><td>16.90</td><td></td><td>bernstein/amd64-2/1</td></tr>
|
||||
<tr><td>amd64</td><td>Intel Pentium D (f64); nmi0104</td><td>16.90</td><td></td><td>bernstein/amd64-2/1</td></tr>
|
||||
<tr><td>amd64</td><td>Intel Pentium D (f64); nmi0241</td><td>16.93</td><td></td><td>bernstein/amd64-2/1</td></tr>
|
||||
<tr><td>ppc64</td><td>IBM POWER5; nmi0154</td><td>16.93</td><td></td><td>bernstein/big-1/1</td></tr>
|
||||
<tr><td>x86</td><td>Intel Pentium 4 (f24); nmi0086</td><td>16.96</td><td></td><td>bernstein/x86-mmx-1/1</td></tr>
|
||||
<tr><td>x86</td><td>Intel Pentium 4 (f12); fireball</td><td>16.98</td><td></td><td>bernstein/x86-mmx-1/1</td></tr>
|
||||
<tr><td>x86</td><td>Intel Pentium 4 (f24); nmitest4</td><td>17.01</td><td></td><td>bernstein/x86-mmx-1/1</td></tr>
|
||||
<tr><td>ppc64</td><td>IBM PowerPC G5 970; nmi0048</td><td>17.17</td><td></td><td>bernstein/big-1/1</td></tr>
|
||||
<tr><td>x86</td><td>Intel Pentium 2 (652); boris</td><td>17.33</td><td></td><td>bernstein/x86-mmx-1/1</td></tr>
|
||||
<tr><td>x86</td><td>Intel Pentium 3 (68a)</td><td>17.49</td><td></td><td>Bernstein aes-128/x86-mmx-1</td></tr>
|
||||
<tr><td>x86</td><td>Intel Pentium 3 (672); orpheus</td><td>17.55</td><td></td><td>bernstein/x86-mmx-1/1</td></tr>
|
||||
<tr><td>x86</td><td>Intel Pentium M (6d8)</td><td>17.57</td><td></td><td>Wu v0/1</td></tr>
|
||||
<tr><td>x86</td><td>Intel Pentium 4 (f33)?</td><td></td><td>17.75 (284/block)</td><td>Matsui/Fukuda (FSE 2005)</td></tr>
|
||||
<tr><td>x86</td><td>Intel Xeon (f29); nmibuild40</td><td>17.79</td><td></td><td>bernstein/x86-mmx-1/1</td></tr>
|
||||
<tr><td>x86</td><td>Intel Xeon (f27); nmi0059</td><td>17.79</td><td></td><td>bernstein/x86-mmx-1/1</td></tr>
|
||||
<tr><td>x86</td><td>Intel Xeon (f25); nmibuild16</td><td>17.79</td><td></td><td>bernstein/x86-mmx-1/1</td></tr>
|
||||
<tr><td>x86</td><td>Intel Xeon (f25); nmi0013</td><td>17.79</td><td></td><td>bernstein/x86-mmx-1/1</td></tr>
|
||||
<tr><td>x86</td><td>Intel Xeon (f29); nmi0059</td><td>17.80</td><td></td><td>bernstein/x86-mmx-1/1</td></tr>
|
||||
<tr><td>x86</td><td>Intel Xeon (f29); nmibuild17</td><td>17.81</td><td></td><td>bernstein/x86-mmx-1/1</td></tr>
|
||||
<tr><td>x86</td><td>Intel Xeon (f25); nmibuild15</td><td>17.82</td><td></td><td>bernstein/x86-mmx-1/1</td></tr>
|
||||
<tr><td>x86</td><td>Intel Xeon (f25); nmibuild26</td><td>17.83</td><td></td><td>bernstein/x86-mmx-1/1</td></tr>
|
||||
<tr><td>x86</td><td>Intel Xeon (f25); nmibuild21</td><td>17.83</td><td></td><td>bernstein/x86-mmx-1/1</td></tr>
|
||||
<tr><td>x86</td><td>Intel Xeon (f25); nmi0036</td><td>17.84</td><td></td><td>bernstein/x86-mmx-1/1</td></tr>
|
||||
<tr><td>x86</td><td>Intel Xeon (f25); nmibuild22</td><td>17.84</td><td></td><td>bernstein/x86-mmx-1/1</td></tr>
|
||||
<tr><td>x86</td><td>AMD Athlon (622); thoth</td><td>18.38</td><td></td><td>bernstein/x86-mmx-1/1</td></tr>
|
||||
<tr><td>ppc32</td><td>IBM POWER4; nmibuild14</td><td>18.55</td><td></td><td>bernstein/little-1/1</td></tr>
|
||||
<tr><td>x86</td><td>Intel Xeon (f41); nmi0079</td><td>18.88</td><td></td><td>bernstein/x86-mmx-1/1</td></tr>
|
||||
<tr><td>x86</td><td>Intel Xeon (f41); nmi0062</td><td>18.89</td><td></td><td>bernstein/x86-mmx-1/1</td></tr>
|
||||
<tr><td>amd64</td><td>Intel Core 2 Duo (6f6)</td><td></td><td>18.9</td><td>OpenSSL 0.9.8e</td></tr>
|
||||
<tr><td>x86</td><td>Intel Xeon (f41); nmi0061</td><td>18.91</td><td></td><td>bernstein/x86-mmx-1/1</td></tr>
|
||||
<tr><td>x86</td><td>Intel Pentium 4 (f41); svlin002</td><td>18.94</td><td></td><td>bernstein/x86-mmx-1/1</td></tr>
|
||||
<tr><td>x86</td><td>Intel Xeon (f41); nmi0076</td><td>18.96</td><td></td><td>bernstein/x86-mmx-1/1</td></tr>
|
||||
<tr><td>x86</td><td>Intel Xeon (f4a); nmi0102</td><td>18.97</td><td></td><td>bernstein/x86-mmx-1/1</td></tr>
|
||||
<tr><td>x86</td><td>Intel Xeon (f41); nmi0060</td><td>18.97</td><td></td><td>bernstein/x86-mmx-1/1</td></tr>
|
||||
<tr><td>x86</td><td>Intel Xeon (f41); nmi0063</td><td>18.95</td><td></td><td>bernstein/x86-mmx-1/1</td></tr>
|
||||
<tr><td>x86</td><td>Intel Pentium 3 (68a)</td><td>19.06</td><td></td><td>Wu v1/1</td></tr>
|
||||
<tr><td>ppc32</td><td>Motorola PowerPC G4 7410; gggg</td><td>19.11</td><td></td><td>bernstein/big-1/1</td></tr>
|
||||
<tr><td>amd64</td><td>Intel Core 2 Duo (6f6)</td><td></td><td>19.5</td><td>OpenSSL 0.9.8a</td></tr>
|
||||
<tr><td>x86</td><td>AMD Athlon (622)?</td><td></td><td>19.9375 (319/block)</td><td>Lipmaa</td></tr>
|
||||
<tr><td>x86</td><td>Intel Pentium 1 (52c)</td><td></td><td>20 (320/block)</td><td>Lipmaa</td></tr>
|
||||
<tr><td>sparc</td><td>Sun UltraSPARC III</td><td>20.75</td><td></td><td>Bernstein big-1/1</td></tr>
|
||||
<tr><td>amd64</td><td>AMD Athlon 64 (15,75,2)</td><td></td><td>20.9</td><td>OpenSSL 0.9.8e</td></tr>
|
||||
<tr><td>ppc32</td><td>Motorola PowerPC G4 7400; nmi0042</td><td>20.92</td><td></td><td>bernstein/big-1/1</td></tr>
|
||||
<tr><td>x86</td><td>Intel Pentium M (6d8)</td><td></td><td>21</td><td>OpenSSL 0.9.8a</td></tr>
|
||||
<tr><td>x86</td><td>Intel Pentium D (f47); shell</td><td>21.58</td><td></td><td>bernstein/x86-mmx-1/1</td></tr>
|
||||
<tr><td>x86</td><td>AMD Athlon (622)</td><td></td><td>22</td><td>OpenSSL 0.9.8a</td></tr>
|
||||
<tr><td>x86</td><td>Intel Pentium 4 (f29)</td><td></td><td>22</td><td>OpenSSL 0.9.8b</td></tr>
|
||||
<tr><td>amd64</td><td>AMD Athlon 64 (15,75,2)?</td><td></td><td>23.5</td><td>OpenSSL 0.9.7e</td></tr>
|
||||
<tr><td>x86</td><td>Intel Pentium 4 (f41)</td><td></td><td>23.5</td><td>OpenSSL 0.9.8a</td></tr>
|
||||
<tr><td>x86</td><td>Intel Pentium 3 (672); orpheus</td><td></td><td>23.62</td><td>OpenSSL 0.9.8e</td></tr>
|
||||
<tr><td>ppc32</td><td>Motorola PowerPC G4 7410</td><td></td><td>24.0625 (385/block)</td><td>Ahrens</td></tr>
|
||||
<tr><td>x86</td><td>Intel Pentium 4 (f12)</td><td></td><td>24.4</td><td>OpenSSL 0.9.8a</td></tr>
|
||||
<tr><td>sparc</td><td>Sun UltraSPARC III</td><td></td><td>25</td><td>OpenSSL</td></tr>
|
||||
<tr><td>ppc32</td><td>Motorola PowerPC G4 7410</td><td></td><td>25.0625 (401/block)</td><td>Ahrens</td></tr>
|
||||
<tr><td>x86</td><td>Intel Core Duo; nmi0068</td><td>25.74</td><td></td><td>gladman/1</td></tr>
|
||||
<tr><td>amd64</td><td>Intel Pentium D (f64); speed</td><td></td><td>27.33</td><td>OpenSSL 0.9.8e</td></tr>
|
||||
<tr><td>ppc32</td><td>Motorola PowerPC G4 7410; gggg</td><td></td><td>29.32</td><td>OpenSSL 0.9.8c</td></tr>
|
||||
<tr><td>sparcv9</td><td>Sun UltraSPARC III; nmi0051</td><td>29.45</td><td></td><td>bernstein/big-1/1</td></tr>
|
||||
<tr><td>sparcv9</td><td>Sun UltraSPARC III; nmisolaris10</td><td>29.46</td><td></td><td>bernstein/big-1/1</td></tr>
|
||||
<tr><td>ppc64</td><td>IBM Cell PPE; nmips3</td><td>35.20</td><td></td><td>bernstein/big-1/1</td></tr>
|
||||
<tr><td>amd64</td><td>Intel Pentium 4 (f64)</td><td></td><td>37</td><td>OpenSSL 0.9.7f</td></tr>
|
||||
<tr><td>x86</td><td>Intel Pentium 4 (f29)</td><td></td><td>39</td><td>OpenSSL 0.9.7e</td></tr>
|
||||
<tr><td>sparc</td><td>Sun UltraSPARC III</td><td></td><td>46.875 (750/block)</td><td>Bassham</td></tr>
|
||||
<tr><td>x86</td><td>Intel Pentium 1 (52c); cruncher</td><td>38.20</td><td></td><td>hongjun/v1/1</td></tr>
|
||||
</tbody></table>
|
||||
<p>
|
||||
Regarding amd64 Intel Pentium 4,
|
||||
Matsui writes:
|
||||
"The number of memory reads
|
||||
for one block encryption of AES
|
||||
is 4 (for plaintext loads)
|
||||
+ 11 x 4 (for subkey loads)
|
||||
+ 16 x 10 (for table lookups)
|
||||
= 208,
|
||||
which means that Pentium 4 takes at least 208 cycles/block for one block encryption."
|
||||
But this lower bound ignores the possibility of loading partially expanded keys,
|
||||
saving as many as 30 loads,
|
||||
and using 64-bit loads for keys and plaintext,
|
||||
saving 9 more loads.
|
||||
</p><p>
|
||||
Regarding amd64 AMD Athlon 64,
|
||||
Matsui writes:
|
||||
"Considering an instruction latency of Athlon 64, the theoretical limit of AES
|
||||
performance on this processor seems around 16 cycles/round = 160 cycles/block.
|
||||
Our result is hence reaching closely this limit."
|
||||
|
||||
|
||||
</p></body></html>
|
||||
|
|
@ -0,0 +1,32 @@
|
|||
# archive/manifest.yaml — curated list of works to preserve.
|
||||
# Edited by hand. Tools never write to this file. See ARCHIVE.md.
|
||||
#
|
||||
# Per-artifact cap: 25 MB. Above that, archive.py warns and skips the fetch;
|
||||
# commit an oversize artifact deliberately with `git add -f`.
|
||||
#
|
||||
# To evict an entry, see archive/removed.yaml — record there FIRST, then
|
||||
# delete the line here, then run `make archive-gc`.
|
||||
|
||||
- url: "https://nvlpubs.nist.gov/nistpubs/FIPS/NIST.FIPS.203.pdf"
|
||||
slug: nist-fips-203
|
||||
title: "FIPS 203 — Module-Lattice-Based Key-Encapsulation Mechanism Standard"
|
||||
type: pdf
|
||||
aliases:
|
||||
# The DOI form the simd paper cites; URL normalisation cannot equate
|
||||
# a DOI with the landing URL it resolves to, so it is authored here.
|
||||
- "https://doi.org/10.6028/NIST.FIPS.203"
|
||||
tags: [research]
|
||||
note: >
|
||||
The ML-KEM standard. Cited in the SIMD / post-quantum systems work;
|
||||
archived so the citation survives any future reorganization of the
|
||||
NIST publications site.
|
||||
|
||||
- url: "https://cr.yp.to/aes-speed.html"
|
||||
slug: djb-aes-speed
|
||||
title: "Cache-timing attacks on AES (cr.yp.to)"
|
||||
# type: html — auto-detected from the .html extension; no override needed.
|
||||
tags: [research]
|
||||
note: >
|
||||
Bernstein's cache-timing-attacks page, cited in the SIMD work. The
|
||||
Phase 2 bootstrap entry: a stable, JavaScript-free static page, so its
|
||||
monolith snapshot is reproducible and classifies cleanly as `ok`.
|
||||
|
|
@ -0,0 +1,14 @@
|
|||
{
|
||||
"url": "https://nvlpubs.nist.gov/nistpubs/FIPS/NIST.FIPS.203.pdf",
|
||||
"slug": "nist-fips-203",
|
||||
"title": "FIPS 203 — Module-Lattice-Based Key-Encapsulation Mechanism Standard",
|
||||
"type": "pdf",
|
||||
"artifact": "document.pdf",
|
||||
"sha256": "fe1f12f32a7e44ec9fdebbf400cda843a40b506dee676725234dc6f7923b6cac",
|
||||
"previous-sha256": null,
|
||||
"bytes": 1252341,
|
||||
"archived": "2026-05-22",
|
||||
"source-date": null,
|
||||
"snapshot-quality": "ok",
|
||||
"wayback": "http://web.archive.org/web/20260515100505/https://nvlpubs.nist.gov/nistpubs/FIPS/NIST.FIPS.203.pdf"
|
||||
}
|
||||
Binary file not shown.
|
|
@ -0,0 +1,19 @@
|
|||
# archive/removed.yaml — record of evicted archive entries.
|
||||
#
|
||||
# Append an entry here BEFORE deleting its line from manifest.yaml, then
|
||||
# run `make archive-gc`. The GC deletes only archive/<slug>/ directories
|
||||
# whose slug is recorded here; an orphaned directory absent from this file
|
||||
# is reported, never deleted. See ARCHIVE.md § Eviction & removal.
|
||||
#
|
||||
# Schema (all fields but `note` required):
|
||||
# url: original URL at time of removal
|
||||
# slug: the archive/<slug>/ directory archive-gc may delete
|
||||
# removed: ISO date of removal
|
||||
# reason: takedown | author-request | legal | quality
|
||||
# note: optional free-text context
|
||||
#
|
||||
# This is not a hostile-tracking list — it exists so GC knows what is safe
|
||||
# to delete, re-adding a removed URL is surfaced loudly, and the link-rot
|
||||
# scanner and `archive-suggest` skip removed works.
|
||||
|
||||
[]
|
||||
|
|
@ -0,0 +1,643 @@
|
|||
{-# LANGUAGE GHC2021 #-}
|
||||
{-# LANGUAGE OverloadedStrings #-}
|
||||
-- | Archive section — the link-archiving system. Phases 1-2: PDF and HTML.
|
||||
--
|
||||
-- Authored input: archive/manifest.yaml (one line per archived link)
|
||||
-- Generated, committed: archive/<slug>/{document.pdf | snapshot.html}
|
||||
-- + PROVENANCE.json
|
||||
-- Generated, gitignored: archive/<slug>/{document,snapshot}.txt
|
||||
-- + data/archive-index.json
|
||||
--
|
||||
-- @tools/archive.py fetch@ runs before the Hakyll build: it downloads
|
||||
-- PDFs / snapshots HTML pages with @monolith@, extracts text, and writes
|
||||
-- each PROVENANCE.json. This module then routes the artifacts and renders
|
||||
-- one @/archive/<slug>/@ page per entry plus the @/archive/@ index.
|
||||
--
|
||||
-- An entry whose artifact has not been fetched (no PROVENANCE.json, or
|
||||
-- no artifact file on disk) is skipped — it produces no page, and an
|
||||
-- orphaned @archive/<slug>/@ directory with no manifest line is inert
|
||||
-- (no page, not deployed). Artifact-integrity (SHA-256) verification
|
||||
-- runs on both sides: @archive.py fetch@ re-hashes before the Hakyll
|
||||
-- build, and 'verifyArtifactSha' (below) re-hashes again in
|
||||
-- 'loadArchiveEntries' — so the guarantee holds even when @archive.py@
|
||||
-- does not run first (no @.venv@, a direct @cabal run site -- build@,
|
||||
-- or a deploy host without the Python toolchain).
|
||||
--
|
||||
-- See @ARCHIVE.md@ at the repo root for the full design and phase plan.
|
||||
module Archive (archiveRules, archiveBuildStats) where
|
||||
|
||||
import Control.Exception (SomeException, catch)
|
||||
import Control.Monad (filterM, forM, forM_, when)
|
||||
import Data.Function (on)
|
||||
import Data.List (groupBy, intercalate, sort, sortBy)
|
||||
import qualified Data.Map.Strict as Map
|
||||
import Data.Maybe (catMaybes, fromMaybe)
|
||||
import Data.Ord (Down (..), comparing)
|
||||
import qualified Data.Set as Set
|
||||
import qualified Data.Text as T
|
||||
import Data.Time (Day, diffDays, fromGregorian,
|
||||
getCurrentTime, utctDay)
|
||||
import qualified Data.Aeson as A
|
||||
import Data.Aeson ((.:), (.:?))
|
||||
import qualified Data.ByteString.Lazy.Char8 as LBS
|
||||
import qualified Data.Yaml as Y
|
||||
import System.Directory (doesDirectoryExist, doesFileExist,
|
||||
listDirectory)
|
||||
import System.Exit (exitFailure)
|
||||
import System.IO (hPutStrLn, readFile', stderr)
|
||||
import System.Process (readProcess)
|
||||
import Text.Read (readMaybe)
|
||||
import Hakyll
|
||||
import Contexts (siteCtx)
|
||||
import Backlinks (referencedByField)
|
||||
import SimilarLinks (similarLinksField)
|
||||
import ArchiveIndex (ArchiveStatus (..), statusName,
|
||||
archiveStatusForSlug, normalizeUrl)
|
||||
|
||||
-- ---------------------------------------------------------------------------
|
||||
-- Data model
|
||||
-- ---------------------------------------------------------------------------
|
||||
|
||||
-- | One authored entry in @archive/manifest.yaml@ — only the fields this
|
||||
-- module consumes. @title:@, @type:@ and @tags:@ are read by
|
||||
-- @tools/archive.py@ (title and type fold into PROVENANCE.json; tags are
|
||||
-- Phase 4) and need no Haskell-side binding.
|
||||
data ManifestEntry = ManifestEntry
|
||||
{ meUrl :: String
|
||||
, meAliases :: [String] -- ^ authored equivalent URLs of the
|
||||
-- same work (e.g. its DOI form);
|
||||
-- matching metadata, not identity
|
||||
, meNote :: Maybe String
|
||||
, mePaywalled :: Bool
|
||||
, meVisibility :: String -- ^ "public" (default) | "private"
|
||||
}
|
||||
|
||||
instance A.FromJSON ManifestEntry where
|
||||
parseJSON = A.withObject "ManifestEntry" $ \o -> do
|
||||
url <- o .: "url"
|
||||
aliases <- fromMaybe [] <$> o .:? "aliases"
|
||||
note <- o .:? "note"
|
||||
paywalled <- fromMaybe False <$> o .:? "paywalled"
|
||||
visibility <- fromMaybe "public" <$> o .:? "visibility"
|
||||
-- A publication/privacy field must fail closed: an unknown value
|
||||
-- (e.g. a typo'd "privte") would otherwise be treated as public
|
||||
-- and publish an artifact the author intended to keep offline.
|
||||
when (visibility `notElem` ["public", "private"]) $ fail $
|
||||
"manifest entry " ++ url
|
||||
++ ": visibility must be \"public\" or \"private\", got "
|
||||
++ show visibility
|
||||
return (ManifestEntry url aliases note paywalled visibility)
|
||||
|
||||
newtype RemovedEntry = RemovedEntry { reUrl :: String }
|
||||
|
||||
instance A.FromJSON RemovedEntry where
|
||||
parseJSON = A.withObject "RemovedEntry" $ \o ->
|
||||
RemovedEntry <$> o .: "url"
|
||||
|
||||
-- | One generated @archive/<slug>/PROVENANCE.json@ — the immutable
|
||||
-- record of an archival event, written by @tools/archive.py@.
|
||||
data Provenance = Provenance
|
||||
{ pvUrl :: String
|
||||
, pvSlug :: String
|
||||
, pvTitle :: String
|
||||
, pvType :: String -- ^ "pdf" | "html"
|
||||
, pvArtifact :: String -- ^ "document.pdf" | "snapshot.html"
|
||||
, pvSha256 :: String
|
||||
, pvBytes :: Integer
|
||||
, pvArchived :: String
|
||||
, pvQuality :: String -- ^ "ok" | "degraded" | "js-required"
|
||||
, pvWayback :: Maybe String
|
||||
}
|
||||
|
||||
instance A.FromJSON Provenance where
|
||||
parseJSON = A.withObject "Provenance" $ \o -> Provenance
|
||||
<$> o .: "url"
|
||||
<*> o .: "slug"
|
||||
<*> o .: "title"
|
||||
<*> o .: "type"
|
||||
<*> o .: "artifact"
|
||||
<*> o .: "sha256"
|
||||
<*> o .: "bytes"
|
||||
<*> o .: "archived"
|
||||
<*> (fromMaybe "ok" <$> o .:? "snapshot-quality")
|
||||
<*> o .:? "wayback"
|
||||
|
||||
-- | A renderable archive entry: the authored manifest line joined with
|
||||
-- its generated provenance and extracted full text. @aeTextId@ is the
|
||||
-- on-disk path of the extracted-text sidecar when it exists (it is
|
||||
-- gitignored, so a no-@.venv@ build may lack it).
|
||||
data ArchiveEntry = ArchiveEntry
|
||||
{ aeManifest :: ManifestEntry
|
||||
, aeProv :: Provenance
|
||||
, aeFulltext :: String
|
||||
, aeTextId :: Maybe FilePath
|
||||
, aeStatus :: ArchiveStatus -- ^ link-rot status of the original
|
||||
}
|
||||
|
||||
-- | The extracted-text sidecar name for an artifact type.
|
||||
textFileFor :: Provenance -> String
|
||||
textFileFor pv
|
||||
| pvType pv == "html" = "snapshot.txt"
|
||||
| otherwise = "document.txt"
|
||||
|
||||
-- | True for a @visibility: private@ entry — kept in-repo as a local
|
||||
-- preservation copy, but its artifact is never routed to @_site/@ and
|
||||
-- its extracted text is never rendered into the page.
|
||||
isPrivate :: ArchiveEntry -> Bool
|
||||
isPrivate = (== "private") . meVisibility . aeManifest
|
||||
|
||||
-- ---------------------------------------------------------------------------
|
||||
-- Rule-generation-time IO (runs inside 'preprocess')
|
||||
-- ---------------------------------------------------------------------------
|
||||
|
||||
manifestPath, removedPath :: FilePath
|
||||
manifestPath = "archive/manifest.yaml"
|
||||
removedPath = "archive/removed.yaml"
|
||||
|
||||
-- | Read @archive/manifest.yaml@. An absent file yields an empty list
|
||||
-- (the archive degrades to invisible, matching the @.venv@-gated
|
||||
-- silent-skip convention). A *parse error on a present file* halts the
|
||||
-- build: the file exists but is broken — degrading to invisible would
|
||||
-- swallow real errors like a typo'd @visibility@ value or a malformed
|
||||
-- entry, both of which are publication-relevant.
|
||||
readManifest :: IO [ManifestEntry]
|
||||
readManifest = do
|
||||
exists <- doesFileExist manifestPath
|
||||
if not exists
|
||||
then return []
|
||||
else do
|
||||
parsed <- Y.decodeFileEither manifestPath
|
||||
case parsed of
|
||||
-- An empty or all-comments file decodes as YAML @Null@,
|
||||
-- not as a list. That is the legitimate "drained to zero
|
||||
-- entries" state, not a broken file — treat it as the
|
||||
-- empty manifest the absent-file branch already supports.
|
||||
Right A.Null -> return []
|
||||
Right v -> case A.fromJSON v of
|
||||
A.Success es -> return es
|
||||
A.Error msg -> fatal msg
|
||||
Left e -> fatal (show e)
|
||||
where
|
||||
fatal msg = do
|
||||
hPutStrLn stderr $ "[archive] FATAL: manifest.yaml: " ++ msg
|
||||
exitFailure
|
||||
|
||||
readRemovedUrls :: IO (Set.Set T.Text)
|
||||
readRemovedUrls = do
|
||||
exists <- doesFileExist removedPath
|
||||
if not exists
|
||||
then return Set.empty
|
||||
else do
|
||||
parsed <- Y.decodeFileEither removedPath
|
||||
case parsed of
|
||||
Right entries -> return . Set.fromList $
|
||||
map (normalizeUrl . T.pack . reUrl) (entries :: [RemovedEntry])
|
||||
Left e -> do
|
||||
hPutStrLn stderr $
|
||||
"[archive] FATAL: removed.yaml: " ++ show e
|
||||
exitFailure
|
||||
|
||||
validateManifestEntries :: [ManifestEntry] -> Set.Set T.Text -> IO ()
|
||||
validateManifestEntries manifest removed = go Map.empty manifest
|
||||
where
|
||||
go _ [] = return ()
|
||||
go seen (entry : rest) = do
|
||||
-- The canonical URL and every authored alias must be unique
|
||||
-- across the manifest and absent from removed.yaml — the same
|
||||
-- equivalence the Python pre-scan enforces, so a direct Hakyll
|
||||
-- build fails just as closed. Within-entry duplicates (an alias
|
||||
-- normalising to its own entry's URL) are deduped, not errors.
|
||||
let url = meUrl entry
|
||||
norms = Set.toList . Set.fromList $
|
||||
map (normalizeUrl . T.pack) (url : meAliases entry)
|
||||
forM_ norms $ \norm -> do
|
||||
when (norm `Set.member` removed) $ do
|
||||
hPutStrLn stderr $
|
||||
"[archive] FATAL: manifest entry " ++ show url
|
||||
++ " matches removed.yaml (directly or via `aliases:`); "
|
||||
++ "refusing to publish a deliberately removed work."
|
||||
exitFailure
|
||||
case Map.lookup norm seen of
|
||||
Just prior -> do
|
||||
hPutStrLn stderr $
|
||||
"[archive] FATAL: manifest entries " ++ show prior
|
||||
++ " and " ++ show url ++ " normalise to the same "
|
||||
++ "archive target (directly or via `aliases:`)."
|
||||
exitFailure
|
||||
Nothing -> return ()
|
||||
go (foldr (\n m -> Map.insert n url m) seen norms) rest
|
||||
|
||||
-- | Scan @archive/<slug>/PROVENANCE.json@ into a @url -> (slug, Provenance)@
|
||||
-- map. The directory name is the slug; the join key is the URL.
|
||||
readProvenances :: IO (Map.Map String (String, Provenance))
|
||||
readProvenances = do
|
||||
exists <- doesDirectoryExist "archive"
|
||||
if not exists
|
||||
then return Map.empty
|
||||
else do
|
||||
names <- listDirectory "archive"
|
||||
entries <- forM names $ \name -> do
|
||||
let provPath = "archive/" ++ name ++ "/PROVENANCE.json"
|
||||
isFile <- doesFileExist provPath
|
||||
if not isFile
|
||||
then return Nothing
|
||||
else do
|
||||
decoded <- A.eitherDecodeFileStrict' provPath
|
||||
case decoded of
|
||||
Right p -> return (Just (pvUrl p, (name, p)))
|
||||
Left e -> do
|
||||
hPutStrLn stderr $
|
||||
"[archive] FATAL: " ++ provPath ++ ": " ++ show e
|
||||
exitFailure
|
||||
return (Map.fromList (catMaybes entries))
|
||||
|
||||
-- | Read a file, returning "" on any error (e.g. an absent text sidecar).
|
||||
readFileSafe :: FilePath -> IO String
|
||||
readFileSafe path =
|
||||
catch (readFile' path) (\(_ :: SomeException) -> return "")
|
||||
|
||||
-- | Verify a committed artifact's SHA-256 against its recorded value.
|
||||
-- The build halts with a clear message on mismatch — so the integrity
|
||||
-- guarantee holds even when @tools/archive.py@ does not run first
|
||||
-- (e.g. no @.venv@, or a direct @cabal run site -- build@), and a
|
||||
-- tampered or corrupted artifact can never be deployed.
|
||||
--
|
||||
-- Shells out to @sha256sum@ (GNU coreutils — same toolchain the rest of
|
||||
-- the build assumes); a missing or non-zero @sha256sum@ surfaces as an
|
||||
-- exception that also halts the build.
|
||||
verifyArtifactSha :: String -> FilePath -> String -> IO ()
|
||||
verifyArtifactSha slug path expected = do
|
||||
out <- readProcess "sha256sum" [path] ""
|
||||
let actual = takeWhile (/= ' ') out
|
||||
when (actual /= expected) $ do
|
||||
hPutStrLn stderr $
|
||||
"[archive] FATAL: " ++ slug ++ ": " ++ path
|
||||
++ " SHA-256 mismatch (recorded " ++ expected
|
||||
++ ", found " ++ actual
|
||||
++ "). The committed artifact is corrupt or was replaced; "
|
||||
++ "halting build."
|
||||
exitFailure
|
||||
|
||||
-- | Join the authored manifest with generated provenance. A manifest
|
||||
-- entry with no matching provenance — or whose artifact is not on disk
|
||||
-- — is dropped, so it produces no page.
|
||||
loadArchiveEntries :: IO [ArchiveEntry]
|
||||
loadArchiveEntries = do
|
||||
manifest <- readManifest
|
||||
removed <- readRemovedUrls
|
||||
validateManifestEntries manifest removed
|
||||
provByUrl <- readProvenances
|
||||
-- Join on normalised URLs, like every other URL comparison in the
|
||||
-- archive system: editing a manifest URL to a normalisation-
|
||||
-- equivalent form (http->https, trailing slash, tracking params)
|
||||
-- must keep matching its provenance — an exact-string join would
|
||||
-- silently unpublish the page while ArchiveIndex's normalised
|
||||
-- filter keeps links pointing at it. Key collisions can't occur:
|
||||
-- validateManifestEntries rejects normalised duplicates.
|
||||
let normKey = T.unpack . normalizeUrl . T.pack
|
||||
provByNorm = Map.mapKeys normKey provByUrl
|
||||
fmap catMaybes $ forM manifest $ \me ->
|
||||
case Map.lookup (normKey (meUrl me)) provByNorm of
|
||||
Nothing -> return Nothing
|
||||
Just (slug, pv) -> do
|
||||
let dir = "archive/" ++ slug
|
||||
txtPath = dir ++ "/" ++ textFileFor pv
|
||||
let artPath = dir ++ "/" ++ pvArtifact pv
|
||||
artifactThere <- doesFileExist artPath
|
||||
if not artifactThere
|
||||
then do
|
||||
hPutStrLn stderr $
|
||||
"[archive] FATAL: " ++ slug ++ ": " ++ artPath
|
||||
++ " is missing although PROVENANCE.json exists; "
|
||||
++ "restore the committed artifact before building."
|
||||
exitFailure
|
||||
else do
|
||||
verifyArtifactSha slug artPath (pvSha256 pv)
|
||||
txtThere <- doesFileExist txtPath
|
||||
txt <- if txtThere then readFileSafe txtPath
|
||||
else return ""
|
||||
return $ Just ArchiveEntry
|
||||
{ aeManifest = me
|
||||
, aeProv = pv
|
||||
, aeFulltext = txt
|
||||
, aeTextId = if txtThere then Just txtPath
|
||||
else Nothing
|
||||
, aeStatus = archiveStatusForSlug slug
|
||||
}
|
||||
|
||||
-- ---------------------------------------------------------------------------
|
||||
-- Rules
|
||||
-- ---------------------------------------------------------------------------
|
||||
|
||||
-- | All archive rules. Called once from 'Site.rules'.
|
||||
--
|
||||
-- The manifest is read here in 'preprocess' (and 'ArchiveIndex' reads
|
||||
-- its sidecars in once-per-process CAFs), so archive state is fixed at
|
||||
-- rule-generation time: under @site watch@, edits to @manifest.yaml@,
|
||||
-- @removed.yaml@, or the regenerated state JSONs are not picked up
|
||||
-- until the process restarts. One-shot builds are unaffected.
|
||||
archiveRules :: Rules ()
|
||||
archiveRules = do
|
||||
entries <- preprocess loadArchiveEntries
|
||||
|
||||
-- Raw artifacts: the PDF / HTML snapshot of every *public* entry,
|
||||
-- served at its own path (/archive/<slug>/...). Routing this explicit
|
||||
-- list rather than a glob means a `visibility: private` entry's
|
||||
-- artifact is never deployed, and an orphan directory's artifact
|
||||
-- (no manifest line) is not deployed either.
|
||||
let publicArtifacts =
|
||||
[ fromFilePath ("archive/" ++ pvSlug (aeProv e)
|
||||
++ "/" ++ pvArtifact (aeProv e))
|
||||
| e <- entries, not (isPrivate e) ]
|
||||
match (fromList publicArtifacts) $ do
|
||||
route idRoute
|
||||
compile copyFileCompiler
|
||||
|
||||
-- Provenance, extracted text, and the manifest: matched (not routed)
|
||||
-- so the generated pages can `load` them as dependencies and recompile
|
||||
-- when they change.
|
||||
match "archive/*/PROVENANCE.json" $ compile getResourceBody
|
||||
match "archive/*/document.txt" $ compile getResourceBody
|
||||
match "archive/*/snapshot.txt" $ compile getResourceBody
|
||||
match "archive/manifest.yaml" $ compile getResourceBody
|
||||
|
||||
mapM_ archiveEntryRule entries
|
||||
archiveIndexRule entries
|
||||
archiveMetaRule entries
|
||||
|
||||
-- | @data/archive-meta.json@ — routed page path -> link-rot status, the
|
||||
-- client-side manifest behind the search page's archive filter (same
|
||||
-- pattern as @data/epistemic-meta.json@). Keys use the routed
|
||||
-- @.../index.html@ form to match @search-filters.js@'s @normUrl@.
|
||||
-- Named @archive-meta@, and its field @status@ scoped under it, so the
|
||||
-- epistemic @status@ filter namespace is untouched.
|
||||
archiveMetaRule :: [ArchiveEntry] -> Rules ()
|
||||
archiveMetaRule entries =
|
||||
create ["data/archive-meta.json"] $ do
|
||||
route idRoute
|
||||
compile $ do
|
||||
_ <- loadAll "archive/*/PROVENANCE.json" :: Compiler [Item String]
|
||||
let metaMap = Map.fromList
|
||||
[ ( "/archive/" ++ pvSlug (aeProv e) ++ "/index.html"
|
||||
, Map.singleton ("status" :: String)
|
||||
(statusName (aeStatus e)) )
|
||||
| e <- entries ]
|
||||
makeItem (LBS.unpack (A.encode metaMap))
|
||||
|
||||
-- | One @/archive/<slug>/@ page.
|
||||
archiveEntryRule :: ArchiveEntry -> Rules ()
|
||||
archiveEntryRule ae =
|
||||
create [fromFilePath ("archive/" ++ slug ++ "/index.html")] $ do
|
||||
route idRoute
|
||||
compile $ do
|
||||
-- Dependency edges: recompile when provenance or the manifest
|
||||
-- changes. The extracted-text sidecar is gitignored and may be
|
||||
-- absent (no .venv / fetch never ran); load it as a dependency
|
||||
-- only when present, so the build never fails for a missing
|
||||
-- generated file.
|
||||
_ <- load provId :: Compiler (Item String)
|
||||
_ <- load manifestId :: Compiler (Item String)
|
||||
case aeTextId ae of
|
||||
Just tp -> do
|
||||
_ <- load (fromFilePath tp) :: Compiler (Item String)
|
||||
return ()
|
||||
Nothing -> return ()
|
||||
makeItem ""
|
||||
>>= loadAndApplyTemplate "templates/archive.html" ctx
|
||||
>>= loadAndApplyTemplate "templates/default.html" ctx
|
||||
>>= relativizeUrls
|
||||
where
|
||||
slug = pvSlug (aeProv ae)
|
||||
provId = fromFilePath ("archive/" ++ slug ++ "/PROVENANCE.json")
|
||||
manifestId = fromFilePath manifestPath
|
||||
ctx = archiveEntryCtx ae
|
||||
|
||||
-- | The @/archive/@ index — every archived work, newest snapshot first.
|
||||
archiveIndexRule :: [ArchiveEntry] -> Rules ()
|
||||
archiveIndexRule entries =
|
||||
create ["archive/index.html"] $ do
|
||||
route idRoute
|
||||
compile $ do
|
||||
-- Recompile when any provenance appears / changes, or the
|
||||
-- manifest changes.
|
||||
_ <- loadAll "archive/*/PROVENANCE.json" :: Compiler [Item String]
|
||||
_ <- load (fromFilePath manifestPath) :: Compiler (Item String)
|
||||
let sorted = sortBy (comparing (Down . pvArchived . aeProv)) entries
|
||||
items = map (\e -> Item (fromFilePath ("archive/" ++ pvSlug (aeProv e))) e)
|
||||
sorted
|
||||
ctx = listField "entries" entryListCtx (return items)
|
||||
<> constField "title" "Archive"
|
||||
<> constField "archive" "true"
|
||||
<> constField "noindex" "true"
|
||||
<> (if null entries then mempty
|
||||
else constField "has-entries" "true")
|
||||
<> siteCtx
|
||||
makeItem ""
|
||||
>>= loadAndApplyTemplate "templates/archive-index.html" ctx
|
||||
>>= loadAndApplyTemplate "templates/default.html" ctx
|
||||
>>= relativizeUrls
|
||||
|
||||
-- ---------------------------------------------------------------------------
|
||||
-- Contexts
|
||||
-- ---------------------------------------------------------------------------
|
||||
|
||||
-- | Per-entry context for the @/archive/<slug>/@ page.
|
||||
archiveEntryCtx :: ArchiveEntry -> Context String
|
||||
archiveEntryCtx ae = mconcat
|
||||
[ constField "title" (pvTitle pv)
|
||||
, constField "archive" "true"
|
||||
, constField "noindex" "true"
|
||||
, constField "original-url" (meUrl me)
|
||||
, constField "archived" (pvArchived pv)
|
||||
, constField "archive-type" (pvType pv)
|
||||
, constField "sha-short" (take 12 (pvSha256 pv))
|
||||
, constField "size" (formatBytes (pvBytes pv))
|
||||
, constField "snapshot-quality" (pvQuality pv)
|
||||
, constField "status" (statusName (aeStatus ae))
|
||||
, qualityFlag
|
||||
, maybeField "status-note" (statusNote (aeStatus ae))
|
||||
, maybeField "note" (meNote me)
|
||||
, maybeField "wayback" (pvWayback pv)
|
||||
, maybeField "paywalled" (if mePaywalled me then Just "true" else Nothing)
|
||||
, visibilityFields
|
||||
-- "Referenced by" (the pages that cite this work) and "Related"
|
||||
-- (semantically near content). Both resolve by this page's route, so
|
||||
-- they need no archive-specific wiring; each is a $if(...)$-guarded
|
||||
-- section in archive.html.
|
||||
, referencedByField
|
||||
, similarLinksField
|
||||
, siteCtx
|
||||
]
|
||||
where
|
||||
me = aeManifest ae
|
||||
pv = aeProv ae
|
||||
slug = pvSlug pv
|
||||
artUrl = "/archive/" ++ slug ++ "/" ++ pvArtifact pv
|
||||
-- A non-'ok' snapshot raises a visible flag on the page.
|
||||
qualityFlag
|
||||
| pvQuality pv == "ok" = mempty
|
||||
| otherwise = constField "degraded" "true"
|
||||
-- A private entry keeps a local preservation copy but publishes none
|
||||
-- of it: no embed, no extracted text — only the provenance metadata
|
||||
-- and a 'held offline' note. A public entry embeds the artifact raw
|
||||
-- (the browser renders the PDF natively, the snapshot loads directly;
|
||||
-- no PDF.js wrapper) and renders its extracted text into the page.
|
||||
-- The is-pdf / is-html flag drives only the iframe sandbox: a
|
||||
-- third-party HTML snapshot is sandboxed, our own committed PDF is not.
|
||||
visibilityFields
|
||||
| isPrivate ae = constField "private" "true"
|
||||
| otherwise = typeField
|
||||
<> constField "artifact-url" artUrl
|
||||
<> constField "artifact-name" (pvArtifact pv)
|
||||
<> fulltextField (pvType pv) (aeFulltext ae)
|
||||
typeField
|
||||
| pvType pv == "html" = constField "is-html" "true"
|
||||
| otherwise = constField "is-pdf" "true"
|
||||
|
||||
-- | Renders the extracted full text into the page DOM so embed.py and
|
||||
-- Pagefind index real text, not an opaque iframe. PDF text keeps its
|
||||
-- pdftotext layout in a @<pre>@; HTML text is block-separated prose, so
|
||||
-- it renders as escaped @<p>@ paragraphs. Absent when the text is empty
|
||||
-- / whitespace, so the @$if(fulltext)$@ guard hides the section.
|
||||
fulltextField :: String -> String -> Context String
|
||||
fulltextField ftype txt
|
||||
| all isBlank txt = mempty
|
||||
| ftype == "html" = constField "fulltext" (htmlParagraphs txt)
|
||||
| otherwise = constField "fulltext" preBlock
|
||||
where
|
||||
isBlank c = c == ' ' || c == '\n' || c == '\t' || c == '\r'
|
||||
preBlock = "<pre class=\"archive-fulltext\">"
|
||||
++ escapeHtml txt ++ "</pre>"
|
||||
|
||||
-- | Block-separated text (paragraphs delimited by blank lines, as
|
||||
-- @archive.py@'s HTML extractor writes it) → escaped @<p>@ elements.
|
||||
htmlParagraphs :: String -> String
|
||||
htmlParagraphs = concatMap para . paragraphsOf
|
||||
where
|
||||
para p = "<p>" ++ escapeHtml p ++ "</p>\n"
|
||||
paragraphsOf = map (unwords . concatMap words)
|
||||
. filter (not . blankGroup)
|
||||
. groupBy ((==) `on` blankLine)
|
||||
. lines
|
||||
blankGroup g = null g || blankLine (head g)
|
||||
blankLine = all (`elem` (" \t\r" :: String))
|
||||
|
||||
-- | List-item context for the @/archive/@ index.
|
||||
entryListCtx :: Context ArchiveEntry
|
||||
entryListCtx = mconcat
|
||||
[ field "entry-title" (return . pvTitle . aeProv . itemBody)
|
||||
, field "entry-archived" (return . pvArchived . aeProv . itemBody)
|
||||
, field "entry-type" (return . pvType . aeProv . itemBody)
|
||||
, field "entry-quality" (return . pvQuality . aeProv . itemBody)
|
||||
, boolField "entry-degraded" ((/= "ok") . pvQuality . aeProv . itemBody)
|
||||
, boolField "entry-private" (isPrivate . itemBody)
|
||||
, field "entry-status" (return . statusName . aeStatus . itemBody)
|
||||
, boolField "entry-rotted" ((== Rotted) . aeStatus . itemBody)
|
||||
, field "entry-url" (\i -> return $
|
||||
"/archive/" ++ pvSlug (aeProv (itemBody i)) ++ "/")
|
||||
]
|
||||
|
||||
-- | Provide a field only when the value is present; otherwise contribute
|
||||
-- nothing, so the template's @$if(...)$@ guard is false.
|
||||
maybeField :: String -> Maybe String -> Context String
|
||||
maybeField k = maybe mempty (constField k)
|
||||
|
||||
-- | A prose note for a non-live link-rot status, shown on the archive
|
||||
-- page; 'Nothing' for 'Live' / 'Error' (no note rendered).
|
||||
statusNote :: ArchiveStatus -> Maybe String
|
||||
statusNote Rotted = Just "The original is no longer reachable. This archived \
|
||||
\copy is now the live link."
|
||||
statusNote Moved = Just "The original page has moved since this snapshot was \
|
||||
\taken; the link above may redirect."
|
||||
statusNote _ = Nothing
|
||||
|
||||
-- ---------------------------------------------------------------------------
|
||||
-- Formatting
|
||||
-- ---------------------------------------------------------------------------
|
||||
|
||||
-- | Human-readable byte count (mirrors the helper in build/Stats.hs).
|
||||
formatBytes :: Integer -> String
|
||||
formatBytes b
|
||||
| b < 1024 = show b ++ " B"
|
||||
| b < 1024 * 1024 = showD (b * 10 `div` 1024) ++ " KB"
|
||||
| otherwise = showD (b * 10 `div` (1024 * 1024)) ++ " MB"
|
||||
where
|
||||
showD n = show (n `div` 10) ++ "." ++ show (n `mod` 10)
|
||||
|
||||
-- ---------------------------------------------------------------------------
|
||||
-- /build/ telemetry
|
||||
-- ---------------------------------------------------------------------------
|
||||
|
||||
-- | Archive metrics for the @/build/@ telemetry page — count, total size,
|
||||
-- median artifact age, breakdowns by link-rot status / snapshot quality
|
||||
-- / visibility, the paywalled count, and any orphan directories.
|
||||
-- Rendered by @Stats.hs@; an empty archive yields just the count.
|
||||
archiveBuildStats :: IO [(String, String)]
|
||||
archiveBuildStats = do
|
||||
entries <- loadArchiveEntries
|
||||
today <- utctDay <$> getCurrentTime
|
||||
orphans <- findOrphanDirs entries
|
||||
let n = length entries
|
||||
bytes = sum (map (pvBytes . aeProv) entries)
|
||||
ages = [ fromInteger (diffDays today d)
|
||||
| e <- entries
|
||||
, Just d <- [parseIsoDay (pvArchived (aeProv e))] ]
|
||||
paywalled = length (filter (mePaywalled . aeManifest) entries)
|
||||
return $
|
||||
[ ("Entries", show n) ]
|
||||
++ (if n == 0 then [] else
|
||||
[ ("Total size", formatBytes bytes)
|
||||
, ("Median age", medianAge ages)
|
||||
, ("By status", tallyOf (map (statusName . aeStatus) entries))
|
||||
, ("By quality", tallyOf (map (pvQuality . aeProv) entries))
|
||||
, ("By visibility", tallyOf (map (meVisibility . aeManifest) entries))
|
||||
])
|
||||
++ [ ("Paywalled", show paywalled) | paywalled > 0 ]
|
||||
++ [ ("Orphan directories", unwords orphans) | not (null orphans) ]
|
||||
|
||||
-- | Directory names under @archive/@ that hold a @PROVENANCE.json@ but are
|
||||
-- not a live manifest entry — drift the @/build/@ page should surface.
|
||||
findOrphanDirs :: [ArchiveEntry] -> IO [String]
|
||||
findOrphanDirs entries = do
|
||||
exists <- doesDirectoryExist "archive"
|
||||
if not exists
|
||||
then return []
|
||||
else do
|
||||
names <- listDirectory "archive"
|
||||
let live = map (pvSlug . aeProv) entries
|
||||
filterM
|
||||
(\name -> do
|
||||
hasProv <- doesFileExist
|
||||
("archive/" ++ name ++ "/PROVENANCE.json")
|
||||
return (hasProv && name `notElem` live))
|
||||
(sort names)
|
||||
|
||||
-- | Format a multiset of string values as @"a 2 \183 b 1"@.
|
||||
tallyOf :: [String] -> String
|
||||
tallyOf xs = intercalate " \183 "
|
||||
[ k ++ " " ++ show c
|
||||
| (k, c) <- Map.toList (Map.fromListWith (+) [ (x, 1 :: Int) | x <- xs ]) ]
|
||||
|
||||
-- | The median of a list of ages, as @"N days"@; an em dash when empty.
|
||||
-- An even-length list takes the mean of the two middle elements,
|
||||
-- rounded to the nearest whole day.
|
||||
medianAge :: [Int] -> String
|
||||
medianAge [] = "\8212"
|
||||
medianAge xs =
|
||||
let sorted = sort xs
|
||||
n = length sorted
|
||||
upper = sorted !! (n `div` 2)
|
||||
lower = sorted !! (n `div` 2 - 1) -- forced only when n is even
|
||||
m | odd n = upper
|
||||
| otherwise = (lower + upper + 1) `div` 2
|
||||
in show m ++ if m == 1 then " day" else " days"
|
||||
|
||||
-- | Parse a @YYYY-MM-DD@ date; 'Nothing' on malformed input.
|
||||
parseIsoDay :: String -> Maybe Day
|
||||
parseIsoDay s = case splitOnDash s of
|
||||
[y, m, d] -> fromGregorian <$> readMaybe y <*> readMaybe m <*> readMaybe d
|
||||
_ -> Nothing
|
||||
where
|
||||
splitOnDash str = case break (== '-') str of
|
||||
(a, '-' : rest) -> a : splitOnDash rest
|
||||
(a, _) -> [a]
|
||||
|
|
@ -0,0 +1,281 @@
|
|||
{-# LANGUAGE GHC2021 #-}
|
||||
{-# LANGUAGE OverloadedStrings #-}
|
||||
-- | ArchiveIndex — shared read-only access to the archive's two JSON
|
||||
-- sidecars: @data/archive-index.json@ (the @url\/alias -> slug@ map
|
||||
-- written by @archive.py fetch@) and @data/archive-state.json@ (the
|
||||
-- per-URL link-rot status written by @archive.py check@).
|
||||
--
|
||||
-- Consumers:
|
||||
--
|
||||
-- * @Filters.Archive@ — appends the archive affordance to body links
|
||||
-- whose target is archived, and flips a @rotted@ link to the local
|
||||
-- copy.
|
||||
-- * @Backlinks@ — keeps archived external links through pass 1 and
|
||||
-- canonicalises them to their @/archive/<slug>/@ page in pass 2.
|
||||
-- * @Archive@ — surfaces each entry's rot status on its page, the
|
||||
-- @/archive/@ index, and the @/build/@ telemetry.
|
||||
--
|
||||
-- Both files are loaded once per *process* via NOINLINE
|
||||
-- @unsafePerformIO@ CAFs (as are the manifest/removed URL sets below).
|
||||
-- An absent or malformed file degrades safely: an empty index makes the
|
||||
-- link consumers no-op; an absent state file makes every entry @Live@
|
||||
-- (the safe default — no link flip). @archive.py check@ is decoupled
|
||||
-- from @make build@; a build consumes whatever state file exists.
|
||||
--
|
||||
-- Consequence of the once-per-process read (shared with the manifest
|
||||
-- read in 'Archive.archiveRules'): under @site watch@, edits to
|
||||
-- @manifest.yaml@, @removed.yaml@, or the regenerated state JSONs are
|
||||
-- not re-read — the server renders stale archive state until restart.
|
||||
-- One-shot builds (@make build@ / @make deploy@) are unaffected.
|
||||
module ArchiveIndex
|
||||
( ArchiveStatus (..)
|
||||
, statusName
|
||||
, archiveSlugFor
|
||||
, archiveStatusForSlug
|
||||
, archiveIndexIsEmpty
|
||||
, normalizeUrl
|
||||
) where
|
||||
|
||||
import Data.Map.Strict (Map)
|
||||
import qualified Data.Map.Strict as Map
|
||||
import Data.Maybe (fromMaybe)
|
||||
import Data.Set (Set)
|
||||
import qualified Data.Set as Set
|
||||
import Data.Text (Text)
|
||||
import qualified Data.Text as T
|
||||
import qualified Data.Aeson as A
|
||||
import Data.Aeson ((.!=), (.:), (.:?))
|
||||
import qualified Data.Yaml as Y
|
||||
import System.Directory (doesFileExist)
|
||||
import System.IO.Unsafe (unsafePerformIO)
|
||||
|
||||
-- ---------------------------------------------------------------------------
|
||||
-- Link-rot status
|
||||
-- ---------------------------------------------------------------------------
|
||||
|
||||
-- | The link-rot status of an archived work's original URL, as set by
|
||||
-- @archive.py check@. 'Live' is the safe default for an unscanned or
|
||||
-- unknown entry.
|
||||
data ArchiveStatus = Live | Moved | Rotted | Error
|
||||
deriving (Eq, Show)
|
||||
|
||||
-- | The lower-case wire name, matching @archive-state.json@ and the
|
||||
-- @status:@ Pagefind filter tag.
|
||||
statusName :: ArchiveStatus -> String
|
||||
statusName Live = "live"
|
||||
statusName Moved = "moved"
|
||||
statusName Rotted = "rotted"
|
||||
statusName Error = "error"
|
||||
|
||||
parseStatus :: Text -> ArchiveStatus
|
||||
parseStatus "moved" = Moved
|
||||
parseStatus "rotted" = Rotted
|
||||
parseStatus "error" = Error
|
||||
parseStatus _ = Live
|
||||
|
||||
-- ---------------------------------------------------------------------------
|
||||
-- JSON shapes
|
||||
-- ---------------------------------------------------------------------------
|
||||
|
||||
-- | One @archive-index.json@ entry. Only @slug@ and @aliases@ are used.
|
||||
data IdxEntry = IdxEntry
|
||||
{ ieSlug :: String
|
||||
, ieAliases :: [Text]
|
||||
}
|
||||
|
||||
instance A.FromJSON IdxEntry where
|
||||
parseJSON = A.withObject "IdxEntry" $ \o -> IdxEntry
|
||||
<$> o .: "slug"
|
||||
<*> (o .:? "aliases" .!= [])
|
||||
|
||||
-- | One @archive-state.json@ entry — only the @status@ is consumed here.
|
||||
newtype StateEntry = StateEntry { seStatus :: ArchiveStatus }
|
||||
|
||||
instance A.FromJSON StateEntry where
|
||||
parseJSON = A.withObject "StateEntry" $ \o ->
|
||||
StateEntry . parseStatus <$> (o .:? "status" .!= "live")
|
||||
|
||||
newtype UrlEntry = UrlEntry { ueUrl :: Text }
|
||||
|
||||
instance A.FromJSON UrlEntry where
|
||||
parseJSON = A.withObject "UrlEntry" $ \o ->
|
||||
UrlEntry <$> o .: "url"
|
||||
|
||||
-- ---------------------------------------------------------------------------
|
||||
-- Loaded-once CAFs
|
||||
-- ---------------------------------------------------------------------------
|
||||
|
||||
indexPath, statePath, manifestPath, removedPath :: FilePath
|
||||
indexPath = "data/archive-index.json"
|
||||
statePath = "data/archive-state.json"
|
||||
manifestPath = "archive/manifest.yaml"
|
||||
removedPath = "archive/removed.yaml"
|
||||
|
||||
readUrlSet :: FilePath -> IO (Set Text)
|
||||
readUrlSet path = do
|
||||
exists <- doesFileExist path
|
||||
if not exists
|
||||
then return Set.empty
|
||||
else do
|
||||
decoded <- Y.decodeFileEither path
|
||||
case decoded of
|
||||
Right entries -> return . Set.fromList $
|
||||
map (normalizeUrl . ueUrl) (entries :: [UrlEntry])
|
||||
Left e -> ioError . userError $
|
||||
"[archive] FATAL: " ++ path ++ ": " ++ show e
|
||||
|
||||
-- | Normalised URLs recorded as deliberate takedowns in @removed.yaml@.
|
||||
-- Consulted independently of 'activeUrls': a takedown must also knock
|
||||
-- out any *alias* key a stale index still carries for another entry.
|
||||
{-# NOINLINE removedUrls #-}
|
||||
removedUrls :: Set Text
|
||||
removedUrls = unsafePerformIO (readUrlSet removedPath)
|
||||
|
||||
-- | Canonical URLs still permitted to participate in link annotation.
|
||||
-- Filtering the generated index at build time makes a direct Hakyll build
|
||||
-- respect authored manifest/removal state even when archive.py did not run.
|
||||
{-# NOINLINE activeUrls #-}
|
||||
activeUrls :: Set Text
|
||||
activeUrls = unsafePerformIO $ do
|
||||
manifest <- readUrlSet manifestPath
|
||||
return (manifest `Set.difference` removedUrls)
|
||||
|
||||
-- | @canonical-url -> entry@. Absent/malformed file -> empty; entries no
|
||||
-- longer permitted by the authored manifest/removal state are removed.
|
||||
{-# NOINLINE rawIndex #-}
|
||||
rawIndex :: Map Text IdxEntry
|
||||
rawIndex = unsafePerformIO $ do
|
||||
exists <- doesFileExist indexPath
|
||||
if not exists
|
||||
then return Map.empty
|
||||
else do
|
||||
decoded <- A.eitherDecodeFileStrict' indexPath
|
||||
let parsed = either (const Map.empty) id decoded
|
||||
return $ Map.filterWithKey
|
||||
(\canon _ -> normalizeUrl canon `Set.member` activeUrls)
|
||||
parsed
|
||||
|
||||
-- | @url -> status@. Absent/malformed file -> empty (every entry 'Live').
|
||||
{-# NOINLINE rawState #-}
|
||||
rawState :: Map Text ArchiveStatus
|
||||
rawState = unsafePerformIO $ do
|
||||
exists <- doesFileExist statePath
|
||||
if not exists
|
||||
then return Map.empty
|
||||
else do
|
||||
decoded <- A.eitherDecodeFileStrict' statePath
|
||||
return $ either (const Map.empty) (Map.map seStatus) decoded
|
||||
|
||||
-- | @normalised-url -> slug@: the canonical key and every alias from
|
||||
-- @archive-index.json@, each fed through 'normalizeUrl'. Both keys and
|
||||
-- lookups are normalised, so a citation form the alias set cannot
|
||||
-- enumerate (e.g. an unbounded arXiv version, or any tracking-laden
|
||||
-- variant of a clean manifest URL) still resolves. Alias keys that
|
||||
-- match a recorded takedown are dropped, so a stale index (archive.py
|
||||
-- not run since removed.yaml changed) cannot keep annotating a removed
|
||||
-- work through an alias of a still-active entry.
|
||||
{-# NOINLINE flatIndex #-}
|
||||
flatIndex :: Map Text String
|
||||
flatIndex = Map.fromList
|
||||
[ (nkey, ieSlug e)
|
||||
| (canon, e) <- Map.toList rawIndex
|
||||
, key <- canon : ieAliases e
|
||||
, let nkey = normalizeUrl key
|
||||
, nkey `Set.notMember` removedUrls
|
||||
]
|
||||
|
||||
-- | @slug -> status@: each entry's status, looked up by its canonical URL
|
||||
-- in the state file (the two files share the manifest URL as key).
|
||||
{-# NOINLINE slugStatus #-}
|
||||
slugStatus :: Map String ArchiveStatus
|
||||
slugStatus = Map.fromList
|
||||
[ (ieSlug e, Map.findWithDefault Live canon rawState)
|
||||
| (canon, e) <- Map.toList rawIndex
|
||||
]
|
||||
|
||||
-- ---------------------------------------------------------------------------
|
||||
-- Public lookups
|
||||
-- ---------------------------------------------------------------------------
|
||||
|
||||
-- | True when no archive index is available — the link consumers no-op.
|
||||
archiveIndexIsEmpty :: Bool
|
||||
archiveIndexIsEmpty = Map.null rawIndex
|
||||
|
||||
-- | The archive slug for an outbound URL, or 'Nothing'. Both the index
|
||||
-- keys and the input go through 'normalizeUrl', so a citation form that
|
||||
-- the alias set cannot enumerate — an unbounded arXiv version, or any
|
||||
-- tracking-laden variant of a clean manifest URL — still resolves.
|
||||
archiveSlugFor :: Text -> Maybe String
|
||||
archiveSlugFor url = Map.lookup (normalizeUrl url) flatIndex
|
||||
|
||||
-- | The link-rot status of an archived entry, by slug. 'Live' for an
|
||||
-- unknown slug or when no scan has run.
|
||||
archiveStatusForSlug :: String -> ArchiveStatus
|
||||
archiveStatusForSlug slug = Map.findWithDefault Live slug slugStatus
|
||||
|
||||
-- ---------------------------------------------------------------------------
|
||||
-- URL normalisation (matching, not display)
|
||||
-- ---------------------------------------------------------------------------
|
||||
|
||||
-- | Tracking-only query parameters: their presence or absence is
|
||||
-- semantically irrelevant; the lookup strips them before matching.
|
||||
-- Sync with @TRACKING_PARAMS@ in @tools/archive.py@.
|
||||
trackingParams :: [Text]
|
||||
trackingParams =
|
||||
[ "utm_source", "utm_medium", "utm_campaign", "utm_term", "utm_content"
|
||||
, "fbclid", "gclid", "mc_eid", "mc_cid", "ref", "igshid"
|
||||
, "_hsenc", "_hsmi", "mkt_tok"
|
||||
]
|
||||
|
||||
-- | Remove tracking-only query parameters; preserve every other parameter
|
||||
-- in its original order.
|
||||
stripTracking :: Text -> Text
|
||||
stripTracking url = case T.breakOn "?" url of
|
||||
(_, "") -> url
|
||||
(path, q) ->
|
||||
let kept = filter notTracking (T.splitOn "&" (T.drop 1 q))
|
||||
in if null kept then path
|
||||
else path <> "?" <> T.intercalate "&" kept
|
||||
where
|
||||
notTracking p = T.takeWhile (/= '=') p `notElem` trackingParams
|
||||
|
||||
-- | The canonical form of an arXiv URL: @https://arxiv.org/abs/<id>@ with
|
||||
-- no version suffix and no @.pdf@. Maps every member of the
|
||||
-- abs/pdf/versioned/@.pdf@ family to the same key. Non-arXiv passes through.
|
||||
arxivCanonical :: Text -> Text
|
||||
arxivCanonical url
|
||||
| Just rest <- T.stripPrefix "https://arxiv.org/" url
|
||||
, Just key <- arxivKey rest = key
|
||||
| Just rest <- T.stripPrefix "http://arxiv.org/" url
|
||||
, Just key <- arxivKey rest = key
|
||||
| otherwise = url
|
||||
where
|
||||
arxivKey rest = case T.breakOn "/" rest of
|
||||
(kind, slashId)
|
||||
| kind `elem` ["abs", "pdf"], not (T.null slashId) ->
|
||||
Just $ "https://arxiv.org/abs/"
|
||||
<> stripVer (stripPdfSuf (T.tail slashId))
|
||||
_ -> Nothing
|
||||
stripPdfSuf t = fromMaybe t (T.stripSuffix ".pdf" t)
|
||||
stripVer t = case T.breakOnEnd "v" t of
|
||||
(before, ver)
|
||||
| not (T.null before)
|
||||
, not (T.null ver)
|
||||
, T.all isAsciiDigit ver
|
||||
-> T.dropEnd 1 before
|
||||
_ -> t
|
||||
isAsciiDigit c = c >= '0' && c <= '9'
|
||||
|
||||
-- | The full normalisation: drop fragment, strip tracking, fold
|
||||
-- @http://@→@https://@, arXiv-canonicalise, trim a trailing slash. Both
|
||||
-- 'flatIndex' keys and 'archiveSlugFor' inputs go through this so the
|
||||
-- index never misses a citation form the design promises to match.
|
||||
normalizeUrl :: Text -> Text
|
||||
normalizeUrl url =
|
||||
let noFrag = T.takeWhile (/= '#') url
|
||||
clean = stripTracking noFrag
|
||||
https = case T.stripPrefix "http://" clean of
|
||||
Just rest -> "https://" <> rest
|
||||
Nothing -> clean
|
||||
arxiv = arxivCanonical https
|
||||
in T.dropWhileEnd (== '/') arxiv
|
||||
|
|
@ -0,0 +1,109 @@
|
|||
{-# LANGUAGE GHC2021 #-}
|
||||
{-# LANGUAGE OverloadedStrings #-}
|
||||
-- | Author system — treats authors like tags.
|
||||
--
|
||||
-- Author pages live at /authors/{slug}/index.html.
|
||||
-- Items with no "authors" frontmatter key default to Levi Neuwirth.
|
||||
--
|
||||
-- Frontmatter format (name-only or name|url — url part is ignored now):
|
||||
-- authors:
|
||||
-- - "Levi Neuwirth"
|
||||
-- - "Alice Smith | https://alice.example" -- url ignored; link goes to /authors/alice-smith/
|
||||
module Authors
|
||||
( buildAllAuthors
|
||||
, applyAuthorRules
|
||||
) where
|
||||
|
||||
import Data.Maybe (fromMaybe)
|
||||
import Hakyll
|
||||
import Pagination (sortAndGroup)
|
||||
import Patterns (authorIndexable)
|
||||
import Contexts (abstractField, tagLinksField)
|
||||
import Utils (authorSlugify, authorNameOf)
|
||||
|
||||
|
||||
-- ---------------------------------------------------------------------------
|
||||
-- Slug helpers
|
||||
--
|
||||
-- The slugify and nameOf helpers used to live here in their own
|
||||
-- definitions; they now defer to 'Utils' so that they cannot drift from
|
||||
-- the 'Contexts' versions on Unicode edge cases.
|
||||
-- ---------------------------------------------------------------------------
|
||||
|
||||
slugify :: String -> String
|
||||
slugify = authorSlugify
|
||||
|
||||
nameOf :: String -> String
|
||||
nameOf = authorNameOf
|
||||
|
||||
|
||||
-- ---------------------------------------------------------------------------
|
||||
-- Constants
|
||||
-- ---------------------------------------------------------------------------
|
||||
|
||||
defaultAuthor :: String
|
||||
defaultAuthor = "Levi Neuwirth"
|
||||
|
||||
-- | Content patterns indexed by author. Sourced from 'Patterns.authorIndexable'
|
||||
-- so this stays in lockstep with Tags.hs and Backlinks.hs.
|
||||
allContent :: Pattern
|
||||
allContent = authorIndexable
|
||||
|
||||
|
||||
-- ---------------------------------------------------------------------------
|
||||
-- Tag-like helpers (mirror of Tags.hs)
|
||||
-- ---------------------------------------------------------------------------
|
||||
|
||||
-- | Returns all author names for an identifier.
|
||||
-- Defaults to ["Levi Neuwirth"] when no "authors" key is present.
|
||||
getAuthors :: MonadMetadata m => Identifier -> m [String]
|
||||
getAuthors ident = do
|
||||
meta <- getMetadata ident
|
||||
let entries = fromMaybe [] (lookupStringList "authors" meta)
|
||||
return $ if null entries
|
||||
then [defaultAuthor]
|
||||
else map nameOf entries
|
||||
|
||||
-- | Canonical identifier for an author's index page (page 1).
|
||||
authorIdentifier :: String -> Identifier
|
||||
authorIdentifier name = fromFilePath $ "authors/" ++ slugify name ++ "/index.html"
|
||||
|
||||
-- | Paginated identifier: page 1 → authors/{slug}/index.html
|
||||
-- page N → authors/{slug}/page/N/index.html
|
||||
authorPageId :: String -> PageNumber -> Identifier
|
||||
authorPageId slug 1 = fromFilePath $ "authors/" ++ slug ++ "/index.html"
|
||||
authorPageId slug n = fromFilePath $ "authors/" ++ slug ++ "/page/" ++ show n ++ "/index.html"
|
||||
|
||||
|
||||
-- ---------------------------------------------------------------------------
|
||||
-- Build + rules
|
||||
-- ---------------------------------------------------------------------------
|
||||
|
||||
buildAllAuthors :: Rules Tags
|
||||
buildAllAuthors = buildTagsWith getAuthors allContent authorIdentifier
|
||||
|
||||
applyAuthorRules :: Tags -> Context String -> Rules ()
|
||||
applyAuthorRules authors baseCtx = tagsRules authors $ \name pat -> do
|
||||
let slug = slugify name
|
||||
paginate <- buildPaginateWith sortAndGroup pat (authorPageId slug)
|
||||
paginateRules paginate $ \pageNum pat' -> do
|
||||
route idRoute
|
||||
compile $ do
|
||||
items <- recentFirst =<< loadAll (pat' .&&. hasNoVersion)
|
||||
let ctx = listField "items" itemCtx (return items)
|
||||
<> paginateContext paginate pageNum
|
||||
<> constField "author" name
|
||||
<> constField "title" name
|
||||
<> constField "portal" "true"
|
||||
<> baseCtx
|
||||
makeItem ""
|
||||
>>= loadAndApplyTemplate "templates/author-index.html" ctx
|
||||
>>= loadAndApplyTemplate "templates/default.html" ctx
|
||||
>>= relativizeUrls
|
||||
where
|
||||
itemCtx = dateField "date" "%-d %B %Y"
|
||||
<> tagLinksField "item-tags"
|
||||
<> abstractField
|
||||
<> defaultContext
|
||||
|
||||
|
||||
|
|
@ -0,0 +1,515 @@
|
|||
{-# LANGUAGE GHC2021 #-}
|
||||
{-# LANGUAGE OverloadedStrings #-}
|
||||
-- | Backlinks with context: build-time computation of which pages link to
|
||||
-- each page, including the sentence that contains each link.
|
||||
--
|
||||
-- Architecture (dependency-correct, no circular deps):
|
||||
--
|
||||
-- 1. Each content file is compiled under @version "links"@: a lightweight
|
||||
-- pass that parses the source, walks the AST block-by-block, splits
|
||||
-- each paragraph into sentences, and for every internal link records
|
||||
-- the URL *and* the HTML of the sentence that contains it. The result
|
||||
-- is serialised as a JSON array of @{url, context}@ objects.
|
||||
--
|
||||
-- 2. A @create ["data/backlinks.json"]@ rule loads all "links" items,
|
||||
-- inverts the map, and serialises
|
||||
-- @target → [{url, title, abstract, context}]@ as JSON.
|
||||
--
|
||||
-- 3. @backlinksField@ loads that JSON at page render time and injects
|
||||
-- an HTML list showing each source's title and a quoted sentence of
|
||||
-- context. The @load@ call establishes a proper Hakyll dependency so
|
||||
-- pages recompile when backlinks change.
|
||||
--
|
||||
-- Dependency order (no cycles):
|
||||
-- content "links" versions → data/backlinks.json → content default versions
|
||||
module Backlinks
|
||||
( backlinkRules
|
||||
, backlinksField
|
||||
, referencedByField
|
||||
) where
|
||||
|
||||
import Data.List (nubBy, partition, sortBy,
|
||||
stripPrefix)
|
||||
import Data.Ord (comparing)
|
||||
import Data.Maybe (fromMaybe)
|
||||
import qualified Data.Map.Strict as Map
|
||||
import Data.Map.Strict (Map)
|
||||
import qualified Data.ByteString as BS
|
||||
import qualified Data.Text as T
|
||||
import qualified Data.Text.Lazy as TL
|
||||
import qualified Data.Text.Lazy.Encoding as TLE
|
||||
import qualified Data.Text.Encoding as TE
|
||||
import qualified Data.Text.Encoding.Error as TE
|
||||
import qualified Data.Aeson as Aeson
|
||||
import Data.Aeson ((.=))
|
||||
import Text.Pandoc.Class (runPure)
|
||||
import Text.Pandoc.Writers (writeHtml5String)
|
||||
import Text.Pandoc.Definition (Block (..), Inline (..), Pandoc (..),
|
||||
nullMeta)
|
||||
import Text.Pandoc.Options (WriterOptions (..), HTMLMathMethod (..))
|
||||
import Text.Pandoc.Walk (query)
|
||||
import Hakyll
|
||||
import Compilers (readerOpts, writerOpts)
|
||||
import Filters (preprocessSource)
|
||||
import qualified Patterns as P
|
||||
import ArchiveIndex (archiveSlugFor)
|
||||
|
||||
-- ---------------------------------------------------------------------------
|
||||
-- Link-with-context entry (intermediate, saved by the "links" pass)
|
||||
-- ---------------------------------------------------------------------------
|
||||
|
||||
data LinkEntry = LinkEntry
|
||||
{ leUrl :: T.Text -- internal URL (as found in the AST)
|
||||
, leSentence :: String -- HTML of the sentence containing the link
|
||||
, leParagraph :: String -- HTML of the full surrounding paragraph
|
||||
} deriving (Show, Eq)
|
||||
|
||||
instance Aeson.ToJSON LinkEntry where
|
||||
toJSON e = Aeson.object
|
||||
[ "url" .= leUrl e
|
||||
, "sentence" .= leSentence e
|
||||
, "paragraph" .= leParagraph e
|
||||
]
|
||||
|
||||
instance Aeson.FromJSON LinkEntry where
|
||||
parseJSON = Aeson.withObject "LinkEntry" $ \o ->
|
||||
LinkEntry
|
||||
<$> o Aeson..: "url"
|
||||
<*> o Aeson..: "sentence"
|
||||
<*> o Aeson..: "paragraph"
|
||||
|
||||
-- ---------------------------------------------------------------------------
|
||||
-- Backlink source record (stored in data/backlinks.json)
|
||||
-- ---------------------------------------------------------------------------
|
||||
|
||||
data BacklinkSource = BacklinkSource
|
||||
{ blUrl :: String
|
||||
, blTitle :: String
|
||||
, blAbstract :: String
|
||||
, blSentence :: String -- raw HTML of the sentence containing the link
|
||||
, blParagraph :: String -- raw HTML of the full paragraph (hover popup)
|
||||
, blFragment :: String -- archived-target fragment (no '#'), else ""
|
||||
} deriving (Show, Eq, Ord)
|
||||
|
||||
instance Aeson.ToJSON BacklinkSource where
|
||||
toJSON bl = Aeson.object
|
||||
[ "url" .= blUrl bl
|
||||
, "title" .= blTitle bl
|
||||
, "abstract" .= blAbstract bl
|
||||
, "sentence" .= blSentence bl
|
||||
, "paragraph" .= blParagraph bl
|
||||
, "fragment" .= blFragment bl
|
||||
]
|
||||
|
||||
instance Aeson.FromJSON BacklinkSource where
|
||||
parseJSON = Aeson.withObject "BacklinkSource" $ \o ->
|
||||
BacklinkSource
|
||||
<$> o Aeson..: "url"
|
||||
<*> o Aeson..: "title"
|
||||
<*> o Aeson..: "abstract"
|
||||
<*> o Aeson..: "sentence"
|
||||
<*> o Aeson..: "paragraph"
|
||||
<*> o Aeson..:? "fragment" Aeson..!= ""
|
||||
|
||||
-- ---------------------------------------------------------------------------
|
||||
-- Writer options for context rendering
|
||||
-- ---------------------------------------------------------------------------
|
||||
|
||||
-- | Minimal writer options for rendering paragraph context: no template
|
||||
-- (fragment only), plain math fallback (context excerpts are previews, not
|
||||
-- full renders, and KaTeX CSS may not be loaded on all target pages).
|
||||
contextWriterOpts :: WriterOptions
|
||||
contextWriterOpts = writerOpts
|
||||
{ writerTemplate = Nothing
|
||||
, writerHTMLMathMethod = PlainMath
|
||||
}
|
||||
|
||||
-- ---------------------------------------------------------------------------
|
||||
-- Context extraction
|
||||
-- ---------------------------------------------------------------------------
|
||||
|
||||
-- | URL filter: skip external links, pseudo-schemes, anchor-only fragments,
|
||||
-- and static-asset paths.
|
||||
isPageLink :: T.Text -> Bool
|
||||
isPageLink u
|
||||
-- An archived external URL is kept regardless of scheme or extension:
|
||||
-- pass 2 inverts it to its /archive/<slug>/ page.
|
||||
| isArchived = True
|
||||
| otherwise =
|
||||
not (T.isPrefixOf "http://" u) &&
|
||||
not (T.isPrefixOf "https://" u) &&
|
||||
-- protocol-relative //host/path is external, not a page path
|
||||
not (T.isPrefixOf "//" u) &&
|
||||
not (T.isPrefixOf "#" u) &&
|
||||
not (T.isPrefixOf "mailto:" u) &&
|
||||
not (T.isPrefixOf "tel:" u) &&
|
||||
not (T.null u) &&
|
||||
not (hasStaticExt u)
|
||||
where
|
||||
isArchived = case archiveSlugFor u of
|
||||
Just _ -> True
|
||||
Nothing -> False
|
||||
staticExts = [".pdf",".svg",".png",".jpg",".jpeg",".webp",
|
||||
".mp3",".mp4",".woff2",".woff",".ttf",".ico",
|
||||
".json",".asc",".xml",".gz",".zip"]
|
||||
hasStaticExt x = any (`T.isSuffixOf` T.toLower x) staticExts
|
||||
|
||||
-- | Render a list of inlines to an HTML fragment string.
|
||||
-- Uses Plain (not Para) to avoid a wrapping <p> — callers add their own.
|
||||
renderInlines :: [Inline] -> String
|
||||
renderInlines inlines =
|
||||
case runPure (writeHtml5String contextWriterOpts doc) of
|
||||
Left _ -> ""
|
||||
Right txt -> T.unpack txt
|
||||
where
|
||||
doc = Pandoc nullMeta [Plain inlines]
|
||||
|
||||
-- | Split a list of inlines into sentences by terminator punctuation.
|
||||
--
|
||||
-- A @Str@ whose last non-closing-punctuation character is @.@, @!@, or @?@
|
||||
-- ends a sentence when followed by @Space@, @SoftBreak@, @LineBreak@, or
|
||||
-- end-of-list. Closing quote/bracket characters after the terminator
|
||||
-- (e.g. @right-double-quote@, @)@, @]@) are tolerated.
|
||||
--
|
||||
-- The splitter is deliberately simple: abbreviations like "e.g." or "Dr."
|
||||
-- will cause occasional over-splitting. That is acceptable for backlink
|
||||
-- previews, where a slightly short context is preferable to the complexity
|
||||
-- of abbreviation detection.
|
||||
splitSentences :: [Inline] -> [[Inline]]
|
||||
splitSentences = go []
|
||||
where
|
||||
go acc [] = if null acc then [] else [reverse acc]
|
||||
go acc (tok : rest)
|
||||
| isTerminator tok && leadingBreak rest =
|
||||
reverse (tok : acc) : go [] (dropLeadingBreak rest)
|
||||
| otherwise =
|
||||
go (tok : acc) rest
|
||||
|
||||
isTerminator :: Inline -> Bool
|
||||
isTerminator (Str s) = endsWithTerminator s
|
||||
isTerminator _ = False
|
||||
|
||||
endsWithTerminator :: T.Text -> Bool
|
||||
endsWithTerminator t =
|
||||
case T.unsnoc (T.dropWhileEnd isClosingPunct t) of
|
||||
Just (_, c) -> c == '.' || c == '!' || c == '?'
|
||||
Nothing -> False
|
||||
|
||||
isClosingPunct :: Char -> Bool
|
||||
isClosingPunct c = c `elem` (")]\"'\x201D\x2019" :: String)
|
||||
|
||||
leadingBreak :: [Inline] -> Bool
|
||||
leadingBreak [] = True
|
||||
leadingBreak (Space : _) = True
|
||||
leadingBreak (SoftBreak : _) = True
|
||||
leadingBreak (LineBreak : _) = True
|
||||
leadingBreak _ = False
|
||||
|
||||
dropLeadingBreak :: [Inline] -> [Inline]
|
||||
dropLeadingBreak (Space : xs) = xs
|
||||
dropLeadingBreak (SoftBreak : xs) = xs
|
||||
dropLeadingBreak (LineBreak : xs) = xs
|
||||
dropLeadingBreak xs = xs
|
||||
|
||||
-- | Extract @LinkEntry@ records from a Pandoc document.
|
||||
-- For every internal link in a paragraph, emit an entry carrying the HTML
|
||||
-- of the sentence containing the link (default display) and the HTML of
|
||||
-- the full paragraph (hover/popup context).
|
||||
-- Recurses into Div, BlockQuote, BulletList, OrderedList, and
|
||||
-- DefinitionList. @Plain@ matters as much as @Para@: Pandoc renders
|
||||
-- tight list items (the default @- item@ Markdown form) as @Plain@
|
||||
-- blocks, so without it every link written in a tight list would be
|
||||
-- invisible to the backlinks system.
|
||||
extractLinksWithContext :: Pandoc -> [LinkEntry]
|
||||
extractLinksWithContext (Pandoc _ blocks) = concatMap go blocks
|
||||
where
|
||||
go :: Block -> [LinkEntry]
|
||||
go (Para inlines) = paraEntries inlines
|
||||
go (Plain inlines) = paraEntries inlines
|
||||
go (BlockQuote bs) = concatMap go bs
|
||||
go (Div _ bs) = concatMap go bs
|
||||
go (BulletList items) = concatMap (concatMap go) items
|
||||
go (OrderedList _ items) = concatMap (concatMap go) items
|
||||
go (DefinitionList defs) = concatMap defEntries defs
|
||||
go _ = []
|
||||
|
||||
defEntries :: ([Inline], [[Block]]) -> [LinkEntry]
|
||||
defEntries (term, bodies) =
|
||||
paraEntries term ++ concatMap (concatMap go) bodies
|
||||
|
||||
paraEntries :: [Inline] -> [LinkEntry]
|
||||
paraEntries inlines =
|
||||
let paraHtml = renderInlines inlines
|
||||
sentences = splitSentences inlines
|
||||
in concatMap (sentenceEntries paraHtml) sentences
|
||||
|
||||
sentenceEntries :: String -> [Inline] -> [LinkEntry]
|
||||
sentenceEntries paraHtml sentence =
|
||||
let urls = filter isPageLink (query getUrl sentence)
|
||||
in if null urls then []
|
||||
else
|
||||
let sentHtml = renderInlines sentence
|
||||
in map (\u -> LinkEntry u sentHtml paraHtml) urls
|
||||
|
||||
getUrl :: Inline -> [T.Text]
|
||||
getUrl (Link _ _ (url, _)) = [url]
|
||||
getUrl _ = []
|
||||
|
||||
-- ---------------------------------------------------------------------------
|
||||
-- Lightweight links compiler
|
||||
-- ---------------------------------------------------------------------------
|
||||
|
||||
-- | Compile a source file lightly: parse the Markdown (wikilinks preprocessed),
|
||||
-- extract internal links with their paragraph context, and serialise as JSON.
|
||||
linksCompiler :: Compiler (Item String)
|
||||
linksCompiler = do
|
||||
body <- getResourceBody
|
||||
let src = itemBody body
|
||||
let body' = itemSetBody (preprocessSource src) body
|
||||
pandocItem <- readPandocWith readerOpts body'
|
||||
let entries = nubBy sameEntry
|
||||
(extractLinksWithContext (itemBody pandocItem))
|
||||
makeItem . TL.unpack . TLE.decodeUtf8 . Aeson.encode $ entries
|
||||
where
|
||||
sameEntry a b =
|
||||
leUrl a == leUrl b &&
|
||||
leSentence a == leSentence b &&
|
||||
leParagraph a == leParagraph b
|
||||
|
||||
-- ---------------------------------------------------------------------------
|
||||
-- URL normalisation
|
||||
-- ---------------------------------------------------------------------------
|
||||
|
||||
-- | Normalise an internal URL as a map key: strip query string and
|
||||
-- fragment; ensure a leading slash; strip a trailing @index.html@
|
||||
-- (keeping the directory slash) before the bare @.html@ extension, so a
|
||||
-- page routed @essays\/foo\/index.html@ and a body link authored in the
|
||||
-- canonical directory form @\/essays\/foo\/@ collide on the same key
|
||||
-- (mirrors 'SimilarLinks.normaliseUrl'); percent-decode the path so that
|
||||
-- @\/essays\/caf%C3%A9@ and @\/essays\/café@ collide on the same key.
|
||||
--
|
||||
-- Both sides of the backlink join go through this function: page keys
|
||||
-- via 'backlinksFieldWith' (@normaliseUrl ("/" ++ route)@) and link
|
||||
-- targets via 'targetKey' — so the two always agree.
|
||||
normaliseUrl :: String -> String
|
||||
normaliseUrl url =
|
||||
let t = T.pack url
|
||||
t1 = fst (T.breakOn "?" (fst (T.breakOn "#" t)))
|
||||
t2 = if T.isPrefixOf "/" t1 then t1 else "/" `T.append` t1
|
||||
t3 = fromMaybe t2 (T.stripSuffix "index.html" t2)
|
||||
t4 = fromMaybe t3 (T.stripSuffix ".html" t3)
|
||||
in percentDecode (T.unpack t4)
|
||||
|
||||
-- | Decode percent-escapes (@%XX@) into raw bytes, then re-interpret the
|
||||
-- resulting bytestring as UTF-8. Invalid escapes are passed through
|
||||
-- verbatim so this is safe to call on already-decoded input.
|
||||
percentDecode :: String -> String
|
||||
percentDecode = T.unpack . TE.decodeUtf8With lenientDecode . pack . go
|
||||
where
|
||||
go [] = []
|
||||
go ('%':a:b:rest)
|
||||
| Just hi <- hexDigit a
|
||||
, Just lo <- hexDigit b
|
||||
= fromIntegral (hi * 16 + lo) : go rest
|
||||
go (c:rest) = fromIntegral (fromEnum c) : go rest
|
||||
|
||||
hexDigit c
|
||||
| c >= '0' && c <= '9' = Just (fromEnum c - fromEnum '0')
|
||||
| c >= 'a' && c <= 'f' = Just (fromEnum c - fromEnum 'a' + 10)
|
||||
| c >= 'A' && c <= 'F' = Just (fromEnum c - fromEnum 'A' + 10)
|
||||
| otherwise = Nothing
|
||||
|
||||
pack = BS.pack
|
||||
lenientDecode = TE.lenientDecode
|
||||
|
||||
-- ---------------------------------------------------------------------------
|
||||
-- Archive-aware target keying
|
||||
-- ---------------------------------------------------------------------------
|
||||
|
||||
-- | The @data/backlinks.json@ key an outbound URL inverts to. An archived
|
||||
-- external URL canonicalises to its @/archive/<slug>/@ page key — computed
|
||||
-- exactly as 'backlinksFieldWith' computes the archive page's own key (the
|
||||
-- same string fed through 'normaliseUrl'), so the two always agree. Every
|
||||
-- other URL is normalised as before.
|
||||
targetKey :: T.Text -> T.Text
|
||||
targetKey u = case archiveSlugFor u of
|
||||
Just slug -> T.pack (normaliseUrl ("/archive/" ++ slug ++ "/index.html"))
|
||||
Nothing -> T.pack (normaliseUrl (T.unpack u))
|
||||
|
||||
-- | The fragment (without @#@) of an archived URL, for granular grouping
|
||||
-- of "Referenced by". Empty for a non-archived URL or one with no fragment
|
||||
-- — so granular grouping stays an archive-only behaviour.
|
||||
archiveFragment :: T.Text -> String
|
||||
archiveFragment u = case archiveSlugFor u of
|
||||
Just _ -> T.unpack (T.drop 1 (T.dropWhile (/= '#') u))
|
||||
Nothing -> ""
|
||||
|
||||
-- ---------------------------------------------------------------------------
|
||||
-- Content patterns (must match the rules in Site.hs — sourced from
|
||||
-- Patterns.allContent so additions to the canonical list automatically
|
||||
-- propagate to backlinks).
|
||||
-- ---------------------------------------------------------------------------
|
||||
|
||||
allContent :: Pattern
|
||||
allContent = P.allContent
|
||||
|
||||
-- ---------------------------------------------------------------------------
|
||||
-- Hakyll rules
|
||||
-- ---------------------------------------------------------------------------
|
||||
|
||||
-- | Register the @version "links"@ rules for all content and the
|
||||
-- @create ["data/backlinks.json"]@ rule. Call this from 'Site.rules'.
|
||||
backlinkRules :: Rules ()
|
||||
backlinkRules = do
|
||||
-- Pass 1: extract links + context from each content file.
|
||||
match allContent $ version "links" $
|
||||
compile linksCompiler
|
||||
|
||||
-- Pass 2: invert the map and write the backlinks JSON.
|
||||
create ["data/backlinks.json"] $ do
|
||||
route idRoute
|
||||
compile $ do
|
||||
items <- loadAll (allContent .&&. hasVersion "links")
|
||||
:: Compiler [Item String]
|
||||
pairs <- concat <$> mapM toSourcePairs items
|
||||
makeItem . TL.unpack . TLE.decodeUtf8 . Aeson.encode
|
||||
$ Map.fromListWith (++) [(k, [v]) | (k, v) <- pairs]
|
||||
|
||||
-- | For one "links" item, produce @(normalised-target-url, BacklinkSource)@
|
||||
-- pairs — one per internal link found in the source file.
|
||||
toSourcePairs :: Item String -> Compiler [(T.Text, BacklinkSource)]
|
||||
toSourcePairs item = do
|
||||
let ident0 = setVersion Nothing (itemIdentifier item)
|
||||
mRoute <- getRoute ident0
|
||||
meta <- getMetadata ident0
|
||||
let srcUrl = maybe "" (\r -> "/" ++ r) mRoute
|
||||
let title = fromMaybe "(untitled)" (lookupString "title" meta)
|
||||
let abstract = fromMaybe "" (lookupString "abstract" meta)
|
||||
case mRoute of
|
||||
Nothing -> return []
|
||||
Just _ ->
|
||||
case Aeson.decodeStrict (TE.encodeUtf8 (T.pack (itemBody item)))
|
||||
:: Maybe [LinkEntry] of
|
||||
Nothing -> return []
|
||||
Just entries ->
|
||||
return [ ( targetKey (leUrl e)
|
||||
, BacklinkSource srcUrl title abstract
|
||||
(leSentence e)
|
||||
(leParagraph e)
|
||||
(archiveFragment (leUrl e))
|
||||
)
|
||||
| e <- entries ]
|
||||
|
||||
-- ---------------------------------------------------------------------------
|
||||
-- Context field
|
||||
-- ---------------------------------------------------------------------------
|
||||
|
||||
-- | Context field @$backlinks$@ that injects an HTML list of pages that link
|
||||
-- to the current page, each with its paragraph context.
|
||||
-- Returns @noResult@ (so @$if(backlinks)$@ is false) when there are none.
|
||||
backlinksField :: Context String
|
||||
backlinksField = backlinksFieldWith renderBacklinks "backlinks"
|
||||
|
||||
-- | "Referenced by" for archive pages. Same lookup as 'backlinksField',
|
||||
-- but the sources are grouped by the fragment each citation targets, so an
|
||||
-- archived work's page can show which section/page each citing essay points
|
||||
-- at (granular backlinks).
|
||||
referencedByField :: Context String
|
||||
referencedByField = backlinksFieldWith renderReferencedBy "referenced-by"
|
||||
|
||||
-- | Shared machinery for 'backlinksField' and 'referencedByField': look the
|
||||
-- page up in @data/backlinks.json@ by its normalised route, then hand the
|
||||
-- sorted sources to the given renderer.
|
||||
backlinksFieldWith :: ([BacklinkSource] -> String) -> String -> Context String
|
||||
backlinksFieldWith renderSources name = field name $ \item -> do
|
||||
blItem <- load (fromFilePath "data/backlinks.json") :: Compiler (Item String)
|
||||
case Aeson.decodeStrict (TE.encodeUtf8 (T.pack (itemBody blItem)))
|
||||
:: Maybe (Map T.Text [BacklinkSource]) of
|
||||
Nothing -> noResult "backlinks: could not parse data/backlinks.json"
|
||||
Just blMap -> do
|
||||
mRoute <- getRoute (itemIdentifier item)
|
||||
case mRoute of
|
||||
Nothing -> fail "backlinks: item has no route"
|
||||
Just r ->
|
||||
let key = T.pack (normaliseUrl ("/" ++ r))
|
||||
sources = fromMaybe [] (Map.lookup key blMap)
|
||||
sorted = sortBy (comparing blTitle) sources
|
||||
in if null sorted
|
||||
then fail "no backlinks"
|
||||
else return (renderSources sorted)
|
||||
|
||||
-- ---------------------------------------------------------------------------
|
||||
-- HTML rendering
|
||||
-- ---------------------------------------------------------------------------
|
||||
|
||||
-- | Render backlink sources as an HTML list. Each item shows:
|
||||
-- * the source title as a link (serif body face),
|
||||
-- * a <blockquote> of the sentence containing the link (default context),
|
||||
-- * a small hoverable "¶" affordance that reveals the full paragraph in
|
||||
-- a CSS-driven popup when hovered or keyboard-focused.
|
||||
--
|
||||
-- 'blSentence' and 'blParagraph' are already HTML fragments produced by
|
||||
-- the Pandoc writer, so they are emitted unescaped.
|
||||
renderBacklinks :: [BacklinkSource] -> String
|
||||
renderBacklinks sources =
|
||||
"<ul class=\"backlinks-list\">\n"
|
||||
++ concatMap renderBacklinkItem sources
|
||||
++ "</ul>"
|
||||
|
||||
-- | "Referenced by", grouped by the fragment each citation targets.
|
||||
-- Sources citing the work with no fragment render first as a plain list;
|
||||
-- each distinct fragment then gets its own subheading. With no fragments
|
||||
-- anywhere (the common case) this collapses to exactly the flat list.
|
||||
renderReferencedBy :: [BacklinkSource] -> String
|
||||
renderReferencedBy sources =
|
||||
let (general, fragmented) = partition (null . blFragment) sources
|
||||
groups = Map.toList $ Map.fromListWith (flip (++))
|
||||
[ (blFragment s, [s]) | s <- fragmented ]
|
||||
in renderList general ++ concatMap renderGroup groups
|
||||
where
|
||||
renderList [] = ""
|
||||
renderList ss = "<ul class=\"backlinks-list\">\n"
|
||||
++ concatMap renderBacklinkItem ss ++ "</ul>\n"
|
||||
renderGroup (frag, ss) =
|
||||
"<div class=\"referenced-by-group\">"
|
||||
++ "<h3 class=\"referenced-by-fragment\">"
|
||||
++ escapeHtml (fragmentLabel frag) ++ "</h3>"
|
||||
++ renderList ss
|
||||
++ "</div>\n"
|
||||
|
||||
-- | Human label for a cited fragment: a PDF @#page=N@ becomes "Page N";
|
||||
-- any other @#anchor@ is shown verbatim behind a section mark.
|
||||
fragmentLabel :: String -> String
|
||||
fragmentLabel frag =
|
||||
case stripPrefix "page=" frag of
|
||||
Just n -> "Page " ++ n
|
||||
Nothing -> "\x00A7 " ++ frag
|
||||
|
||||
-- | One backlink @<li>@: the source title as a link, the sentence of
|
||||
-- context as a blockquote, and a hover affordance revealing the full
|
||||
-- paragraph. 'blSentence' / 'blParagraph' are already HTML fragments from
|
||||
-- the Pandoc writer, so they are emitted unescaped.
|
||||
renderBacklinkItem :: BacklinkSource -> String
|
||||
renderBacklinkItem bl =
|
||||
"<li class=\"backlink-item\">"
|
||||
++ "<a class=\"backlink-source\" href=\""
|
||||
++ escapeHtml (blUrl bl) ++ "\">"
|
||||
++ escapeHtml (blTitle bl) ++ "</a>"
|
||||
++ ( if null (blSentence bl) then ""
|
||||
else "<blockquote class=\"backlink-quote\">"
|
||||
++ blSentence bl
|
||||
++ paragraphAffordance
|
||||
++ "</blockquote>" )
|
||||
++ "</li>\n"
|
||||
where
|
||||
paragraphAffordance
|
||||
| null (blParagraph bl) = ""
|
||||
| blParagraph bl == blSentence bl = ""
|
||||
| otherwise =
|
||||
"<span class=\"backlink-full\">"
|
||||
++ "<button type=\"button\" class=\"backlink-full-trigger\""
|
||||
++ " aria-label=\"Show full paragraph\" tabindex=\"0\">\x00B6</button>"
|
||||
++ "<span class=\"backlink-full-popup\" role=\"tooltip\">"
|
||||
++ blParagraph bl
|
||||
++ "</span>"
|
||||
++ "</span>"
|
||||
|
|
@ -0,0 +1,183 @@
|
|||
{-# LANGUAGE GHC2021 #-}
|
||||
-- | Parser for custom fields on BibLaTeX entries that citeproc doesn't
|
||||
-- surface on its own: @file:@ (path to a hosted PDF) and @keywords:@
|
||||
-- (comma-separated list, shared vocabulary with essay-frontmatter
|
||||
-- @keywords:@ for bibliography-page cross-linking). Also captures
|
||||
-- @author:@ and @year:@ used for bibliography-page sorting.
|
||||
--
|
||||
-- Character-based scanner with brace-balance tracking, so fields
|
||||
-- whose values span multiple lines parse correctly — e.g.:
|
||||
--
|
||||
-- @
|
||||
-- \@inproceedings{kyber2018,
|
||||
-- author = {Bos, Joppe W. and Ducas, Léo and ...
|
||||
-- and Stehlé, Damien},
|
||||
-- title = {{CRYSTALS -- Kyber}},
|
||||
-- year = {2018}
|
||||
-- }
|
||||
-- @
|
||||
--
|
||||
-- Field values enclosed in @{...}@ (balanced) or @"..."@ are both
|
||||
-- recognized. Unknown fields are ignored.
|
||||
module BibExtras
|
||||
( BibExtra (..)
|
||||
, emptyBibExtra
|
||||
, parseBibExtras
|
||||
, firstAuthorSurname
|
||||
) where
|
||||
|
||||
import Data.Char (isAlphaNum, isSpace, toLower)
|
||||
import Data.List (dropWhileEnd)
|
||||
import Data.Map.Strict (Map)
|
||||
import qualified Data.Map.Strict as Map
|
||||
import System.IO (readFile')
|
||||
|
||||
|
||||
-- | Custom fields we extract per citekey. Fields absent from the
|
||||
-- entry normalize to @Nothing@ / @[]@.
|
||||
data BibExtra = BibExtra
|
||||
{ bibFile :: Maybe FilePath -- ^ @file:@ — URL path to a hosted PDF.
|
||||
, bibKeywords :: [String] -- ^ @keywords:@ — comma-split, trimmed.
|
||||
, bibAuthor :: Maybe String -- ^ @author:@ — raw value, sort key only.
|
||||
, bibYear :: Maybe String -- ^ @year:@ — raw value, sort key only.
|
||||
} deriving (Show)
|
||||
|
||||
-- | Neutral default for a citekey with no custom fields.
|
||||
emptyBibExtra :: BibExtra
|
||||
emptyBibExtra = BibExtra Nothing [] Nothing Nothing
|
||||
|
||||
-- | First-author surname for alphabetic sort. Conservative extraction:
|
||||
-- take everything up to the first comma of the first author entry.
|
||||
-- BibLaTeX author format separates authors with " and ", so
|
||||
-- "Nietzsche, Friedrich and Holub, Robert C." → "Nietzsche".
|
||||
-- Corporate authors like "{National Institute of ...}" strip the
|
||||
-- outer braces (the parser drops them) and sort by the full name.
|
||||
-- Entries without an author sort under the empty string.
|
||||
firstAuthorSurname :: BibExtra -> String
|
||||
firstAuthorSurname extra = case bibAuthor extra of
|
||||
Just s -> trim (takeWhile (/= ',') (stripOuterBraces s))
|
||||
Nothing -> ""
|
||||
where
|
||||
stripOuterBraces ('{':rest) = dropWhileEnd (== '}') rest
|
||||
stripOuterBraces s = s
|
||||
|
||||
|
||||
-- | Parse a @.bib@ file; returns a map @citekey -> 'BibExtra'@.
|
||||
parseBibExtras :: FilePath -> IO (Map String BibExtra)
|
||||
parseBibExtras path = Map.fromList . parseBib <$> readFile' path
|
||||
|
||||
|
||||
-- ---------------------------------------------------------------------------
|
||||
-- Character-based scanner
|
||||
-- ---------------------------------------------------------------------------
|
||||
|
||||
-- | Enumerate all entries in a .bib file as (citekey, extra) pairs.
|
||||
-- @\@string@ \/ @\@comment@ \/ @\@preamble@ blocks (case-insensitive)
|
||||
-- carry no citekey and are skipped wholesale.
|
||||
parseBib :: String -> [(String, BibExtra)]
|
||||
parseBib input = go (dropTo '@' input)
|
||||
where
|
||||
-- Advance past any non-entry prefix to the first '@'.
|
||||
dropTo c = dropWhile (/= c)
|
||||
|
||||
go [] = []
|
||||
go ('@':rest) =
|
||||
let -- Entry type, then '{', then citekey, then ',', then fields, then '}'.
|
||||
(typeName, r1) = span isAlphaNum rest
|
||||
r2 = dropWhile isSpace r1
|
||||
in case r2 of
|
||||
'{':r3
|
||||
-- Not citekey entries: a @string macro name (or the body
|
||||
-- of a @comment/@preamble) must never be parsed as a
|
||||
-- citekey. Skip the balanced brace group and carry on.
|
||||
| map toLower typeName `elem` ["string", "comment", "preamble"] ->
|
||||
let (_, r4) = readBraces 1 "" r3
|
||||
in go (dropTo '@' r4)
|
||||
| otherwise ->
|
||||
let (citekey, r4) = span (\c -> c /= ',' && not (isSpace c)) r3
|
||||
r5 = dropWhile (\c -> c /= ',' && c /= '}') r4
|
||||
in case r5 of
|
||||
',':r6 ->
|
||||
let (flds, r7) = parseFields r6
|
||||
in (trim citekey, toExtra flds) : go (dropTo '@' r7)
|
||||
-- Fieldless entries: walk past and carry on.
|
||||
'}':r6 -> (trim citekey, emptyBibExtra) : go (dropTo '@' r6)
|
||||
_ -> []
|
||||
_ -> go (dropTo '@' r2)
|
||||
go (_:rest) = go (dropTo '@' rest)
|
||||
|
||||
-- | Parse fields until the closing '}' of the enclosing entry.
|
||||
-- Accepts @name = {value}@, @name = "value"@, or trailing commas.
|
||||
parseFields :: String -> ([(String, String)], String)
|
||||
parseFields = go
|
||||
where
|
||||
go s =
|
||||
let s' = dropWhile isSkippable s
|
||||
in case s' of
|
||||
[] -> ([], [])
|
||||
'}':rest -> ([], rest)
|
||||
_ -> case parseField s' of
|
||||
Nothing -> ([], s') -- malformed; stop collecting
|
||||
Just (nv, rest) ->
|
||||
let (more, rest') = go rest
|
||||
in (nv : more, rest')
|
||||
|
||||
isSkippable c = isSpace c || c == ','
|
||||
|
||||
-- | Parse a single @name = value@ field.
|
||||
parseField :: String -> Maybe ((String, String), String)
|
||||
parseField s =
|
||||
let (name, r1) = span (\c -> isAlphaNum c || c == '_') (dropWhile isSpace s)
|
||||
r2 = dropWhile isSpace r1
|
||||
in case r2 of
|
||||
'=':r3 -> do
|
||||
let r4 = dropWhile isSpace r3
|
||||
(value, r5) <- readFieldValue r4
|
||||
return ((map toLower (trim name), value), r5)
|
||||
_ -> Nothing
|
||||
|
||||
-- | Read a field's value, honoring nested braces and quoted forms.
|
||||
readFieldValue :: String -> Maybe (String, String)
|
||||
readFieldValue ('{':rest) = Just (readBraces 1 "" rest)
|
||||
readFieldValue ('"':rest) = Just (readQuote "" rest)
|
||||
readFieldValue _ = Nothing
|
||||
|
||||
-- | Read characters up to the matching @}@ that closes the outermost
|
||||
-- @{@; preserves interior @{@ / @}@ pairs as part of the value.
|
||||
readBraces :: Int -> String -> String -> (String, String)
|
||||
readBraces 0 acc r = (reverse acc, r)
|
||||
readBraces _ acc [] = (reverse acc, [])
|
||||
readBraces 1 acc ('}':r) = (reverse acc, r) -- outer close
|
||||
readBraces n acc ('{':r) = readBraces (n + 1) ('{' : acc) r
|
||||
readBraces n acc ('}':r) = readBraces (n - 1) ('}' : acc) r
|
||||
readBraces n acc (c:r) = readBraces n (c : acc) r
|
||||
|
||||
-- | Read characters up to the closing @"@.
|
||||
readQuote :: String -> String -> (String, String)
|
||||
readQuote acc ('"':r) = (reverse acc, r)
|
||||
readQuote acc [] = (reverse acc, [])
|
||||
readQuote acc (c:r) = readQuote (c : acc) r
|
||||
|
||||
-- | Build a 'BibExtra' from the parsed fields list.
|
||||
toExtra :: [(String, String)] -> BibExtra
|
||||
toExtra flds = BibExtra
|
||||
{ bibFile = lookup "file" flds
|
||||
, bibKeywords = case lookup "keywords" flds of
|
||||
Nothing -> []
|
||||
Just s -> filter (not . null) (map trim (splitOn ',' s))
|
||||
, bibAuthor = lookup "author" flds
|
||||
, bibYear = lookup "year" flds
|
||||
}
|
||||
|
||||
|
||||
-- ---------------------------------------------------------------------------
|
||||
-- Utilities
|
||||
-- ---------------------------------------------------------------------------
|
||||
|
||||
trim :: String -> String
|
||||
trim = dropWhile isSpace . dropWhileEnd isSpace
|
||||
|
||||
splitOn :: Eq a => a -> [a] -> [[a]]
|
||||
splitOn c xs = case break (== c) xs of
|
||||
(before, []) -> [before]
|
||||
(before, _ : rest) -> before : splitOn c rest
|
||||
|
|
@ -0,0 +1,253 @@
|
|||
{-# LANGUAGE GHC2021 #-}
|
||||
{-# LANGUAGE OverloadedStrings #-}
|
||||
-- | Music catalog: featured works + grouped-by-category listing.
|
||||
-- Renders HTML directly (same pattern as Backlinks.hs) to avoid the
|
||||
-- complexity of nested listFieldWith.
|
||||
module Catalog
|
||||
( musicCatalogCtx
|
||||
) where
|
||||
|
||||
import Data.Char (isSpace, toLower)
|
||||
import Data.List (groupBy, isPrefixOf, sortBy)
|
||||
import Data.Maybe (fromMaybe)
|
||||
import Data.Ord (comparing)
|
||||
import Data.Aeson (Value (..))
|
||||
import qualified Data.Aeson.KeyMap as KM
|
||||
import qualified Data.Vector as V
|
||||
import qualified Data.Text as T
|
||||
import Hakyll
|
||||
import Contexts (scorePageList, siteCtx)
|
||||
|
||||
-- ---------------------------------------------------------------------------
|
||||
-- Entry type
|
||||
-- ---------------------------------------------------------------------------
|
||||
|
||||
data CatalogEntry = CatalogEntry
|
||||
{ ceTitle :: String
|
||||
, ceUrl :: String
|
||||
, ceYear :: Maybe String
|
||||
, ceDuration :: Maybe String
|
||||
, ceInstrumentation :: Maybe String
|
||||
, ceCategory :: String -- defaults to "other"
|
||||
, ceFeatured :: Bool
|
||||
, ceHasScore :: Bool
|
||||
, ceHasRecording :: Bool
|
||||
}
|
||||
|
||||
-- ---------------------------------------------------------------------------
|
||||
-- Category helpers
|
||||
-- ---------------------------------------------------------------------------
|
||||
|
||||
categoryOrder :: [String]
|
||||
categoryOrder = ["orchestral","chamber","solo","vocal","choral","electronic","other"]
|
||||
|
||||
categoryLabel :: String -> String
|
||||
categoryLabel "orchestral" = "Orchestral"
|
||||
categoryLabel "chamber" = "Chamber"
|
||||
categoryLabel "solo" = "Solo"
|
||||
categoryLabel "vocal" = "Vocal"
|
||||
categoryLabel "choral" = "Choral"
|
||||
categoryLabel "electronic" = "Electronic"
|
||||
categoryLabel _ = "Other"
|
||||
|
||||
categoryRank :: String -> Int
|
||||
categoryRank c = fromMaybe (length categoryOrder)
|
||||
(lookup c (zip categoryOrder [0..]))
|
||||
|
||||
-- ---------------------------------------------------------------------------
|
||||
-- Parsing helpers
|
||||
-- ---------------------------------------------------------------------------
|
||||
|
||||
-- | @featured: true@ in YAML becomes Bool True in Aeson; also accept the
|
||||
-- string "true" in case the author quotes it.
|
||||
isFeatured :: Metadata -> Bool
|
||||
isFeatured meta =
|
||||
case KM.lookup "featured" meta of
|
||||
Just (Bool True) -> True
|
||||
Just (String "true") -> True
|
||||
_ -> False
|
||||
|
||||
-- | True if a @recording@ key is present, or any movement has an @audio@ key.
|
||||
hasRecordingMeta :: Metadata -> Bool
|
||||
hasRecordingMeta meta =
|
||||
KM.member "recording" meta || anyMovHasAudio meta
|
||||
where
|
||||
anyMovHasAudio m =
|
||||
case KM.lookup "movements" m of
|
||||
Just (Array v) -> any movHasAudio (V.toList v)
|
||||
_ -> False
|
||||
movHasAudio (Object o) = KM.member "audio" o
|
||||
movHasAudio _ = False
|
||||
|
||||
-- | Parse a year: accepts Number (e.g. @year: 2019@) or String.
|
||||
parseYear :: Metadata -> Maybe String
|
||||
parseYear meta =
|
||||
case KM.lookup "year" meta of
|
||||
Just (Number n) -> Just $ show (floor (fromRational (toRational n) :: Double) :: Int)
|
||||
Just (String t) -> Just (T.unpack t)
|
||||
_ -> Nothing
|
||||
|
||||
parseCatalogEntry :: Item String -> Compiler (Maybe CatalogEntry)
|
||||
parseCatalogEntry item = do
|
||||
meta <- getMetadata (itemIdentifier item)
|
||||
mRoute <- getRoute (itemIdentifier item)
|
||||
-- Through 'scorePageList' rather than reading @score-pages@ here, so the
|
||||
-- catalog's score indicator cannot disagree with what the reader shows:
|
||||
-- a composition that declares its pages with @score-dir@ has no
|
||||
-- @score-pages@ key to find.
|
||||
pages <- scorePageList item
|
||||
case mRoute of
|
||||
Nothing -> return Nothing
|
||||
Just r -> do
|
||||
let title = fromMaybe "(untitled)" (lookupString "title" meta)
|
||||
url = "/" ++ r
|
||||
year = parseYear meta
|
||||
dur = lookupString "duration" meta
|
||||
instr = lookupString "instrumentation" meta
|
||||
-- Fold unknown categories into the canonical "other"
|
||||
-- bucket here: two distinct unknown values share a rank
|
||||
-- but would groupBy into separate groups, rendering as
|
||||
-- adjacent duplicate "Other" sections.
|
||||
rawCat = fromMaybe "other" (lookupString "category" meta)
|
||||
cat = if rawCat `elem` categoryOrder then rawCat else "other"
|
||||
return $ Just CatalogEntry
|
||||
{ ceTitle = title
|
||||
, ceUrl = url
|
||||
, ceYear = year
|
||||
, ceDuration = dur
|
||||
, ceInstrumentation = instr
|
||||
, ceCategory = cat
|
||||
, ceFeatured = isFeatured meta
|
||||
, ceHasScore = not (null pages)
|
||||
, ceHasRecording = hasRecordingMeta meta
|
||||
}
|
||||
|
||||
-- ---------------------------------------------------------------------------
|
||||
-- HTML rendering
|
||||
-- ---------------------------------------------------------------------------
|
||||
--
|
||||
-- Trust model: per the site convention (see also Stats.hs:pageLink),
|
||||
-- frontmatter @title@ values are author-controlled trusted HTML and may
|
||||
-- contain inline markup such as @<em>...</em>@. They are emitted
|
||||
-- pre-escaped — but we still escape every other interpolated frontmatter
|
||||
-- value (year, duration, instrumentation) and sanitize hrefs through
|
||||
-- 'safeHref', so a stray @<@ in those fields cannot break the markup.
|
||||
|
||||
-- | Defense-in-depth href sanitiser. Mirrors 'Stats.isSafeUrl'.
|
||||
safeHref :: String -> String
|
||||
safeHref u =
|
||||
let norm = map toLower (dropWhile isSpace u)
|
||||
in if not ("//" `isPrefixOf` norm)
|
||||
&& any (`isPrefixOf` norm) ["/", "https://", "mailto:", "#"]
|
||||
then escAttr u
|
||||
else "#"
|
||||
|
||||
escAttr :: String -> String
|
||||
escAttr = concatMap esc
|
||||
where
|
||||
esc '&' = "&"
|
||||
esc '<' = "<"
|
||||
esc '>' = ">"
|
||||
esc '"' = """
|
||||
esc '\'' = "'"
|
||||
esc c = [c]
|
||||
|
||||
escText :: String -> String
|
||||
escText = concatMap esc
|
||||
where
|
||||
esc '&' = "&"
|
||||
esc '<' = "<"
|
||||
esc '>' = ">"
|
||||
esc c = [c]
|
||||
|
||||
renderIndicators :: CatalogEntry -> String
|
||||
renderIndicators e = concatMap render
|
||||
[ (ceHasScore e, "<span class=\"catalog-ind\" title=\"Score available\">◼</span>")
|
||||
, (ceHasRecording e, "<span class=\"catalog-ind\" title=\"Recording available\">♪</span>")
|
||||
]
|
||||
where
|
||||
render (True, s) = s
|
||||
render (False, _) = ""
|
||||
|
||||
renderEntry :: CatalogEntry -> String
|
||||
renderEntry e = concat
|
||||
[ "<li class=\"catalog-entry\">"
|
||||
, "<div class=\"catalog-entry-main\">"
|
||||
, "<a class=\"catalog-title\" href=\"", safeHref (ceUrl e), "\">"
|
||||
, ceTitle e
|
||||
, "</a>"
|
||||
, renderIndicators e
|
||||
, maybe "" (\y -> "<span class=\"catalog-year\">" ++ escText y ++ "</span>") (ceYear e)
|
||||
, maybe "" (\d -> "<span class=\"catalog-duration\">" ++ escText d ++ "</span>") (ceDuration e)
|
||||
, "</div>"
|
||||
, maybe "" (\i -> "<div class=\"catalog-instrumentation\">" ++ escText i ++ "</div>") (ceInstrumentation e)
|
||||
, "</li>"
|
||||
]
|
||||
|
||||
renderCategorySection :: String -> [CatalogEntry] -> String
|
||||
renderCategorySection cat entries = concat
|
||||
[ "<section class=\"catalog-section\">"
|
||||
, "<h2 class=\"catalog-section-title\">", escText (categoryLabel cat), "</h2>"
|
||||
, "<ul class=\"catalog-list\">"
|
||||
, concatMap renderEntry entries
|
||||
, "</ul>"
|
||||
, "</section>"
|
||||
]
|
||||
|
||||
-- ---------------------------------------------------------------------------
|
||||
-- Load all compositions (excluding the catalog index itself)
|
||||
-- ---------------------------------------------------------------------------
|
||||
|
||||
loadEntries :: Compiler [CatalogEntry]
|
||||
loadEntries = do
|
||||
items <- loadAll ("content/music/*/index.md" .&&. hasNoVersion)
|
||||
mItems <- mapM parseCatalogEntry items
|
||||
return [e | Just e <- mItems]
|
||||
|
||||
-- ---------------------------------------------------------------------------
|
||||
-- Context fields
|
||||
-- ---------------------------------------------------------------------------
|
||||
|
||||
-- | @$featured-works$@: HTML list of featured entries; noResult when none.
|
||||
featuredWorksField :: Context String
|
||||
featuredWorksField = field "featured-works" $ \_ -> do
|
||||
entries <- loadEntries
|
||||
let featured = filter ceFeatured entries
|
||||
if null featured
|
||||
then fail "no featured works"
|
||||
else return $
|
||||
"<ul class=\"catalog-list catalog-featured-list\">"
|
||||
++ concatMap renderEntry featured
|
||||
++ "</ul>"
|
||||
|
||||
-- | @$has-featured$@: present when at least one composition is featured.
|
||||
hasFeaturedField :: Context String
|
||||
hasFeaturedField = field "has-featured" $ \_ -> do
|
||||
entries <- loadEntries
|
||||
if any ceFeatured entries then return "true" else fail "no featured works"
|
||||
|
||||
-- | @$catalog-by-category$@: HTML for all category sections.
|
||||
-- Sorted by canonical category order; if no compositions exist yet,
|
||||
-- returns a placeholder paragraph.
|
||||
catalogByCategoryField :: Context String
|
||||
catalogByCategoryField = field "catalog-by-category" $ \_ -> do
|
||||
entries <- loadEntries
|
||||
if null entries
|
||||
then return "<p class=\"catalog-empty\">Works forthcoming.</p>"
|
||||
else do
|
||||
let sorted = sortBy (comparing (categoryRank . ceCategory)) entries
|
||||
grouped = groupBy (\a b -> ceCategory a == ceCategory b) sorted
|
||||
return $ concatMap renderGroup grouped
|
||||
where
|
||||
-- groupBy on a non-empty list yields non-empty sublists, so the
|
||||
-- (e:_) pattern is structurally guaranteed in this call site.
|
||||
renderGroup g@(e : _) = renderCategorySection (ceCategory e) g
|
||||
renderGroup [] = "" -- unreachable; satisfies coverage checker
|
||||
|
||||
musicCatalogCtx :: Context String
|
||||
musicCatalogCtx =
|
||||
constField "catalog" "true"
|
||||
<> hasFeaturedField
|
||||
<> featuredWorksField
|
||||
<> catalogByCategoryField
|
||||
<> siteCtx
|
||||
|
|
@ -0,0 +1,393 @@
|
|||
{-# LANGUAGE GHC2021 #-}
|
||||
{-# LANGUAGE LambdaCase #-}
|
||||
{-# LANGUAGE OverloadedStrings #-}
|
||||
-- | Citation processing pipeline.
|
||||
--
|
||||
-- Steps:
|
||||
-- 1. Skip if the document contains no Cite nodes and frKeys is empty.
|
||||
-- 2. Inject default bibliography / CSL metadata if absent.
|
||||
-- 3. Inject nocite entries for further-reading keys.
|
||||
-- 4. Run Pandoc's citeproc to resolve references and generate bibliography.
|
||||
-- 5. Walk the AST and replace Cite nodes with numbered superscripts.
|
||||
-- 6. Extract the citeproc bibliography div from the body, reorder by
|
||||
-- first-appearance, split into cited / further-reading sections,
|
||||
-- and render to an HTML string for the template's $bibliography$ field.
|
||||
--
|
||||
-- Returns (Pandoc without refs div, bibliography HTML).
|
||||
-- The bibliography HTML is empty when there are no citations.
|
||||
--
|
||||
-- NOTE: processCitations with in-text CSL leaves Cite nodes as Cite nodes
|
||||
-- in the AST — it only populates their inline content and creates the refs
|
||||
-- div. The HTML writer later wraps them in <span class="citation">. We must
|
||||
-- therefore match Cite nodes (not Span nodes) in our transform pass.
|
||||
--
|
||||
-- NOTE: Hakyll strips YAML frontmatter before passing to readPandocWith, so
|
||||
-- the Pandoc Meta is empty. further-reading keys are passed explicitly by the
|
||||
-- caller (read from Hakyll's own metadata via lookupStringList).
|
||||
--
|
||||
-- NOTE: Does not import Contexts to avoid cycles.
|
||||
module Citations
|
||||
( applyCitations
|
||||
-- * For synthetic bibliography pages (Phase 6b)
|
||||
, renderBibliographyHtml
|
||||
) where
|
||||
|
||||
import Data.List (intercalate, intersperse, nub, partition, sortBy)
|
||||
import Data.Map.Strict (Map)
|
||||
import qualified Data.Map.Strict as Map
|
||||
import Data.Maybe (fromMaybe, mapMaybe)
|
||||
import Data.Ord (comparing)
|
||||
import Data.Text (Text)
|
||||
import qualified Data.Text as T
|
||||
import Text.Pandoc
|
||||
import Text.Pandoc.Citeproc (processCitations)
|
||||
import Text.Pandoc.Walk
|
||||
|
||||
import BibExtras (BibExtra (..), emptyBibExtra, parseBibExtras)
|
||||
import qualified Filters.Archive (annotateBlock)
|
||||
|
||||
|
||||
-- ---------------------------------------------------------------------------
|
||||
-- Public API
|
||||
-- ---------------------------------------------------------------------------
|
||||
|
||||
-- | Process citations in a Pandoc document.
|
||||
-- @frKeys@: further-reading citation keys (read from Hakyll metadata by
|
||||
-- the caller, since Hakyll strips YAML frontmatter before parsing).
|
||||
-- Returns @(body, citedHtml, furtherHtml)@ where @body@ has Cite nodes
|
||||
-- replaced with numbered superscripts and no bibliography div,
|
||||
-- @citedHtml@ is the inline-cited references HTML, and @furtherHtml@ is
|
||||
-- the further-reading-only references HTML (each empty when absent).
|
||||
applyCitations :: [Text] -> Text -> Pandoc -> IO (Pandoc, Text, Text)
|
||||
applyCitations frKeys bibPath doc
|
||||
| not (hasCitations frKeys doc) = return (doc, "", "")
|
||||
| otherwise = do
|
||||
-- Read custom fields (@file:@, @keywords:@) from the .bib file
|
||||
-- in parallel with citeproc. These don't affect citation
|
||||
-- resolution — they enhance the rendered bibliography entries.
|
||||
extras <- parseBibExtras (T.unpack bibPath)
|
||||
let doc1 = injectMeta frKeys bibPath doc
|
||||
processed <- runIOorExplode $ processCitations doc1
|
||||
let (body, citedHtml, furtherHtml) = transformAndExtract extras frKeys processed
|
||||
return (body, citedHtml, furtherHtml)
|
||||
|
||||
-- | Render a standalone bibliography section from a list of citekeys and
|
||||
-- a set of @.bib@ file paths. Used by the synthetic @\/bibliography\/@
|
||||
-- pages (Phase 6b) to produce CSL-formatted entries outside of any
|
||||
-- essay's citation context.
|
||||
--
|
||||
-- Given citekeys are passed to citeproc via a synthesized @nocite@
|
||||
-- metadata entry on an otherwise empty document; citeproc emits a
|
||||
-- @refs@ Div whose children are the rendered entries. We then reorder
|
||||
-- the children to match the caller-supplied @keys@ list (citeproc's
|
||||
-- own ordering is overridden so callers control sort), enhance each
|
||||
-- entry with the Phase 6a PDF-link and keyword-strip hooks, and
|
||||
-- render to HTML wrapped in @\<div class="csl-bib-body"\>@.
|
||||
--
|
||||
-- @extras@ is the combined 'BibExtra' map for the same @.bib@ files;
|
||||
-- passed in so that 'enhanceEntry' can consult @file:@ and
|
||||
-- @keywords:@ without each entry re-parsing the files.
|
||||
renderBibliographyHtml :: [FilePath] -- ^ .bib paths
|
||||
-> Map String BibExtra -- ^ enhancement map
|
||||
-> [String] -- ^ citekeys, in desired order
|
||||
-> IO Text
|
||||
renderBibliographyHtml _ _ [] = return ""
|
||||
renderBibliographyHtml bibPaths extras keys = do
|
||||
let doc = synthesizeNociteDoc bibPaths keys
|
||||
processed <- runIOorExplode $ processCitations doc
|
||||
let refsDivs = concatMap unwrapRefs (pandocBlocks processed)
|
||||
ordered = reorderByKeys keys refsDivs
|
||||
enhanced = map (annotateArchive . enhanceEntry extras) ordered
|
||||
return (renderEntries "csl-bib-body" enhanced)
|
||||
where
|
||||
pandocBlocks (Pandoc _ bs) = bs
|
||||
unwrapRefs (Div ("refs", _, _) children) = children
|
||||
unwrapRefs _ = []
|
||||
|
||||
-- | Build a Pandoc doc whose only citation-relevant content is a
|
||||
-- @nocite@ metadata entry listing every supplied citekey. Runs
|
||||
-- through 'processCitations' to emit a fully-formatted @refs@ Div
|
||||
-- containing every entry.
|
||||
synthesizeNociteDoc :: [FilePath] -> [String] -> Pandoc
|
||||
synthesizeNociteDoc bibPaths keys =
|
||||
let meta = Meta $ Map.fromList
|
||||
[ ("bibliography", bibPathMeta bibPaths)
|
||||
, ("csl", MetaString "data/chicago-notes.csl")
|
||||
, ("nocite", nociteVal (map T.pack keys))
|
||||
]
|
||||
in Pandoc meta []
|
||||
where
|
||||
bibPathMeta [p] = MetaString (T.pack p)
|
||||
bibPathMeta ps = MetaList (map (MetaString . T.pack) ps)
|
||||
|
||||
nociteVal ks = MetaInlines (intercalate [Space] (map mkCite ks))
|
||||
mkCite k = [Cite [Citation k [] [] AuthorInText 1 0] [Str ("@" <> k)]]
|
||||
|
||||
-- | Reorder a list of @csl-entry@ Divs to match a requested key order.
|
||||
-- Divs not in the key list (shouldn't happen in practice, but safe
|
||||
-- by construction) drop to the end in their original order.
|
||||
reorderByKeys :: [String] -> [Block] -> [Block]
|
||||
reorderByKeys keys divs =
|
||||
let divMap = Map.fromList [ (T.unpack (stripRefPrefix d), blk)
|
||||
| blk@(Div (d, _, _) _) <- divs ]
|
||||
found = mapMaybe (`Map.lookup` divMap) keys
|
||||
leftovers = filter (\blk -> case blk of
|
||||
Div (d, _, _) _ ->
|
||||
T.unpack (stripRefPrefix d) `notElem` keys
|
||||
_ -> True) divs
|
||||
in found ++ leftovers
|
||||
|
||||
|
||||
-- ---------------------------------------------------------------------------
|
||||
-- Detection
|
||||
-- ---------------------------------------------------------------------------
|
||||
|
||||
-- | True if the document has inline [@key] cites or a further-reading list.
|
||||
hasCitations :: [Text] -> Pandoc -> Bool
|
||||
hasCitations frKeys doc =
|
||||
not (null (query collectCites doc))
|
||||
|| not (null frKeys)
|
||||
where
|
||||
collectCites (Cite {}) = [()]
|
||||
collectCites _ = []
|
||||
|
||||
|
||||
-- ---------------------------------------------------------------------------
|
||||
-- Metadata injection
|
||||
-- ---------------------------------------------------------------------------
|
||||
|
||||
-- | Inject default bibliography / CSL paths and nocite for further-reading.
|
||||
injectMeta :: [Text] -> Text -> Pandoc -> Pandoc
|
||||
injectMeta frKeys bibPath (Pandoc meta blocks) =
|
||||
let meta1 = if null frKeys then meta
|
||||
else insertMeta "nocite" (nociteVal frKeys) meta
|
||||
meta2 = case lookupMeta "bibliography" meta1 of
|
||||
Nothing -> insertMeta "bibliography"
|
||||
(MetaString bibPath) meta1
|
||||
Just _ -> meta1
|
||||
meta3 = case lookupMeta "csl" meta2 of
|
||||
Nothing -> insertMeta "csl"
|
||||
(MetaString "data/chicago-notes.csl") meta2
|
||||
Just _ -> meta2
|
||||
in Pandoc meta3 blocks
|
||||
where
|
||||
-- Each key becomes its own Cite node (matching what pandoc parses from
|
||||
-- nocite: "@key1 @key2" in YAML frontmatter).
|
||||
nociteVal keys = MetaInlines (intercalate [Space] (map mkCiteNode keys))
|
||||
mkCiteNode k = [Cite [Citation k [] [] AuthorInText 1 0] [Str ("@" <> k)]]
|
||||
|
||||
-- | Insert a key/value pair into Pandoc Meta.
|
||||
insertMeta :: Text -> MetaValue -> Meta -> Meta
|
||||
insertMeta k v (Meta m) = Meta (Map.insert k v m)
|
||||
|
||||
|
||||
-- ---------------------------------------------------------------------------
|
||||
-- Transform pass
|
||||
-- ---------------------------------------------------------------------------
|
||||
|
||||
-- | Number citation Cite nodes and extract the bibliography div.
|
||||
transformAndExtract :: Map String BibExtra -> [Text] -> Pandoc -> (Pandoc, Text, Text)
|
||||
transformAndExtract extras frKeys doc@(Pandoc meta _) =
|
||||
let citeOrder = collectCiteOrder doc -- keys, first-appearance order
|
||||
keyNums = Map.fromList (zip citeOrder [1 :: Int ..])
|
||||
-- Replace Cite nodes with numbered superscript markers
|
||||
doc' = walk (transformInline keyNums) doc
|
||||
-- Pull bibliography div out of body and render to HTML
|
||||
(bodyBlocks, citedHtml, furtherHtml) = extractBibliography extras citeOrder frKeys
|
||||
(pandocBlocks doc')
|
||||
in (Pandoc meta bodyBlocks, citedHtml, furtherHtml)
|
||||
where
|
||||
pandocBlocks (Pandoc _ bs) = bs
|
||||
|
||||
-- | Collect citation keys in order of first appearance (body only).
|
||||
-- NOTE: after processCitations, Cite nodes remain as Cite in the AST;
|
||||
-- they are not converted to Span nodes with in-text CSL.
|
||||
-- We query only blocks (not metadata) so that nocite Cite nodes injected
|
||||
-- into the 'nocite' meta field are not mistakenly treated as inline citations.
|
||||
collectCiteOrder :: Pandoc -> [Text]
|
||||
collectCiteOrder (Pandoc _ blocks) = nub (query extractKeys blocks)
|
||||
where
|
||||
extractKeys (Cite citations _) = map citationId citations
|
||||
extractKeys _ = []
|
||||
|
||||
-- | Replace a Cite node with a numbered superscript marker.
|
||||
transformInline :: Map Text Int -> Inline -> Inline
|
||||
transformInline keyNums (Cite citations _) =
|
||||
let keys = map citationId citations
|
||||
nums = mapMaybe (`Map.lookup` keyNums) keys
|
||||
in case (keys, nums) of
|
||||
-- Both lists are guaranteed non-empty by the @null nums@ check
|
||||
-- below, but pattern-match to keep this total instead of
|
||||
-- relying on @head@.
|
||||
(firstKey : _, firstNum : _) ->
|
||||
RawInline "html" (markerHtml keys firstKey firstNum nums)
|
||||
_ ->
|
||||
Str ""
|
||||
transformInline _ x = x
|
||||
|
||||
markerHtml :: [Text] -> Text -> Int -> [Int] -> Text
|
||||
markerHtml keys firstKey firstNum nums =
|
||||
let label = "[" <> T.intercalate "," (map tshow nums) <> "]"
|
||||
allIds = T.intercalate " " (map ("ref-" <>) keys)
|
||||
in "<sup class=\"cite-marker\" id=\"cite-back-" <> tshow firstNum <> "\">"
|
||||
<> "<a href=\"#ref-" <> firstKey <> "\" class=\"cite-link\""
|
||||
<> " data-cite-keys=\"" <> allIds <> "\">"
|
||||
<> label <> "</a></sup>"
|
||||
where tshow = T.pack . show
|
||||
|
||||
|
||||
-- ---------------------------------------------------------------------------
|
||||
-- Bibliography extraction + rendering
|
||||
-- ---------------------------------------------------------------------------
|
||||
|
||||
-- | Separate the @refs@ div from body blocks and render it to HTML.
|
||||
-- Returns @(bodyBlocks, citedHtml, furtherHtml)@.
|
||||
extractBibliography :: Map String BibExtra -> [Text] -> [Text] -> [Block]
|
||||
-> ([Block], Text, Text)
|
||||
extractBibliography extras citeOrder frKeys blocks =
|
||||
let (bodyBlocks, refDivs) = partition (not . isRefsDiv) blocks
|
||||
(citedHtml, furtherHtml) = case refDivs of
|
||||
[] -> ("", "")
|
||||
(d:_) -> renderBibDiv extras citeOrder frKeys d
|
||||
in (bodyBlocks, citedHtml, furtherHtml)
|
||||
where
|
||||
isRefsDiv (Div ("refs", _, _) _) = True
|
||||
isRefsDiv _ = False
|
||||
|
||||
-- | Render the citeproc @refs@ Div into two HTML strings:
|
||||
-- @(citedHtml, furtherHtml)@ — each is empty when there are no entries
|
||||
-- in that section. Headings are rendered in the template, not here.
|
||||
--
|
||||
-- Entry bodies are enhanced before numbering: title-wrapped as a
|
||||
-- @.pdf-link[data-pdf-src]@ when the .bib @file:@ field is set (so
|
||||
-- popups.js's PDF hover preview fires), and a trailing
|
||||
-- @\<div class="bib-keywords"\>@ appended when @keywords:@ is set.
|
||||
renderBibDiv :: Map String BibExtra -> [Text] -> [Text] -> Block -> (Text, Text)
|
||||
renderBibDiv extras citeOrder _frKeys (Div _ children) =
|
||||
let enhanced = map (annotateArchive . enhanceEntry extras) children
|
||||
keyIndex = Map.fromList (zip citeOrder [0 :: Int ..])
|
||||
(citedEntries, furtherEntries) =
|
||||
partition (isCited keyIndex) enhanced
|
||||
sorted = sortBy (comparing (entryOrder keyIndex)) citedEntries
|
||||
numbered = zipWith addNumber [1..] sorted
|
||||
citedHtml = renderEntries "csl-bib-body cite-refs" numbered
|
||||
furtherHtml
|
||||
| null furtherEntries = ""
|
||||
| otherwise = renderEntries "csl-bib-body further-reading-refs" furtherEntries
|
||||
in (citedHtml, furtherHtml)
|
||||
renderBibDiv _ _ _ _ = ("", "")
|
||||
|
||||
|
||||
-- ---------------------------------------------------------------------------
|
||||
-- Bib entry enhancement (Phase 6a)
|
||||
-- ---------------------------------------------------------------------------
|
||||
|
||||
-- | Bibliography-side archive annotation: the same affordance (and
|
||||
-- rotted-link flip) 'Filters.Archive' gives body links, applied to a
|
||||
-- rendered CSL entry. The bibliography never travels the body filter
|
||||
-- chain — it is extracted and rendered to an HTML string for the
|
||||
-- template's @$bibliography$@ field — so the entry Divs are annotated
|
||||
-- here, after 'enhanceEntry' (whose PDF-link title wrap must see the
|
||||
-- entry's original inline shape).
|
||||
annotateArchive :: Block -> Block
|
||||
annotateArchive = Filters.Archive.annotateBlock
|
||||
|
||||
-- | Augment a single @csl-entry@ Div with the custom fields we parsed
|
||||
-- from the .bib file. Other Blocks pass through unchanged.
|
||||
enhanceEntry :: Map String BibExtra -> Block -> Block
|
||||
enhanceEntry extras b@(Div attrs@(divId, _, _) blocks) =
|
||||
let key = T.unpack (stripRefPrefix divId)
|
||||
extra = fromMaybe emptyBibExtra (Map.lookup key extras)
|
||||
withLink = case bibFile extra of
|
||||
Nothing -> blocks
|
||||
Just fp -> map (wrapFirstTitleBlock (T.pack fp)) blocks
|
||||
withKw = withLink ++ keywordsBlocks (bibKeywords extra)
|
||||
in case (bibFile extra, bibKeywords extra) of
|
||||
(Nothing, []) -> b
|
||||
_ -> Div attrs withKw
|
||||
enhanceEntry _ b = b
|
||||
|
||||
-- | In one block of an entry, wrap the first title-bearing inline
|
||||
-- with a @.pdf-link@ anchor. Pandoc's CSL-formatted references
|
||||
-- render the title as either a @Quoted@ (article titles in
|
||||
-- Chicago-notes: "Paper Title") or an @Emph@ (book titles:
|
||||
-- /Book Title/), and those are the first such inline in each
|
||||
-- entry. We wrap at the block level and fall back to passing the
|
||||
-- block through if no matching inline appears.
|
||||
wrapFirstTitleBlock :: Text -> Block -> Block
|
||||
wrapFirstTitleBlock href = \case
|
||||
Para ils -> Para (wrapFirstTitle href ils)
|
||||
Plain ils -> Plain (wrapFirstTitle href ils)
|
||||
other -> other
|
||||
|
||||
-- | Left-to-right scan: wrap the first title-bearing inline in a link
|
||||
-- pointing at the PDF. Pandoc's CSL renderer emits article titles as
|
||||
-- @Span@ nodes (whose rendered HTML wraps quotation marks around the
|
||||
-- title text) and book titles as @Emph@; @Quoted@ appears in some
|
||||
-- other CSL styles. First match of any of these is treated as the
|
||||
-- title; subsequent ones pass through — journal names are also
|
||||
-- @Emph@ on @\@article@ entries but come after the @Span@ title, so
|
||||
-- the article case picks the right target.
|
||||
wrapFirstTitle :: Text -> [Inline] -> [Inline]
|
||||
wrapFirstTitle href inls = reverse . fst $ foldl step ([], False) inls
|
||||
where
|
||||
step (acc, True) inl = (inl:acc, True)
|
||||
step (acc, False) inl = case inl of
|
||||
Span _ _ -> (asPdfLink href [inl] : acc, True)
|
||||
Quoted _ _ -> (asPdfLink href [inl] : acc, True)
|
||||
Emph _ -> (asPdfLink href [inl] : acc, True)
|
||||
_ -> (inl:acc, False)
|
||||
|
||||
-- | Build the @.pdf-link[data-pdf-src]@ anchor that popups.js binds to.
|
||||
-- See @static/js/popups.js:112@ for the matching selector.
|
||||
asPdfLink :: Text -> [Inline] -> Inline
|
||||
asPdfLink href content =
|
||||
Link ("", ["pdf-link"], [("data-pdf-src", href)])
|
||||
content
|
||||
(href, "")
|
||||
|
||||
-- | Trailing keyword strip, linking each keyword to the future
|
||||
-- @/bibliography/\<keyword\>/@ page. Returns @[]@ when the keyword
|
||||
-- list is empty so the entry gets no extra block at all.
|
||||
keywordsBlocks :: [String] -> [Block]
|
||||
keywordsBlocks [] = []
|
||||
keywordsBlocks ks =
|
||||
[ Div ("", ["bib-keywords"], [])
|
||||
[Plain (intersperse (Str ", ") (map keywordLink ks))]
|
||||
]
|
||||
where
|
||||
keywordLink k =
|
||||
Link ("", ["bib-keyword"], [])
|
||||
[Str (T.pack k)]
|
||||
(T.pack ("/bibliography/" ++ k ++ "/"), "")
|
||||
|
||||
isCited :: Map Text Int -> Block -> Bool
|
||||
isCited keyIndex (Div (rid, _, _) _) = Map.member (stripRefPrefix rid) keyIndex
|
||||
isCited _ _ = False
|
||||
|
||||
entryOrder :: Map Text Int -> Block -> Int
|
||||
entryOrder keyIndex (Div (rid, _, _) _) =
|
||||
fromMaybe maxBound $ Map.lookup (stripRefPrefix rid) keyIndex
|
||||
entryOrder _ _ = maxBound
|
||||
|
||||
-- | Prepend [N] marker to a bibliography entry block.
|
||||
addNumber :: Int -> Block -> Block
|
||||
addNumber n (Div attrs@(divId, _, _) content) =
|
||||
Div attrs
|
||||
( Plain [ RawInline "html"
|
||||
("<a class=\"ref-num\" href=\"#" <> divId <> "\">[" <> T.pack (show n) <> "]</a>") ]
|
||||
: content )
|
||||
addNumber _ b = b
|
||||
|
||||
-- | Strip the @ref-@ prefix that citeproc adds to div IDs.
|
||||
stripRefPrefix :: Text -> Text
|
||||
stripRefPrefix t = fromMaybe t (T.stripPrefix "ref-" t)
|
||||
|
||||
-- | Render a list of blocks as an HTML string (used for bibliography sections).
|
||||
renderEntries :: Text -> [Block] -> Text
|
||||
renderEntries cls entries =
|
||||
case runPure (writeHtml5String wOpts (Pandoc nullMeta entries)) of
|
||||
Left _ -> ""
|
||||
Right html -> "<div class=\"" <> cls <> "\">\n" <> html <> "</div>\n"
|
||||
where
|
||||
wOpts = def { writerWrapText = WrapNone }
|
||||
|
|
@ -0,0 +1,165 @@
|
|||
{-# LANGUAGE GHC2021 #-}
|
||||
{-# LANGUAGE OverloadedStrings #-}
|
||||
-- | Commonplace book: loads data/commonplace.yaml and renders
|
||||
-- themed and chronological HTML views for /commonplace.
|
||||
module Commonplace
|
||||
( commonplaceCtx
|
||||
) where
|
||||
|
||||
import Data.Aeson (FromJSON (..), withObject, (.:), (.:?), (.!=))
|
||||
import Data.List (nub, sortBy)
|
||||
import Data.Ord (comparing, Down (..))
|
||||
import qualified Data.Text as T
|
||||
import qualified Data.Text.Encoding as TE
|
||||
import qualified Data.Yaml as Y
|
||||
import Hakyll hiding (escapeHtml, renderTags)
|
||||
import Contexts (siteCtx)
|
||||
import Utils (escapeHtml)
|
||||
|
||||
-- ---------------------------------------------------------------------------
|
||||
-- Entry type
|
||||
-- ---------------------------------------------------------------------------
|
||||
|
||||
data CPEntry = CPEntry
|
||||
{ cpText :: String
|
||||
, cpAttribution :: String
|
||||
, cpSource :: Maybe String
|
||||
, cpSourceUrl :: Maybe String
|
||||
, cpTags :: [String]
|
||||
, cpCommentary :: Maybe String
|
||||
, cpDateAdded :: String
|
||||
}
|
||||
|
||||
instance FromJSON CPEntry where
|
||||
parseJSON = withObject "CPEntry" $ \o -> CPEntry
|
||||
<$> o .: "text"
|
||||
<*> o .: "attribution"
|
||||
<*> o .:? "source"
|
||||
<*> o .:? "source-url"
|
||||
<*> o .:? "tags" .!= []
|
||||
<*> o .:? "commentary"
|
||||
<*> o .:? "date-added" .!= ""
|
||||
|
||||
-- ---------------------------------------------------------------------------
|
||||
-- HTML rendering
|
||||
-- ---------------------------------------------------------------------------
|
||||
|
||||
-- | Escape HTML, then replace newlines with <br> for multi-line verse.
|
||||
renderText :: String -> String
|
||||
renderText = concatMap tr . escapeHtml . stripTrailingNL
|
||||
where
|
||||
tr '\n' = "<br>\n"
|
||||
tr c = [c]
|
||||
stripTrailingNL = reverse . dropWhile (== '\n') . reverse
|
||||
|
||||
renderAttribution :: CPEntry -> String
|
||||
renderAttribution e =
|
||||
"<p class=\"cp-attribution\">\x2014\x202f"
|
||||
++ escapeHtml (cpAttribution e)
|
||||
++ maybe "" renderSource (cpSource e)
|
||||
++ "</p>"
|
||||
where
|
||||
renderSource src = case cpSourceUrl e of
|
||||
Just url -> ", <a href=\"" ++ escapeHtml url ++ "\">"
|
||||
++ escapeHtml src ++ "</a>"
|
||||
Nothing -> ", " ++ escapeHtml src
|
||||
|
||||
renderTags :: [String] -> String
|
||||
renderTags [] = ""
|
||||
renderTags ts =
|
||||
"<div class=\"cp-tags\">"
|
||||
++ concatMap (\t -> "<span class=\"cp-tag\">" ++ escapeHtml t ++ "</span>") ts
|
||||
++ "</div>"
|
||||
|
||||
renderEntry :: CPEntry -> String
|
||||
renderEntry e = concat
|
||||
[ "<article class=\"cp-entry\">"
|
||||
, "<blockquote class=\"cp-quote\"><p>"
|
||||
, renderText (cpText e)
|
||||
, "</p></blockquote>"
|
||||
, renderAttribution e
|
||||
, maybe "" renderCommentary (cpCommentary e)
|
||||
, renderTags (cpTags e)
|
||||
, "</article>"
|
||||
]
|
||||
where
|
||||
renderCommentary c =
|
||||
"<p class=\"cp-commentary\">" ++ escapeHtml c ++ "</p>"
|
||||
|
||||
-- ---------------------------------------------------------------------------
|
||||
-- Themed view
|
||||
-- ---------------------------------------------------------------------------
|
||||
|
||||
-- | All distinct tags in first-occurrence order (preserves YAML ordering).
|
||||
allTags :: [CPEntry] -> [String]
|
||||
allTags = nub . concatMap cpTags
|
||||
|
||||
renderTagSection :: String -> [CPEntry] -> String
|
||||
renderTagSection tag entries = concat
|
||||
[ "<section class=\"cp-theme-section\">"
|
||||
, "<h2 class=\"cp-theme-heading\">" ++ escapeHtml tag ++ "</h2>"
|
||||
, concatMap renderEntry entries
|
||||
, "</section>"
|
||||
]
|
||||
|
||||
renderThemedView :: [CPEntry] -> String
|
||||
renderThemedView [] =
|
||||
"<div class=\"cp-themed\" id=\"cp-themed\">"
|
||||
++ "<p class=\"cp-empty\">No entries yet.</p>"
|
||||
++ "</div>"
|
||||
renderThemedView entries =
|
||||
"<div class=\"cp-themed\" id=\"cp-themed\">"
|
||||
++ concatMap renderSection (allTags entries)
|
||||
++ (if null untagged then ""
|
||||
else renderTagSection "miscellany" untagged)
|
||||
++ "</div>"
|
||||
where
|
||||
renderSection t =
|
||||
let es = filter (elem t . cpTags) entries
|
||||
in if null es then "" else renderTagSection t es
|
||||
untagged = filter (null . cpTags) entries
|
||||
|
||||
-- ---------------------------------------------------------------------------
|
||||
-- Chronological view
|
||||
-- ---------------------------------------------------------------------------
|
||||
|
||||
renderChronoView :: [CPEntry] -> String
|
||||
renderChronoView entries =
|
||||
"<div class=\"cp-chrono\" id=\"cp-chrono\" hidden>"
|
||||
++ (if null sorted
|
||||
then "<p class=\"cp-empty\">No entries yet.</p>"
|
||||
else concatMap renderEntry sorted)
|
||||
++ "</div>"
|
||||
where
|
||||
sorted = sortBy (comparing (Down . cpDateAdded)) entries
|
||||
|
||||
-- ---------------------------------------------------------------------------
|
||||
-- Load entries from data/commonplace.yaml
|
||||
-- ---------------------------------------------------------------------------
|
||||
|
||||
loadCommonplace :: Compiler [CPEntry]
|
||||
loadCommonplace = do
|
||||
rawItem <- load (fromFilePath "data/commonplace.yaml") :: Compiler (Item String)
|
||||
let raw = itemBody rawItem
|
||||
-- encodeUtf8, not Char8.pack: Char8 truncates each Char to 8 bits,
|
||||
-- silently corrupting any codepoint above 0x7F (same hazard Now.hs
|
||||
-- documents — em-dash 0x2014 would become control char 0x14).
|
||||
case Y.decodeEither' (TE.encodeUtf8 (T.pack raw)) of
|
||||
Left err -> fail ("commonplace.yaml: " ++ show err)
|
||||
Right entries -> return entries
|
||||
|
||||
-- ---------------------------------------------------------------------------
|
||||
-- Context
|
||||
-- ---------------------------------------------------------------------------
|
||||
|
||||
commonplaceCtx :: Context String
|
||||
commonplaceCtx =
|
||||
constField "commonplace" "true"
|
||||
<> themedField
|
||||
<> chronoField
|
||||
<> siteCtx
|
||||
where
|
||||
themedField = field "cp-themed-html" $ \_ ->
|
||||
renderThemedView <$> loadCommonplace
|
||||
chronoField = field "cp-chrono-html" $ \_ ->
|
||||
renderChronoView <$> loadCommonplace
|
||||
|
|
@ -0,0 +1,260 @@
|
|||
{-# LANGUAGE GHC2021 #-}
|
||||
{-# LANGUAGE OverloadedStrings #-}
|
||||
module Compilers
|
||||
( essayCompiler
|
||||
, postCompiler
|
||||
, pageCompiler
|
||||
, poetryCompiler
|
||||
, fictionCompiler
|
||||
, compositionCompiler
|
||||
, photographyCompiler
|
||||
, sidecarCompiler
|
||||
, readerOpts
|
||||
, writerOpts
|
||||
) where
|
||||
|
||||
import Hakyll
|
||||
import Text.Pandoc.Definition (Pandoc (..), Block (..),
|
||||
Inline (..))
|
||||
import Text.Pandoc.Options (ReaderOptions (..), WriterOptions (..),
|
||||
HTMLMathMethod (..))
|
||||
import Text.Pandoc.Extensions (enableExtension, Extension (..))
|
||||
import qualified Data.Text as T
|
||||
import Data.Maybe (fromMaybe)
|
||||
import System.FilePath (takeDirectory)
|
||||
import Utils (wordCount, readingTime, escapeHtml)
|
||||
import Filters (applyAll, preprocessSource)
|
||||
import qualified Citations
|
||||
import qualified Filters.Score as Score
|
||||
import qualified Filters.Viz as Viz
|
||||
|
||||
-- ---------------------------------------------------------------------------
|
||||
-- Reader / writer options
|
||||
-- ---------------------------------------------------------------------------
|
||||
|
||||
readerOpts :: ReaderOptions
|
||||
readerOpts = defaultHakyllReaderOptions
|
||||
|
||||
-- | Reader options with hard_line_breaks enabled — every source newline within
|
||||
-- a paragraph becomes a <br>. Used for poetry so stanza lines render as-is.
|
||||
poetryReaderOpts :: ReaderOptions
|
||||
poetryReaderOpts = readerOpts
|
||||
{ readerExtensions = enableExtension Ext_hard_line_breaks
|
||||
(readerExtensions readerOpts) }
|
||||
|
||||
writerOpts :: WriterOptions
|
||||
writerOpts = defaultHakyllWriterOptions
|
||||
{ writerHTMLMathMethod = KaTeX ""
|
||||
, writerHighlightStyle = Nothing
|
||||
, writerNumberSections = False
|
||||
, writerTableOfContents = False
|
||||
}
|
||||
|
||||
-- ---------------------------------------------------------------------------
|
||||
-- Inline stringification (local, avoids depending on Text.Pandoc.Shared)
|
||||
-- ---------------------------------------------------------------------------
|
||||
|
||||
stringify :: [Inline] -> T.Text
|
||||
stringify = T.concat . map inlineToText
|
||||
where
|
||||
inlineToText (Str t) = t
|
||||
inlineToText Space = " "
|
||||
inlineToText SoftBreak = " "
|
||||
inlineToText LineBreak = " "
|
||||
inlineToText (Emph ils) = stringify ils
|
||||
inlineToText (Strong ils) = stringify ils
|
||||
inlineToText (Strikeout ils) = stringify ils
|
||||
inlineToText (Superscript ils) = stringify ils
|
||||
inlineToText (Subscript ils) = stringify ils
|
||||
inlineToText (SmallCaps ils) = stringify ils
|
||||
inlineToText (Quoted _ ils) = stringify ils
|
||||
inlineToText (Cite _ ils) = stringify ils
|
||||
inlineToText (Code _ t) = t
|
||||
inlineToText (RawInline _ t) = t
|
||||
inlineToText (Link _ ils _) = stringify ils
|
||||
inlineToText (Image _ ils _) = stringify ils
|
||||
inlineToText (Note _) = ""
|
||||
inlineToText (Span _ ils) = stringify ils
|
||||
inlineToText _ = ""
|
||||
|
||||
-- ---------------------------------------------------------------------------
|
||||
-- TOC extraction
|
||||
-- ---------------------------------------------------------------------------
|
||||
|
||||
-- | Collect (level, identifier, title-text) for h2/h3 headings.
|
||||
collectHeadings :: Pandoc -> [(Int, T.Text, String)]
|
||||
collectHeadings (Pandoc _ blocks) = concatMap go blocks
|
||||
where
|
||||
go (Header lvl (ident, _, _) inlines)
|
||||
| lvl == 2 || lvl == 3
|
||||
= [(lvl, ident, T.unpack (stringify inlines))]
|
||||
go _ = []
|
||||
|
||||
-- ---------------------------------------------------------------------------
|
||||
-- TOC tree
|
||||
-- ---------------------------------------------------------------------------
|
||||
|
||||
data TOCNode = TOCNode T.Text String [TOCNode]
|
||||
|
||||
buildTree :: [(Int, T.Text, String)] -> [TOCNode]
|
||||
buildTree = go 2
|
||||
where
|
||||
go _ [] = []
|
||||
go lvl ((l, i, t) : rest)
|
||||
| l == lvl =
|
||||
let (childItems, remaining) = span (\(l', _, _) -> l' > lvl) rest
|
||||
children = go (lvl + 1) childItems
|
||||
in TOCNode i t children : go lvl remaining
|
||||
| l < lvl = []
|
||||
| otherwise = go lvl rest -- skip unexpected deeper items at this level
|
||||
|
||||
renderTOC :: [TOCNode] -> String
|
||||
renderTOC [] = ""
|
||||
renderTOC nodes = "<ol>\n" ++ concatMap renderNode nodes ++ "</ol>\n"
|
||||
where
|
||||
renderNode (TOCNode i t children) =
|
||||
"<li><a href=\"#" ++ T.unpack i ++ "\" data-target=\"" ++ T.unpack i ++ "\">"
|
||||
++ Utils.escapeHtml t ++ "</a>" ++ renderTOC children ++ "</li>\n"
|
||||
|
||||
-- | Build a TOC HTML string from a Pandoc document.
|
||||
buildTOC :: Pandoc -> String
|
||||
buildTOC doc = renderTOC (buildTree (collectHeadings doc))
|
||||
|
||||
-- ---------------------------------------------------------------------------
|
||||
-- Compilers
|
||||
-- ---------------------------------------------------------------------------
|
||||
|
||||
-- | Shared compiler pipeline parameterised on reader options.
|
||||
-- Saves toc/word-count/reading-time/bibliography snapshots.
|
||||
essayCompilerWith :: ReaderOptions -> Compiler (Item String)
|
||||
essayCompilerWith rOpts = do
|
||||
-- Raw Markdown source (used for word count / reading time).
|
||||
body <- getResourceBody
|
||||
let src = itemBody body
|
||||
|
||||
-- Apply source-level preprocessors (wikilinks, etc.) before parsing.
|
||||
let body' = itemSetBody (preprocessSource src) body
|
||||
|
||||
-- Parse to Pandoc AST.
|
||||
pandocItem <- readPandocWith rOpts body'
|
||||
|
||||
-- Get further-reading keys from Hakyll metadata (YAML frontmatter is stripped
|
||||
-- before being passed to readPandocWith, so we read it from Hakyll instead).
|
||||
ident <- getUnderlying
|
||||
meta <- getMetadata ident
|
||||
let frKeys = map T.pack $ fromMaybe [] (lookupStringList "further-reading" meta)
|
||||
let bibPath = T.pack $ fromMaybe "data/bibliography.bib" (lookupString "bibliography" meta)
|
||||
|
||||
-- Run citeproc, transform citation spans → superscripts, extract bibliography.
|
||||
(pandocWithCites, bibHtml, furtherHtml) <- unsafeCompiler $
|
||||
Citations.applyCitations frKeys bibPath (itemBody pandocItem)
|
||||
|
||||
-- Inline SVG score fragments and data visualizations (both read files
|
||||
-- relative to the source file's directory).
|
||||
filePath <- getResourceFilePath
|
||||
let srcDir = takeDirectory filePath
|
||||
pandocWithScores <- unsafeCompiler $
|
||||
Score.inlineScores srcDir pandocWithCites
|
||||
pandocWithViz <- unsafeCompiler $
|
||||
Viz.inlineViz srcDir pandocWithScores
|
||||
|
||||
-- Apply remaining AST-level filters (sidenotes, smallcaps, links, etc.).
|
||||
-- applyAll touches the filesystem via Images.apply (webp existence
|
||||
-- check), so it runs through unsafeCompiler.
|
||||
pandocFiltered <- unsafeCompiler $ applyAll srcDir pandocWithViz
|
||||
let pandocItem' = itemSetBody pandocFiltered pandocItem
|
||||
|
||||
-- Build TOC from the filtered AST.
|
||||
let toc = buildTOC pandocFiltered
|
||||
|
||||
-- Write HTML.
|
||||
let htmlItem = writePandocWith writerOpts pandocItem'
|
||||
|
||||
-- Save snapshots keyed to this item's identifier.
|
||||
_ <- saveSnapshot "toc" (itemSetBody toc htmlItem)
|
||||
_ <- saveSnapshot "word-count" (itemSetBody (show (wordCount src)) htmlItem)
|
||||
_ <- saveSnapshot "reading-time" (itemSetBody (show (readingTime src)) htmlItem)
|
||||
_ <- saveSnapshot "bibliography" (itemSetBody (T.unpack bibHtml) htmlItem)
|
||||
_ <- saveSnapshot "further-reading-refs" (itemSetBody (T.unpack furtherHtml) htmlItem)
|
||||
|
||||
return htmlItem
|
||||
|
||||
-- | Compiler for essays.
|
||||
essayCompiler :: Compiler (Item String)
|
||||
essayCompiler = essayCompilerWith readerOpts
|
||||
|
||||
-- | Compiler for blog posts: same pipeline as essays.
|
||||
postCompiler :: Compiler (Item String)
|
||||
postCompiler = essayCompiler
|
||||
|
||||
-- | Compiler for poetry: enables hard_line_breaks so each source line becomes
|
||||
-- a <br>, preserving verse line endings without manual trailing-space markup.
|
||||
poetryCompiler :: Compiler (Item String)
|
||||
poetryCompiler = essayCompilerWith poetryReaderOpts
|
||||
|
||||
-- | Compiler for fiction: same pipeline as essays; visual differences are
|
||||
-- handled entirely by the reading template and reading.css.
|
||||
fictionCompiler :: Compiler (Item String)
|
||||
fictionCompiler = essayCompiler
|
||||
|
||||
-- | Compiler for music composition landing pages: full essay pipeline
|
||||
-- (TOC, sidenotes, score fragments, citations, smallcaps, etc.).
|
||||
compositionCompiler :: Compiler (Item String)
|
||||
compositionCompiler = essayCompiler
|
||||
|
||||
-- | Compiler for photography pages: body prose runs through the same
|
||||
-- source preprocessors and AST filters as other content (so wikilinks,
|
||||
-- smallcaps, sidenotes, image @<picture>@ wrapping, etc. all work in
|
||||
-- caption / process-note prose), but skips TOC, word-count,
|
||||
-- reading-time, citations, and further-reading. Visual content has no
|
||||
-- meaningful word count, and the epistemic / bibliography surfaces in
|
||||
-- 'essayCtx' don't apply here.
|
||||
photographyCompiler :: Compiler (Item String)
|
||||
photographyCompiler = do
|
||||
body <- getResourceBody
|
||||
let src = itemBody body
|
||||
body' = itemSetBody (preprocessSource src) body
|
||||
filePath <- getResourceFilePath
|
||||
let srcDir = takeDirectory filePath
|
||||
pandocItem <- readPandocWith readerOpts body'
|
||||
pandocFiltered <- unsafeCompiler $ applyAll srcDir (itemBody pandocItem)
|
||||
let pandocItem' = itemSetBody pandocFiltered pandocItem
|
||||
return (writePandocWith writerOpts pandocItem')
|
||||
|
||||
-- | Reduced pipeline for tag-meta sidecar markdown files. Applies
|
||||
-- source-level preprocessors and AST filters (wikilinks, sidenotes,
|
||||
-- smallcaps, links, etc.) so sidecar prose can use the same rich
|
||||
-- markdown features as essays, then saves the rendered HTML under
|
||||
-- the @"body"@ snapshot. Skips TOC, word count, reading time, and
|
||||
-- citations — none of those belong in a portal intro. The item
|
||||
-- itself is not routed; the body is consumed only via snapshot
|
||||
-- loads by the tag-index rule and the home-page grid.
|
||||
sidecarCompiler :: Compiler (Item String)
|
||||
sidecarCompiler = do
|
||||
body <- getResourceBody
|
||||
let src = itemBody body
|
||||
body' = itemSetBody (preprocessSource src) body
|
||||
filePath <- getResourceFilePath
|
||||
let srcDir = takeDirectory filePath
|
||||
pandocItem <- readPandocWith readerOpts body'
|
||||
pandocFiltered <- unsafeCompiler $ applyAll srcDir (itemBody pandocItem)
|
||||
let pandocItem' = itemSetBody pandocFiltered pandocItem
|
||||
let htmlItem = writePandocWith writerOpts pandocItem'
|
||||
_ <- saveSnapshot "body" htmlItem
|
||||
return htmlItem
|
||||
|
||||
-- | Compiler for simple pages: filters applied, no TOC snapshot.
|
||||
pageCompiler :: Compiler (Item String)
|
||||
pageCompiler = do
|
||||
body <- getResourceBody
|
||||
let src = itemBody body
|
||||
body' = itemSetBody (preprocessSource src) body
|
||||
filePath <- getResourceFilePath
|
||||
let srcDir = takeDirectory filePath
|
||||
pandocItem <- readPandocWith readerOpts body'
|
||||
pandocFiltered <- unsafeCompiler $ applyAll srcDir (itemBody pandocItem)
|
||||
let pandocItem' = itemSetBody pandocFiltered pandocItem
|
||||
let htmlItem = writePandocWith writerOpts pandocItem'
|
||||
_ <- saveSnapshot "word-count" (itemSetBody (show (wordCount src)) htmlItem)
|
||||
_ <- saveSnapshot "reading-time" (itemSetBody (show (readingTime src)) htmlItem)
|
||||
return htmlItem
|
||||
File diff suppressed because it is too large
Load Diff
|
|
@ -0,0 +1,79 @@
|
|||
{-# LANGUAGE GHC2021 #-}
|
||||
{-# LANGUAGE OverloadedStrings #-}
|
||||
|
||||
-- | Section-break ornament selection.
|
||||
--
|
||||
-- Every page exposes a @$dingbat$@ template variable naming the ornament to
|
||||
-- use in place of the standard @<hr>@. The template renders this as
|
||||
-- @data-dingbat="..."@ on @<body>@; CSS attribute selectors then swap the
|
||||
-- @hr::after@ glyph.
|
||||
--
|
||||
-- Resolution order:
|
||||
--
|
||||
-- 1. @dingbat:@ frontmatter key on the page (must be in 'knownDingbats').
|
||||
-- 2. Section default derived from the item's route ('sectionDefault').
|
||||
-- 3. Fallback ('fallbackDingbat').
|
||||
--
|
||||
-- Best practice: set @dingbat:@ explicitly in frontmatter. The section
|
||||
-- defaults are a safety net, not a substitute.
|
||||
--
|
||||
-- Adding a new ornament:
|
||||
--
|
||||
-- 1. Add its name to 'knownDingbats'.
|
||||
-- 2. Optionally assign a section default in 'sectionDefault'.
|
||||
-- 3. Add a matching @body[data-dingbat="…"]@ rule in typography.css.
|
||||
module Dingbat
|
||||
( dingbatField
|
||||
, knownDingbats
|
||||
) where
|
||||
|
||||
import Data.List (isPrefixOf)
|
||||
import Data.Maybe (fromMaybe)
|
||||
import Hakyll
|
||||
|
||||
-- | Curated palette. Extend here when adding a new ornament.
|
||||
knownDingbats :: [String]
|
||||
knownDingbats =
|
||||
[ "asterism" -- ⁂ typographic asterism (neutral fallback)
|
||||
, "asterisks" -- * * * spaced asterisks (classic scene break)
|
||||
, "fleuron" -- Aldine leaf (literary essays)
|
||||
, "trefoil" -- three-lobed ornament (poetry)
|
||||
, "lozenge" -- diamond/rhombus (blog)
|
||||
, "clef" -- musical ornament (music)
|
||||
, "memento" -- mourning ornament (memento-mori)
|
||||
, "tech" -- tech ornament
|
||||
, "ai" -- AI ornament (the cute robot)
|
||||
]
|
||||
|
||||
-- | Last-resort default when neither frontmatter nor section rule applies.
|
||||
fallbackDingbat :: String
|
||||
fallbackDingbat = "asterism"
|
||||
|
||||
-- | Section defaults matched against the item's route prefix.
|
||||
-- First matching prefix wins. Unmatched routes use 'fallbackDingbat'.
|
||||
sectionDefault :: String -> String
|
||||
sectionDefault r
|
||||
| "essays/" `isPrefixOf` r = "fleuron"
|
||||
| "blog/" `isPrefixOf` r = "lozenge"
|
||||
| "poetry/" `isPrefixOf` r = "trefoil"
|
||||
| "fiction/" `isPrefixOf` r = "asterisks"
|
||||
| "music/" `isPrefixOf` r = "clef"
|
||||
| "memento-mori/" `isPrefixOf` r = "memento"
|
||||
| otherwise = fallbackDingbat
|
||||
|
||||
-- | @$dingbat$@: name of the ornament to use on this page.
|
||||
dingbatField :: Context a
|
||||
dingbatField = field "dingbat" $ \item -> do
|
||||
meta <- getMetadata (itemIdentifier item)
|
||||
r <- fromMaybe "" <$> getRoute (itemIdentifier item)
|
||||
let sectionD = sectionDefault r
|
||||
case lookupString "dingbat" meta of
|
||||
Nothing -> return sectionD
|
||||
Just name
|
||||
| name `elem` knownDingbats -> return name
|
||||
| otherwise -> do
|
||||
let ident = toFilePath (itemIdentifier item)
|
||||
unsafeCompiler $ putStrLn $
|
||||
"[Dingbat] " ++ ident ++ ": unknown dingbat \""
|
||||
++ name ++ "\" — using \"" ++ sectionD ++ "\""
|
||||
return sectionD
|
||||
|
|
@ -0,0 +1,55 @@
|
|||
{-# LANGUAGE GHC2021 #-}
|
||||
-- | Re-exports all Pandoc AST filter modules and provides a single
|
||||
-- @applyAll@ combinator that chains them in the correct order.
|
||||
module Filters
|
||||
( applyAll
|
||||
, preprocessSource
|
||||
) where
|
||||
|
||||
import Text.Pandoc.Definition (Pandoc)
|
||||
|
||||
import qualified Filters.Sidenotes as Sidenotes
|
||||
import qualified Filters.Typography as Typography
|
||||
import qualified Filters.Links as Links
|
||||
import qualified Filters.SourceRefs as SourceRefs
|
||||
import qualified Filters.Smallcaps as Smallcaps
|
||||
import qualified Filters.Archive as Archive
|
||||
import qualified Filters.Dropcaps as Dropcaps
|
||||
import qualified Filters.Math as Math
|
||||
import qualified Filters.Wikilinks as Wikilinks
|
||||
import qualified Filters.Transclusion as Transclusion
|
||||
import qualified Filters.EmbedPdf as EmbedPdf
|
||||
import qualified Filters.Code as Code
|
||||
import qualified Filters.Images as Images
|
||||
import qualified Filters.Aftermatter as Aftermatter
|
||||
|
||||
-- | Apply all AST-level filters in pipeline order.
|
||||
-- Run on the Pandoc document after reading, before writing.
|
||||
--
|
||||
-- 'Filters.Images.apply' is the only IO-performing filter (it probes the
|
||||
-- filesystem for @.webp@ companions before deciding whether to emit
|
||||
-- @<picture>@). It runs first — i.e. innermost in the composition — and
|
||||
-- every downstream filter stays pure. @srcDir@ is the directory of the
|
||||
-- source Markdown file, passed through to Images for relative-path
|
||||
-- resolution of co-located assets.
|
||||
applyAll :: FilePath -> Pandoc -> IO Pandoc
|
||||
applyAll srcDir doc = do
|
||||
imagesDone <- Images.apply srcDir doc
|
||||
sourceRefsDone <- SourceRefs.apply imagesDone
|
||||
pure
|
||||
. Aftermatter.apply
|
||||
. Sidenotes.apply
|
||||
. Typography.apply
|
||||
. Links.apply
|
||||
. Archive.apply
|
||||
. Smallcaps.apply
|
||||
. Dropcaps.apply
|
||||
. Math.apply
|
||||
. Code.apply
|
||||
$ sourceRefsDone
|
||||
|
||||
-- | Apply source-level preprocessors to the raw Markdown string.
|
||||
-- Order matters: EmbedPdf must run before Transclusion, because the
|
||||
-- transclusion parser would otherwise treat {{pdf:...}} as a broken slug.
|
||||
preprocessSource :: String -> String
|
||||
preprocessSource = Transclusion.preprocess . EmbedPdf.preprocess . Wikilinks.preprocess
|
||||
|
|
@ -0,0 +1,19 @@
|
|||
{-# LANGUAGE GHC2021 #-}
|
||||
{-# LANGUAGE OverloadedStrings #-}
|
||||
module Filters.Aftermatter (apply) where
|
||||
|
||||
import Text.Pandoc.Definition (Pandoc (..), Block (..), Format (..))
|
||||
|
||||
apply :: Pandoc -> Pandoc
|
||||
apply (Pandoc meta blocks) = Pandoc meta (concatMap go blocks)
|
||||
where
|
||||
go (Div attr@(_, classes, _) content)
|
||||
| "aftermatter" `elem` classes
|
||||
= [dividerBlock, Div attr content]
|
||||
go b = [b]
|
||||
|
||||
dividerBlock :: Block
|
||||
dividerBlock = RawBlock (Format "html")
|
||||
"<div class=\"aftermatter-divider\" aria-hidden=\"true\">\
|
||||
\<a href=\"/new.html\" class=\"aftermatter-logo\" aria-label=\"New\"></a>\
|
||||
\</div>"
|
||||
|
|
@ -0,0 +1,119 @@
|
|||
{-# LANGUAGE GHC2021 #-}
|
||||
{-# LANGUAGE OverloadedStrings #-}
|
||||
-- | Filters.Archive — annotate (and, for dead links, redirect) body links
|
||||
-- to archived works.
|
||||
--
|
||||
-- For every @Link@ whose URL matches an entry in @data/archive-index.json@
|
||||
-- (the equivalent-URL alias set included):
|
||||
--
|
||||
-- * a 'live', 'moved' or (inconclusive) 'error' target keeps its
|
||||
-- original link and gains a small superscript affordance pointing at
|
||||
-- the local @/archive/<slug>/@ page — purely additive;
|
||||
--
|
||||
-- * a 'rotted' target (confirmed dead by @archive.py check@'s
|
||||
-- hysteresis) has its primary link flipped to the archived copy, so
|
||||
-- a reader of an old essay reaches a working snapshot instead of a
|
||||
-- 404. A "archived" marker replaces the affordance.
|
||||
--
|
||||
-- Registered in 'Filters.applyAll' immediately after @Smallcaps@ and
|
||||
-- before @Links@: it must see the smallcaps-rewritten text, and it emits
|
||||
-- the affordance/marker as @RawInline@ so the downstream @Links@ pass
|
||||
-- never re-classifies it.
|
||||
--
|
||||
-- No-op when @data/archive-index.json@ is absent. When no rot scan has
|
||||
-- run, every entry is 'Live' — no link is ever flipped.
|
||||
--
|
||||
-- 'annotateBlock' exposes the same pass for rendered blocks that never
|
||||
-- travel the body filter chain — @Citations@ applies it to each
|
||||
-- CSL-rendered bibliography entry, so a bibliography URL gets the same
|
||||
-- affordance (and the same rotted-link flip) as a body link.
|
||||
module Filters.Archive (apply, annotateBlock) where
|
||||
|
||||
import qualified Data.Text as T
|
||||
import Text.Pandoc.Definition
|
||||
import Text.Pandoc.Walk (walk)
|
||||
import ArchiveIndex (ArchiveStatus (..), archiveIndexIsEmpty,
|
||||
archiveSlugFor, archiveStatusForSlug)
|
||||
|
||||
-- | Annotate body links. Links inside headings are left alone at
|
||||
-- /every/ nesting depth — an affordance there would be noise, and a
|
||||
-- top-level pattern match would miss a @Header@ inside a @Div@ or
|
||||
-- @BlockQuote@. Header links are tagged with a sentinel class before
|
||||
-- the annotation walk and stripped of it afterwards, so the sentinel
|
||||
-- can never leak into the writer. Identity when the index is empty.
|
||||
apply :: Pandoc -> Pandoc
|
||||
apply doc
|
||||
| archiveIndexIsEmpty = doc
|
||||
| otherwise =
|
||||
walk unprotectLink . walk annotateInlines . walk protectHeader $ doc
|
||||
|
||||
-- | The annotation pass for a single already-rendered block, outside the
|
||||
-- body filter chain. No header protection — the callers' blocks
|
||||
-- (CSL bibliography entries) contain none. Identity when the index is
|
||||
-- absent.
|
||||
annotateBlock :: Block -> Block
|
||||
annotateBlock b
|
||||
| archiveIndexIsEmpty = b
|
||||
| otherwise = walk annotateInlines b
|
||||
|
||||
-- | Sentinel class marking a link the annotation walk must skip. It
|
||||
-- only exists between the protect and unprotect walks inside 'apply'.
|
||||
skipClass :: T.Text
|
||||
skipClass = "archive-header-skip"
|
||||
|
||||
protectHeader :: Block -> Block
|
||||
protectHeader (Header lvl attr ils) = Header lvl attr (walk protect ils)
|
||||
where
|
||||
protect (Link (ident, cls, kvs) text target) =
|
||||
Link (ident, skipClass : cls, kvs) text target
|
||||
protect x = x
|
||||
protectHeader b = b
|
||||
|
||||
unprotectLink :: Inline -> Inline
|
||||
unprotectLink (Link (ident, cls, kvs) text target)
|
||||
| skipClass `elem` cls =
|
||||
Link (ident, filter (/= skipClass) cls, kvs) text target
|
||||
unprotectLink x = x
|
||||
|
||||
-- | For each archived @Link@: flip it if the target is 'Rotted', else
|
||||
-- append the affordance. Non-archived links — and links protected by
|
||||
-- 'protectHeader' — pass through untouched.
|
||||
annotateInlines :: [Inline] -> [Inline]
|
||||
annotateInlines = concatMap expand
|
||||
where
|
||||
expand l@(Link (_, cls, _) _ _)
|
||||
| skipClass `elem` cls = [l]
|
||||
expand l@(Link attr text (url, _)) =
|
||||
case archiveSlugFor url of
|
||||
Nothing -> [l]
|
||||
Just slug -> case archiveStatusForSlug slug of
|
||||
Rotted -> [flipped slug attr text, marker slug "rotted"
|
||||
"The original is a dead link — \
|
||||
\opens the local archived copy"]
|
||||
_ -> [l, marker slug "" "Archived — \
|
||||
\local preservation copy"]
|
||||
expand x = [x]
|
||||
|
||||
-- | A 'Rotted' link, redirected to the local archived copy. Keeps the
|
||||
-- link text; the @archive-rotted@ class lets CSS mark it.
|
||||
flipped :: String -> Attr -> [Inline] -> Inline
|
||||
flipped slug (ident, classes, kvs) text =
|
||||
Link (ident, "archive-rotted" : classes, kvs) text
|
||||
( T.pack ("/archive/" ++ slug ++ "/")
|
||||
, "Original link is dead \8212 opens the local archived copy" )
|
||||
|
||||
-- | The superscript marker after the link: "A" for a normal affordance,
|
||||
-- "archived" for a flipped dead link. Emitted as raw HTML so the
|
||||
-- downstream @Links@ filter (which classifies @Link@ nodes) leaves it
|
||||
-- alone. Slugs are @[a-z0-9-]@ by construction in @archive.py@.
|
||||
marker :: String -> String -> T.Text -> Inline
|
||||
marker slug modifier title = RawInline "html" $ T.concat
|
||||
[ "<sup class=\"archive-affordance", modifierClass, "\">"
|
||||
, "<a href=\"/archive/", T.pack slug, "/\" title=\"", title, "\">"
|
||||
, label, "</a></sup>"
|
||||
]
|
||||
where
|
||||
modifierClass = if null modifier
|
||||
then ""
|
||||
else " archive-affordance--" <> T.pack modifier
|
||||
label = if null modifier then "A" else "archived"
|
||||
|
|
@ -0,0 +1,29 @@
|
|||
{-# LANGUAGE GHC2021 #-}
|
||||
{-# LANGUAGE OverloadedStrings #-}
|
||||
-- | Prepend "language-" to fenced-code-block class names so that
|
||||
-- Prism.js can find and highlight them.
|
||||
--
|
||||
-- Pandoc (with writerHighlightStyle = Nothing) outputs
|
||||
-- <pre class="python"><code>
|
||||
-- Prism.js requires
|
||||
-- <pre class="language-python"><code class="language-python">
|
||||
--
|
||||
-- We transform the AST before writing rather than post-processing HTML,
|
||||
-- so the class appears on both <pre> and <code> via Pandoc's normal output.
|
||||
module Filters.Code (apply) where
|
||||
|
||||
import qualified Data.Text as T
|
||||
import Text.Pandoc.Definition
|
||||
import Text.Pandoc.Walk (walk)
|
||||
|
||||
apply :: Pandoc -> Pandoc
|
||||
apply = walk addLangPrefix
|
||||
|
||||
addLangPrefix :: Block -> Block
|
||||
addLangPrefix (CodeBlock (ident, classes, kvs) code) =
|
||||
CodeBlock (ident, map prefix classes, kvs) code
|
||||
where
|
||||
prefix c
|
||||
| "language-" `T.isPrefixOf` c = c
|
||||
| otherwise = "language-" <> c
|
||||
addLangPrefix x = x
|
||||
|
|
@ -0,0 +1,15 @@
|
|||
{-# LANGUAGE GHC2021 #-}
|
||||
-- | Dropcap support.
|
||||
--
|
||||
-- The dropcap on the opening paragraph is implemented entirely in CSS
|
||||
-- via @#markdownBody > p:first-of-type::first-letter@, so no AST
|
||||
-- transformation is required. This module is a placeholder for future
|
||||
-- work (e.g. adding a @.lead-paragraph@ class when the first block is
|
||||
-- not a Para, or decorative initial-capital images).
|
||||
module Filters.Dropcaps (apply) where
|
||||
|
||||
import Text.Pandoc.Definition (Pandoc)
|
||||
|
||||
-- | Identity — dropcaps are handled by CSS.
|
||||
apply :: Pandoc -> Pandoc
|
||||
apply = id
|
||||
|
|
@ -0,0 +1,89 @@
|
|||
{-# LANGUAGE GHC2021 #-}
|
||||
-- | Source-level preprocessor for inline PDF embeds.
|
||||
--
|
||||
-- Rewrites block-level @{{pdf:...}}@ directives to raw HTML that renders the
|
||||
-- named file inside a vendored PDF.js viewer iframe.
|
||||
--
|
||||
-- Syntax (must be the sole content of a line after trimming):
|
||||
--
|
||||
-- > {{pdf:/papers/foo.pdf}} — embed from page 1
|
||||
-- > {{pdf:/papers/foo.pdf#5}} — start at page 5 (bare integer)
|
||||
-- > {{pdf:/papers/foo.pdf#page=5}} — start at page 5 (explicit form)
|
||||
--
|
||||
-- The file path must be root-relative (begins with @/@).
|
||||
-- PDF.js is expected to be vendored at @/pdfjs/web/viewer.html@.
|
||||
--
|
||||
-- Code protection (honest scope): lines inside /fenced/ code blocks
|
||||
-- are passed through untouched ('Filters.Wikilinks.mapOutsideFences'),
|
||||
-- so fenced examples can show @{{pdf:…}}@ literally. Indented code
|
||||
-- blocks and inline code spans are NOT recognised — a full-line
|
||||
-- directive inside either is still rewritten.
|
||||
module Filters.EmbedPdf (preprocess) where
|
||||
|
||||
import Data.Char (isDigit)
|
||||
import Data.List (isPrefixOf, isSuffixOf)
|
||||
import Filters.Wikilinks (mapOutsideFences)
|
||||
import qualified Utils as U
|
||||
|
||||
-- | Apply PDF-embed substitution to the raw Markdown source string,
|
||||
-- skipping lines inside fenced code blocks.
|
||||
preprocess :: String -> String
|
||||
preprocess = mapOutsideFences processLine
|
||||
|
||||
processLine :: String -> String
|
||||
processLine line =
|
||||
case parseDirective (U.trim line) of
|
||||
Nothing -> line
|
||||
Just (filePath, pageHash) -> renderEmbed filePath pageHash
|
||||
|
||||
-- | Parse a @{{pdf:/path/to/file.pdf}}@ or @{{pdf:/path.pdf#N}}@ directive.
|
||||
-- Returns @(filePath, pageHash)@ where @pageHash@ is either @""@ or @"#page=N"@.
|
||||
parseDirective :: String -> Maybe (String, String)
|
||||
parseDirective s
|
||||
| not ("{{pdf:" `isPrefixOf` s) = Nothing
|
||||
| not ("}}" `isSuffixOf` s) = Nothing
|
||||
| otherwise =
|
||||
let inner = take (length s - 2) (drop 6 s) -- strip "{{pdf:" and "}}"
|
||||
(path, frag) = break (== '#') inner
|
||||
in if null path
|
||||
then Nothing
|
||||
else Just (path, parsePageHash frag)
|
||||
|
||||
-- | Convert the fragment part of the directive (e.g. @#5@ or @#page=5@) to a
|
||||
-- PDF.js-compatible @#page=N@ hash, or @""@ if absent/invalid.
|
||||
parsePageHash :: String -> String
|
||||
parsePageHash ('#' : rest)
|
||||
| "page=" `isPrefixOf` rest =
|
||||
let n = takeWhile isDigit (drop 5 rest)
|
||||
in if null n then "" else "#page=" ++ n
|
||||
| all isDigit rest && not (null rest) = "#page=" ++ rest
|
||||
parsePageHash _ = ""
|
||||
|
||||
-- | Render the HTML for a PDF embed.
|
||||
renderEmbed :: String -> String -> String
|
||||
renderEmbed filePath pageHash =
|
||||
let viewerUrl = "/pdfjs/web/viewer.html?file=" ++ encodeQueryValue filePath ++ pageHash
|
||||
in "<div class=\"pdf-embed-wrapper\">"
|
||||
++ "<iframe class=\"pdf-embed\""
|
||||
++ " src=\"" ++ viewerUrl ++ "\""
|
||||
++ " title=\"PDF document\""
|
||||
++ " loading=\"lazy\""
|
||||
++ " allowfullscreen></iframe>"
|
||||
++ "</div>"
|
||||
|
||||
-- | Percent-encode characters that would break a query-string value.
|
||||
-- Slashes are left unencoded so root-relative paths remain readable and
|
||||
-- work correctly with PDF.js's internal fetch. @#@ is encoded for
|
||||
-- defense-in-depth even though the directive parser already splits on it
|
||||
-- before this function is called.
|
||||
encodeQueryValue :: String -> String
|
||||
encodeQueryValue = concatMap enc
|
||||
where
|
||||
enc ' ' = "%20"
|
||||
enc '&' = "%26"
|
||||
enc '?' = "%3F"
|
||||
enc '+' = "%2B"
|
||||
enc '"' = "%22"
|
||||
enc '#' = "%23"
|
||||
enc c = [c]
|
||||
|
||||
|
|
@ -0,0 +1,394 @@
|
|||
{-# LANGUAGE GHC2021 #-}
|
||||
{-# LANGUAGE OverloadedStrings #-}
|
||||
-- | Image filter: lazy loading, lightbox markers, WebP <picture>
|
||||
-- wrappers, and CLS-preventing width/height attrs.
|
||||
--
|
||||
-- For local raster images (JPG, JPEG, PNG, GIF) whose @.webp@ companion
|
||||
-- exists on disk at build time, emits a @<picture>@ element with a WebP
|
||||
-- @<source>@ and the original format as the @<img>@ fallback. When the
|
||||
-- webp companion is absent (cwebp not installed, @convert-images.sh@ not
|
||||
-- yet run, or a single file missed), the filter emits a plain @<img>@ so
|
||||
-- the image still renders. This matters because browsers do NOT fall back
|
||||
-- from a 404'd @<source>@ inside @<picture>@ to the nested @<img>@ — the
|
||||
-- source is selected up front and a broken one leaves the area blank.
|
||||
--
|
||||
-- @tools/convert-images.sh@ produces the companion .webp files at build
|
||||
-- time. When cwebp is not installed the script is a no-op, and this
|
||||
-- filter degrades gracefully to plain @<img>@.
|
||||
--
|
||||
-- SVG files and external URLs are passed through with only lazy loading
|
||||
-- (and lightbox markers for standalone images).
|
||||
--
|
||||
-- Width / height attrs are looked up from @{image}.dims.yaml@ sidecars
|
||||
-- produced by @tools/extract-dimensions.py@ at build time, on the same
|
||||
-- path-resolution rules as the WebP companion check (absolute paths
|
||||
-- under @static/@, relative under the source-file directory). When a
|
||||
-- sidecar is missing the filter emits an attr-free <img> rather than
|
||||
-- guessing — partial dimensions are worse than no dimensions, since
|
||||
-- the browser would then size the image wrong on first paint.
|
||||
module Filters.Images (apply) where
|
||||
|
||||
import Data.Char (toLower)
|
||||
import Data.Default (def)
|
||||
import Data.List (isPrefixOf)
|
||||
import Data.Text (Text)
|
||||
import qualified Data.Text as T
|
||||
import qualified Data.Aeson.KeyMap as KM
|
||||
import qualified Data.Scientific as Sci
|
||||
import qualified Data.Yaml as Y
|
||||
import Text.Pandoc.Definition
|
||||
import qualified Text.Pandoc as Pandoc
|
||||
import Text.Pandoc.Walk (walkM)
|
||||
import System.Directory (doesFileExist)
|
||||
import System.FilePath (replaceExtension, takeExtension, (</>))
|
||||
import qualified Utils as U
|
||||
|
||||
-- | Apply image attribute injection and WebP wrapping to the entire document.
|
||||
--
|
||||
-- @srcDir@ is the directory of the source Markdown file, used to resolve
|
||||
-- relative image paths when probing for the corresponding @.webp@
|
||||
-- companion file. Absolute paths (leading @/@) are resolved against
|
||||
-- @static/@ instead, matching the layout @convert-images.sh@ writes to.
|
||||
--
|
||||
-- Two-pass walk:
|
||||
--
|
||||
-- 1. Block-level pass (@transformBlock@) intercepts standalone
|
||||
-- figures so we can synthesize the entire @<figure>@ ourselves
|
||||
-- when WebP wrapping kicks in. Without this pass, replacing the
|
||||
-- inner @Image@ with a @RawInline@ would break Pandoc's
|
||||
-- alt-vs-caption comparison and we'd lose the
|
||||
-- @aria-hidden="true"@ hint on identical-text figcaptions.
|
||||
-- 2. Inline-level pass (@transformInline@) handles every remaining
|
||||
-- @Image@ — inline-in-prose, inside @Link@s, etc. Pandoc's writer
|
||||
-- still applies its accessibility heuristics for figures we
|
||||
-- didn't synthesize (notably the no-WebP case).
|
||||
apply :: FilePath -> Pandoc -> IO Pandoc
|
||||
apply srcDir doc = do
|
||||
doc' <- walkM (transformBlock srcDir) doc
|
||||
walkM (transformInline srcDir) doc'
|
||||
|
||||
-- ---------------------------------------------------------------------------
|
||||
-- Core transformations
|
||||
-- ---------------------------------------------------------------------------
|
||||
|
||||
-- | Block-level pass. Currently only acts on the simple-figure shape
|
||||
-- that Pandoc's Markdown reader produces for @@ standalone:
|
||||
--
|
||||
-- @Figure attr caption [Plain [Image imgAttr alt target]]@
|
||||
--
|
||||
-- When the image has a WebP companion on disk, we replace the whole
|
||||
-- Figure with a @RawBlock@ containing the equivalent HTML — but with
|
||||
-- the @<picture>@ wrapper inside and a manually-emitted
|
||||
-- @aria-hidden="true"@ on the figcaption when alt text equals the
|
||||
-- caption text. Anything more exotic (multi-image figures, mixed
|
||||
-- block content inside the figure, no-WebP images) is left to
|
||||
-- Pandoc's default emission, which is already correct for those
|
||||
-- cases.
|
||||
transformBlock :: FilePath -> Block -> IO Block
|
||||
transformBlock srcDir b@(Figure figAttr caption [Plain [Image imgAttr alt target]]) = do
|
||||
let src = T.unpack (fst target)
|
||||
if not (isLocalRaster src)
|
||||
then pure b
|
||||
else do
|
||||
hasWebp <- doesFileExist (webpPhysicalPath srcDir (fst target))
|
||||
if not hasWebp
|
||||
then pure b -- Pandoc handles aria-hidden naturally on the no-WebP path.
|
||||
else synthesizeFigure srcDir figAttr caption imgAttr alt target
|
||||
transformBlock _ b = pure b
|
||||
|
||||
-- | Build a @<figure>@ block from an Image and its surrounding
|
||||
-- metadata. Used only on the WebP branch; the no-WebP branch leaves
|
||||
-- Pandoc to emit the figure naturally.
|
||||
--
|
||||
-- Aria-hiding rule: when the caption's plain-text content equals the
|
||||
-- alt text and both are non-empty, mark the @<figcaption>@ with
|
||||
-- @aria-hidden="true"@. Screen readers then announce the alt
|
||||
-- (via the @<img>@) and skip the figcaption (which would just
|
||||
-- duplicate it). Non-matching captions render as visible content.
|
||||
--
|
||||
-- Caption inline rendering goes through Pandoc's HTML writer, so
|
||||
-- formatting (italic, links, code, etc.) is preserved.
|
||||
synthesizeFigure :: FilePath -> Attr -> Caption -> Attr -> [Inline] -> Target -> IO Block
|
||||
synthesizeFigure srcDir figAttr caption imgAttr alt target = do
|
||||
dims <- readDims srcDir (fst target)
|
||||
let pictureHtml = renderPicture imgAttr alt target True dims
|
||||
capInlines = captionInlines caption
|
||||
capText = stringify capInlines
|
||||
altText = stringify alt
|
||||
useAriaHide = capText == altText && not (T.null altText)
|
||||
pure $ RawBlock (Format "html") $
|
||||
renderFigure figAttr pictureHtml (renderFigcaption capInlines useAriaHide)
|
||||
|
||||
transformInline :: FilePath -> Inline -> IO Inline
|
||||
transformInline srcDir (Link lAttr ils lTarget) = do
|
||||
-- Recurse into link contents; images inside a link get no lightbox marker.
|
||||
ils' <- mapM (wrapLinkedImg srcDir) ils
|
||||
pure (Link lAttr ils' lTarget)
|
||||
transformInline srcDir (Image attr alt target) =
|
||||
renderImg srcDir attr alt target True
|
||||
transformInline _ x = pure x
|
||||
|
||||
wrapLinkedImg :: FilePath -> Inline -> IO Inline
|
||||
wrapLinkedImg srcDir (Image iAttr alt iTarget) =
|
||||
renderImg srcDir iAttr alt iTarget False
|
||||
wrapLinkedImg _ x = pure x
|
||||
|
||||
-- | Dispatch on image type:
|
||||
-- * Local raster with webp companion on disk → @<picture>@ with WebP @<source>@
|
||||
-- * Local raster without companion → plain @<img>@ (graceful degradation)
|
||||
-- * Everything else (SVG, URL) → plain @<img>@ with loading/lightbox attrs
|
||||
--
|
||||
-- In all three branches, when a @{image}.dims.yaml@ sidecar is
|
||||
-- present, @width@ and @height@ attrs are emitted on the rendered
|
||||
-- @<img>@. The sidecar lookup is skipped for non-local sources
|
||||
-- (HTTP URLs, data URIs) since there's no local file to measure.
|
||||
renderImg :: FilePath -> Attr -> [Inline] -> Target -> Bool -> IO Inline
|
||||
renderImg srcDir attr alt target@(src, _) lightbox = do
|
||||
let s = T.unpack src
|
||||
isRaster = isLocalRaster s
|
||||
local = not (isUrl s)
|
||||
dims <- if local then readDims srcDir src else pure Nothing
|
||||
if isRaster
|
||||
then do
|
||||
hasWebp <- doesFileExist (webpPhysicalPath srcDir src)
|
||||
if hasWebp
|
||||
then pure $ RawInline (Format "html")
|
||||
(renderPicture attr alt target lightbox dims)
|
||||
else pure $ Image (commonAttrs dims) alt target
|
||||
else
|
||||
pure $ Image (commonAttrs dims) alt target
|
||||
where
|
||||
commonAttrs dims =
|
||||
withDims dims
|
||||
$ addAttr "decoding" "async"
|
||||
$ addLightbox lightbox
|
||||
$ addAttr "loading" "lazy" attr
|
||||
|
||||
addLightbox True a = addAttr "data-lightbox" "true" a
|
||||
addLightbox False a = a
|
||||
|
||||
withDims Nothing a = a
|
||||
withDims (Just (w, h)) a =
|
||||
addAttr "width" (T.pack (show w))
|
||||
(addAttr "height" (T.pack (show h)) a)
|
||||
|
||||
-- | Physical on-disk path of the @.webp@ companion for a Markdown image src.
|
||||
--
|
||||
-- Absolute paths (@/images/foo.jpg@) resolve under @static/@ because that
|
||||
-- is where Hakyll's static-asset rule writes them from. Relative paths
|
||||
-- resolve against the source file's directory, where Pandoc already
|
||||
-- expects co-located assets to live.
|
||||
webpPhysicalPath :: FilePath -> Text -> FilePath
|
||||
webpPhysicalPath srcDir src =
|
||||
let s = T.unpack src
|
||||
physical = if "/" `isPrefixOf` s
|
||||
then "static" ++ s
|
||||
else srcDir </> s
|
||||
in replaceExtension physical ".webp"
|
||||
|
||||
-- | Physical on-disk path of the @.dims.yaml@ sidecar for a Markdown
|
||||
-- image src. Same path-resolution rules as 'webpPhysicalPath'; the
|
||||
-- sidecar lives next to the original image with the literal
|
||||
-- extension @.dims.yaml@ appended.
|
||||
dimsPhysicalPath :: FilePath -> Text -> FilePath
|
||||
dimsPhysicalPath srcDir src =
|
||||
let s = T.unpack src
|
||||
physical = if "/" `isPrefixOf` s
|
||||
then "static" ++ s
|
||||
else srcDir </> s
|
||||
in physical ++ ".dims.yaml"
|
||||
|
||||
-- | Read the @{image}.dims.yaml@ sidecar and return @(width, height)@
|
||||
-- when present and parseable. Returns 'Nothing' on absent file,
|
||||
-- parse error, missing keys, or non-integer values — all of which
|
||||
-- cause the filter to emit no width/height attrs (rather than a
|
||||
-- guess that would size the image wrong on first paint).
|
||||
readDims :: FilePath -> Text -> IO (Maybe (Int, Int))
|
||||
readDims srcDir src = do
|
||||
let path = dimsPhysicalPath srcDir src
|
||||
exists <- doesFileExist path
|
||||
if not exists
|
||||
then pure Nothing
|
||||
else do
|
||||
decoded <- Y.decodeFileEither path
|
||||
pure $ case decoded of
|
||||
Right (Y.Object obj) -> do
|
||||
w <- intValue =<< KM.lookup "width" obj
|
||||
h <- intValue =<< KM.lookup "height" obj
|
||||
Just (w, h)
|
||||
_ -> Nothing
|
||||
where
|
||||
intValue :: Y.Value -> Maybe Int
|
||||
intValue (Y.Number n) = Sci.toBoundedInteger n
|
||||
intValue _ = Nothing
|
||||
|
||||
-- ---------------------------------------------------------------------------
|
||||
-- <picture> rendering
|
||||
-- ---------------------------------------------------------------------------
|
||||
|
||||
-- | Emit a @<picture>@ element with a WebP @<source>@ and an @<img>@ fallback.
|
||||
renderPicture :: Attr -> [Inline] -> Target -> Bool -> Maybe (Int, Int) -> Text
|
||||
renderPicture (ident, classes, kvs) alt (src, title) lightbox dims =
|
||||
T.concat
|
||||
[ "<picture>"
|
||||
, "<source srcset=\"", esc (T.pack webpSrc), "\" type=\"image/webp\">"
|
||||
, "<img"
|
||||
, attrId ident
|
||||
, attrClasses classes
|
||||
, " src=\"", esc src, "\""
|
||||
, attrAlt alt
|
||||
, attrTitle title
|
||||
, dimsAttrs dims
|
||||
, " loading=\"lazy\""
|
||||
, " decoding=\"async\""
|
||||
, if lightbox then " data-lightbox=\"true\"" else ""
|
||||
, renderKvs passedKvs
|
||||
, ">"
|
||||
, "</picture>"
|
||||
]
|
||||
where
|
||||
webpSrc = replaceExtension (T.unpack src) ".webp"
|
||||
-- Strip attrs we handle explicitly above (id/class/alt/title) and the
|
||||
-- attrs we always emit ourselves (loading, decoding, data-lightbox,
|
||||
-- width, height), so they don't appear twice on the <img>.
|
||||
passedKvs = filter
|
||||
(\(k, _) -> k `notElem`
|
||||
[ "loading", "decoding", "data-lightbox"
|
||||
, "id", "class", "alt", "title", "src"
|
||||
, "width", "height"
|
||||
])
|
||||
kvs
|
||||
|
||||
dimsAttrs Nothing = ""
|
||||
dimsAttrs (Just (w, h)) =
|
||||
" width=\"" <> T.pack (show w)
|
||||
<> "\" height=\"" <> T.pack (show h) <> "\""
|
||||
|
||||
-- ---------------------------------------------------------------------------
|
||||
-- <figure> synthesis (Block walk, WebP path only)
|
||||
-- ---------------------------------------------------------------------------
|
||||
|
||||
-- | Build a @<figure>@ HTML element wrapping pre-rendered inner
|
||||
-- content (typically a @<picture>@) and a pre-rendered figcaption.
|
||||
-- Preserves any id / classes / kvs from the surrounding Pandoc
|
||||
-- 'Figure' attr.
|
||||
renderFigure :: Attr -> Text -> Text -> Text
|
||||
renderFigure (figId, figClasses, figKvs) inner figcaption =
|
||||
T.concat
|
||||
[ "<figure"
|
||||
, attrId figId
|
||||
, attrClasses figClasses
|
||||
, renderKvs figKvs
|
||||
, ">\n"
|
||||
, inner
|
||||
, "\n"
|
||||
, figcaption
|
||||
, "\n</figure>"
|
||||
]
|
||||
|
||||
-- | Build a @<figcaption>@ element. When @ariaHidden@ is true, emits
|
||||
-- @aria-hidden="true"@ — used when the caption text exactly
|
||||
-- duplicates the image alt (so screen readers don't announce the
|
||||
-- same content twice). Caption inlines render through Pandoc's HTML
|
||||
-- writer to preserve formatting.
|
||||
renderFigcaption :: [Inline] -> Bool -> Text
|
||||
renderFigcaption ils ariaHidden =
|
||||
let body = renderInlinesToHtml ils
|
||||
attrs = if ariaHidden then " aria-hidden=\"true\"" else ""
|
||||
in "<figcaption" <> attrs <> ">" <> body <> "</figcaption>"
|
||||
|
||||
-- | Pandoc 'Caption' has a long form (@[Block]@) and an optional short
|
||||
-- form (@Maybe ShortCaption@). We use the long form, flattening any
|
||||
-- @Plain@ / @Para@ blocks into a single inline list. Multi-block
|
||||
-- captions (rare) collapse to the inlines of their text-bearing
|
||||
-- blocks; non-text blocks (like nested lists) are dropped, since
|
||||
-- they don't make sense in a figcaption anyway.
|
||||
captionInlines :: Caption -> [Inline]
|
||||
captionInlines (Caption _ blocks) = concatMap go blocks
|
||||
where
|
||||
go (Plain ils) = ils
|
||||
go (Para ils) = ils
|
||||
go _ = []
|
||||
|
||||
-- | Render Pandoc 'Inline' nodes to HTML using Pandoc's own writer.
|
||||
-- Wrapping the inlines in a @Plain@ block (rather than @Para@)
|
||||
-- avoids the surrounding @<p>@ tag the writer would otherwise emit.
|
||||
-- On writer failure (extremely unlikely for inline-only input),
|
||||
-- falls back to the plain-text 'stringify' rendering — a worse but
|
||||
-- still safe figcaption.
|
||||
renderInlinesToHtml :: [Inline] -> Text
|
||||
renderInlinesToHtml ils =
|
||||
case Pandoc.runPure (Pandoc.writeHtml5String def doc) of
|
||||
Right t -> T.strip t
|
||||
Left _ -> stringify ils
|
||||
where
|
||||
doc = Pandoc mempty [Plain ils]
|
||||
|
||||
attrId :: Text -> Text
|
||||
attrId t = if T.null t then "" else " id=\"" <> esc t <> "\""
|
||||
|
||||
attrClasses :: [Text] -> Text
|
||||
attrClasses [] = ""
|
||||
attrClasses cs = " class=\"" <> T.intercalate " " (map esc cs) <> "\""
|
||||
|
||||
attrAlt :: [Inline] -> Text
|
||||
attrAlt ils = let t = stringify ils
|
||||
in if T.null t then "" else " alt=\"" <> esc t <> "\""
|
||||
|
||||
attrTitle :: Text -> Text
|
||||
attrTitle t = if T.null t then "" else " title=\"" <> esc t <> "\""
|
||||
|
||||
renderKvs :: [(Text, Text)] -> Text
|
||||
renderKvs = T.concat . map (\(k, v) -> " " <> k <> "=\"" <> esc v <> "\"")
|
||||
|
||||
-- ---------------------------------------------------------------------------
|
||||
-- Helpers
|
||||
-- ---------------------------------------------------------------------------
|
||||
|
||||
-- | True for local (non-URL) images with a raster format we can convert.
|
||||
isLocalRaster :: FilePath -> Bool
|
||||
isLocalRaster src = not (isUrl src) && lowerExt src `elem` [".jpg", ".jpeg", ".png", ".gif"]
|
||||
|
||||
isUrl :: String -> Bool
|
||||
isUrl s = any (`isPrefixOf` s) ["http://", "https://", "//", "data:"]
|
||||
|
||||
-- | Extension of a path, lowercased (e.g. ".JPG" → ".jpg").
|
||||
-- Returns the empty string for paths with no extension.
|
||||
lowerExt :: FilePath -> String
|
||||
lowerExt = map toLower . takeExtension
|
||||
|
||||
-- | Prepend a key=value pair if not already present.
|
||||
addAttr :: Text -> Text -> Attr -> Attr
|
||||
addAttr k v (i, cs, kvs)
|
||||
| any ((== k) . fst) kvs = (i, cs, kvs)
|
||||
| otherwise = (i, cs, (k, v) : kvs)
|
||||
|
||||
-- | Plain-text content of a list of inlines (for alt text).
|
||||
stringify :: [Inline] -> Text
|
||||
stringify = T.concat . map go
|
||||
where
|
||||
go (Str t) = t
|
||||
go Space = " "
|
||||
go SoftBreak = " "
|
||||
go LineBreak = " "
|
||||
go (Emph ils) = stringify ils
|
||||
go (Strong ils) = stringify ils
|
||||
go (Strikeout ils) = stringify ils
|
||||
go (Superscript ils) = stringify ils
|
||||
go (Subscript ils) = stringify ils
|
||||
go (SmallCaps ils) = stringify ils
|
||||
go (Underline ils) = stringify ils
|
||||
go (Quoted _ ils) = stringify ils
|
||||
go (Cite _ ils) = stringify ils
|
||||
go (Code _ t) = t
|
||||
go (Math _ t) = t
|
||||
go (RawInline _ _) = ""
|
||||
go (Link _ ils _) = stringify ils
|
||||
go (Image _ ils _) = stringify ils
|
||||
go (Span _ ils) = stringify ils
|
||||
go (Note _) = ""
|
||||
|
||||
-- | HTML-escape a text value for use in attribute values.
|
||||
-- Defers to the canonical 'Utils.escapeHtmlText'.
|
||||
esc :: Text -> Text
|
||||
esc = U.escapeHtmlText
|
||||
|
|
@ -0,0 +1,187 @@
|
|||
{-# LANGUAGE GHC2021 #-}
|
||||
{-# LANGUAGE OverloadedStrings #-}
|
||||
-- | External link classification.
|
||||
--
|
||||
-- Walks all @Link@ inlines and:
|
||||
-- * Adds @class="link-external"@ to any link whose URL starts with
|
||||
-- @http://@ or @https://@ and is not on the site's own domain.
|
||||
-- * Adds @data-link-icon@ / @data-link-icon-type@ attributes for
|
||||
-- per-domain brand icons (see 'domainIcon' for the full list).
|
||||
-- * Adds @target="_blank" rel="noopener noreferrer"@ to external links.
|
||||
module Filters.Links (apply) where
|
||||
|
||||
import Data.Text (Text)
|
||||
import qualified Data.Text as T
|
||||
import Text.Pandoc.Definition
|
||||
import Text.Pandoc.Walk (walk)
|
||||
|
||||
-- | Apply link classification to the entire document.
|
||||
-- Two passes: PDF links first (rewrites href to the viewer URL and tags
|
||||
-- the anchor @pdf-link@), then general classification. The second pass
|
||||
-- explicitly skips anchors the PDF pass already claimed — the viewer URL
|
||||
-- is root-relative, so without that guard it would also be classified as
|
||||
-- an internal page link and get double chrome.
|
||||
apply :: Pandoc -> Pandoc
|
||||
apply = walk classifyLink . walk classifyPdfLink
|
||||
|
||||
-- | Rewrite root-relative PDF links to open via the vendored PDF.js viewer.
|
||||
-- Preserves the original path in @data-pdf-src@ so the popup thumbnail
|
||||
-- provider can locate the corresponding @.thumb.png@ file.
|
||||
-- Skips links that are already pointing at the viewer (idempotent).
|
||||
--
|
||||
-- Handles fragment identifiers (e.g. @/papers/foo.pdf#page=5@): the
|
||||
-- fragment is stripped before the @.pdf@ suffix check and re-attached
|
||||
-- after the viewer URL so PDF.js's anchor handling works.
|
||||
classifyPdfLink :: Inline -> Inline
|
||||
classifyPdfLink (Link (ident, classes, kvs) ils (url, title))
|
||||
| "/" `T.isPrefixOf` url
|
||||
, let (path, fragment) = T.break (== '#') url
|
||||
, ".pdf" `T.isSuffixOf` T.toLower path
|
||||
, "pdf-link" `notElem` classes =
|
||||
let viewerUrl = "/pdfjs/web/viewer.html?file="
|
||||
<> encodeQueryValue path <> fragment
|
||||
classes' = classes ++ ["pdf-link"]
|
||||
kvs' = kvs ++ [("data-pdf-src", path)]
|
||||
in Link (ident, classes', kvs') ils (viewerUrl, title)
|
||||
classifyPdfLink x = x
|
||||
|
||||
classifyLink :: Inline -> Inline
|
||||
classifyLink l@(Link (_, classes, _) _ _)
|
||||
-- Source-ref links are owned by Filters.SourceRefs: they keep the
|
||||
-- inline-code chrome of their body, must not receive an external
|
||||
-- brand icon stamp, and have their own popup provider. Leave them
|
||||
-- entirely alone.
|
||||
| "source-ref" `elem` classes = l
|
||||
-- PDF links were already rewritten to the (root-relative) viewer URL
|
||||
-- and given their own chrome by 'classifyPdfLink' in the preceding
|
||||
-- pass; without this guard they would be double-classified as
|
||||
-- internal page links.
|
||||
| "pdf-link" `elem` classes = l
|
||||
classifyLink (Link (ident, classes, kvs) ils (url, title))
|
||||
| isExternal url =
|
||||
let icon = domainIcon url
|
||||
classes' = classes ++ ["link-external"]
|
||||
kvs' = kvs
|
||||
++ [("target", "_blank")]
|
||||
++ [("rel", "noopener noreferrer")]
|
||||
++ [("data-link-icon", icon)]
|
||||
++ [("data-link-icon-type", "svg")]
|
||||
in Link (ident, classes', kvs') ils (url, title)
|
||||
| isInternalPage url =
|
||||
let classes' = classes ++ ["link-internal"]
|
||||
kvs' = kvs
|
||||
++ [("data-link-icon", "internal")]
|
||||
++ [("data-link-icon-type", "svg")]
|
||||
in Link (ident, classes', kvs') ils (url, title)
|
||||
classifyLink x = x
|
||||
|
||||
-- ---------------------------------------------------------------------------
|
||||
-- Helpers
|
||||
-- ---------------------------------------------------------------------------
|
||||
|
||||
-- | True if the URL is a root-relative or relative path to another page
|
||||
-- (not an anchor-only link like @#section@ or @#ref-foo@).
|
||||
isInternalPage :: Text -> Bool
|
||||
isInternalPage url
|
||||
| T.null url = False
|
||||
| "#" `T.isPrefixOf` url = False -- anchor-only
|
||||
| "mailto:" `T.isPrefixOf` url = False
|
||||
| "http://" `T.isPrefixOf` url = False -- handled by isExternal
|
||||
| "https://" `T.isPrefixOf` url = False
|
||||
| otherwise = True
|
||||
|
||||
-- | True if the URL points outside the site's content host.
|
||||
--
|
||||
-- Only @levineuwirth.org@ and @www.levineuwirth.org@ count as the content
|
||||
-- site itself. Sibling subdomains like @git.levineuwirth.org@ (Forgejo) are
|
||||
-- distinct services and are classified as external so they receive their
|
||||
-- brand icon, @target=_blank@, and @rel=noopener noreferrer@.
|
||||
--
|
||||
-- Uses strict hostname comparison rather than substring matching, so a
|
||||
-- hostile lookalike like @evil-levineuwirth.org.attacker.com@ is also
|
||||
-- correctly classified as external.
|
||||
isExternal :: Text -> Bool
|
||||
isExternal url =
|
||||
case extractHost url of
|
||||
Nothing -> False
|
||||
Just host -> host /= siteHost && host /= "www." <> siteHost
|
||||
where
|
||||
siteHost = "levineuwirth.org"
|
||||
|
||||
-- | Extract the lowercased hostname from an absolute http(s) URL,
|
||||
-- stripping any userinfo (@user:pass\@@) and port. Returns 'Nothing'
|
||||
-- for non-http(s) URLs (relative paths, mailto:, etc.).
|
||||
extractHost :: Text -> Maybe Text
|
||||
extractHost url
|
||||
| Just rest <- T.stripPrefix "https://" url = Just (hostOf rest)
|
||||
| Just rest <- T.stripPrefix "http://" url = Just (hostOf rest)
|
||||
| otherwise = Nothing
|
||||
where
|
||||
hostOf rest =
|
||||
let authority = T.takeWhile (\c -> c /= '/' && c /= '?' && c /= '#') rest
|
||||
-- 'T.breakOnEnd' yields the segment after the last @\@@, or
|
||||
-- the whole authority when there is no userinfo.
|
||||
(_, hostPort) = T.breakOnEnd "@" authority
|
||||
host = T.takeWhile (/= ':') hostPort
|
||||
in T.toLower host
|
||||
|
||||
-- | Icon name for the link, matching a file in /images/link-icons/<name>.svg.
|
||||
--
|
||||
-- Matches on the URL's host only, never on the full URL — a path like
|
||||
-- @https://example.org/why-x.com-failed@ must not get the Twitter
|
||||
-- icon. URLs with no extractable host get the generic icon.
|
||||
domainIcon :: Text -> Text
|
||||
domainIcon url = maybe "external" iconForHost (extractHost url)
|
||||
|
||||
iconForHost :: Text -> Text
|
||||
iconForHost host
|
||||
-- Scholarly / reference
|
||||
| m "wikipedia.org" = "wikipedia"
|
||||
| m "arxiv.org" = "arxiv"
|
||||
| m "doi.org" = "doi"
|
||||
| m "worldcat.org" = "worldcat"
|
||||
| m "orcid.org" = "orcid"
|
||||
| m "archive.org" = "internet-archive"
|
||||
-- Code / software
|
||||
| m "github.com" = "github"
|
||||
| m "git.levineuwirth.org" = "forgejo"
|
||||
| m "tensorflow.org" = "tensorflow"
|
||||
-- AI companies (consumer products share a brand icon with the lab)
|
||||
| m "anthropic.com" = "anthropic"
|
||||
| m "claude.ai" = "anthropic"
|
||||
| m "openai.com" = "openai"
|
||||
| m "chatgpt.com" = "openai"
|
||||
-- Social / media
|
||||
| m "twitter.com" = "twitter"
|
||||
| m "x.com" = "twitter"
|
||||
| m "reddit.com" = "reddit"
|
||||
| m "youtube.com" = "youtube"
|
||||
| m "youtu.be" = "youtube"
|
||||
| m "tiktok.com" = "tiktok"
|
||||
| m "substack.com" = "substack"
|
||||
| m "news.ycombinator.com" = "hacker-news"
|
||||
| m "lesswrong.com" = "lesswrong"
|
||||
-- News
|
||||
| m "nytimes.com" = "new-york-times"
|
||||
-- Institutions
|
||||
| m "nasa.gov" = "nasa"
|
||||
| m "apple.com" = "apple"
|
||||
| otherwise = "external"
|
||||
where
|
||||
-- Label-suffix match: the host is the domain itself or a subdomain
|
||||
-- of it. Never fires on a lookalike label (@notx.com@) or on text
|
||||
-- in the path or query.
|
||||
m d = host == d || ("." <> d) `T.isSuffixOf` host
|
||||
|
||||
-- | Percent-encode characters that would break a @?file=@ query-string value.
|
||||
-- Slashes are intentionally left unencoded so root-relative paths remain
|
||||
-- readable and work correctly with PDF.js's internal fetch.
|
||||
encodeQueryValue :: Text -> Text
|
||||
encodeQueryValue = T.concatMap enc
|
||||
where
|
||||
enc ' ' = "%20"
|
||||
enc '&' = "%26"
|
||||
enc '?' = "%3F"
|
||||
enc '+' = "%2B"
|
||||
enc '"' = "%22"
|
||||
enc c = T.singleton c
|
||||
|
|
@ -0,0 +1,14 @@
|
|||
{-# LANGUAGE GHC2021 #-}
|
||||
-- | Math filter placeholder.
|
||||
--
|
||||
-- The spec calls for converting simple LaTeX to Unicode at build time.
|
||||
-- For now, all math (inline and display) is handled client-side by KaTeX,
|
||||
-- which is loaded conditionally on pages that contain math. Server-side
|
||||
-- KaTeX rendering is a Phase 3 task.
|
||||
module Filters.Math (apply) where
|
||||
|
||||
import Text.Pandoc.Definition (Pandoc)
|
||||
|
||||
-- | Identity — math rendering is handled by KaTeX.
|
||||
apply :: Pandoc -> Pandoc
|
||||
apply = id
|
||||
|
|
@ -0,0 +1,146 @@
|
|||
{-# LANGUAGE GHC2021 #-}
|
||||
{-# LANGUAGE OverloadedStrings #-}
|
||||
-- | Inline SVG score fragments into the Pandoc AST.
|
||||
--
|
||||
-- Fenced-div syntax in Markdown:
|
||||
--
|
||||
-- > :::score-fragment{score-name="Main Theme, mm. 1–8" score-caption="The opening gesture."}
|
||||
-- > 
|
||||
-- > :::
|
||||
--
|
||||
-- The filter reads the referenced SVG from disk (path resolved relative to
|
||||
-- the source file's directory), replaces hardcoded black fills/strokes with
|
||||
-- @currentColor@ for dark-mode compatibility, and emits a @\<figure\>@ with
|
||||
-- the appropriate exhibit attributes for gallery.js TOC integration.
|
||||
module Filters.Score (inlineScores) where
|
||||
|
||||
import Control.Exception (IOException, try)
|
||||
import Data.Char (isHexDigit)
|
||||
import Data.Maybe (listToMaybe)
|
||||
import qualified Data.Text as T
|
||||
import qualified Data.Text.IO as TIO
|
||||
import System.Directory (doesFileExist)
|
||||
import System.FilePath ((</>))
|
||||
import System.IO (hPutStrLn, stderr)
|
||||
import Text.Pandoc.Definition
|
||||
import Text.Pandoc.Walk (walkM)
|
||||
import qualified Utils as U
|
||||
|
||||
-- | Walk the Pandoc AST and inline all score-fragment divs.
|
||||
-- @baseDir@ is the directory of the source file; image paths in the
|
||||
-- fenced-div are resolved relative to it.
|
||||
inlineScores :: FilePath -> Pandoc -> IO Pandoc
|
||||
inlineScores baseDir = walkM (inlineScore baseDir)
|
||||
|
||||
inlineScore :: FilePath -> Block -> IO Block
|
||||
inlineScore baseDir (Div (_, cls, attrs) blocks)
|
||||
| "score-fragment" `elem` cls = do
|
||||
let mName = lookup "score-name" attrs
|
||||
mCaption = lookup "score-caption" attrs
|
||||
mPath = findImagePath blocks
|
||||
case mPath of
|
||||
Nothing -> return $ Div ("", cls, attrs) blocks
|
||||
Just path -> do
|
||||
let fullPath = baseDir </> T.unpack path
|
||||
exists <- doesFileExist fullPath
|
||||
if not exists
|
||||
then do
|
||||
hPutStrLn stderr $
|
||||
"[Score] missing SVG: " ++ fullPath
|
||||
++ " (referenced from a score-fragment in " ++ baseDir ++ ")"
|
||||
return (errorBlock mName ("Missing score: " <> path))
|
||||
else do
|
||||
result <- try (TIO.readFile fullPath) :: IO (Either IOException T.Text)
|
||||
case result of
|
||||
Left e -> do
|
||||
hPutStrLn stderr $
|
||||
"[Score] read error on " ++ fullPath ++ ": " ++ show e
|
||||
return (errorBlock mName ("Could not read score: " <> path))
|
||||
Right svgRaw -> do
|
||||
let html = buildHtml mName mCaption (processColors svgRaw)
|
||||
return $ RawBlock (Format "html") html
|
||||
inlineScore _ block = return block
|
||||
|
||||
-- | Render an inline error block in place of a missing or unreadable score.
|
||||
-- Mirrors the convention in 'Filters.Viz.errorBlock' so build failures are
|
||||
-- visible to the author without aborting the entire site build.
|
||||
errorBlock :: Maybe T.Text -> T.Text -> Block
|
||||
errorBlock mName message =
|
||||
RawBlock (Format "html") $ T.concat
|
||||
[ "<figure class=\"score-fragment score-fragment--error\""
|
||||
, maybe "" (\n -> " data-exhibit-name=\"" <> escHtml n <> "\"") mName
|
||||
, ">"
|
||||
, "<div class=\"score-fragment-error\">"
|
||||
, escHtml message
|
||||
, "</div>"
|
||||
, "</figure>"
|
||||
]
|
||||
|
||||
-- | Extract the image src from the first Para that contains an Image inline.
|
||||
findImagePath :: [Block] -> Maybe T.Text
|
||||
findImagePath blocks = listToMaybe
|
||||
[ src
|
||||
| Para inlines <- blocks
|
||||
, Image _ _ (src, _) <- inlines
|
||||
]
|
||||
|
||||
-- | Replace hardcoded black fill/stroke values with @currentColor@ so the
|
||||
-- SVG inherits the CSS @color@ property in both light and dark modes.
|
||||
--
|
||||
-- Quoted attribute forms (@fill="#000"@) are self-delimiting — the
|
||||
-- closing quote bounds the match — so plain 'T.replace' is safe for
|
||||
-- them. Unquoted style-property forms (@fill:#000@) are not: naive
|
||||
-- replacement would also fire on the prefix of a longer hex colour
|
||||
-- (@fill:#000080@ → @fill:currentColor80@, invalid CSS). Those go
|
||||
-- through 'replaceHexColor', which rewrites a match only when it is
|
||||
-- not followed by another hex digit; the boundary check also makes
|
||||
-- the 3-digit/6-digit application order irrelevant.
|
||||
processColors :: T.Text -> T.Text
|
||||
processColors
|
||||
-- 3-digit hex and keyword patterns
|
||||
= T.replace "fill=\"#000\"" "fill=\"currentColor\""
|
||||
. T.replace "fill=\"black\"" "fill=\"currentColor\""
|
||||
. T.replace "stroke=\"#000\"" "stroke=\"currentColor\""
|
||||
. T.replace "stroke=\"black\"" "stroke=\"currentColor\""
|
||||
. replaceHexColor "fill:#000" "fill:currentColor"
|
||||
. T.replace "fill:black" "fill:currentColor"
|
||||
. replaceHexColor "stroke:#000" "stroke:currentColor"
|
||||
. T.replace "stroke:black" "stroke:currentColor"
|
||||
-- 6-digit hex patterns (applied first — bottom of the chain)
|
||||
. T.replace "fill=\"#000000\"" "fill=\"currentColor\""
|
||||
. T.replace "stroke=\"#000000\"" "stroke=\"currentColor\""
|
||||
. replaceHexColor "fill:#000000" "fill:currentColor"
|
||||
. replaceHexColor "stroke:#000000" "stroke:currentColor"
|
||||
|
||||
-- | 'T.replace' restricted to hex-boundary-terminated matches: an
|
||||
-- occurrence of @needle@ is rewritten only when the character after
|
||||
-- it is not another hex digit, so @fill:#000@ never fires inside the
|
||||
-- longer colours @fill:#0008@, @fill:#000080@, or @fill:#00000080@.
|
||||
replaceHexColor :: T.Text -> T.Text -> T.Text -> T.Text
|
||||
replaceHexColor needle replacement = go
|
||||
where
|
||||
go t =
|
||||
let (pre, rest) = T.breakOn needle t
|
||||
in if T.null rest
|
||||
then pre
|
||||
else
|
||||
let after = T.drop (T.length needle) rest
|
||||
in case T.uncons after of
|
||||
Just (c, _) | isHexDigit c ->
|
||||
pre <> needle <> go after
|
||||
_ -> pre <> replacement <> go after
|
||||
|
||||
buildHtml :: Maybe T.Text -> Maybe T.Text -> T.Text -> T.Text
|
||||
buildHtml mName mCaption svgContent = T.concat
|
||||
[ "<figure class=\"score-fragment exhibit\""
|
||||
, maybe "" (\n -> " data-exhibit-name=\"" <> escHtml n <> "\"") mName
|
||||
, " data-exhibit-type=\"score\">"
|
||||
, "<div class=\"score-fragment-inner\">"
|
||||
, svgContent
|
||||
, "</div>"
|
||||
, maybe "" (\c -> "<figcaption class=\"score-caption\">" <> escHtml c <> "</figcaption>") mCaption
|
||||
, "</figure>"
|
||||
]
|
||||
|
||||
escHtml :: T.Text -> T.Text
|
||||
escHtml = U.escapeHtmlText
|
||||
|
|
@ -0,0 +1,162 @@
|
|||
{-# LANGUAGE GHC2021 #-}
|
||||
{-# LANGUAGE OverloadedStrings #-}
|
||||
-- | Convert Pandoc @Note@ inlines to inline sidenote HTML.
|
||||
--
|
||||
-- Each footnote becomes:
|
||||
-- * A @<sup class="sidenote-ref">@ anchor in the body text.
|
||||
-- * A @<span class="sidenote">@ immediately following it, containing
|
||||
-- the rendered note content.
|
||||
--
|
||||
-- Additionally, every consumed note is re-emitted in a
|
||||
-- @<section class="footnotes">@ appended at the document end. The
|
||||
-- filter swallows Pandoc's own @Note@ inlines, so Pandoc's writer
|
||||
-- never produces that section itself — without this re-emission,
|
||||
-- narrow viewports with JavaScript disabled (where sidenotes.css
|
||||
-- hides @.sidenote@ and sidenotes.js's bottom sheet never runs)
|
||||
-- would lose footnote content entirely.
|
||||
--
|
||||
-- On wide viewports, sidenotes.css floats the spans into the right
|
||||
-- margin and hides @section.footnotes@; on narrow viewports the
|
||||
-- spans are hidden and the section is shown. The in-text anchor
|
||||
-- targets the footnotes item (the only target visible on narrow
|
||||
-- no-JS viewports); sidenotes.js intercepts clicks and pairs
|
||||
-- ref\/note by element id, so the href is purely the no-JS path.
|
||||
module Filters.Sidenotes (apply) where
|
||||
|
||||
import Control.Monad.State.Strict
|
||||
import Data.Default (def)
|
||||
import Data.Text (Text)
|
||||
import qualified Data.Text as T
|
||||
import Text.Pandoc.Class (runPure)
|
||||
import Text.Pandoc.Definition
|
||||
import Text.Pandoc.Options (WriterOptions (..),
|
||||
HTMLMathMethod (KaTeX))
|
||||
import Text.Pandoc.Walk (walkM)
|
||||
import Text.Pandoc.Writers.HTML (writeHtml5String)
|
||||
|
||||
-- | Accumulator: next label counter plus collected notes
|
||||
-- (newest-first; reversed before rendering the fallback section).
|
||||
type NoteState = (Int, [(Text, [Block])])
|
||||
|
||||
-- | Transform all @Note@ inlines in the document to inline sidenote
|
||||
-- HTML, and append the collected notes as a @section.footnotes@
|
||||
-- fallback block.
|
||||
apply :: Pandoc -> Pandoc
|
||||
apply doc =
|
||||
let (Pandoc m blocks, (_, collected)) =
|
||||
runState (walkM convertNote doc) (1, [])
|
||||
notes = reverse collected
|
||||
in Pandoc m $
|
||||
if null notes
|
||||
then blocks
|
||||
else blocks ++ [footnotesSection notes]
|
||||
|
||||
convertNote :: Inline -> State NoteState Inline
|
||||
convertNote (Note blocks) = do
|
||||
(n, acc) <- get
|
||||
put (n + 1, (toLabel n, blocks) : acc)
|
||||
return $ RawInline "html" (renderNote n blocks)
|
||||
convertNote x = return x
|
||||
|
||||
-- | The end-of-document fallback list. Letter labels are rendered
|
||||
-- explicitly (an @<ol>@'s automatic numbering would disagree with
|
||||
-- the in-text letters), so the list itself is unstyled.
|
||||
footnotesSection :: [(Text, [Block])] -> Block
|
||||
footnotesSection notes = RawBlock "html" $ T.concat $
|
||||
[ "<section class=\"footnotes\" role=\"doc-endnotes\">"
|
||||
, "<ol class=\"footnotes-list\">"
|
||||
]
|
||||
++ map item notes ++
|
||||
[ "</ol>"
|
||||
, "</section>"
|
||||
]
|
||||
where
|
||||
item (lbl, blocks) = T.concat
|
||||
[ "<li id=\"fn-", lbl, "\" class=\"footnote-item\">"
|
||||
, "<span class=\"footnote-label\" aria-hidden=\"true\">", lbl, "</span>"
|
||||
, blocksToHtml blocks
|
||||
, "<a href=\"#snref-", lbl
|
||||
, "\" class=\"footnote-back\" role=\"doc-backlink\""
|
||||
, " aria-label=\"Back to reference ", lbl, "\">\x21a9\xfe0e</a>"
|
||||
, "</li>"
|
||||
]
|
||||
|
||||
-- | Convert a 1-based counter to a letter label using base-26 expansion
|
||||
-- (Excel-column style): 1→a, 2→b, … 26→z, 27→aa, 28→ab, … 52→az,
|
||||
-- 53→ba, … 702→zz, 703→aaa. Guarantees a unique label per counter so
|
||||
-- no two sidenotes in a single document collide on @id="sn-…"@.
|
||||
toLabel :: Int -> Text
|
||||
toLabel n
|
||||
| n <= 0 = "?"
|
||||
| otherwise = T.pack (go n)
|
||||
where
|
||||
go k
|
||||
| k <= 0 = ""
|
||||
| otherwise =
|
||||
let (q, r) = (k - 1) `divMod` 26
|
||||
in go q ++ [toEnum (fromEnum 'a' + r)]
|
||||
|
||||
renderNote :: Int -> [Block] -> Text
|
||||
renderNote n blocks =
|
||||
let inner = blocksToInlineHtml blocks
|
||||
lbl = toLabel n
|
||||
in T.concat
|
||||
-- href targets the footnotes-section item: on narrow no-JS
|
||||
-- viewports that is the only visible rendering of the note
|
||||
-- (the adjacent .sidenote span is display:none there, and on
|
||||
-- wide viewports the note is already visible in the margin).
|
||||
-- sidenotes.js pairs ref/note by id and preventDefaults the
|
||||
-- click, so the href only ever navigates without JS.
|
||||
[ "<sup class=\"sidenote-ref\" id=\"snref-", lbl, "\">"
|
||||
, "<a href=\"#fn-", lbl, "\">", lbl, "</a>"
|
||||
, "</sup>"
|
||||
, "<span class=\"sidenote\" id=\"sn-", lbl, "\">"
|
||||
, "<sup class=\"sidenote-num\">", lbl, "</sup>\x00a0"
|
||||
, inner
|
||||
, "</span>"
|
||||
]
|
||||
|
||||
-- | Render a list of Pandoc blocks for inclusion inside an inline @<span
|
||||
-- class="sidenote">@. Each top-level @Para@ is wrapped in a
|
||||
-- @<span class="sidenote-para">@ instead of a @<p>@ (which would be
|
||||
-- invalid inside a @<span>@); other block types are rendered with the
|
||||
-- regular Pandoc HTML writer.
|
||||
--
|
||||
-- Operating on the AST is preferred over post-rendered string
|
||||
-- substitution because the latter mangles content that legitimately
|
||||
-- contains the literal text @<p>@ (e.g. code samples discussing HTML).
|
||||
blocksToInlineHtml :: [Block] -> Text
|
||||
blocksToInlineHtml = T.concat . map renderOne
|
||||
where
|
||||
renderOne :: Block -> Text
|
||||
renderOne (Para inlines) =
|
||||
"<span class=\"sidenote-para\">"
|
||||
<> inlinesToHtml inlines
|
||||
<> "</span>"
|
||||
renderOne (Plain inlines) =
|
||||
inlinesToHtml inlines
|
||||
renderOne b =
|
||||
blocksToHtml [b]
|
||||
|
||||
-- | Writer options for note bodies. Must agree with the math method in
|
||||
-- 'Compilers.writerOpts' (KaTeX), or math inside a footnote silently
|
||||
-- degrades to the writer default (PlainMath -> italics) and the
|
||||
-- client-side KaTeX pass never sees it. Defined locally because
|
||||
-- importing Compilers from here would create a module cycle
|
||||
-- (Compilers -> Filters -> Filters.Sidenotes).
|
||||
noteWriterOpts :: WriterOptions
|
||||
noteWriterOpts = def { writerHTMLMathMethod = KaTeX "" }
|
||||
|
||||
-- | Render a list of inlines to HTML (no surrounding @<p>@).
|
||||
inlinesToHtml :: [Inline] -> Text
|
||||
inlinesToHtml inlines =
|
||||
case runPure (writeHtml5String noteWriterOpts (Pandoc mempty [Plain inlines])) of
|
||||
Left _ -> T.empty
|
||||
Right t -> t
|
||||
|
||||
-- | Render a list of Pandoc blocks to an HTML fragment via a pure writer run.
|
||||
blocksToHtml :: [Block] -> Text
|
||||
blocksToHtml blocks =
|
||||
case runPure (writeHtml5String noteWriterOpts (Pandoc mempty blocks)) of
|
||||
Left _ -> T.empty
|
||||
Right t -> t
|
||||
|
|
@ -0,0 +1,85 @@
|
|||
{-# LANGUAGE GHC2021 #-}
|
||||
{-# LANGUAGE OverloadedStrings #-}
|
||||
-- | Automatic small-caps wrapping for abbreviations in body text.
|
||||
--
|
||||
-- Any @Str@ token that consists entirely of uppercase letters (and
|
||||
-- hyphens) and is at least three characters long is wrapped in
|
||||
-- @<abbr class="smallcaps">@. This catches CSS, HTML, API, NASA, etc.
|
||||
-- while avoiding single-character tokens (\"I\", \"A\") and mixed-case
|
||||
-- words.
|
||||
--
|
||||
-- Authors can also use Pandoc span syntax for explicit control:
|
||||
-- @[TEXT]{.smallcaps}@ — Pandoc already emits the @smallcaps@ class on
|
||||
-- those spans, and typography.css styles @.smallcaps@ directly, so no
|
||||
-- extra filter logic is needed for that case.
|
||||
--
|
||||
-- The filter is /not/ applied inside headings (where Fira Sans uppercase
|
||||
-- text looks intentional, at any nesting depth — including headings
|
||||
-- inside divs and block quotes) or inside @Code@/@RawInline@ inlines.
|
||||
module Filters.Smallcaps (apply) where
|
||||
|
||||
import Data.Char (isUpper, isAlpha)
|
||||
import Data.Text (Text)
|
||||
import qualified Data.Text as T
|
||||
import Text.Pandoc.Definition
|
||||
import Text.Pandoc.Walk (walk)
|
||||
import qualified Utils as U
|
||||
|
||||
-- | Apply smallcaps detection to paragraph-level content.
|
||||
-- Heading blocks are skipped at /every/ nesting level (a top-level
|
||||
-- pattern match would miss a @Header@ inside a @Div@ or
|
||||
-- @BlockQuote@): each header's @Str@ content is swapped for a
|
||||
-- sentinel 'RawInline' before the wrapping walk and restored
|
||||
-- afterwards, so 'wrapCaps' can never see it, wherever the header
|
||||
-- sits in the block tree.
|
||||
apply :: Pandoc -> Pandoc
|
||||
apply = walk restoreStr . walk wrapCaps . walk protectHeader
|
||||
|
||||
-- | Sentinel format marking a @Str@ that must not be wrapped. It only
|
||||
-- exists between the protect and restore walks inside 'apply' and
|
||||
-- can never leak into the writer.
|
||||
skipFmt :: Format
|
||||
skipFmt = Format "smallcaps-skip"
|
||||
|
||||
protectHeader :: Block -> Block
|
||||
protectHeader (Header lvl attr ils) = Header lvl attr (walk protectStr ils)
|
||||
where
|
||||
protectStr (Str t) = RawInline skipFmt t
|
||||
protectStr x = x
|
||||
protectHeader b = b
|
||||
|
||||
restoreStr :: Inline -> Inline
|
||||
restoreStr (RawInline fmt t) | fmt == skipFmt = Str t
|
||||
restoreStr x = x
|
||||
|
||||
-- | Wrap an all-caps Str token in an abbr element, preserving any trailing
|
||||
-- punctuation (comma, period, colon, semicolon, closing paren/bracket)
|
||||
-- outside the abbr element.
|
||||
wrapCaps :: Inline -> Inline
|
||||
wrapCaps (Str t) =
|
||||
let (core, trail) = stripTrailingPunct t
|
||||
in if isAbbreviation core
|
||||
then RawInline "html" $
|
||||
"<abbr class=\"smallcaps\">" <> escHtml core <> "</abbr>"
|
||||
<> trail
|
||||
else Str t
|
||||
wrapCaps x = x
|
||||
|
||||
-- | Split trailing punctuation from the token body.
|
||||
stripTrailingPunct :: Text -> (Text, Text)
|
||||
stripTrailingPunct t =
|
||||
let isPunct c = c `elem` (",.:;!?)]\'" :: String)
|
||||
trail = T.takeWhileEnd isPunct t
|
||||
core = T.dropEnd (T.length trail) t
|
||||
in (core, trail)
|
||||
|
||||
-- | True if the token looks like an abbreviation: all uppercase (plus
|
||||
-- hyphens), at least 3 characters, contains at least one alpha character.
|
||||
isAbbreviation :: Text -> Bool
|
||||
isAbbreviation t =
|
||||
T.length t >= 3
|
||||
&& T.all (\c -> isUpper c || c == '-') t
|
||||
&& T.any isAlpha t
|
||||
|
||||
escHtml :: Text -> Text
|
||||
escHtml = U.escapeHtmlText
|
||||
|
|
@ -0,0 +1,218 @@
|
|||
{-# LANGUAGE GHC2021 #-}
|
||||
{-# LANGUAGE OverloadedStrings #-}
|
||||
-- | Detect repo-relative source-file references in prose and wrap them
|
||||
-- in a link that triggers a hover-preview popup of the file's contents.
|
||||
--
|
||||
-- Two trigger forms:
|
||||
--
|
||||
-- * Inline @\`build\/Filters\/Links.hs\`@ — Markdown inline code whose
|
||||
-- text passes a conservative source-path heuristic.
|
||||
-- * A Markdown link to
|
||||
-- @https:\/\/git.levineuwirth.org\/neuwirth\/levineuwirth.org\/(src|raw)\/branch\/<branch>\/<path>@.
|
||||
--
|
||||
-- Both produce
|
||||
-- @\<a class="source-ref" data-source-path="…" href="…">@. The href
|
||||
-- points to the Forgejo source viewer so a click without JS — or a
|
||||
-- popup that fails to fetch — still resolves to a useful target.
|
||||
-- The popup provider in @static\/js\/popups.js@ fetches
|
||||
-- @\/source\/\<path\>@ (a same-origin copy emitted by the Hakyll
|
||||
-- source-preview rule in 'Site.rules') and renders a
|
||||
-- syntax-highlighted snippet via Prism.
|
||||
--
|
||||
-- Conservative-by-design: the trigger only fires on paths the
|
||||
-- @/source/@ serving rule actually publishes ('isServedPath', a
|
||||
-- mirror of @sourcePreviewable@ in 'Site.rules'), or a small set of
|
||||
-- named root files. This keeps the parser cheap, avoids false
|
||||
-- positives on words that happen to contain a slash and a dot, and
|
||||
-- guarantees every wrapped path has a fetchable @/source/…@ copy.
|
||||
module Filters.SourceRefs (apply, isSourcePath, forgejoSourceUrl) where
|
||||
|
||||
import Control.Monad (when)
|
||||
import Data.IORef (IORef, atomicModifyIORef', newIORef, readIORef)
|
||||
import qualified Data.Map.Strict as Map
|
||||
import Data.Text (Text)
|
||||
import qualified Data.Text as T
|
||||
import System.Directory (doesFileExist)
|
||||
import System.IO.Unsafe (unsafePerformIO)
|
||||
import Text.Pandoc.Definition
|
||||
import Text.Pandoc.Walk (walkM)
|
||||
|
||||
-- | Two passes: lift Forgejo source URLs in existing Markdown links
|
||||
-- first, then wrap inline-code source paths. Both passes only add
|
||||
-- the @source-ref@ class when it is not already present, so re-runs
|
||||
-- are idempotent.
|
||||
--
|
||||
-- Runs in 'IO' because the heuristic confirms each candidate is a
|
||||
-- real on-disk file before wrapping. This rules out paths like
|
||||
-- @data/backlinks.json@ that look like source but are Hakyll build
|
||||
-- artifacts produced into @_site/@ — wrapping those would emit a
|
||||
-- link whose popup is guaranteed to 404.
|
||||
apply :: Pandoc -> IO Pandoc
|
||||
apply doc = do
|
||||
afterLinks <- walkM classifyExistingLink doc
|
||||
walkM wrapInlineCode afterLinks
|
||||
|
||||
-- | Inline @`path`@ → @\<a class="source-ref" data-source-path="path"\>\<code\>path\<\/code\>\<\/a\>@.
|
||||
-- The original 'Code' node is preserved as the link's body so the
|
||||
-- inline-code chrome (mono font, background) survives unchanged.
|
||||
wrapInlineCode :: Inline -> IO Inline
|
||||
wrapInlineCode orig@(Code (cIdent, cClasses, cKvs) txt)
|
||||
| "source-ref" `notElem` cClasses
|
||||
, isSourcePath txt = do
|
||||
exists <- existsCached txt
|
||||
if exists
|
||||
then pure $ Link
|
||||
( ""
|
||||
, ["source-ref"]
|
||||
, [ ("data-source-path", txt)
|
||||
, ("target", "_blank")
|
||||
, ("rel", "noopener noreferrer")
|
||||
]
|
||||
)
|
||||
[Code (cIdent, cClasses, cKvs) txt]
|
||||
(forgejoSourceUrl txt, "")
|
||||
else pure orig
|
||||
wrapInlineCode x = pure x
|
||||
|
||||
-- | Existing Markdown link to a Forgejo source URL on this site's git
|
||||
-- host → tagged @source-ref@ and given a @data-source-path@ pointing
|
||||
-- at the same path the popup provider expects.
|
||||
classifyExistingLink :: Inline -> IO Inline
|
||||
classifyExistingLink orig@(Link (ident, classes, kvs) ils (url, title))
|
||||
| "source-ref" `notElem` classes
|
||||
, Just path <- forgejoSourcePath url
|
||||
, isSourcePath path = do
|
||||
exists <- existsCached path
|
||||
if exists
|
||||
then pure $ Link
|
||||
( ident
|
||||
, classes ++ ["source-ref"]
|
||||
, kvs ++ [("data-source-path", path)]
|
||||
)
|
||||
ils (url, title)
|
||||
else pure orig
|
||||
classifyExistingLink x = pure x
|
||||
|
||||
-- ---------------------------------------------------------------------------
|
||||
-- Heuristic
|
||||
-- ---------------------------------------------------------------------------
|
||||
|
||||
-- | True when the text looks like a repo-relative path that the
|
||||
-- @/source/@ serving rule actually publishes (or is a whitelisted
|
||||
-- root file), ends in a known source extension, and contains only
|
||||
-- safe path characters. Conservative by design — the goal is no
|
||||
-- false positives on prose that incidentally contains a slash and a
|
||||
-- dot, and no wrapped path whose popup fetch would 404.
|
||||
isSourcePath :: Text -> Bool
|
||||
isSourcePath t = and
|
||||
[ not (T.null t)
|
||||
, T.all safeChar t
|
||||
, (isServedPath t && hasKnownExt t) || isKnownRootFile t
|
||||
]
|
||||
where
|
||||
safeChar c =
|
||||
('a' <= c && c <= 'z')
|
||||
|| ('A' <= c && c <= 'Z')
|
||||
|| ('0' <= c && c <= '9')
|
||||
|| c == '/' || c == '.' || c == '_' || c == '-' || c == '+'
|
||||
|
||||
-- | Mirror of the @sourcePreviewable@ whitelist in 'Site.rules' (the
|
||||
-- rule that copies files to @/source/<path>@) — the two must stay
|
||||
-- aligned so every link this filter emits has a corresponding
|
||||
-- @/source/…@ target for the popup to fetch. Directories Site.hs
|
||||
-- does not serve (e.g. @content/@) are deliberately absent here:
|
||||
-- wrapping them would emit popups that are guaranteed to 404.
|
||||
isServedPath :: Text -> Bool
|
||||
isServedPath t = or
|
||||
[ "build/" `T.isPrefixOf` t && hasExt ".hs"
|
||||
, "static/js/" `T.isPrefixOf` t
|
||||
, "static/css/" `T.isPrefixOf` t
|
||||
, "templates/" `T.isPrefixOf` t
|
||||
, "tools/" `T.isPrefixOf` t && (hasExt ".sh" || hasExt ".py")
|
||||
, "nginx/" `T.isPrefixOf` t && hasExt ".conf"
|
||||
, "data/" `T.isPrefixOf` t
|
||||
&& not ("/" `T.isInfixOf` T.drop 5 t) -- top-level data files only
|
||||
&& (hasExt ".json" || hasExt ".yaml" || hasExt ".md" || hasExt ".bib")
|
||||
]
|
||||
where
|
||||
hasExt e = e `T.isSuffixOf` T.toLower t
|
||||
|
||||
hasKnownExt :: Text -> Bool
|
||||
hasKnownExt t =
|
||||
let lower = T.toLower t
|
||||
in any (`T.isSuffixOf` lower)
|
||||
[ ".hs", ".js", ".mjs", ".css", ".html"
|
||||
, ".py", ".cabal", ".md", ".yaml", ".yml"
|
||||
, ".toml", ".sh", ".bash", ".svg", ".conf"
|
||||
, ".json", ".ini", ".tex", ".bib"
|
||||
]
|
||||
|
||||
isKnownRootFile :: Text -> Bool
|
||||
isKnownRootFile t = t `elem`
|
||||
[ "Makefile"
|
||||
, "levineuwirth.cabal"
|
||||
, "cabal.project", "cabal.project.freeze"
|
||||
, "pyproject.toml", "uv.lock"
|
||||
, "WRITING.md", "HOMEPAGE.md", "PHOTOGRAPHY.md", "README.md"
|
||||
, "LICENSE", "checklist.md"
|
||||
]
|
||||
|
||||
-- ---------------------------------------------------------------------------
|
||||
-- File existence cache
|
||||
-- ---------------------------------------------------------------------------
|
||||
|
||||
-- | Process-wide memo of /positive/ @doesFileExist@ results, keyed by
|
||||
-- the same path the popup will fetch. Hakyll runs this filter once
|
||||
-- per compiled page and the same source-file references recur across
|
||||
-- many pages (e.g. @build\/Filters\/Links.hs@ in the Links page,
|
||||
-- the Colophon, several essays); the cache turns N stats into one
|
||||
-- per distinct path. Only existence is memoized: a missing file is
|
||||
-- re-stat'ed on every miss, so a source file created during a
|
||||
-- long-lived @make watch@ session is picked up on the next rebuild
|
||||
-- instead of staying "absent" for the process lifetime. (A file
|
||||
-- /deleted/ mid-watch stays cached as present until restart — the
|
||||
-- benign direction: the popup fetch 404s and simply never appears.)
|
||||
-- The build process's working directory is the project root, so the
|
||||
-- path can be passed straight to 'doesFileExist' without prefixing.
|
||||
{-# NOINLINE existsCacheRef #-}
|
||||
existsCacheRef :: IORef (Map.Map Text Bool)
|
||||
existsCacheRef = unsafePerformIO (newIORef Map.empty)
|
||||
|
||||
existsCached :: Text -> IO Bool
|
||||
existsCached path = do
|
||||
cache <- readIORef existsCacheRef
|
||||
case Map.lookup path cache of
|
||||
Just b -> pure b
|
||||
Nothing -> do
|
||||
b <- doesFileExist (T.unpack path)
|
||||
when b $
|
||||
atomicModifyIORef' existsCacheRef (\m -> (Map.insert path b m, ()))
|
||||
pure b
|
||||
|
||||
-- ---------------------------------------------------------------------------
|
||||
-- Forgejo URL helpers
|
||||
-- ---------------------------------------------------------------------------
|
||||
|
||||
-- | Forgejo source-viewer URL for a repo-relative path. Pinned to the
|
||||
-- @main@ branch so previews always reflect the deployed tip.
|
||||
forgejoSourceUrl :: Text -> Text
|
||||
forgejoSourceUrl path =
|
||||
"https://git.levineuwirth.org/neuwirth/levineuwirth.org/src/branch/main/"
|
||||
<> path
|
||||
|
||||
-- | Inverse of 'forgejoSourceUrl': extract the repo-relative path from
|
||||
-- a Forgejo URL on this site's git host. Recognises both the
|
||||
-- @\/src\/branch\/<b>\/@ web view and the @\/raw\/branch\/<b>\/@
|
||||
-- variants. Returns 'Nothing' for any other URL.
|
||||
forgejoSourcePath :: Text -> Maybe Text
|
||||
forgejoSourcePath url = do
|
||||
rest <- T.stripPrefix repoBase url
|
||||
afterBranch <-
|
||||
case T.stripPrefix "src/branch/" rest of
|
||||
Just r -> Just r
|
||||
Nothing -> T.stripPrefix "raw/branch/" rest
|
||||
let (_branch, slashAndPath) = T.breakOn "/" afterBranch
|
||||
path = T.drop 1 slashAndPath
|
||||
if T.null path then Nothing else Just path
|
||||
where
|
||||
repoBase = "https://git.levineuwirth.org/neuwirth/levineuwirth.org/"
|
||||
|
|
@ -0,0 +1,83 @@
|
|||
{-# LANGUAGE GHC2021 #-}
|
||||
-- | Source-level transclusion preprocessor.
|
||||
--
|
||||
-- Rewrites block-level {{slug}} and {{slug#section}} directives to raw
|
||||
-- HTML placeholders that transclude.js resolves at runtime.
|
||||
--
|
||||
-- A directive must be the sole content of a line (after trimming) to be
|
||||
-- replaced — this prevents accidental substitution inside prose.
|
||||
--
|
||||
-- Code protection (honest scope): lines inside /fenced/ code blocks
|
||||
-- are passed through untouched ('Filters.Wikilinks.mapOutsideFences'),
|
||||
-- so fenced examples can show @{{slug}}@ literally. Indented code
|
||||
-- blocks and inline code spans are NOT recognised — a full-line
|
||||
-- directive inside either is still rewritten.
|
||||
--
|
||||
-- Examples:
|
||||
-- {{my-essay}} → full-page transclusion of /my-essay.html
|
||||
-- {{essays/deep-dive}} → /essays/deep-dive.html (full body)
|
||||
-- {{my-essay#introduction}} → section "introduction" of /my-essay.html
|
||||
module Filters.Transclusion (preprocess) where
|
||||
|
||||
import Data.List (isSuffixOf, isPrefixOf, stripPrefix)
|
||||
import Filters.Wikilinks (mapOutsideFences)
|
||||
import qualified Utils as U
|
||||
|
||||
-- | Apply transclusion substitution to the raw Markdown source string,
|
||||
-- skipping lines inside fenced code blocks.
|
||||
preprocess :: String -> String
|
||||
preprocess = mapOutsideFences processLine
|
||||
|
||||
processLine :: String -> String
|
||||
processLine line =
|
||||
case parseDirective (U.trim line) of
|
||||
Nothing -> line
|
||||
Just (url, secAttr) ->
|
||||
"<div class=\"transclude\" data-src=\"" ++ escAttr url ++ "\""
|
||||
++ secAttr ++ "></div>"
|
||||
|
||||
-- | Parse a {{slug}} or {{slug#section}} directive.
|
||||
-- Returns (absolute-url, section-attribute-string) or Nothing.
|
||||
--
|
||||
-- The section name is HTML-escaped before being interpolated into the
|
||||
-- @data-section@ attribute, so a stray @\"@, @&@, @<@, or @>@ in a
|
||||
-- section name cannot break the surrounding markup.
|
||||
parseDirective :: String -> Maybe (String, String)
|
||||
parseDirective s = do
|
||||
inner <- stripPrefix "{{" s >>= stripSuffix "}}"
|
||||
case break (== '#') inner of
|
||||
("", _) -> Nothing
|
||||
(slug, "") -> Just (slugToUrl slug, "")
|
||||
(slug, '#' : sec)
|
||||
| null sec -> Just (slugToUrl slug, "")
|
||||
| otherwise -> Just (slugToUrl slug,
|
||||
" data-section=\"" ++ escAttr sec ++ "\"")
|
||||
_ -> Nothing
|
||||
|
||||
-- | Convert a slug (possibly with leading slash, possibly with path segments)
|
||||
-- to a root-relative .html URL. Idempotent for slugs that already end in
|
||||
-- @.html@ so callers can safely pass either form.
|
||||
slugToUrl :: String -> String
|
||||
slugToUrl slug
|
||||
| ".html" `isSuffixOf` slug, "/" `isPrefixOf` slug = slug
|
||||
| ".html" `isSuffixOf` slug = "/" ++ slug
|
||||
| "/" `isPrefixOf` slug = slug ++ ".html"
|
||||
| otherwise = "/" ++ slug ++ ".html"
|
||||
|
||||
-- | Minimal HTML attribute-value escape.
|
||||
escAttr :: String -> String
|
||||
escAttr = concatMap esc
|
||||
where
|
||||
esc '&' = "&"
|
||||
esc '<' = "<"
|
||||
esc '>' = ">"
|
||||
esc '"' = """
|
||||
esc '\'' = "'"
|
||||
esc c = [c]
|
||||
|
||||
-- | Strip a suffix from a string, returning Nothing if not present.
|
||||
stripSuffix :: String -> String -> Maybe String
|
||||
stripSuffix suf str
|
||||
| suf `isSuffixOf` str = Just (take (length str - length suf) str)
|
||||
| otherwise = Nothing
|
||||
|
||||
|
|
@ -0,0 +1,54 @@
|
|||
{-# LANGUAGE GHC2021 #-}
|
||||
{-# LANGUAGE OverloadedStrings #-}
|
||||
-- | Typographic refinements applied to the Pandoc AST.
|
||||
--
|
||||
-- Currently: expands common Latin abbreviations to @<abbr>@ elements
|
||||
-- (e.g. → exempli gratia, i.e. → id est, etc.). Pandoc's @smart@
|
||||
-- reader extension already handles em-dashes, en-dashes, ellipses,
|
||||
-- and curly quotes, so those are not repeated here.
|
||||
module Filters.Typography (apply) where
|
||||
|
||||
import Data.Text (Text)
|
||||
import Text.Pandoc.Definition
|
||||
import Text.Pandoc.Walk (walk)
|
||||
import Utils (escapeHtmlText)
|
||||
|
||||
-- | Apply all typographic transformations to the document.
|
||||
apply :: Pandoc -> Pandoc
|
||||
apply = walk expandAbbrev
|
||||
|
||||
-- ---------------------------------------------------------------------------
|
||||
-- Abbreviation expansion
|
||||
-- ---------------------------------------------------------------------------
|
||||
|
||||
-- | Abbreviations that should be wrapped in @<abbr title="…">@.
|
||||
-- Each entry is (verbatim text as it appears in the Pandoc Str token,
|
||||
-- long-form title for the tooltip).
|
||||
abbrevMap :: [(Text, Text)]
|
||||
abbrevMap =
|
||||
[ ("e.g.", "exempli gratia")
|
||||
, ("i.e.", "id est")
|
||||
, ("cf.", "confer")
|
||||
, ("viz.", "videlicet")
|
||||
, ("ibid.", "ibidem")
|
||||
, ("op.", "opere") -- usually followed by "cit." in a separate token
|
||||
, ("NB", "nota bene")
|
||||
, ("NB:", "nota bene")
|
||||
]
|
||||
|
||||
-- | If the Str token exactly matches a known abbreviation, replace it with
|
||||
-- a @RawInline "html"@ @<abbr>@ element; otherwise leave it unchanged.
|
||||
--
|
||||
-- Both the @title@ attribute and the visible body pass through
|
||||
-- 'escapeHtmlText' for consistency with every other raw-HTML emitter
|
||||
-- in the filter pipeline. The abbreviations themselves are ASCII-safe
|
||||
-- so this is defense-in-depth rather than a live hazard.
|
||||
expandAbbrev :: Inline -> Inline
|
||||
expandAbbrev (Str t) =
|
||||
case lookup t abbrevMap of
|
||||
Just title ->
|
||||
RawInline "html" $
|
||||
"<abbr title=\"" <> escapeHtmlText title <> "\">"
|
||||
<> escapeHtmlText t <> "</abbr>"
|
||||
Nothing -> Str t
|
||||
expandAbbrev x = x
|
||||
|
|
@ -0,0 +1,238 @@
|
|||
{-# LANGUAGE GHC2021 #-}
|
||||
{-# LANGUAGE OverloadedStrings #-}
|
||||
-- | Inline data visualizations into the Pandoc AST.
|
||||
--
|
||||
-- Two fenced-div classes are recognized in Markdown:
|
||||
--
|
||||
-- __Static figure__ (Matplotlib → SVG, no client-side JS required):
|
||||
--
|
||||
-- > ::: {.figure script="figures/myplot.py" caption="Caption text"}
|
||||
-- > :::
|
||||
--
|
||||
-- Runs the Python script; stdout must be an SVG document with a
|
||||
-- transparent background. Black fills and strokes are replaced with
|
||||
-- @currentColor@ so figures adapt to dark mode automatically.
|
||||
-- See @tools/viz_theme.py@ for the recommended matplotlib setup.
|
||||
--
|
||||
-- __Interactive figure__ (Altair/Vega-Lite → JSON spec):
|
||||
--
|
||||
-- > ::: {.visualization script="figures/myplot.py" caption="Caption text"}
|
||||
-- > :::
|
||||
--
|
||||
-- Runs the Python script; stdout must be a Vega-Lite JSON spec. The spec
|
||||
-- is embedded verbatim inside a @\<script type=\"application\/json\"\>@ tag;
|
||||
-- @viz.js@ picks it up and renders it via Vega-Embed, applying a
|
||||
-- monochrome theme that responds to the site\'s light/dark toggle.
|
||||
--
|
||||
-- __Authoring conventions:__
|
||||
--
|
||||
-- * Scripts are run from the project root; paths are relative to it.
|
||||
-- * Scripts run under @.venv\/bin\/python3@ when that virtualenv exists
|
||||
-- (@uv sync@ creates it), otherwise under @python3@ from @PATH@.
|
||||
-- * @script=@ paths are resolved relative to the source file\'s directory.
|
||||
-- * For @.figure@ scripts: use pure black (@#000000@) for all drawn
|
||||
-- elements and transparent backgrounds so @processColors@ and CSS
|
||||
-- @currentColor@ handle dark mode.
|
||||
-- * For @.visualization@ scripts: set encoding colours to @\"black\"@;
|
||||
-- @viz.js@ applies the site palette via Vega-Lite @config@.
|
||||
-- * Set @viz: true@ in the page\'s YAML frontmatter to load Vega JS.
|
||||
module Filters.Viz (inlineViz) where
|
||||
|
||||
import Control.Exception (IOException, catch)
|
||||
import Data.Char (isHexDigit)
|
||||
import Data.Maybe (fromMaybe)
|
||||
import qualified Data.Text as T
|
||||
import System.Directory (doesFileExist)
|
||||
import System.Exit (ExitCode (..))
|
||||
import System.FilePath ((</>))
|
||||
import System.IO (hPutStrLn, stderr)
|
||||
import System.Process (readProcessWithExitCode)
|
||||
import Text.Pandoc.Definition
|
||||
import Text.Pandoc.Walk (walkM)
|
||||
import qualified Utils as U
|
||||
|
||||
-- ---------------------------------------------------------------------------
|
||||
-- Public entry point
|
||||
-- ---------------------------------------------------------------------------
|
||||
|
||||
-- | Walk the Pandoc AST and inline all @.figure@ and @.visualization@ divs.
|
||||
-- @baseDir@ is the directory of the source file; @script=@ paths are
|
||||
-- resolved relative to it.
|
||||
inlineViz :: FilePath -> Pandoc -> IO Pandoc
|
||||
inlineViz baseDir = walkM (transformBlock baseDir)
|
||||
|
||||
-- ---------------------------------------------------------------------------
|
||||
-- Block transformation
|
||||
-- ---------------------------------------------------------------------------
|
||||
|
||||
transformBlock :: FilePath -> Block -> IO Block
|
||||
transformBlock baseDir blk@(Div (_, cls, attrs) _)
|
||||
| "figure" `elem` cls = do
|
||||
result <- runScript baseDir attrs
|
||||
case result of
|
||||
Left err ->
|
||||
warn "figure" err >> return (errorBlock err)
|
||||
Right out ->
|
||||
let caption = attr "caption" attrs
|
||||
in return $ RawBlock (Format "html")
|
||||
(staticFigureHtml (processColors out) caption)
|
||||
| "visualization" `elem` cls = do
|
||||
result <- runScript baseDir attrs
|
||||
case result of
|
||||
Left err ->
|
||||
warn "visualization" err >> return (errorBlock err)
|
||||
Right out ->
|
||||
let caption = attr "caption" attrs
|
||||
in return $ RawBlock (Format "html")
|
||||
(interactiveFigureHtml (escScriptTag out) caption)
|
||||
| otherwise = return blk
|
||||
transformBlock _ b = return b
|
||||
|
||||
-- ---------------------------------------------------------------------------
|
||||
-- Script execution
|
||||
-- ---------------------------------------------------------------------------
|
||||
|
||||
-- | Path to the Python interpreter used for figure scripts.
|
||||
--
|
||||
-- Prefers the project virtualenv over whatever @python3@ happens to be
|
||||
-- first on @PATH@. Scripts run from the project root, so the relative
|
||||
-- path matches the @[ -d .venv ]@ gates the Makefile uses for its other
|
||||
-- Python steps.
|
||||
--
|
||||
-- Without this preference a bare @make build@ breaks every figure: it
|
||||
-- runs @cabal run site@ outside @uv run@, so the scripts inherit a
|
||||
-- system interpreter that has no numpy or matplotlib and each one exits
|
||||
-- with @ModuleNotFoundError@. Falling back to @PATH@ keeps a
|
||||
-- system-wide install (or an already-activated venv) working.
|
||||
pythonExe :: IO FilePath
|
||||
pythonExe = do
|
||||
let venvPython = ".venv" </> "bin" </> "python3"
|
||||
inVenv <- doesFileExist venvPython
|
||||
return (if inVenv then venvPython else "python3")
|
||||
|
||||
-- | Run @\<python\> <script>@. Returns the script\'s stdout on success, or an
|
||||
-- error message on failure (non-zero exit, missing @script=@ attribute, or
|
||||
-- missing script file). See 'pythonExe' for interpreter selection.
|
||||
runScript :: FilePath -> [(T.Text, T.Text)] -> IO (Either String T.Text)
|
||||
runScript baseDir attrs =
|
||||
case lookup "script" attrs of
|
||||
Nothing -> return (Left "missing script= attribute")
|
||||
Just p -> do
|
||||
let fullPath = baseDir </> T.unpack p
|
||||
exists <- doesFileExist fullPath
|
||||
if not exists
|
||||
then return (Left ("script not found: " ++ fullPath))
|
||||
else do
|
||||
py <- pythonExe
|
||||
(ec, out, err) <-
|
||||
readProcessWithExitCode py [fullPath] ""
|
||||
`catch` (\e -> return (ExitFailure 1, "", show (e :: IOException)))
|
||||
return $ case ec of
|
||||
ExitSuccess -> Right (T.pack out)
|
||||
ExitFailure _ -> Left $
|
||||
"in " ++ fullPath ++ ": "
|
||||
++ (if null err then "non-zero exit" else err)
|
||||
|
||||
-- ---------------------------------------------------------------------------
|
||||
-- SVG colour post-processing (mirrors Filters.Score.processColors)
|
||||
-- ---------------------------------------------------------------------------
|
||||
|
||||
-- | Replace hardcoded black fill/stroke values with @currentColor@ so the
|
||||
-- embedded SVG inherits the CSS text colour in both light and dark modes.
|
||||
--
|
||||
-- Quoted attribute forms (@fill="#000"@) are self-delimiting — the
|
||||
-- closing quote bounds the match — so plain 'T.replace' is safe for
|
||||
-- them. Unquoted style-property forms (@fill:#000@) are not: naive
|
||||
-- replacement would also fire on the prefix of a longer hex colour
|
||||
-- (@fill:#000080@ → @fill:currentColor80@, invalid CSS). Those go
|
||||
-- through 'replaceHexColor', which rewrites a match only when it is
|
||||
-- not followed by another hex digit.
|
||||
processColors :: T.Text -> T.Text
|
||||
processColors
|
||||
= T.replace "fill=\"#000\"" "fill=\"currentColor\""
|
||||
. T.replace "fill=\"black\"" "fill=\"currentColor\""
|
||||
. T.replace "stroke=\"#000\"" "stroke=\"currentColor\""
|
||||
. T.replace "stroke=\"black\"" "stroke=\"currentColor\""
|
||||
. replaceHexColor "fill:#000" "fill:currentColor"
|
||||
. T.replace "fill:black" "fill:currentColor"
|
||||
. replaceHexColor "stroke:#000" "stroke:currentColor"
|
||||
. T.replace "stroke:black" "stroke:currentColor"
|
||||
. T.replace "fill=\"#000000\"" "fill=\"currentColor\""
|
||||
. T.replace "stroke=\"#000000\"" "stroke=\"currentColor\""
|
||||
. replaceHexColor "fill:#000000" "fill:currentColor"
|
||||
. replaceHexColor "stroke:#000000" "stroke:currentColor"
|
||||
|
||||
-- | 'T.replace' restricted to hex-boundary-terminated matches: an
|
||||
-- occurrence of @needle@ is rewritten only when the character after
|
||||
-- it is not another hex digit, so @fill:#000@ never fires inside the
|
||||
-- longer colours @fill:#0008@, @fill:#000080@, or @fill:#00000080@.
|
||||
-- (Mirrors 'Filters.Score.replaceHexColor'.)
|
||||
replaceHexColor :: T.Text -> T.Text -> T.Text -> T.Text
|
||||
replaceHexColor needle replacement = go
|
||||
where
|
||||
go t =
|
||||
let (pre, rest) = T.breakOn needle t
|
||||
in if T.null rest
|
||||
then pre
|
||||
else
|
||||
let after = T.drop (T.length needle) rest
|
||||
in case T.uncons after of
|
||||
Just (c, _) | isHexDigit c ->
|
||||
pre <> needle <> go after
|
||||
_ -> pre <> replacement <> go after
|
||||
|
||||
-- ---------------------------------------------------------------------------
|
||||
-- JSON safety for <script> embedding
|
||||
-- ---------------------------------------------------------------------------
|
||||
|
||||
-- | Replace @<\/@ with the JSON Unicode escape @\u003c\/@ so that Vega-Lite
|
||||
-- JSON embedded inside a @\<script\>@ tag cannot accidentally close it.
|
||||
-- JSON.parse decodes the escape back to @<\/@ transparently.
|
||||
escScriptTag :: T.Text -> T.Text
|
||||
escScriptTag = T.replace "</" "\\u003c/"
|
||||
|
||||
-- ---------------------------------------------------------------------------
|
||||
-- HTML output
|
||||
-- ---------------------------------------------------------------------------
|
||||
|
||||
staticFigureHtml :: T.Text -> T.Text -> T.Text
|
||||
staticFigureHtml svgContent caption = T.concat
|
||||
[ "<figure class=\"viz-figure\">"
|
||||
, svgContent
|
||||
, if T.null caption then ""
|
||||
else "<figcaption class=\"viz-caption\">" <> escHtml caption <> "</figcaption>"
|
||||
, "</figure>"
|
||||
]
|
||||
|
||||
interactiveFigureHtml :: T.Text -> T.Text -> T.Text
|
||||
interactiveFigureHtml jsonSpec caption = T.concat
|
||||
[ "<figure class=\"viz-interactive\">"
|
||||
, "<div class=\"vega-container\">"
|
||||
, "<script type=\"application/json\" class=\"vega-spec\">"
|
||||
, jsonSpec
|
||||
, "</script>"
|
||||
, "</div>"
|
||||
, if T.null caption then ""
|
||||
else "<figcaption class=\"viz-caption\">" <> escHtml caption <> "</figcaption>"
|
||||
, "</figure>"
|
||||
]
|
||||
|
||||
errorBlock :: String -> Block
|
||||
errorBlock msg = RawBlock (Format "html") $ T.concat
|
||||
[ "<div class=\"viz-error\"><strong>Visualization error:</strong> "
|
||||
, escHtml (T.pack msg)
|
||||
, "</div>"
|
||||
]
|
||||
|
||||
-- ---------------------------------------------------------------------------
|
||||
-- Helpers
|
||||
-- ---------------------------------------------------------------------------
|
||||
|
||||
attr :: T.Text -> [(T.Text, T.Text)] -> T.Text
|
||||
attr key kvs = fromMaybe "" (lookup key kvs)
|
||||
|
||||
warn :: String -> String -> IO ()
|
||||
warn kind msg = hPutStrLn stderr $ "[Viz] " ++ kind ++ " error: " ++ msg
|
||||
|
||||
escHtml :: T.Text -> T.Text
|
||||
escHtml = U.escapeHtmlText
|
||||
|
|
@ -0,0 +1,193 @@
|
|||
{-# LANGUAGE GHC2021 #-}
|
||||
{-# LANGUAGE OverloadedStrings #-}
|
||||
-- | Wikilink syntax preprocessor.
|
||||
--
|
||||
-- Applied to the raw Markdown source string /before/ Pandoc parsing.
|
||||
-- Transforms:
|
||||
--
|
||||
-- * @[[Page Title]]@ → @[Page Title](/page-title)@
|
||||
-- * @[[Page Title|Display]]@ → @[Display](/page-title)@
|
||||
--
|
||||
-- The URL slug is derived from the page title: lowercased, spaces
|
||||
-- replaced with hyphens, non-alphanumeric characters stripped, and
|
||||
-- a @.html@ suffix appended so the link resolves identically under
|
||||
-- the dev server, file:// previews, and nginx in production.
|
||||
--
|
||||
-- Code protection (honest scope): lines inside /fenced/ code blocks
|
||||
-- are passed through untouched (see 'mapOutsideFences'), and within a
|
||||
-- line, inline code spans (backtick runs, CommonMark equal-length
|
||||
-- matching) are skipped — so both fenced and @`inline`@ examples can
|
||||
-- show @[[…]]@ literally. Indented code blocks and code spans that
|
||||
-- cross a line break are NOT recognised; a wikilink inside those is
|
||||
-- still rewritten.
|
||||
module Filters.Wikilinks (preprocess, mapOutsideFences) where
|
||||
|
||||
import Data.Char (isAlphaNum, toLower, isSpace)
|
||||
import Data.List (intercalate)
|
||||
import qualified Utils as U
|
||||
|
||||
-- | Scan the raw Markdown source for @[[…]]@ wikilinks and replace them
|
||||
-- with standard Markdown link syntax. Processing is line-by-line and
|
||||
-- skips fenced code blocks; a wikilink therefore cannot span a line
|
||||
-- break (which was never a sensible authoring form).
|
||||
preprocess :: String -> String
|
||||
preprocess = mapOutsideFences replaceWikilinks
|
||||
|
||||
replaceWikilinks :: String -> String
|
||||
replaceWikilinks = go
|
||||
where
|
||||
go [] = []
|
||||
-- Inline code span: a backtick run opens a span closed by a run of
|
||||
-- exactly the same length (CommonMark). Its body passes through
|
||||
-- verbatim so documentation can quote @`[[…]]`@ literally. An
|
||||
-- unclosed run is literal text — and then a following @[[…]]@ is
|
||||
-- genuinely a wikilink, matching how Pandoc will read the line.
|
||||
go s@('`':_) =
|
||||
let (run, afterRun) = span (== '`') s
|
||||
in case codeSpan (length run) afterRun of
|
||||
Just (body, after) -> run ++ body ++ run ++ go after
|
||||
Nothing -> run ++ go afterRun
|
||||
go ('[':'[':rest) =
|
||||
case break (== ']') rest of
|
||||
(inner, ']':']':after)
|
||||
| not (null inner) ->
|
||||
toMarkdownLink inner ++ go after
|
||||
_ -> '[' : '[' : go rest
|
||||
go (c:rest) = c : go rest
|
||||
|
||||
-- @codeSpan n s@: the span body and the remainder after a closing
|
||||
-- run of exactly @n@ backticks; 'Nothing' when no closer exists on
|
||||
-- this line.
|
||||
codeSpan :: Int -> String -> Maybe (String, String)
|
||||
codeSpan n = loop
|
||||
where
|
||||
loop [] = Nothing
|
||||
loop s@('`':_) =
|
||||
let (run, rest) = span (== '`') s
|
||||
in if length run == n
|
||||
then Just ("", rest)
|
||||
else prepend run <$> loop rest
|
||||
loop (c:cs) = prepend [c] <$> loop cs
|
||||
prepend pre (body, after) = (pre ++ body, after)
|
||||
|
||||
-- ---------------------------------------------------------------------------
|
||||
-- Fence-aware line mapping (shared by all source-level preprocessors)
|
||||
-- ---------------------------------------------------------------------------
|
||||
|
||||
-- | Apply a line transformation to every line that is not part of a
|
||||
-- fenced code block. Shared by the three source-level preprocessors
|
||||
-- (wikilinks here, 'Filters.Transclusion', 'Filters.EmbedPdf') so
|
||||
-- their directive syntax can be quoted literally inside fenced code.
|
||||
--
|
||||
-- Fence tracking follows CommonMark: an opener is at most three
|
||||
-- spaces of indentation followed by a run of at least three backticks
|
||||
-- or tildes (longer runs allowed); for backtick fences the info
|
||||
-- string may not contain a backtick. The closer uses the same fence
|
||||
-- character, a run at least as long as the opener, and nothing but
|
||||
-- whitespace after it. An unclosed fence extends to the end of the
|
||||
-- document. Fence delimiter lines themselves pass through untouched.
|
||||
--
|
||||
-- Honest scope: only /fenced/ code blocks are protected. Indented
|
||||
-- code blocks and inline code spans are not recognised here — a
|
||||
-- directive inside either is still rewritten.
|
||||
mapOutsideFences :: (String -> String) -> String -> String
|
||||
mapOutsideFences f = unlines . go Nothing . lines
|
||||
where
|
||||
go _ [] = []
|
||||
go Nothing (l:ls) =
|
||||
case openingFence l of
|
||||
Just fence -> l : go (Just fence) ls
|
||||
Nothing -> f l : go Nothing ls
|
||||
go st@(Just fence) (l:ls)
|
||||
| closesFence fence l = l : go Nothing ls
|
||||
| otherwise = l : go st ls
|
||||
|
||||
-- | The fence character and run length of a CommonMark fence opener,
|
||||
-- or 'Nothing' when the line does not open a fence.
|
||||
openingFence :: String -> Maybe (Char, Int)
|
||||
openingFence l = do
|
||||
rest <- stripFenceIndent l
|
||||
case rest of
|
||||
(c:_) | c == '`' || c == '~' ->
|
||||
let run = takeWhile (== c) rest
|
||||
n = length run
|
||||
info = drop n rest
|
||||
in if n >= 3 && (c == '~' || '`' `notElem` info)
|
||||
then Just (c, n)
|
||||
else Nothing
|
||||
_ -> Nothing
|
||||
|
||||
-- | True when the line closes the fence opened by @(c, n)@: the same
|
||||
-- fence character, a run at least as long as the opener, and only
|
||||
-- whitespace after it.
|
||||
closesFence :: (Char, Int) -> String -> Bool
|
||||
closesFence (c, n) l =
|
||||
case stripFenceIndent l of
|
||||
Nothing -> False
|
||||
Just rest ->
|
||||
let run = takeWhile (== c) rest
|
||||
in length run >= n && all isSpace (drop (length run) rest)
|
||||
|
||||
-- | Strip up to three leading spaces (the indentation CommonMark allows
|
||||
-- on a fence line); 'Nothing' for four or more, which would be an
|
||||
-- indented code block rather than a fence.
|
||||
stripFenceIndent :: String -> Maybe String
|
||||
stripFenceIndent l =
|
||||
let (indent, rest) = span (== ' ') l
|
||||
in if length indent <= 3 then Just rest else Nothing
|
||||
|
||||
-- | Convert the inner content of @[[…]]@ to a Markdown link.
|
||||
--
|
||||
-- Display text is escaped via 'escMdLinkText' so that a literal @]@, @[@,
|
||||
-- or backslash in the display does not break the surrounding Markdown
|
||||
-- link syntax. The URL itself is produced by 'slugify' and therefore only
|
||||
-- ever contains @[a-z0-9-]@, so no URL-side encoding is needed — adding
|
||||
-- one would be defense against a character set we can't produce.
|
||||
toMarkdownLink :: String -> String
|
||||
toMarkdownLink inner =
|
||||
let (title, display) = splitOnPipe inner
|
||||
url = "/" ++ slugify title ++ ".html"
|
||||
in "[" ++ escMdLinkText display ++ "](" ++ url ++ ")"
|
||||
|
||||
-- | Escape the minimum set of characters that would prematurely terminate
|
||||
-- a Markdown link's display-text segment: backslash (escape char), @[@,
|
||||
-- and @]@. Backslash MUST be escaped first so the escapes we introduce
|
||||
-- for @[@ and @]@ are not themselves re-escaped.
|
||||
--
|
||||
-- Deliberately NOT escaped: @_@, @*@, @\`@, @<@. Those are inline
|
||||
-- formatting markers in Markdown and escaping them would strip the
|
||||
-- author's ability to put emphasis, code, or inline HTML in a wikilink's
|
||||
-- display text.
|
||||
escMdLinkText :: String -> String
|
||||
escMdLinkText = concatMap esc
|
||||
where
|
||||
esc '\\' = "\\\\"
|
||||
esc '[' = "\\["
|
||||
esc ']' = "\\]"
|
||||
esc c = [c]
|
||||
|
||||
-- | Split on the first @|@; if none, display = title.
|
||||
splitOnPipe :: String -> (String, String)
|
||||
splitOnPipe s =
|
||||
case break (== '|') s of
|
||||
(title, '|':display) -> (U.trim title, U.trim display)
|
||||
_ -> (U.trim s, U.trim s)
|
||||
|
||||
-- | Produce a URL slug: lowercase, words joined by hyphens,
|
||||
-- non-alphanumeric characters removed.
|
||||
--
|
||||
-- Trailing punctuation is dropped rather than preserved as a dangling
|
||||
-- hyphen — @slugify "end." == "end"@, not @"end-"@. This is intentional:
|
||||
-- author-authored wikilinks tend to end sentences with a period and the
|
||||
-- desired URL is almost always the terminal-punctuation-free form.
|
||||
slugify :: String -> String
|
||||
slugify = intercalate "-" . words . map toLowerAlnum
|
||||
where
|
||||
toLowerAlnum c
|
||||
| isAlphaNum c = toLower c
|
||||
| isSpace c = ' '
|
||||
| c == '-' = '-'
|
||||
| otherwise = ' ' -- replace punctuation with a space so words
|
||||
-- split correctly and double-hyphens are
|
||||
-- collapsed by 'words'
|
||||
|
||||
|
|
@ -0,0 +1,23 @@
|
|||
module Main where
|
||||
|
||||
import Data.Time.Clock.POSIX (getPOSIXTime)
|
||||
import System.Directory (createDirectoryIfMissing)
|
||||
import Hakyll (hakyll)
|
||||
import Site (rules)
|
||||
|
||||
-- | Stamp the start of this build into @data/build-stamp.txt@ before
|
||||
-- Hakyll scans the provider directory. The file therefore always exists
|
||||
-- and always differs from the previous run. The telemetry pages
|
||||
-- (@/build/@, @/stats/@) @load@ it as a dependency so Hakyll recompiles
|
||||
-- them on every build instead of serving a stale cached copy when no
|
||||
-- tracked content changed. See build/Stats.hs and build/Site.hs.
|
||||
writeBuildStamp :: IO ()
|
||||
writeBuildStamp = do
|
||||
createDirectoryIfMissing True "data"
|
||||
t <- getPOSIXTime
|
||||
writeFile "data/build-stamp.txt" (show t ++ "\n")
|
||||
|
||||
main :: IO ()
|
||||
main = do
|
||||
writeBuildStamp
|
||||
hakyll rules
|
||||
|
|
@ -0,0 +1,638 @@
|
|||
{-# LANGUAGE GHC2021 #-}
|
||||
{-# LANGUAGE OverloadedStrings #-}
|
||||
-- | Frontmatter marks: the monogram (a hand-authored SVG per piece) and
|
||||
-- the epistemic figure (a build-time SVG generated from frontmatter).
|
||||
-- See MARKS.md for the full specification.
|
||||
--
|
||||
-- Two Hakyll context fields are exported:
|
||||
--
|
||||
-- * @$monogramSvg$@ — the inlined monogram for the current item, or
|
||||
-- 'noResult' when no co-located @mark.svg@ exists.
|
||||
-- * @$epistemicSvg$@ — the generated epistemic figure, or 'noResult'
|
||||
-- when the item has no @status:@ frontmatter
|
||||
-- (MARKS.md §3.1).
|
||||
--
|
||||
-- Both fields are deterministic: byte-identical inputs produce
|
||||
-- byte-identical SVGs, so the GPG signing pipeline is undisturbed.
|
||||
module Marks
|
||||
( monogramSvgField
|
||||
, hasMonogramField
|
||||
, monogramSvgFieldFor
|
||||
, hasMonogramFieldFor
|
||||
, epistemicSvgField
|
||||
, hasMonogram
|
||||
) where
|
||||
|
||||
import Control.Exception (IOException, try)
|
||||
import Data.Char (toLower)
|
||||
import Data.Maybe (catMaybes, isJust)
|
||||
import qualified Data.Text as T
|
||||
import qualified Data.Text.IO as TIO
|
||||
import Numeric (showFFloat)
|
||||
import System.Directory (doesFileExist)
|
||||
import System.FilePath (takeBaseName, takeDirectory,
|
||||
takeFileName, (</>))
|
||||
import System.IO (hPutStrLn, stderr)
|
||||
import Text.Read (readMaybe)
|
||||
|
||||
import Hakyll
|
||||
import Stability (resolveStability)
|
||||
|
||||
-- ---------------------------------------------------------------------------
|
||||
-- Monogram path resolution
|
||||
-- ---------------------------------------------------------------------------
|
||||
|
||||
-- | Candidate monogram paths for a given source path. The build picks the
|
||||
-- first one that exists on disk. This dual-form resolver matches the
|
||||
-- site's mixed flat / directory essay convention:
|
||||
--
|
||||
-- > content/essays/foo.md → content/essays/foo.mark.svg
|
||||
-- > content/essays/foo/index.md → content/essays/foo/mark.svg
|
||||
monogramCandidates :: FilePath -> [FilePath]
|
||||
monogramCandidates fp =
|
||||
let dir = takeDirectory fp
|
||||
fname = takeFileName fp
|
||||
in if fname == "index.md"
|
||||
then [dir </> "mark.svg"]
|
||||
else [dir </> takeBaseName fp ++ ".mark.svg"]
|
||||
|
||||
-- | Predicate form of 'resolveMonogramPath' — used by Stats.hs to
|
||||
-- compute monogram coverage on @/build/@. Returns 'True' when at
|
||||
-- least one of the dual-form candidate paths exists on disk.
|
||||
hasMonogram :: Item a -> Compiler Bool
|
||||
hasMonogram item = isJust <$> resolveMonogramPath item
|
||||
|
||||
-- | @$has-monogram$@ — present (renders as @"true"@) only when the
|
||||
-- item has an actual @mark.svg@ on disk; 'noResult' for the
|
||||
-- placeholder-roundel case. Templates that don't want to display
|
||||
-- placeholder roundels (e.g. item-card listings, popup previews)
|
||||
-- gate on this flag instead of @$monogramSvg$@, which the
|
||||
-- frontmatter header relies on always rendering for symmetric
|
||||
-- column layout.
|
||||
hasMonogramField :: Context String
|
||||
hasMonogramField = field "has-monogram" $ \item -> do
|
||||
has <- hasMonogram item
|
||||
if has then return "true" else noResult "no real monogram"
|
||||
|
||||
-- | Return the first candidate path that exists on disk, or 'Nothing'.
|
||||
resolveMonogramPath :: Item a -> Compiler (Maybe FilePath)
|
||||
resolveMonogramPath item =
|
||||
unsafeCompiler $ firstExisting (monogramCandidates fp)
|
||||
where
|
||||
fp = toFilePath (itemIdentifier item)
|
||||
firstExisting [] = return Nothing
|
||||
firstExisting (p:ps) = do
|
||||
e <- doesFileExist p
|
||||
if e then return (Just p) else firstExisting ps
|
||||
|
||||
-- ---------------------------------------------------------------------------
|
||||
-- Monogram inlining
|
||||
-- ---------------------------------------------------------------------------
|
||||
|
||||
-- | @$monogramSvg$@. Reads the resolved @mark.svg@, normalizes black
|
||||
-- fills/strokes to @currentColor@ (defensive — authors using AI-assist
|
||||
-- tools may produce hardcoded blacks; the contract still holds), strips
|
||||
-- the @width@/@height@ presentation attributes from the root @<svg>@,
|
||||
-- and wraps the result in @<figure class="frontmatter-mark
|
||||
-- frontmatter-mark--monogram">@.
|
||||
--
|
||||
-- When no @mark.svg@ exists, returns the placeholder roundel — an
|
||||
-- empty outer ring at lower opacity that visually balances the
|
||||
-- epistemic-figure column and signals "monogram not yet authored".
|
||||
-- Read failures fall back to the same placeholder.
|
||||
monogramSvgField :: Context String
|
||||
monogramSvgField = field "monogramSvg" $ \item -> do
|
||||
mPath <- resolveMonogramPath item
|
||||
case mPath of
|
||||
Nothing -> return $ T.unpack monogramPlaceholder
|
||||
Just path -> do
|
||||
result <- unsafeCompiler $ try (TIO.readFile path)
|
||||
:: Compiler (Either IOException T.Text)
|
||||
case result of
|
||||
Left e -> do
|
||||
unsafeCompiler $ hPutStrLn stderr $
|
||||
"[Marks] " ++ toFilePath (itemIdentifier item) ++
|
||||
": failed to read " ++ path ++ ": " ++ show e
|
||||
return $ T.unpack monogramPlaceholder
|
||||
Right svg -> return $ T.unpack $ wrapMonogram (processSvg svg)
|
||||
|
||||
-- | Empty-roundel placeholder used while a piece's monogram is still
|
||||
-- to be authored (Phase 2 of MARKS.md). The @--placeholder@ modifier
|
||||
-- class lets CSS render it at reduced opacity so it reads as a
|
||||
-- neutral frame rather than a finished glyph.
|
||||
monogramPlaceholder :: T.Text
|
||||
monogramPlaceholder = T.concat
|
||||
[ "<figure class=\"frontmatter-mark frontmatter-mark--monogram"
|
||||
, " frontmatter-mark--placeholder\" aria-hidden=\"true\">"
|
||||
, "<svg xmlns=\"http://www.w3.org/2000/svg\" viewBox=\"0 0 280 280\">"
|
||||
, "<circle cx=\"140\" cy=\"140\" r=\"128\" fill=\"none\""
|
||||
, " stroke=\"currentColor\" stroke-width=\"0.6\"/>"
|
||||
, "</svg>"
|
||||
, "</figure>"
|
||||
]
|
||||
|
||||
-- | Wrap inlined monogram SVG in its outer figure element.
|
||||
wrapMonogram :: T.Text -> T.Text
|
||||
wrapMonogram svg = T.concat
|
||||
[ "<figure class=\"frontmatter-mark frontmatter-mark--monogram\">"
|
||||
, svg
|
||||
, "</figure>"
|
||||
]
|
||||
|
||||
-- | @$monogramSvg$@ override for synthesized pages whose item identifier
|
||||
-- doesn't live under @content/@ (e.g. @/build/@, @/stats/@), so the
|
||||
-- auto-resolver in 'monogramSvgField' can't find a co-located mark.
|
||||
-- Reads from the supplied path; falls back to the placeholder roundel
|
||||
-- when the file is absent or unreadable.
|
||||
monogramSvgFieldFor :: FilePath -> Context a
|
||||
monogramSvgFieldFor path = field "monogramSvg" $ \_ -> do
|
||||
exists <- unsafeCompiler $ doesFileExist path
|
||||
if not exists
|
||||
then return $ T.unpack monogramPlaceholder
|
||||
else do
|
||||
result <- unsafeCompiler $ try (TIO.readFile path)
|
||||
:: Compiler (Either IOException T.Text)
|
||||
case result of
|
||||
Left e -> do
|
||||
unsafeCompiler $ hPutStrLn stderr $
|
||||
"[Marks] failed to read " ++ path ++ ": " ++ show e
|
||||
return $ T.unpack monogramPlaceholder
|
||||
Right svg -> return $ T.unpack $ wrapMonogram (processSvg svg)
|
||||
|
||||
-- | @$has-monogram$@ override paired with 'monogramSvgFieldFor'. Present
|
||||
-- (as @"true"@) only when the path exists; 'noResult' otherwise.
|
||||
hasMonogramFieldFor :: FilePath -> Context a
|
||||
hasMonogramFieldFor path = field "has-monogram" $ \_ -> do
|
||||
exists <- unsafeCompiler $ doesFileExist path
|
||||
if exists then return "true" else noResult "no real monogram"
|
||||
|
||||
-- | Replace hardcoded black fills/strokes with @currentColor@ and strip
|
||||
-- the root @<svg>@'s @width@/@height@ attributes (presentation lives
|
||||
-- in CSS via the @.frontmatter-mark svg@ selector). Mirrors the color
|
||||
-- substitution in 'Filters.Score.processColors' so the two SVG
|
||||
-- inliners agree on the contract.
|
||||
processSvg :: T.Text -> T.Text
|
||||
processSvg = stripRootDims . normalizeColors
|
||||
|
||||
-- | The same chain 'Filters.Score' applies, kept in sync deliberately.
|
||||
-- 6-digit patterns first so the 3-digit replacement doesn't match
|
||||
-- the prefix of a 6-digit value.
|
||||
normalizeColors :: T.Text -> T.Text
|
||||
normalizeColors
|
||||
= T.replace "fill=\"#000\"" "fill=\"currentColor\""
|
||||
. T.replace "fill=\"black\"" "fill=\"currentColor\""
|
||||
. T.replace "stroke=\"#000\"" "stroke=\"currentColor\""
|
||||
. T.replace "stroke=\"black\"" "stroke=\"currentColor\""
|
||||
. T.replace "fill:#000" "fill:currentColor"
|
||||
. T.replace "fill:black" "fill:currentColor"
|
||||
. T.replace "stroke:#000" "stroke:currentColor"
|
||||
. T.replace "stroke:black" "stroke:currentColor"
|
||||
. T.replace "fill=\"#000000\"" "fill=\"currentColor\""
|
||||
. T.replace "stroke=\"#000000\"" "stroke=\"currentColor\""
|
||||
. T.replace "fill:#000000" "fill:currentColor"
|
||||
. T.replace "stroke:#000000" "stroke:currentColor"
|
||||
|
||||
-- | Remove @width="..."@ and @height="..."@ from the root @<svg>@.
|
||||
-- The substitution is conservative: it walks once and only touches
|
||||
-- the first occurrence of each attribute (the root tag in a
|
||||
-- well-formed monogram).
|
||||
stripRootDims :: T.Text -> T.Text
|
||||
stripRootDims = stripFirst "width" . stripFirst "height"
|
||||
where
|
||||
stripFirst attr txt =
|
||||
case T.breakOn (T.pack (" " ++ attr ++ "=\"")) txt of
|
||||
(before, after)
|
||||
| T.null after -> txt
|
||||
| otherwise ->
|
||||
-- Drop ` attr="..."` including its closing quote.
|
||||
let restAfterEq = T.drop (T.length (T.pack (" " ++ attr ++ "=\""))) after
|
||||
in case T.breakOn "\"" restAfterEq of
|
||||
(_, rest) | T.null rest -> txt
|
||||
| otherwise -> before <> T.drop 1 rest
|
||||
|
||||
-- ---------------------------------------------------------------------------
|
||||
-- Epistemic figure: data extraction
|
||||
-- ---------------------------------------------------------------------------
|
||||
|
||||
-- | Captures the frontmatter inputs the figure consumes. Constructed
|
||||
-- once per item by 'readEpistemicData', then handed to the pure
|
||||
-- geometry below. Keeps the I/O step (metadata + git) separate from
|
||||
-- the SVG-string formatter, so the formatter is testable in isolation
|
||||
-- without mocking Hakyll.
|
||||
data EpistemicData = EpistemicData
|
||||
{ epConfidence :: Maybe Int -- ^ Numeric confidence; 'Nothing' if proved/absent/unparseable.
|
||||
, epConfidenceProved :: Bool -- ^ True when @confidence: proved@ / @proven@.
|
||||
, epImportance :: Maybe Int -- ^ 1–5 ordinal.
|
||||
, epEvidence :: Maybe Int -- ^ 1–5 ordinal.
|
||||
, epScope :: Maybe String -- ^ Validated scope value.
|
||||
, epNovelty :: Maybe String -- ^ Validated novelty value.
|
||||
, epPracticality :: Maybe String -- ^ Validated practicality value.
|
||||
, epPeerStatus :: Maybe String -- ^ Validated peer-status slug ('Nothing' when absent / unreviewed / invalid).
|
||||
, epResultShape :: Maybe String -- ^ Validated result-shape value.
|
||||
, epStability :: String -- ^ Always one of the five stability labels.
|
||||
, epTrust :: Maybe Int -- ^ Trust score 0–100 (60/40 weighted; @proved@ substitutes 100 for confidence). 'Nothing' when confidence or evidence is missing — no label is rendered.
|
||||
}
|
||||
|
||||
-- | Read the figure inputs from a Hakyll item's metadata + git history.
|
||||
readEpistemicData :: Item a -> Compiler EpistemicData
|
||||
readEpistemicData item = do
|
||||
meta <- getMetadata (itemIdentifier item)
|
||||
stab <- resolveStability item
|
||||
let confRaw = lookupString "confidence" meta
|
||||
proved = isProvedConfidenceM confRaw
|
||||
confInt = if proved then Just 100 else readMaybe . trimS =<< confRaw
|
||||
confNumeric = if proved then Nothing else confInt
|
||||
importance = readMaybe . trimS =<< lookupString "importance" meta
|
||||
evidence = readMaybe . trimS =<< lookupString "evidence" meta
|
||||
scope = validate scopeValues =<< lookupString "scope" meta
|
||||
novelty = validate noveltyValues =<< lookupString "novelty" meta
|
||||
practical = validate practicalityValues =<< lookupString "practicality" meta
|
||||
peer = validatePeerStatus =<< lookupString "peer-status" meta
|
||||
resultShape = validate resultShapeValues =<< lookupString "result-shape" meta
|
||||
trust = computeTrust confInt evidence
|
||||
return EpistemicData
|
||||
{ epConfidence = confNumeric
|
||||
, epConfidenceProved = proved
|
||||
, epImportance = importance
|
||||
, epEvidence = evidence
|
||||
, epScope = scope
|
||||
, epNovelty = novelty
|
||||
, epPracticality = practical
|
||||
, epPeerStatus = peer
|
||||
, epResultShape = resultShape
|
||||
, epStability = stab
|
||||
, epTrust = trust
|
||||
}
|
||||
where
|
||||
trimS = trim'
|
||||
|
||||
-- | Trust score: the same 60/40 weighted composite of confidence and
|
||||
-- evidence used by 'Contexts.overallScoreField'. Returns 'Nothing'
|
||||
-- when either input is missing — the figure then renders no trust
|
||||
-- label at all (it collapses to the bare frame), rather than a
|
||||
-- literal "0" indistinguishable from an authored zero score.
|
||||
computeTrust :: Maybe Int -> Maybe Int -> Maybe Int
|
||||
computeTrust (Just c) (Just e) =
|
||||
let raw :: Double
|
||||
raw = fromIntegral c / 100.0 * 0.6 + fromIntegral (e - 1) / 4.0 * 0.4
|
||||
in Just (max 0 (min 100 (round (raw * 100.0))))
|
||||
computeTrust _ _ = Nothing
|
||||
|
||||
-- | Same predicate as 'Contexts.isProvedConfidence' — local copy to keep
|
||||
-- the module's dependency graph light (Marks → Stability only). The
|
||||
-- two are tested against the same vocabulary; if either drifts the
|
||||
-- build still warns via the schema validators in Contexts.hs.
|
||||
isProvedConfidenceM :: Maybe String -> Bool
|
||||
isProvedConfidenceM (Just s) = map toLower (trim' s) `elem` ["proved", "proven"]
|
||||
isProvedConfidenceM _ = False
|
||||
|
||||
trim' :: String -> String
|
||||
trim' = f . f
|
||||
where f = reverse . dropWhile (`elem` (" \t\n\r" :: String))
|
||||
|
||||
-- | Validate a value against an enum list. Returns the lowercase form
|
||||
-- on hit, 'Nothing' otherwise (no warning here — Contexts.hs's parsers
|
||||
-- already warn on invalid frontmatter; the figure simply degrades).
|
||||
validate :: [String] -> String -> Maybe String
|
||||
validate vs raw =
|
||||
let s = map toLower (trim' raw)
|
||||
in if s `elem` vs then Just s else Nothing
|
||||
|
||||
-- | Peer-status validator: matches @peerStatusField@ in Contexts.hs but
|
||||
-- maps @unreviewed@ to 'Nothing' so the figure's outer ring stays
|
||||
-- neutral by default.
|
||||
validatePeerStatus :: String -> Maybe String
|
||||
validatePeerStatus raw =
|
||||
let s = map toLower (trim' raw)
|
||||
in if s `elem` ["under-review", "peer-reviewed", "published", "retracted"]
|
||||
then Just s
|
||||
else Nothing -- includes "unreviewed" and any unknown value
|
||||
|
||||
scopeValues, noveltyValues, practicalityValues, resultShapeValues :: [String]
|
||||
scopeValues = ["personal", "local", "average", "broad", "civilizational"]
|
||||
noveltyValues = ["conventional", "moderate", "idiosyncratic", "innovative"]
|
||||
practicalityValues = ["abstract", "low", "moderate", "high", "exceptional"]
|
||||
resultShapeValues = ["positive", "negative", "mixed", "comparative", "descriptive"]
|
||||
|
||||
-- | Map an ordinal value to its numeric rank (1-based).
|
||||
ordinalRank :: [String] -> String -> Maybe Int
|
||||
ordinalRank vs s = lookup s (zip vs [1..])
|
||||
|
||||
-- ---------------------------------------------------------------------------
|
||||
-- Epistemic figure: geometry
|
||||
-- ---------------------------------------------------------------------------
|
||||
|
||||
-- | Centre of the figure (viewBox coordinates).
|
||||
fxCenter, fyCenter :: Double
|
||||
fxCenter = 100
|
||||
fyCenter = 100
|
||||
|
||||
-- | Inner / outer radii of the roundel and the polygon's full extent.
|
||||
fxOuter, fxOuterPlus, fxAxisFull :: Double
|
||||
fxOuter = 88 -- inner roundel circle
|
||||
fxOuterPlus = 90 -- outer roundel circle
|
||||
fxAxisFull = 80 -- full axis length (polygon vertex when value = 1.0)
|
||||
|
||||
-- | Six axis angles, clockwise from 12 o'clock, in degrees.
|
||||
-- Index → field: 0 confidence, 1 novelty, 2 practicality,
|
||||
-- 3 scope, 4 evidence, 5 importance.
|
||||
axisAngles :: [Double]
|
||||
axisAngles = [0, 60, 120, 180, 240, 300]
|
||||
|
||||
-- | Convert a (clockwise-angle-from-12-o'clock, distance-from-centre) pair
|
||||
-- to absolute viewBox coordinates.
|
||||
polar :: Double -> Double -> (Double, Double)
|
||||
polar angleDeg dist =
|
||||
let theta = (angleDeg - 90) * pi / 180
|
||||
in (fxCenter + dist * cos theta, fyCenter + dist * sin theta)
|
||||
|
||||
-- | Axis index → normalized [0,1] value, or 'Nothing' when the
|
||||
-- underlying frontmatter field is absent / unparseable.
|
||||
axisValue :: EpistemicData -> Int -> Maybe Double
|
||||
axisValue d i = case i of
|
||||
0 -> if epConfidenceProved d
|
||||
then Just 1.0
|
||||
else fmap (\c -> fromIntegral c / 100.0) (epConfidence d)
|
||||
1 -> normalizeOrdinal noveltyValues 4 (epNovelty d)
|
||||
2 -> normalizeOrdinal practicalityValues 5 (epPracticality d)
|
||||
3 -> normalizeOrdinal scopeValues 5 (epScope d)
|
||||
4 -> normalizeIntScale 5 (epEvidence d)
|
||||
5 -> normalizeIntScale 5 (epImportance d)
|
||||
_ -> Nothing
|
||||
|
||||
-- | Map a 1..n ordinal-name value to a [0,1] value via @(rank-1)/(n-1)@.
|
||||
normalizeOrdinal :: [String] -> Int -> Maybe String -> Maybe Double
|
||||
normalizeOrdinal vs n (Just s) = do
|
||||
r <- ordinalRank vs s
|
||||
return $ fromIntegral (r - 1) / fromIntegral (n - 1)
|
||||
normalizeOrdinal _ _ Nothing = Nothing
|
||||
|
||||
-- | Map a 1..n integer to [0,1] via @(v-1)/(n-1)@.
|
||||
normalizeIntScale :: Int -> Maybe Int -> Maybe Double
|
||||
normalizeIntScale n (Just v) = Just $ fromIntegral (v - 1) / fromIntegral (n - 1)
|
||||
normalizeIntScale _ Nothing = Nothing
|
||||
|
||||
-- ---------------------------------------------------------------------------
|
||||
-- Epistemic figure: SVG rendering
|
||||
-- ---------------------------------------------------------------------------
|
||||
|
||||
-- | Format a Double with two decimal places. Determinism (§8.1) requires
|
||||
-- no platform-dependent floating-point formatting.
|
||||
ff :: Double -> T.Text
|
||||
ff x = T.pack (showFFloat (Just 2) x "")
|
||||
|
||||
-- | Format a "x,y" coordinate pair.
|
||||
xy :: Double -> Double -> T.Text
|
||||
xy x y = ff x <> T.singleton ',' <> ff y
|
||||
|
||||
-- | Render the full epistemic figure SVG.
|
||||
renderEpistemicFigure :: EpistemicData -> T.Text
|
||||
renderEpistemicFigure d = T.concat
|
||||
[ "<svg xmlns=\"http://www.w3.org/2000/svg\""
|
||||
, " viewBox=\"0 0 200 200\""
|
||||
, " role=\"img\""
|
||||
, " aria-label=\"Epistemic figure: "
|
||||
, maybe "" (\t -> "trust " <> T.pack (show t) <> ", ") (epTrust d)
|
||||
, "stability ", T.pack (epStability d), "\">"
|
||||
, renderRoundel
|
||||
, renderGuides
|
||||
, renderAxes
|
||||
, renderPolygon d
|
||||
, renderVertexMarks d
|
||||
, renderTicks (epStability d) (epPeerStatus d)
|
||||
, maybe "" renderTrustLabel (epTrust d)
|
||||
, renderResultShape (epResultShape d) (epTrust d)
|
||||
, "</svg>"
|
||||
]
|
||||
|
||||
-- | Two thin concentric circles forming the outer roundel.
|
||||
renderRoundel :: T.Text
|
||||
renderRoundel = T.concat
|
||||
[ "<circle cx=\"", ff fxCenter, "\" cy=\"", ff fyCenter
|
||||
, "\" r=\"", ff fxOuter
|
||||
, "\" fill=\"none\" stroke=\"currentColor\" stroke-width=\"0.5\" opacity=\"0.7\"/>"
|
||||
, "<circle cx=\"", ff fxCenter, "\" cy=\"", ff fyCenter
|
||||
, "\" r=\"", ff fxOuterPlus
|
||||
, "\" fill=\"none\" stroke=\"currentColor\" stroke-width=\"0.5\" opacity=\"0.7\"/>"
|
||||
]
|
||||
|
||||
-- | Four concentric guide circles at 0.2 R, 0.4 R, 0.6 R, 0.8 R.
|
||||
renderGuides :: T.Text
|
||||
renderGuides = T.concat $ map oneGuide [0.2, 0.4, 0.6, 0.8 :: Double]
|
||||
where
|
||||
oneGuide t = T.concat
|
||||
[ "<circle cx=\"", ff fxCenter, "\" cy=\"", ff fyCenter
|
||||
, "\" r=\"", ff (fxAxisFull * t)
|
||||
, "\" fill=\"none\" stroke=\"currentColor\""
|
||||
, " stroke-width=\"0.25\" opacity=\"0.4\"/>"
|
||||
]
|
||||
|
||||
-- | Six radial axes from centre to the inner roundel.
|
||||
renderAxes :: T.Text
|
||||
renderAxes = T.concat $ map oneAxis axisAngles
|
||||
where
|
||||
oneAxis a =
|
||||
let (x, y) = polar a fxAxisFull
|
||||
in T.concat
|
||||
[ "<line x1=\"", ff fxCenter, "\" y1=\"", ff fyCenter
|
||||
, "\" x2=\"", ff x, "\" y2=\"", ff y
|
||||
, "\" stroke=\"currentColor\" stroke-width=\"0.3\" opacity=\"0.55\"/>"
|
||||
]
|
||||
|
||||
-- | Polygon connecting the present field values along their axes.
|
||||
-- When all six axes have a value the polygon is closed; otherwise
|
||||
-- it's an open polyline through the present vertices in axis order.
|
||||
renderPolygon :: EpistemicData -> T.Text
|
||||
renderPolygon d =
|
||||
let pairs = [ (i, axisValue d i) | i <- [0..5] ]
|
||||
verts = [ polar a (fxAxisFull * v)
|
||||
| (i, Just v) <- pairs
|
||||
, let a = axisAngles !! i ]
|
||||
in case verts of
|
||||
[] -> ""
|
||||
_ ->
|
||||
let pointsTxt = T.intercalate " " [ xy x y | (x, y) <- verts ]
|
||||
allPresent = all (isJust . snd) pairs
|
||||
tag = if allPresent then "polygon" else "polyline"
|
||||
fillAttr = if allPresent
|
||||
then " fill=\"currentColor\" fill-opacity=\"0.08\""
|
||||
else " fill=\"none\""
|
||||
in T.concat
|
||||
[ "<", tag
|
||||
, " points=\"", pointsTxt
|
||||
, "\" stroke=\"currentColor\" stroke-width=\"1.1\""
|
||||
, fillAttr
|
||||
, " stroke-linejoin=\"round\" stroke-linecap=\"round\"/>"
|
||||
]
|
||||
|
||||
-- | One vertex point per present axis. Confidence axis gets a 3×3 square
|
||||
-- instead of a 2-px circle when @confidence: proved@ is in effect — the
|
||||
-- "proof cap" marker (MARKS.md §4.3).
|
||||
renderVertexMarks :: EpistemicData -> T.Text
|
||||
renderVertexMarks d = T.concat $ catMaybes
|
||||
[ vertexMark d i | i <- [0..5] ]
|
||||
|
||||
vertexMark :: EpistemicData -> Int -> Maybe T.Text
|
||||
vertexMark d i = do
|
||||
v <- axisValue d i
|
||||
let (x, y) = polar (axisAngles !! i) (fxAxisFull * v)
|
||||
squareCap = i == 0 && epConfidenceProved d
|
||||
return $ if squareCap
|
||||
then T.concat
|
||||
[ "<rect x=\"", ff (x - 1.5), "\" y=\"", ff (y - 1.5)
|
||||
, "\" width=\"3\" height=\"3\""
|
||||
, " fill=\"currentColor\" stroke=\"none\"/>"
|
||||
]
|
||||
else T.concat
|
||||
[ "<circle cx=\"", ff x, "\" cy=\"", ff y
|
||||
, "\" r=\"2\" fill=\"currentColor\" stroke=\"none\"/>"
|
||||
]
|
||||
|
||||
-- | Outer-ring stability ticks at the top of the figure. Always five
|
||||
-- positions; inactive ticks render at opacity 0.4 so the full scale
|
||||
-- stays visible. Peer-status modulates tick *style*; see
|
||||
-- 'renderPeerStatusOverlay'.
|
||||
renderTicks :: String -> Maybe String -> T.Text
|
||||
renderTicks stability peerStatus =
|
||||
let activeCount = case stability of
|
||||
"volatile" -> 1
|
||||
"revising" -> 2
|
||||
"fairly stable" -> 3
|
||||
"stable" -> 4
|
||||
"established" -> 5
|
||||
_ -> 1
|
||||
tickAngles :: [Double]
|
||||
tickAngles = [0, -15, 15, -30, 30]
|
||||
tickOne :: Int -> Double -> T.Text
|
||||
tickOne idx a =
|
||||
let (x1, y1) = polar a fxOuterPlus
|
||||
(x2, y2) = polar a (fxOuterPlus + 1.5)
|
||||
op = if idx < activeCount then "1.0" else "0.4"
|
||||
in T.concat
|
||||
[ "<line x1=\"", ff x1, "\" y1=\"", ff y1
|
||||
, "\" x2=\"", ff x2, "\" y2=\"", ff y2
|
||||
, "\" stroke=\"currentColor\" stroke-width=\"1\""
|
||||
, " stroke-linecap=\"round\" opacity=\"", op, "\"/>"
|
||||
]
|
||||
in T.concat (zipWith tickOne [0..] tickAngles)
|
||||
<> renderPeerStatusOverlay peerStatus
|
||||
|
||||
-- | Per-peer-status decorations layered on top of the tick group.
|
||||
-- Geometry per MARKS.md §4.1.
|
||||
renderPeerStatusOverlay :: Maybe String -> T.Text
|
||||
renderPeerStatusOverlay Nothing = ""
|
||||
renderPeerStatusOverlay (Just "under-review") =
|
||||
-- Small unfilled circle just outside the outermost tick, at the top.
|
||||
let (x, y) = polar 0 (fxOuterPlus + 3.5)
|
||||
in T.concat
|
||||
[ "<circle cx=\"", ff x, "\" cy=\"", ff y
|
||||
, "\" r=\"1\" fill=\"none\" stroke=\"currentColor\""
|
||||
, " stroke-width=\"0.6\"/>"
|
||||
]
|
||||
renderPeerStatusOverlay (Just "peer-reviewed") =
|
||||
-- Single horizontal bar above the outer roundel arc, centred on top.
|
||||
T.concat
|
||||
[ "<line x1=\"", ff (fxCenter - 6), "\" y1=\"", ff (fyCenter - fxOuterPlus - 3)
|
||||
, "\" x2=\"", ff (fxCenter + 6), "\" y2=\"", ff (fyCenter - fxOuterPlus - 3)
|
||||
, "\" stroke=\"currentColor\" stroke-width=\"0.7\""
|
||||
, " stroke-linecap=\"round\"/>"
|
||||
]
|
||||
renderPeerStatusOverlay (Just "published") =
|
||||
-- Printer's-bracket: short vertical marks at ±15° on the outer roundel.
|
||||
let (lx1, ly1) = polar (-15) (fxOuterPlus + 1)
|
||||
(lx2, ly2) = polar (-15) (fxOuterPlus + 4)
|
||||
(rx1, ry1) = polar 15 (fxOuterPlus + 1)
|
||||
(rx2, ry2) = polar 15 (fxOuterPlus + 4)
|
||||
in T.concat
|
||||
[ "<line x1=\"", ff lx1, "\" y1=\"", ff ly1
|
||||
, "\" x2=\"", ff lx2, "\" y2=\"", ff ly2
|
||||
, "\" stroke=\"currentColor\" stroke-width=\"0.8\""
|
||||
, " stroke-linecap=\"round\"/>"
|
||||
, "<line x1=\"", ff rx1, "\" y1=\"", ff ry1
|
||||
, "\" x2=\"", ff rx2, "\" y2=\"", ff ry2
|
||||
, "\" stroke=\"currentColor\" stroke-width=\"0.8\""
|
||||
, " stroke-linecap=\"round\"/>"
|
||||
]
|
||||
renderPeerStatusOverlay (Just "retracted") =
|
||||
-- Horizontal strikethrough across the tick group.
|
||||
T.concat
|
||||
[ "<line x1=\"", ff (fxCenter - 9), "\" y1=\"", ff (fyCenter - fxOuterPlus - 1)
|
||||
, "\" x2=\"", ff (fxCenter + 9), "\" y2=\"", ff (fyCenter - fxOuterPlus - 1)
|
||||
, "\" stroke=\"currentColor\" stroke-width=\"1.5\""
|
||||
, " stroke-linecap=\"round\"/>"
|
||||
]
|
||||
renderPeerStatusOverlay (Just _) = ""
|
||||
|
||||
-- | Trust score (Spectral, 16 px) and the small "TRUST" label below it.
|
||||
renderTrustLabel :: Int -> T.Text
|
||||
renderTrustLabel score = T.concat
|
||||
[ "<text x=\"", ff fxCenter, "\" y=\"", ff (fyCenter + 4)
|
||||
, "\" text-anchor=\"middle\""
|
||||
, " fill=\"currentColor\" stroke=\"none\""
|
||||
, " font-family=\"Spectral, serif\" font-weight=\"500\" font-size=\"16\">"
|
||||
, T.pack (show score)
|
||||
, "</text>"
|
||||
, "<text x=\"", ff fxCenter, "\" y=\"", ff (fyCenter + 14)
|
||||
, "\" text-anchor=\"middle\""
|
||||
, " fill=\"currentColor\" stroke=\"none\""
|
||||
, " font-family=\""Fira Sans", sans-serif\""
|
||||
, " font-size=\"5\" letter-spacing=\"0.18em\""
|
||||
, " opacity=\"0.7\">TRUST</text>"
|
||||
]
|
||||
|
||||
-- | Result-shape glyph immediately to the right of the trust score —
|
||||
-- or centred in its place when no trust score is rendered.
|
||||
renderResultShape :: Maybe String -> Maybe Int -> T.Text
|
||||
renderResultShape Nothing _ = ""
|
||||
renderResultShape (Just shape) mScore =
|
||||
let glyph = case shape of
|
||||
"positive" -> "+"
|
||||
"negative" -> "\x2212" -- minus sign (not hyphen-minus)
|
||||
"mixed" -> "\x00B1" -- ±
|
||||
"comparative" -> "\x223C" -- ∼
|
||||
"descriptive" -> "\x25A1" -- □
|
||||
_ -> ""
|
||||
-- Offset proportional to the trust number's width (digits ≈ 8 px
|
||||
-- each); with no trust label the glyph takes the centre itself.
|
||||
(x, anchor) = case mScore of
|
||||
Just score ->
|
||||
let digitCount = length (show score)
|
||||
offset = fromIntegral digitCount * 4.5 + 3 :: Double
|
||||
in (fxCenter + offset, "start")
|
||||
Nothing -> (fxCenter, "middle")
|
||||
in if T.null (T.pack glyph)
|
||||
then ""
|
||||
else T.concat
|
||||
[ "<text x=\"", ff x
|
||||
, "\" y=\"", ff (fyCenter + 4)
|
||||
, "\" text-anchor=\"", anchor, "\""
|
||||
, " fill=\"currentColor\" stroke=\"none\""
|
||||
, " font-family=\"Spectral, serif\" font-size=\"16\">"
|
||||
, T.pack glyph
|
||||
, "</text>"
|
||||
]
|
||||
|
||||
-- ---------------------------------------------------------------------------
|
||||
-- Field exports
|
||||
-- ---------------------------------------------------------------------------
|
||||
|
||||
-- | @$epistemicSvg$@. Returns 'noResult' when @status:@ is absent
|
||||
-- (matches the existing visibility rule for the epistemic block —
|
||||
-- MARKS.md §3.1). Otherwise returns the inline SVG string ready for
|
||||
-- template interpolation.
|
||||
epistemicSvgField :: Context String
|
||||
epistemicSvgField = field "epistemicSvg" $ \item -> do
|
||||
meta <- getMetadata (itemIdentifier item)
|
||||
case lookupString "status" meta of
|
||||
Nothing -> noResult "no status; epistemic figure suppressed"
|
||||
Just _ -> do
|
||||
d <- readEpistemicData item
|
||||
return $ T.unpack (wrapEpistemic (renderEpistemicFigure d))
|
||||
|
||||
wrapEpistemic :: T.Text -> T.Text
|
||||
wrapEpistemic svg = T.concat
|
||||
[ "<figure class=\"frontmatter-mark frontmatter-mark--epistemic\">"
|
||||
, svg
|
||||
, "</figure>"
|
||||
]
|
||||
|
||||
|
|
@ -0,0 +1,282 @@
|
|||
{-# LANGUAGE GHC2021 #-}
|
||||
{-# LANGUAGE OverloadedStrings #-}
|
||||
-- | Now page: loads data/now.yaml and renders the active-projects view
|
||||
-- and the recently-shipped archive for /current.html. Page-level
|
||||
-- "Last updated" stamp is exposed as a context field; relative time
|
||||
-- ("4 days ago") is computed at build time from getCurrentTime.
|
||||
module Now
|
||||
( nowCtx
|
||||
) where
|
||||
|
||||
import Data.Aeson (FromJSON (..), withObject, (.:), (.:?), (.!=))
|
||||
import Data.Char (toUpper)
|
||||
import Data.List (nub, sortBy)
|
||||
import Data.Maybe (fromMaybe)
|
||||
import Data.Ord (Down (..), comparing)
|
||||
import Data.Time.Calendar (Day, diffDays)
|
||||
import Data.Time.Clock (UTCTime (..), getCurrentTime)
|
||||
import Data.Time.Format (defaultTimeLocale, formatTime, parseTimeM)
|
||||
import qualified Data.Text as T
|
||||
import qualified Data.Text.Encoding as TE
|
||||
import qualified Data.Yaml as Y
|
||||
import Hakyll hiding (escapeHtml)
|
||||
import Contexts (siteCtx)
|
||||
import Utils (escapeHtml)
|
||||
|
||||
-- ---------------------------------------------------------------------------
|
||||
-- Entry types
|
||||
-- ---------------------------------------------------------------------------
|
||||
|
||||
data NowEntry = NowEntry
|
||||
{ neTitle :: String
|
||||
, neSection :: String
|
||||
, neStatus :: String
|
||||
, neUpdated :: String
|
||||
, neLink :: Maybe String
|
||||
, neNote :: Maybe String
|
||||
, nePriority :: Int
|
||||
}
|
||||
|
||||
instance FromJSON NowEntry where
|
||||
parseJSON = withObject "NowEntry" $ \o -> NowEntry
|
||||
<$> o .: "title"
|
||||
<*> o .: "section"
|
||||
<*> o .: "status"
|
||||
<*> o .: "updated"
|
||||
<*> o .:? "link"
|
||||
<*> o .:? "note"
|
||||
<*> o .:? "priority" .!= 0
|
||||
|
||||
data NowShipped = NowShipped
|
||||
{ nsTitle :: String
|
||||
, nsCompleted :: String
|
||||
, nsLink :: Maybe String
|
||||
, nsNote :: Maybe String
|
||||
}
|
||||
|
||||
instance FromJSON NowShipped where
|
||||
parseJSON = withObject "NowShipped" $ \o -> NowShipped
|
||||
<$> o .: "title"
|
||||
<*> o .: "completed"
|
||||
<*> o .:? "link"
|
||||
<*> o .:? "note"
|
||||
|
||||
data NowDoc = NowDoc
|
||||
{ nLastUpdated :: String
|
||||
, nEntries :: [NowEntry]
|
||||
, nShipped :: [NowShipped]
|
||||
}
|
||||
|
||||
instance FromJSON NowDoc where
|
||||
parseJSON = withObject "NowDoc" $ \o -> NowDoc
|
||||
<$> o .: "last-updated"
|
||||
<*> o .:? "entries" .!= []
|
||||
<*> o .:? "shipped" .!= []
|
||||
|
||||
-- ---------------------------------------------------------------------------
|
||||
-- Helpers
|
||||
-- ---------------------------------------------------------------------------
|
||||
|
||||
-- | Section ordering follows first-appearance in entries. Reorder the
|
||||
-- YAML to reorder the page; no separate ordering key required.
|
||||
sectionOrder :: [NowEntry] -> [String]
|
||||
sectionOrder = nub . map neSection
|
||||
|
||||
-- | Status ordering — "how close to shipping." Lower rank sorts first.
|
||||
-- Statuses not listed sort below all known ones (rank 99) so a typo
|
||||
-- surfaces visibly at the bottom of its section instead of silently
|
||||
-- ranking next-to-the-top.
|
||||
statusRanks :: [(String, Int)]
|
||||
statusRanks =
|
||||
[ ("accepted", 0)
|
||||
, ("in-review", 1)
|
||||
, ("revising", 2)
|
||||
, ("drafting", 3)
|
||||
, ("building", 4)
|
||||
, ("early-stage", 5)
|
||||
, ("paused", 6)
|
||||
]
|
||||
|
||||
statusRank :: String -> Int
|
||||
statusRank s = fromMaybe 99 (lookup s statusRanks)
|
||||
|
||||
-- | Three-tier sort key for active entries:
|
||||
-- 1. priority — manual override; higher floats up (default 0)
|
||||
-- 2. statusRank — how close to shipping (lower is closer)
|
||||
-- 3. updated — recency tiebreaker within the same rank
|
||||
-- Sectioning is applied to the *unsorted* list so section ordering
|
||||
-- continues to follow YAML source order; sorting happens within each
|
||||
-- section's filtered slice.
|
||||
entrySortKey :: NowEntry -> (Down Int, Int, Down String)
|
||||
entrySortKey e =
|
||||
( Down (nePriority e)
|
||||
, statusRank (neStatus e)
|
||||
, Down (neUpdated e)
|
||||
)
|
||||
|
||||
-- | "early-stage" → "Early Stage", "research" → "Research".
|
||||
titleCaseWords :: String -> String
|
||||
titleCaseWords = unwords . map cap . wordsOnDash
|
||||
where
|
||||
cap [] = []
|
||||
cap (x:xs) = toUpper x : xs
|
||||
wordsOnDash s = case break (== '-') s of
|
||||
(a, []) -> [a]
|
||||
(a, _:rest) -> a : wordsOnDash rest
|
||||
|
||||
-- ---------------------------------------------------------------------------
|
||||
-- HTML rendering
|
||||
-- ---------------------------------------------------------------------------
|
||||
|
||||
renderStatusChip :: String -> String
|
||||
renderStatusChip s = concat
|
||||
[ "<span class=\"now-status now-status--", escapeHtml s, "\">"
|
||||
, escapeHtml (titleCaseWords s)
|
||||
, "</span>"
|
||||
]
|
||||
|
||||
-- | Active-entry card. Reuses the .item-card / .item-card-* classes from
|
||||
-- item-card.css so the Now page picks up the existing typographic
|
||||
-- register; the .now-* classes layer status-chip + spacing on top.
|
||||
renderEntry :: NowEntry -> String
|
||||
renderEntry e = concat
|
||||
[ "<li class=\"item-card now-card\">"
|
||||
, "<span class=\"item-card-kind now-kind\">"
|
||||
, renderStatusChip (neStatus e)
|
||||
, "</span>"
|
||||
, "<div class=\"item-card-main\">"
|
||||
, "<div class=\"item-card-header\">"
|
||||
, renderTitle (neLink e) (neTitle e)
|
||||
, "<time class=\"item-card-date\" datetime=\"", escapeHtml (neUpdated e), "\">"
|
||||
, escapeHtml (neUpdated e)
|
||||
, "</time>"
|
||||
, "</div>"
|
||||
, maybe "" (\n -> "<p class=\"item-card-abstract is-full\">" ++ escapeHtml n ++ "</p>") (neNote e)
|
||||
, "</div>"
|
||||
, "</li>"
|
||||
]
|
||||
|
||||
renderShippedEntry :: NowShipped -> String
|
||||
renderShippedEntry s = concat
|
||||
[ "<li class=\"item-card now-card now-card--shipped\">"
|
||||
, "<span class=\"item-card-kind now-kind\">"
|
||||
, renderStatusChip "shipped"
|
||||
, "</span>"
|
||||
, "<div class=\"item-card-main\">"
|
||||
, "<div class=\"item-card-header\">"
|
||||
, renderTitle (nsLink s) (nsTitle s)
|
||||
, "<time class=\"item-card-date\" datetime=\"", escapeHtml (nsCompleted s), "\">"
|
||||
, escapeHtml (nsCompleted s)
|
||||
, "</time>"
|
||||
, "</div>"
|
||||
, maybe "" (\n -> "<p class=\"item-card-abstract is-full\">" ++ escapeHtml n ++ "</p>") (nsNote s)
|
||||
, "</div>"
|
||||
, "</li>"
|
||||
]
|
||||
|
||||
renderTitle :: Maybe String -> String -> String
|
||||
renderTitle mu title = case mu of
|
||||
Just url -> "<a class=\"item-card-title\" href=\"" ++ escapeHtml url ++ "\">" ++ escapeHtml title ++ "</a>"
|
||||
Nothing -> "<span class=\"item-card-title\">" ++ escapeHtml title ++ "</span>"
|
||||
|
||||
renderSection :: String -> [NowEntry] -> String
|
||||
renderSection sec es = concat
|
||||
[ "<section class=\"now-section library-section\">"
|
||||
, "<h2 class=\"now-section-heading\">"
|
||||
, escapeHtml (titleCaseWords sec)
|
||||
, "</h2>"
|
||||
, "<ul class=\"item-card-list\">"
|
||||
, concatMap renderEntry es
|
||||
, "</ul>"
|
||||
, "</section>"
|
||||
]
|
||||
|
||||
renderEntries :: [NowEntry] -> String
|
||||
renderEntries [] = ""
|
||||
renderEntries entries = concatMap renderOne (sectionOrder entries)
|
||||
where
|
||||
renderOne sec =
|
||||
let inSec = filter ((== sec) . neSection) entries
|
||||
sorted = sortBy (comparing entrySortKey) inSec
|
||||
in renderSection sec sorted
|
||||
|
||||
renderShippedAll :: [NowShipped] -> String
|
||||
renderShippedAll [] = ""
|
||||
renderShippedAll items = concat
|
||||
[ "<section class=\"now-section now-section--shipped library-section\">"
|
||||
, "<h2 class=\"now-section-heading\">Recently Shipped</h2>"
|
||||
, "<ul class=\"item-card-list\">"
|
||||
, concatMap renderShippedEntry sorted
|
||||
, "</ul>"
|
||||
, "</section>"
|
||||
]
|
||||
where
|
||||
sorted = sortBy (comparing (Down . nsCompleted)) items
|
||||
|
||||
-- ---------------------------------------------------------------------------
|
||||
-- Date formatters — runs at build time
|
||||
-- ---------------------------------------------------------------------------
|
||||
|
||||
-- | "2026-04-26" → "26 April 2026". Falls back to the raw ISO string
|
||||
-- if the date is unparseable, so a typo in @last-updated@ surfaces
|
||||
-- in the rendered page rather than blowing up the build.
|
||||
formatWriterly :: String -> String
|
||||
formatWriterly iso =
|
||||
case parseTimeM True defaultTimeLocale "%Y-%m-%d" iso :: Maybe Day of
|
||||
Nothing -> iso
|
||||
Just d -> formatTime defaultTimeLocale "%-d %B %Y" d
|
||||
|
||||
relativeTime :: Day -> String -> String
|
||||
relativeTime today iso =
|
||||
case parseTimeM True defaultTimeLocale "%Y-%m-%d" iso :: Maybe Day of
|
||||
Nothing -> ""
|
||||
Just d -> bucket (diffDays today d)
|
||||
where
|
||||
bucket n
|
||||
| n < 0 = ""
|
||||
| n == 0 = "today"
|
||||
| n == 1 = "yesterday"
|
||||
| n < 7 = show n ++ " days ago"
|
||||
| n < 28 = pluralize (n `div` 7) "week"
|
||||
| n < 365 = pluralize (n `div` 30) "month"
|
||||
| otherwise = pluralize (n `div` 365) "year"
|
||||
pluralize 1 unit = "1 " ++ unit ++ " ago"
|
||||
pluralize k unit = show k ++ " " ++ unit ++ "s ago"
|
||||
|
||||
-- ---------------------------------------------------------------------------
|
||||
-- Load
|
||||
-- ---------------------------------------------------------------------------
|
||||
|
||||
-- | UTF-8 round-trip String → ByteString. Hakyll's @getResourceBody@
|
||||
-- hands us a 'String' (Unicode codepoints); the yaml library wants
|
||||
-- a UTF-8 'ByteString'. 'Data.ByteString.Char8.pack' would truncate
|
||||
-- each 'Char' to 8 bits — fine for ASCII, silent corruption for any
|
||||
-- codepoint above 0x7F (e.g. em-dash 0x2014 → control char 0x14).
|
||||
loadNow :: Compiler NowDoc
|
||||
loadNow = do
|
||||
rawItem <- load (fromFilePath "data/now.yaml") :: Compiler (Item String)
|
||||
case Y.decodeEither' (TE.encodeUtf8 (T.pack (itemBody rawItem))) of
|
||||
Left err -> fail ("now.yaml: " ++ show err)
|
||||
Right doc -> return doc
|
||||
|
||||
-- ---------------------------------------------------------------------------
|
||||
-- Context
|
||||
-- ---------------------------------------------------------------------------
|
||||
|
||||
nowCtx :: Context String
|
||||
nowCtx =
|
||||
constField "now" "true"
|
||||
<> field "now-last-updated" (\_ -> nLastUpdated <$> loadNow)
|
||||
<> field "now-last-updated-display" (\_ -> formatWriterly . nLastUpdated <$> loadNow)
|
||||
<> field "now-last-updated-relative" (\_ -> do
|
||||
doc <- loadNow
|
||||
nowT <- unsafeCompiler getCurrentTime
|
||||
let today = utctDay nowT
|
||||
rel = relativeTime today (nLastUpdated doc)
|
||||
if null rel
|
||||
then noResult "no relative time"
|
||||
else return rel
|
||||
)
|
||||
<> field "now-entries-html" (\_ -> renderEntries . nEntries <$> loadNow)
|
||||
<> field "now-shipped-html" (\_ -> renderShippedAll . nShipped <$> loadNow)
|
||||
<> siteCtx
|
||||
|
|
@ -0,0 +1,55 @@
|
|||
{-# LANGUAGE GHC2021 #-}
|
||||
{-# LANGUAGE OverloadedStrings #-}
|
||||
-- | Pagination helpers.
|
||||
--
|
||||
-- NOTE: This module must not import Contexts or Tags to avoid cycles.
|
||||
-- Callers (Site.hs) pass contexts in as parameters.
|
||||
module Pagination
|
||||
( pageSize
|
||||
, sortAndGroup
|
||||
, sortAndGroupAt
|
||||
, blogPaginateRules
|
||||
) where
|
||||
|
||||
import Hakyll
|
||||
import Patterns (blogPattern)
|
||||
|
||||
|
||||
-- | Items per page across most paginated lists (e.g. the blog).
|
||||
pageSize :: Int
|
||||
pageSize = 20
|
||||
|
||||
-- | Sort identifiers by date (most recent first) and split into pages.
|
||||
sortAndGroup :: (MonadMetadata m, MonadFail m) => [Identifier] -> m [[Identifier]]
|
||||
sortAndGroup = sortAndGroupAt pageSize
|
||||
|
||||
-- | Like 'sortAndGroup' but with a caller-supplied page size. Used by
|
||||
-- listings that want a different density than the blog default.
|
||||
sortAndGroupAt :: (MonadMetadata m, MonadFail m) => Int -> [Identifier] -> m [[Identifier]]
|
||||
sortAndGroupAt n ids = paginateEvery n <$> sortRecentFirst ids
|
||||
|
||||
-- | Page identifier for the blog index.
|
||||
-- Page 1 → blog/index.html
|
||||
-- Page N → blog/page/N/index.html
|
||||
blogPageId :: PageNumber -> Identifier
|
||||
blogPageId 1 = fromFilePath "blog/index.html"
|
||||
blogPageId n = fromFilePath $ "blog/page/" ++ show n ++ "/index.html"
|
||||
|
||||
-- | Build and rule-ify a paginated blog index.
|
||||
-- @itemCtx@: context for individual posts (postCtx).
|
||||
-- @baseCtx@: site-level context (siteCtx).
|
||||
blogPaginateRules :: Context String -> Context String -> Rules ()
|
||||
blogPaginateRules itemCtx baseCtx = do
|
||||
paginate <- buildPaginateWith sortAndGroup (blogPattern .&&. hasNoVersion) blogPageId
|
||||
paginateRules paginate $ \pageNum pat -> do
|
||||
route idRoute
|
||||
compile $ do
|
||||
posts <- recentFirst =<< loadAll (pat .&&. hasNoVersion)
|
||||
let ctx = listField "posts" itemCtx (return posts)
|
||||
<> paginateContext paginate pageNum
|
||||
<> constField "title" "Blog"
|
||||
<> baseCtx
|
||||
makeItem ""
|
||||
>>= loadAndApplyTemplate "templates/blog-index.html" ctx
|
||||
>>= loadAndApplyTemplate "templates/default.html" ctx
|
||||
>>= relativizeUrls
|
||||
|
|
@ -0,0 +1,200 @@
|
|||
{-# LANGUAGE GHC2021 #-}
|
||||
{-# LANGUAGE OverloadedStrings #-}
|
||||
-- | Canonical content-pattern definitions, shared across modules.
|
||||
--
|
||||
-- Several modules need to enumerate "all author-written content" or
|
||||
-- "all essays". Historically each module hard-coded its own slightly
|
||||
-- different list, which produced silent omissions (e.g. directory-form
|
||||
-- essays not appearing on author pages). This module is the single source
|
||||
-- of truth — every place that needs a content pattern should import from
|
||||
-- here, not write its own.
|
||||
module Patterns
|
||||
( -- * Per-section patterns
|
||||
essayPattern
|
||||
, draftEssayPattern
|
||||
, blogPattern
|
||||
, poetryPattern
|
||||
, fictionPattern
|
||||
, musicPattern
|
||||
, photographyPattern
|
||||
, allPhotoEntries
|
||||
, standalonePagesPattern
|
||||
, pageCollectionPattern
|
||||
-- * Aggregated patterns
|
||||
, allWritings -- essays + blog + poetry + fiction
|
||||
, allContent -- everything that backlinks should index
|
||||
, authorIndexable -- everything that should appear on /authors/{slug}/
|
||||
, tagIndexable -- everything that should appear on /<tag>/
|
||||
) where
|
||||
|
||||
import Hakyll
|
||||
|
||||
-- ---------------------------------------------------------------------------
|
||||
-- Per-section
|
||||
-- ---------------------------------------------------------------------------
|
||||
|
||||
-- | All published essays — flat files, directory-based essays, and entries
|
||||
-- inside one-level collection directories.
|
||||
essayPattern :: Pattern
|
||||
essayPattern =
|
||||
"content/essays/*.md"
|
||||
.||. "content/essays/*/*.md"
|
||||
|
||||
-- | In-progress essay drafts. Matches the flat and directory forms under
|
||||
-- @content/drafts/essays/@. Only 'Site.rules' consumes this, gated on
|
||||
-- @SITE_ENV=dev@ — every other module that enumerates content (Authors,
|
||||
-- Tags, Backlinks, Stats, feeds) sees only 'essayPattern', so drafts are
|
||||
-- automatically invisible to listings, tags, authors, backlinks, and stats.
|
||||
draftEssayPattern :: Pattern
|
||||
draftEssayPattern =
|
||||
"content/drafts/essays/*.md"
|
||||
.||. "content/drafts/essays/*/index.md"
|
||||
|
||||
-- | All blog posts: flat posts plus entries inside collection directories.
|
||||
-- Collection index pages are landing pages and compile separately.
|
||||
blogPattern :: Pattern
|
||||
blogPattern =
|
||||
"content/blog/*.md"
|
||||
.||. ("content/blog/*/*.md" .&&. complement "content/blog/*/index.md")
|
||||
|
||||
-- | All poetry: flat poems plus collection poems, excluding collection
|
||||
-- index pages (which are landing pages, not poems).
|
||||
poetryPattern :: Pattern
|
||||
poetryPattern =
|
||||
"content/poetry/*.md"
|
||||
.||. ("content/poetry/*/*.md" .&&. complement "content/poetry/*/index.md")
|
||||
|
||||
-- | All fiction: flat stories plus entries inside collection directories.
|
||||
-- Collection index pages are landing pages and compile separately.
|
||||
fictionPattern :: Pattern
|
||||
fictionPattern =
|
||||
"content/fiction/*.md"
|
||||
.||. ("content/fiction/*/*.md" .&&. complement "content/fiction/*/index.md")
|
||||
|
||||
-- | Music compositions (landing pages live at @content/music/<slug>/index.md@).
|
||||
musicPattern :: Pattern
|
||||
musicPattern = "content/music/*/index.md"
|
||||
|
||||
-- | All photo entries — flat singles plus directory-form entries.
|
||||
--
|
||||
-- Phase 1 supports two shapes:
|
||||
-- * flat: @content/photography/<slug>.md@
|
||||
-- * directory: @content/photography/<slug>/index.md@
|
||||
--
|
||||
-- The section landing page at @content/photography/index.md@ is
|
||||
-- excluded; it routes via 'Site.rules' as the catalog landing
|
||||
-- (analogous to @content/music/index.md@), not as a photo entry.
|
||||
--
|
||||
-- Phase 5 will extend this pattern with collection-photo files
|
||||
-- (@content/photography/<series>/<photo>.md@) when series support
|
||||
-- lands; until then directory-form @index.md@ files are treated as
|
||||
-- single-photo entries (a series is just a directory with siblings).
|
||||
photographyPattern :: Pattern
|
||||
photographyPattern =
|
||||
("content/photography/*.md" .&&. complement "content/photography/index.md")
|
||||
.||. "content/photography/*/index.md"
|
||||
|
||||
-- | Every photographic entry, including children of series. Distinct
|
||||
-- from 'photographyPattern' (which enumerates only top-level entries
|
||||
-- and series landings) for surfaces that should enumerate every
|
||||
-- photograph individually:
|
||||
--
|
||||
-- * @/photography/by-year/<year>/@ — one frame per file
|
||||
-- * @/photography/contact-sheet/@ — every frame in the roll
|
||||
-- * @/photography/map.json@ — one pin per geotagged photo
|
||||
-- * @/photography/feed.xml@ — one entry per shot
|
||||
-- * Tag indexes — siblings have their own tags
|
||||
--
|
||||
-- The main @/photography/@ landing and the library shelf use
|
||||
-- 'photographyPattern' instead, so a series shows up as a single
|
||||
-- aggregate card rather than once for the landing plus once per child.
|
||||
allPhotoEntries :: Pattern
|
||||
allPhotoEntries =
|
||||
photographyPattern
|
||||
.||. ("content/photography/*/*.md" .&&. complement "content/photography/*/index.md")
|
||||
|
||||
-- | Page collection entries live one directory below @content/@. Known
|
||||
-- section directories are excluded so their files retain specialized
|
||||
-- compilers and routes.
|
||||
pageCollectionPattern :: Pattern
|
||||
pageCollectionPattern =
|
||||
"content/*/*.md" .&&. complement reservedSectionPages
|
||||
where
|
||||
reservedSectionPages =
|
||||
"content/blog/*.md"
|
||||
.||. "content/cv/*.md"
|
||||
.||. "content/drafts/*.md"
|
||||
.||. "content/essays/*.md"
|
||||
.||. "content/fiction/*.md"
|
||||
.||. "content/me/*.md"
|
||||
.||. "content/memento-mori/*.md"
|
||||
.||. "content/music/*.md"
|
||||
.||. "content/photography/*.md"
|
||||
.||. "content/poetry/*.md"
|
||||
.||. "content/scripts/*.md"
|
||||
.||. "content/tag-meta/*.md"
|
||||
|
||||
-- | Top-level standalone pages, curated CV routing pages, and generic page
|
||||
-- collections.
|
||||
standalonePagesPattern :: Pattern
|
||||
standalonePagesPattern =
|
||||
"content/*.md"
|
||||
.||. "content/cv/*.md"
|
||||
.||. pageCollectionPattern
|
||||
|
||||
-- ---------------------------------------------------------------------------
|
||||
-- Aggregations
|
||||
-- ---------------------------------------------------------------------------
|
||||
|
||||
-- | All long-form authored writings.
|
||||
allWritings :: Pattern
|
||||
allWritings = essayPattern .||. blogPattern .||. poetryPattern .||. fictionPattern
|
||||
|
||||
-- | Every content file the backlinks pass should index. Includes music
|
||||
-- landing pages and top-level standalone pages, in addition to writings,
|
||||
-- plus the two directory-form standalone essays (@content/me/index.md@
|
||||
-- and @content/memento-mori/index.md@) — full essays rendered with
|
||||
-- backlinks, whose outgoing links must be visible to the link graph.
|
||||
--
|
||||
-- Photography is deliberately excluded, but note what this pattern
|
||||
-- governs: link *sources*. Photo pages DO render the backlinks block
|
||||
-- now (see 'Contexts.photographyCtx'), and they receive backlinks
|
||||
-- normally — a poem that references a frame surfaces on that frame,
|
||||
-- because inversion keys on the target URL and does not require the
|
||||
-- target to appear here. What they do not do is contribute outbound
|
||||
-- links: a caption-scale entry with an empty body has no prose to
|
||||
-- give the graph, so indexing 240-odd of them would cost a compile
|
||||
-- pass each and return nothing. Revisit if photo bodies start
|
||||
-- carrying real prose.
|
||||
allContent :: Pattern
|
||||
allContent =
|
||||
essayPattern
|
||||
.||. blogPattern
|
||||
.||. poetryPattern
|
||||
.||. fictionPattern
|
||||
.||. musicPattern
|
||||
.||. standalonePagesPattern
|
||||
.||. "content/me/index.md"
|
||||
.||. "content/memento-mori/index.md"
|
||||
|
||||
-- | Content shown on author index pages — essays + blog posts.
|
||||
-- (Poetry and fiction have their own dedicated indexes and are not
|
||||
-- aggregated by author.)
|
||||
authorIndexable :: Pattern
|
||||
authorIndexable = (essayPattern .||. blogPattern) .&&. hasNoVersion
|
||||
|
||||
-- | Content shown on tag index pages — essays + every photographic entry
|
||||
-- (including sibling photos in series). Blog posts are deliberately
|
||||
-- excluded: tags are a topical index for substantial writing, and no
|
||||
-- blog post is ever "about" a topic the way an essay is — including
|
||||
-- them would dilute tag pages. (Blog posts still get related-content
|
||||
-- links via the separate keyword/embedding-based similarity pass.)
|
||||
-- Photography sub-tags (@photography/landscape@, @photography/film@,
|
||||
-- …) generate proper @/<sub-tag>/@ pages from this pattern; the
|
||||
-- bare @photography@ top-level tag is filtered out in
|
||||
-- 'Tags.getExpandedTags' to avoid colliding with the section
|
||||
-- landing's route at @/photography/@.
|
||||
tagIndexable :: Pattern
|
||||
tagIndexable =
|
||||
(essayPattern .||. allPhotoEntries)
|
||||
.&&. hasNoVersion
|
||||
|
|
@ -0,0 +1,775 @@
|
|||
{-# LANGUAGE GHC2021 #-}
|
||||
{-# LANGUAGE OverloadedStrings #-}
|
||||
-- | Photography section — routing and per-page compilation.
|
||||
--
|
||||
-- Phase 1 (current): single-photo entries in flat and directory form,
|
||||
-- plus the @/photography/@ landing page that lists every entry.
|
||||
--
|
||||
-- Phase 5 will extend this module with:
|
||||
-- * collection-photo files (@content/photography/<series>/<photo>.md@)
|
||||
-- * series landing pages
|
||||
-- * @/photography/by-year/@ chronological indexes
|
||||
-- * @/photography/contact-sheet/@ alternate view
|
||||
-- * @/photography/feed.xml@ Atom feed
|
||||
-- * @/photography/map/@ Leaflet map (Phase 4)
|
||||
--
|
||||
-- See @PHOTOGRAPHY.md@ at the repo root for the full design and
|
||||
-- phased implementation plan.
|
||||
module Photography
|
||||
( photographyRules
|
||||
) where
|
||||
|
||||
import Control.Monad (forM, forM_)
|
||||
import Data.List (intercalate, nub, sort, sortBy)
|
||||
import qualified Data.Map.Strict as Map
|
||||
import Data.Map.Strict (Map)
|
||||
import Data.Maybe (mapMaybe, fromMaybe, catMaybes)
|
||||
import qualified Data.Set as Set
|
||||
import Data.Set (Set)
|
||||
import Data.Ord (Down (..), comparing)
|
||||
import System.FilePath (takeBaseName, takeDirectory, takeFileName, replaceExtension, (</>))
|
||||
import System.Directory (doesFileExist, getFileSize)
|
||||
import Text.Printf (printf)
|
||||
import qualified Data.Aeson as Aeson
|
||||
import Data.Aeson (Value (..), (.=))
|
||||
import qualified Data.Aeson.KeyMap as KM
|
||||
import qualified Data.Text.Lazy as TL
|
||||
import qualified Data.Text.Lazy.Encoding as TLE
|
||||
import qualified Data.Vector as V
|
||||
import qualified Data.Scientific as Sci
|
||||
import Hakyll
|
||||
import Compilers (pageCompiler, photographyCompiler)
|
||||
import Contexts (photographyCtx, pageCtx, siteCtx,
|
||||
recentFirstByDisplay)
|
||||
import qualified Patterns as P
|
||||
|
||||
-- ---------------------------------------------------------------------------
|
||||
-- Rules
|
||||
-- ---------------------------------------------------------------------------
|
||||
|
||||
-- | All photography rules. Called from 'Site.rules' once.
|
||||
--
|
||||
-- Order is intentional:
|
||||
--
|
||||
-- 1. Co-located assets first (so the photo file is in @_site/@
|
||||
-- before any page that references it is compiled — Hakyll's
|
||||
-- dependency tracker handles this anyway, but the surface
|
||||
-- ordering reads top-down by data flow).
|
||||
-- 2. Single-photo entries (flat + directory form).
|
||||
-- 3. Section landing at @/photography/@ — loaded after the
|
||||
-- photo entries so its @loadAll photographyPattern@ resolves
|
||||
-- each photo's frontmatter through 'photographyCtx'.
|
||||
photographyRules :: Rules ()
|
||||
photographyRules = do
|
||||
-- A directory is a "series" iff it has @.md@ siblings alongside
|
||||
-- its @index.md@. Collected once at rule-gen time so the entry
|
||||
-- rule can branch on series-landing template selection without
|
||||
-- re-globbing per item.
|
||||
siblingIds <- getMatches
|
||||
( "content/photography/*/*.md"
|
||||
.&&. complement "content/photography/*/index.md"
|
||||
)
|
||||
let seriesSlugs :: Set String
|
||||
seriesSlugs = Set.fromList
|
||||
[ takeFileName (takeDirectory (toFilePath ident))
|
||||
| ident <- siblingIds
|
||||
]
|
||||
|
||||
photographyAssetRules
|
||||
photographyEntryRules seriesSlugs
|
||||
photographySeriesPhotoRules
|
||||
photographyLandingRules
|
||||
photographyMapDataRule
|
||||
photographyMapPageRule
|
||||
photographyFeedRule
|
||||
photographyByYearRules
|
||||
photographyContactSheetRule
|
||||
|
||||
-- ---------------------------------------------------------------------------
|
||||
-- Assets
|
||||
-- ---------------------------------------------------------------------------
|
||||
|
||||
-- | Co-located assets — the photo file itself, and (Phase 3) the
|
||||
-- generated @{photo}.exif.yaml@ + @{photo}.palette.yaml@ sidecars.
|
||||
-- Two patterns are matched in sequence:
|
||||
--
|
||||
-- * @content/photography/<asset>@ — flat-single co-located assets
|
||||
-- * @content/photography/<slug>/<asset>@ — directory-form co-located assets
|
||||
--
|
||||
-- Markdown files are excluded from both rules; they're compiled by
|
||||
-- 'photographyEntryRules' and 'photographyLandingRules'.
|
||||
--
|
||||
-- The @.exif.yaml@ / @.palette.yaml@ sidecars produced by Phase 3
|
||||
-- tooling will be added to the @.gitignore@ defense-in-depth list,
|
||||
-- but copying them through the asset rule is harmless if a stray
|
||||
-- one slips into the repo. The build is not load-bearing on
|
||||
-- sidecar absence.
|
||||
photographyAssetRules :: Rules ()
|
||||
photographyAssetRules = do
|
||||
-- Top-level non-Markdown files (flat-single co-located assets, plus
|
||||
-- any future top-level photography assets like a landing-page hero).
|
||||
--
|
||||
-- Sidecars produced by the Phase 3 Python tooling
|
||||
-- (@{photo}.exif.yaml@, @{photo}.palette.yaml@) are excluded —
|
||||
-- they're consumed by Hakyll at build time and have no role in
|
||||
-- the deployed site.
|
||||
match ("content/photography/*"
|
||||
.&&. complement "content/photography/*.md"
|
||||
.&&. complement "content/photography/*.exif.yaml"
|
||||
.&&. complement "content/photography/*.palette.yaml"
|
||||
.&&. complement "content/photography/*.dims.yaml") $ do
|
||||
route $ gsubRoute "content/" (const "")
|
||||
compile copyFileCompiler
|
||||
|
||||
-- Directory-form entries' co-located assets. Excludes the entry's
|
||||
-- @index.md@, any other Markdown sibling files (collection photos
|
||||
-- in Phase 5), and every build-time YAML sidecar (EXIF, palette,
|
||||
-- dimensions).
|
||||
match ("content/photography/*/*"
|
||||
.&&. complement "content/photography/*/index.md"
|
||||
.&&. complement "content/photography/*/*.md"
|
||||
.&&. complement "content/photography/*/*.exif.yaml"
|
||||
.&&. complement "content/photography/*/*.palette.yaml"
|
||||
.&&. complement "content/photography/*/*.dims.yaml") $ do
|
||||
route $ gsubRoute "content/" (const "")
|
||||
compile copyFileCompiler
|
||||
|
||||
-- ---------------------------------------------------------------------------
|
||||
-- Single-photo entries
|
||||
-- ---------------------------------------------------------------------------
|
||||
|
||||
-- | Compile each single-photo entry. Routing follows the essay
|
||||
-- convention so the URL shape is predictable:
|
||||
--
|
||||
-- * @content/photography/<slug>.md@ → @photography/<slug>.html@
|
||||
-- * @content/photography/<slug>/index.md@ → @photography/<slug>/index.html@
|
||||
--
|
||||
-- The @"content"@ snapshot is saved so a future @/photography/feed.xml@
|
||||
-- (Phase 5) can render the rendered body as feed entry content.
|
||||
photographyEntryRules :: Set String -> Rules ()
|
||||
photographyEntryRules seriesSlugs =
|
||||
match P.photographyPattern $ do
|
||||
route photoEntryRoute
|
||||
compile $ do
|
||||
ident <- getUnderlying
|
||||
let fp = toFilePath ident
|
||||
isIndex = takeFileName fp == "index.md"
|
||||
slug = takeFileName (takeDirectory fp)
|
||||
isSeriesLanding = isIndex && slug `Set.member` seriesSlugs
|
||||
template
|
||||
| isSeriesLanding = "templates/photography-series.html"
|
||||
| otherwise = "templates/photography.html"
|
||||
ctx
|
||||
| isSeriesLanding = seriesCtx
|
||||
| otherwise = photographyCtx
|
||||
photographyCompiler
|
||||
>>= saveSnapshot "content"
|
||||
>>= loadAndApplyTemplate template ctx
|
||||
>>= loadAndApplyTemplate "templates/default.html" ctx
|
||||
>>= relativizeUrls
|
||||
|
||||
-- | Sibling photos inside a series directory:
|
||||
-- @content/photography/<series>/<photo>.md@. Compiled with the
|
||||
-- single-photo template; routed to @<series>/<photo>/index.html@
|
||||
-- so the URL is the canonical directory form (matches the rest of
|
||||
-- the photography section's URL shape).
|
||||
--
|
||||
-- Series landings (@<series>/index.md@) are handled by
|
||||
-- 'photographyEntryRules' with the @photographyPattern@ match;
|
||||
-- they're explicitly excluded here so the two rules don't double-route.
|
||||
photographySeriesPhotoRules :: Rules ()
|
||||
photographySeriesPhotoRules =
|
||||
match ("content/photography/*/*.md"
|
||||
.&&. complement "content/photography/*/index.md") $ do
|
||||
route $ customRoute $ \ident ->
|
||||
-- Drop @"content/"@ prefix and @".md"@ suffix, then append
|
||||
-- @"/index.html"@ to get directory-style URLs.
|
||||
let fp = toFilePath ident
|
||||
rel = drop (length contentPrefix) fp
|
||||
stripped = take (length rel - 3) rel
|
||||
in stripped ++ "/index.html"
|
||||
compile $ photographyCompiler
|
||||
>>= saveSnapshot "content"
|
||||
>>= loadAndApplyTemplate "templates/photography.html" photographyCtx
|
||||
>>= loadAndApplyTemplate "templates/default.html" photographyCtx
|
||||
>>= relativizeUrls
|
||||
where
|
||||
contentPrefix = "content/" :: String
|
||||
|
||||
-- | Context for series-landing pages. Extends 'photographyCtx' with a
|
||||
-- @series-photos@ list field that loads the directory's sibling
|
||||
-- photos (the @<series>/<photo>.md@ files), most-recent-first.
|
||||
--
|
||||
-- The @is-series@ const flag lets the consuming template branch on
|
||||
-- whether to render single-photo chrome (figure + EXIF dl + body)
|
||||
-- or series chrome (intro + photo grid + body).
|
||||
seriesCtx :: Context String
|
||||
seriesCtx =
|
||||
constField "is-series" "true"
|
||||
<> listFieldWith "series-photos" photographyCtx loadSeriesChildren
|
||||
<> seriesStatsCtx
|
||||
<> photographyCtx
|
||||
where
|
||||
loadSeriesChildren parent = do
|
||||
let ident = itemIdentifier parent
|
||||
slug = takeFileName (takeDirectory (toFilePath ident))
|
||||
pat = fromGlob ("content/photography/" ++ slug ++ "/*.md")
|
||||
.&&. complement
|
||||
(fromGlob ("content/photography/" ++ slug ++ "/index.md"))
|
||||
.&&. hasNoVersion
|
||||
recentFirstByDisplay =<< loadAll pat
|
||||
|
||||
|
||||
-- | Statistics for a series landing, in the register of @/build/@ and
|
||||
-- @/stats/@ — the site already treats counting things as worth showing, and
|
||||
-- a body of photographs has plenty to count.
|
||||
--
|
||||
-- Everything is derived at build time from the children's frontmatter and
|
||||
-- the delivery files on disk, so nothing has to be maintained by hand and
|
||||
-- nothing can drift from the set it describes. Fields return 'noResult'
|
||||
-- when a series cannot answer them (no lens recorded, one capture date, an
|
||||
-- image that is gitignored and absent on this machine), so the template's
|
||||
-- @$if(...)$@ guards drop the row rather than printing a blank.
|
||||
seriesStatsCtx :: Context String
|
||||
seriesStatsCtx =
|
||||
field "series-frame-count" (fmap (show . length) . children)
|
||||
<> field "series-bytes" (\i -> do
|
||||
ms <- childMeta i
|
||||
dir <- return (takeDirectory (toFilePath (itemIdentifier i)))
|
||||
sizes <- unsafeCompiler $ mapM (fileSize dir) ms
|
||||
let total = sum sizes
|
||||
if total <= 0 then noResult "no delivery files on disk"
|
||||
else return (humanBytes total))
|
||||
<> field "series-captured-span" capturedSpan
|
||||
<> distinctField "series-cameras" "camera"
|
||||
<> distinctField "series-lenses" "lens"
|
||||
<> field "series-focal-span" (spanOf "focal-length")
|
||||
<> field "series-iso-span" (spanOf "iso")
|
||||
<> field "series-locations" locationList
|
||||
where
|
||||
children i = loadSeriesChildrenFor i
|
||||
|
||||
childMeta i = children i >>= mapM (getMetadata . itemIdentifier)
|
||||
|
||||
fileSize dir meta = case lookupString "photo" meta of
|
||||
Just p | not (null p) -> do
|
||||
let fp = dir </> p
|
||||
ok <- doesFileExist fp
|
||||
if ok then fromIntegral <$> getFileSize fp else return (0 :: Integer)
|
||||
_ -> return 0
|
||||
|
||||
values key i = mapMaybe (lookupString key) <$> childMeta i
|
||||
|
||||
-- Distinct values, in order, comma-joined. One camera reads as a fact
|
||||
-- about the series; five read as a list, which is also a fact about it.
|
||||
distinctField name key = field name $ \i -> do
|
||||
vs <- nub <$> values key i
|
||||
if null vs then noResult (name ++ ": nothing recorded")
|
||||
else return (intercalate ", " vs)
|
||||
|
||||
-- Capture span in prose rather than ISO. Two dates in one month collapse
|
||||
-- to "13–14 August 2026" instead of repeating the month and year; a
|
||||
-- single date prints alone rather than as "13–13 August".
|
||||
capturedSpan i = do
|
||||
vs <- sort . nub <$> values "captured" i
|
||||
case vs of
|
||||
[] -> noResult "no capture dates recorded"
|
||||
[x] -> return (prettyDate x)
|
||||
xs -> return (spanDates (head xs) (last xs))
|
||||
|
||||
spanDates a b =
|
||||
case (splitDate a, splitDate b) of
|
||||
(Just (ya, ma, da), Just (yb, mb, _))
|
||||
| ya == yb && ma == mb ->
|
||||
da ++ "–" ++ dayOf b ++ " " ++ monthName ma ++ " " ++ ya
|
||||
| ya == yb ->
|
||||
da ++ " " ++ monthName ma ++ " – " ++ dayOf b ++ " "
|
||||
++ monthName mb ++ " " ++ ya
|
||||
_ -> prettyDate a ++ " – " ++ prettyDate b
|
||||
|
||||
dayOf d = maybe d (\(_, _, dd) -> dd) (splitDate d)
|
||||
|
||||
prettyDate d = case splitDate d of
|
||||
Just (y, m, dd) -> dd ++ " " ++ monthName m ++ " " ++ y
|
||||
Nothing -> d
|
||||
|
||||
splitDate d = case splitOnDash d of
|
||||
[y, m, dd] -> Just (y, m, dropWhile (== '0') dd)
|
||||
_ -> Nothing
|
||||
|
||||
splitOnDash str = case break (== '-') str of
|
||||
(a, []) -> [a]
|
||||
(a, _ : rest) -> a : splitOnDash rest
|
||||
|
||||
monthName m = case m of
|
||||
"01" -> "January"; "02" -> "February"; "03" -> "March"
|
||||
"04" -> "April"; "05" -> "May"; "06" -> "June"
|
||||
"07" -> "July"; "08" -> "August"; "09" -> "September"
|
||||
"10" -> "October"; "11" -> "November"; "12" -> "December"
|
||||
_ -> m
|
||||
|
||||
-- Locations are stored as "Munich, Germany"; joining the full strings
|
||||
-- with commas produced "Munich, Germany, Nuremberg, Germany", which reads
|
||||
-- as four places. Keep the leading component and separate with a middot.
|
||||
locationList i = do
|
||||
vs <- nub . map (takeWhile (/= ',')) <$> values "location" i
|
||||
if null vs then noResult "no locations recorded"
|
||||
else return (intercalate " · " vs)
|
||||
|
||||
-- Numeric span: strips units, compares as numbers, reprints with the
|
||||
-- unit recovered from the first value. "18mm" and "55mm" must not sort
|
||||
-- as strings, where "18" > "155".
|
||||
spanOf key i = do
|
||||
vs <- nub <$> values key i
|
||||
let parsed = mapMaybe numericPrefix vs
|
||||
unit = case vs of (v:_) -> dropWhile (\c -> c `elem` ("0123456789." :: String)) v
|
||||
[] -> ""
|
||||
case parsed of
|
||||
[] -> noResult "no numeric values"
|
||||
[x] -> return (trimNum x ++ unit)
|
||||
xs -> let lo = minimum xs
|
||||
hi = maximum xs
|
||||
in if lo == hi then return (trimNum lo ++ unit)
|
||||
else return (trimNum lo ++ "–" ++ trimNum hi ++ unit)
|
||||
|
||||
numericPrefix v =
|
||||
let digits = takeWhile (\c -> c `elem` ("0123456789." :: String)) v
|
||||
in if null digits then Nothing else Just (read digits :: Double)
|
||||
|
||||
trimNum x = let r = round x :: Integer
|
||||
in if fromIntegral r == x then show r else printf "%.1f" x
|
||||
|
||||
-- | The same children 'seriesCtx' lists, loadable from a bare item so the
|
||||
-- statistics fields can reach them without duplicating the glob.
|
||||
loadSeriesChildrenFor :: Item a -> Compiler [Item String]
|
||||
loadSeriesChildrenFor parent = do
|
||||
let ident = itemIdentifier parent
|
||||
slug = takeFileName (takeDirectory (toFilePath ident))
|
||||
pat = fromGlob ("content/photography/" ++ slug ++ "/*.md")
|
||||
.&&. complement
|
||||
(fromGlob ("content/photography/" ++ slug ++ "/index.md"))
|
||||
.&&. hasNoVersion
|
||||
loadAll pat
|
||||
|
||||
-- | Bytes as the telemetry pages would print them.
|
||||
humanBytes :: Integer -> String
|
||||
humanBytes n
|
||||
| n >= gb = printf "%.1f GB" (fromIntegral n / fromIntegral gb :: Double)
|
||||
| n >= mb = printf "%.1f MB" (fromIntegral n / fromIntegral mb :: Double)
|
||||
| n >= kb = printf "%.0f KB" (fromIntegral n / fromIntegral kb :: Double)
|
||||
| otherwise = show n ++ " B"
|
||||
where
|
||||
kb = 1024 :: Integer
|
||||
mb = kb * 1024
|
||||
gb = mb * 1024
|
||||
|
||||
-- | Route a photography entry to its public URL. The pattern check on
|
||||
-- @takeFileName@ distinguishes flat (@content/photography/<slug>.md@)
|
||||
-- from directory-form (@content/photography/<slug>/index.md@) without
|
||||
-- re-globbing, since Hakyll has already pre-filtered to entries
|
||||
-- matching 'P.photographyPattern'.
|
||||
--
|
||||
-- Mirrors the essay rule's customRoute (@Site.rules@) but stripped of
|
||||
-- the dev-mode draft branch — drafts are an essay-only concept right
|
||||
-- now.
|
||||
photoEntryRoute :: Routes
|
||||
photoEntryRoute = customRoute $ \ident ->
|
||||
let fp = toFilePath ident
|
||||
fname = takeFileName fp
|
||||
isIndex = fname == "index.md"
|
||||
in if isIndex
|
||||
-- content/photography/<slug>/index.md
|
||||
-- → photography/<slug>/index.html
|
||||
then replaceExtension (drop (length contentPrefix) fp) "html"
|
||||
-- content/photography/<slug>.md → photography/<slug>.html
|
||||
else "photography/" ++ replaceExtension fname "html"
|
||||
where
|
||||
contentPrefix :: String
|
||||
contentPrefix = "content/"
|
||||
|
||||
-- ---------------------------------------------------------------------------
|
||||
-- Landing page
|
||||
-- ---------------------------------------------------------------------------
|
||||
|
||||
-- | Section landing at @/photography/@. Loads all photo entries
|
||||
-- resolved against 'photographyCtx' so each card has access to
|
||||
-- slug / photo-url / captured-display / palette swatches.
|
||||
--
|
||||
-- Sorts by display date (creation date, or most-recent revision
|
||||
-- when the entry has a @revised:@ entry — same ordering authority
|
||||
-- that essay listings use). Phase 2 will replace this listing with
|
||||
-- the masonry/grid/chronological mode toggle, but the underlying
|
||||
-- data feed stays the same — the toggle is a JS layer over the
|
||||
-- already-rendered grid markup.
|
||||
photographyLandingRules :: Rules ()
|
||||
photographyLandingRules =
|
||||
match "content/photography/index.md" $ do
|
||||
route $ constRoute "photography/index.html"
|
||||
compile $ do
|
||||
photos <- recentFirstByDisplay
|
||||
=<< loadAll (P.photographyPattern .&&. hasNoVersion)
|
||||
let ctx =
|
||||
listField "photos" photographyCtx (return photos)
|
||||
<> constField "photography" "true"
|
||||
<> constField "list-page" "true"
|
||||
<> pageCtx
|
||||
pageCompiler
|
||||
>>= loadAndApplyTemplate "templates/photography-index.html" ctx
|
||||
>>= loadAndApplyTemplate "templates/default.html" ctx
|
||||
>>= relativizeUrls
|
||||
|
||||
-- ---------------------------------------------------------------------------
|
||||
-- Map data (Phase 4)
|
||||
-- ---------------------------------------------------------------------------
|
||||
--
|
||||
-- Two artifacts together:
|
||||
--
|
||||
-- * @/photography/map.json@ — JSON array of pin objects, fetched
|
||||
-- by @static/js/photography-map.js@ at view time. Built directly
|
||||
-- from frontmatter; no Python dependency.
|
||||
-- * @/photography/map/@ — the page that renders the Leaflet
|
||||
-- viewport. Lightweight HTML; the heavy lifting lives in the JS.
|
||||
--
|
||||
-- Privacy: every coordinate is rounded to the precision the author
|
||||
-- declares in @geo-precision:@ (default @"city"@) BEFORE it leaves
|
||||
-- this build step. Full-precision coords never reach @map.json@.
|
||||
-- @geo-precision: hidden@ omits the entry entirely.
|
||||
|
||||
-- | Strip a trailing @"index.html"@ component so a Hakyll route
|
||||
-- like @"photography/foo/index.html"@ becomes @"photography/foo/"@.
|
||||
-- Used for map.json click-through URLs.
|
||||
stripIndexHtml :: String -> String
|
||||
stripIndexHtml r
|
||||
| suffixMatches = take (length r - 10) r -- 10 = length "index.html"
|
||||
| otherwise = r
|
||||
where
|
||||
suffix = "/index.html" :: String
|
||||
suffixMatches = suffix == drop (length r - length suffix) r
|
||||
|
||||
-- | Round a decimal coordinate to the precision that matches the
|
||||
-- author's @geo-precision:@ declaration.
|
||||
--
|
||||
-- * @exact@: 4 decimal places (~10 m)
|
||||
-- * @km@ : 2 decimal places (~1 km)
|
||||
-- * @city@ : 1 decimal place (~10 km) — default
|
||||
-- * other : treated as @city@ (defensive only — 'buildPin' validates
|
||||
-- the precision and fails closed before consulting this function)
|
||||
--
|
||||
-- @hidden@ and unrecognised values are handled at the call site by
|
||||
-- skipping the pin entirely; this function is not consulted then.
|
||||
roundCoord :: String -> Double -> Double
|
||||
roundCoord prec x =
|
||||
let n = case prec of
|
||||
"exact" -> 4
|
||||
"km" -> 2
|
||||
"city" -> 1
|
||||
_ -> 1
|
||||
scale = 10 ^^ (n :: Int) :: Double
|
||||
in fromIntegral (round (x * scale) :: Integer) / scale
|
||||
|
||||
-- | Extract @[lat, lon]@ from a frontmatter @geo:@ list. Accepts only
|
||||
-- exactly two numeric entries — anything else returns 'Nothing' so
|
||||
-- the entry is silently skipped on the map.
|
||||
parseGeo :: Aeson.Object -> Maybe (Double, Double)
|
||||
parseGeo meta = case KM.lookup "geo" meta of
|
||||
Just (Array vec) | V.length vec == 2 ->
|
||||
case (asDouble (vec V.! 0), asDouble (vec V.! 1)) of
|
||||
(Just lat, Just lon) -> Just (lat, lon)
|
||||
_ -> Nothing
|
||||
_ -> Nothing
|
||||
where
|
||||
asDouble (Number n) = Just (Sci.toRealFloat n)
|
||||
asDouble _ = Nothing
|
||||
|
||||
-- | Build a single pin object from a photo entry. Returns 'Nothing'
|
||||
-- when:
|
||||
-- * the entry has no @geo:@ frontmatter, or
|
||||
-- * @geo-precision:@ is anything other than @exact@/@km@/@city@ —
|
||||
-- @hidden@ and unrecognised values (typos, wrong case) alike.
|
||||
-- Failing closed means a typo'd \"hidden\" can never publish
|
||||
-- coordinates the author meant to suppress.
|
||||
-- * the entry has no resolvable route (shouldn't happen for
|
||||
-- photographyPattern items, but be defensive).
|
||||
buildPin :: Item String -> Compiler (Maybe Value)
|
||||
buildPin item = do
|
||||
let ident = itemIdentifier item
|
||||
meta <- getMetadata ident
|
||||
mRoute <- getRoute ident
|
||||
case (parseGeo meta, lookupString "geo-precision" meta, mRoute) of
|
||||
(Just (lat, lon), prec, Just r)
|
||||
| maybe True (`elem` ["exact", "km", "city"]) prec ->
|
||||
let prec' = fromMaybe "city" prec
|
||||
rLat = roundCoord prec' lat
|
||||
rLon = roundCoord prec' lon
|
||||
fp = toFilePath ident
|
||||
-- Directory entries (<slug>/index.md) and series children
|
||||
-- (<series>/<photo>.md) both key assets off the parent
|
||||
-- directory; a flat single (content/photography/foo.md)
|
||||
-- has no entry directory, so its slug is its basename and
|
||||
-- its co-located assets route to /photography/ directly.
|
||||
isFlat = takeDirectory fp == "content/photography"
|
||||
&& takeFileName fp /= "index.md"
|
||||
slug = if isFlat then takeBaseName fp
|
||||
else takeFileName (takeDirectory fp)
|
||||
title = fromMaybe slug (lookupString "title" meta)
|
||||
photo = lookupString "photo" meta
|
||||
-- Trim trailing "index.html" so the click-through URL
|
||||
-- is the canonical directory form (no implicit redirect).
|
||||
url = "/" ++ stripIndexHtml r
|
||||
thumb = case photo of
|
||||
Just p | not (null p) ->
|
||||
if isFlat then "/photography/" ++ p
|
||||
else "/photography/" ++ slug ++ "/" ++ p
|
||||
_ -> ""
|
||||
captured = lookupString "captured" meta
|
||||
-- Location and series let the map aggregate: photographs
|
||||
-- rounded to the same city-level coordinate are one place,
|
||||
-- and if they also share a series there is somewhere better
|
||||
-- to send a click than an arbitrary one of them.
|
||||
place = lookupString "location" meta
|
||||
series = lookupString "series" meta
|
||||
in return $ Just $ Aeson.object $
|
||||
[ "slug" .= slug
|
||||
, "title" .= title
|
||||
, "url" .= url
|
||||
, "lat" .= rLat
|
||||
, "lon" .= rLon
|
||||
] ++ (if null thumb then [] else ["thumb" .= thumb])
|
||||
++ maybe [] (\c -> ["captured" .= c]) captured
|
||||
++ maybe [] (\l -> ["location" .= l]) place
|
||||
++ maybe [] (\x -> ["series" .= x]) series
|
||||
_ -> return Nothing
|
||||
|
||||
-- | @/photography/map.json@ — JSON array of geo-tagged photo pins
|
||||
-- for the Leaflet client. Excludes entries with @geo-precision:
|
||||
-- hidden@ and entries with no @geo:@ frontmatter. Walks
|
||||
-- 'allPhotoEntries' so series children with their own GPS land
|
||||
-- on the map alongside top-level photos.
|
||||
photographyMapDataRule :: Rules ()
|
||||
photographyMapDataRule =
|
||||
create ["photography/map.json"] $ do
|
||||
route idRoute
|
||||
compile $ do
|
||||
photos <- loadAll (P.allPhotoEntries .&&. hasNoVersion)
|
||||
:: Compiler [Item String]
|
||||
pins <- mapMaybe id <$> mapM buildPin photos
|
||||
-- LBS.unpack truncates each UTF-8 byte to a Char (Latin-1
|
||||
-- mode), and Hakyll then re-encodes the String to UTF-8 on
|
||||
-- write — producing double-encoded mojibake for any non-
|
||||
-- ASCII title (em-dashes, accents, etc.). Decoding through
|
||||
-- Text gives Hakyll a String of Unicode code points it can
|
||||
-- re-encode cleanly.
|
||||
makeItem $ TL.unpack $ TLE.decodeUtf8 $ Aeson.encode pins
|
||||
|
||||
-- ---------------------------------------------------------------------------
|
||||
-- Map page (Phase 4)
|
||||
-- ---------------------------------------------------------------------------
|
||||
|
||||
-- | @/photography/map/@ — the Leaflet-driven map view. Synthesised
|
||||
-- page; no Markdown source. The @photography-map@ context flag
|
||||
-- gates Leaflet CSS / JS loading in @head.html@ and @default.html@,
|
||||
-- so other photography pages stay lightweight.
|
||||
photographyMapPageRule :: Rules ()
|
||||
photographyMapPageRule =
|
||||
create ["photography/map/index.html"] $ do
|
||||
route idRoute
|
||||
compile $ do
|
||||
let ctx = constField "title" "Map · Photography"
|
||||
<> constField "photography" "true"
|
||||
<> constField "photography-map" "true"
|
||||
<> constField "portal" "true"
|
||||
<> siteCtx
|
||||
makeItem ""
|
||||
>>= loadAndApplyTemplate "templates/photography-map.html" ctx
|
||||
>>= loadAndApplyTemplate "templates/default.html" ctx
|
||||
>>= relativizeUrls
|
||||
|
||||
-- ---------------------------------------------------------------------------
|
||||
-- Atom feed (Phase 5)
|
||||
-- ---------------------------------------------------------------------------
|
||||
|
||||
-- | Configuration for the photography-only Atom feed at
|
||||
-- @/photography/feed.xml@. Distinct from the main @/feed.xml@ so
|
||||
-- text-primary subscribers don't unexpectedly get image-heavy
|
||||
-- entries in their reader.
|
||||
photographyFeedConfig :: FeedConfiguration
|
||||
photographyFeedConfig = FeedConfiguration
|
||||
{ feedTitle = "Levi Neuwirth — Photography"
|
||||
, feedDescription = "New photographs by Levi Neuwirth"
|
||||
, feedAuthorName = "Levi Neuwirth"
|
||||
, feedAuthorEmail = "levi@levineuwirth.org"
|
||||
, feedRoot = "https://levineuwirth.org"
|
||||
}
|
||||
|
||||
-- | Description field for Atom feed entries: prepends an absolute-URL
|
||||
-- @<img>@ tag (so the photograph displays inline in the reader) to
|
||||
-- the rendered prose body. Composed ABOVE 'bodyField' so it wins
|
||||
-- when @$description$@ is consumed by the Atom template.
|
||||
photographyFeedDescription :: Context String
|
||||
photographyFeedDescription = field "description" $ \item -> do
|
||||
let ident = itemIdentifier item
|
||||
body <- itemBody <$> (loadSnapshot ident "content" :: Compiler (Item String))
|
||||
meta <- getMetadata ident
|
||||
let fp = toFilePath ident
|
||||
-- Same asset-path derivation as 'buildPin': directory entries
|
||||
-- (<slug>/index.md) and series children (<series>/<photo>.md)
|
||||
-- both key assets off the parent directory; a flat single
|
||||
-- (content/photography/foo.md) has no entry directory, so its
|
||||
-- co-located assets route to /photography/ directly.
|
||||
isFlat = takeDirectory fp == "content/photography"
|
||||
&& takeFileName fp /= "index.md"
|
||||
slug = takeFileName (takeDirectory fp)
|
||||
imgTag = case lookupString "photo" meta of
|
||||
Just p | not (null p) ->
|
||||
let src = if isFlat then "/photography/" ++ p
|
||||
else "/photography/" ++ slug ++ "/" ++ p
|
||||
in "<p><img src=\"https://levineuwirth.org"
|
||||
++ src ++ "\" alt=\"\"></p>\n"
|
||||
_ -> ""
|
||||
return (imgTag ++ body)
|
||||
|
||||
-- | @/photography/feed.xml@ — Atom feed of the most recent 30 photo
|
||||
-- entries, with each photograph embedded inline at the top of its
|
||||
-- entry description.
|
||||
photographyFeedRule :: Rules ()
|
||||
photographyFeedRule =
|
||||
create ["photography/feed.xml"] $ do
|
||||
route idRoute
|
||||
compile $ do
|
||||
photos <- fmap (take 30) . recentFirst
|
||||
=<< loadAllSnapshots
|
||||
(P.allPhotoEntries .&&. hasNoVersion)
|
||||
"content"
|
||||
let feedCtx =
|
||||
dateField "updated" "%Y-%m-%dT%H:%M:%SZ"
|
||||
<> dateField "published" "%Y-%m-%dT%H:%M:%SZ"
|
||||
<> photographyFeedDescription
|
||||
<> bodyField "description"
|
||||
<> defaultContext
|
||||
renderAtom photographyFeedConfig feedCtx photos
|
||||
|
||||
-- ---------------------------------------------------------------------------
|
||||
-- By-year pages (Phase 5)
|
||||
-- ---------------------------------------------------------------------------
|
||||
--
|
||||
-- @/photography/by-year/@ is the index of years that have photos;
|
||||
-- @/photography/by-year/<year>/@ lists each year's photos
|
||||
-- chronologically. Year is taken from @captured:@ frontmatter
|
||||
-- (when present), falling back to @date:@. Photos with neither
|
||||
-- field — or with a malformed date — are silently dropped from this
|
||||
-- surface; they remain visible on the main grid and any tag pages
|
||||
-- their frontmatter produces.
|
||||
|
||||
-- | Extract a four-digit year from a frontmatter @captured:@ or
|
||||
-- @date:@ field. Returns 'Nothing' when neither is set or both are
|
||||
-- shorter than four characters.
|
||||
yearOfPhoto :: Metadata -> Maybe String
|
||||
yearOfPhoto meta =
|
||||
let firstFour s = if length s >= 4 then Just (take 4 s) else Nothing
|
||||
in case lookupString "captured" meta >>= firstFour of
|
||||
Just yr -> Just yr
|
||||
Nothing -> lookupString "date" meta >>= firstFour
|
||||
|
||||
-- | All by-year rules: collect (year, identifier) pairs once, then
|
||||
-- build the index page and one page per year.
|
||||
photographyByYearRules :: Rules ()
|
||||
photographyByYearRules = do
|
||||
photoIds <- getMatches (P.allPhotoEntries .&&. hasNoVersion)
|
||||
pairs <- forM photoIds $ \ident -> do
|
||||
meta <- getMetadata ident
|
||||
return $ fmap (\yr -> (yr, ident)) (yearOfPhoto meta)
|
||||
let yearMap :: Map String [Identifier]
|
||||
yearMap = Map.fromListWith (++) [(yr, [i]) | (yr, i) <- catMaybes pairs]
|
||||
-- Years sorted descending so the most recent appear first.
|
||||
years = map fst $ sortBy (comparing (Down . fst)) (Map.toList yearMap)
|
||||
|
||||
photographyByYearIndexRule yearMap years
|
||||
forM_ years $ \yr -> photographyByYearPageRule yr (yearMap Map.! yr)
|
||||
|
||||
-- | @/photography/by-year/@ — top-level index. Lists each year that
|
||||
-- has photos with the count, linking to the per-year page.
|
||||
photographyByYearIndexRule :: Map String [Identifier] -> [String] -> Rules ()
|
||||
photographyByYearIndexRule yearMap years =
|
||||
create ["photography/by-year/index.html"] $ do
|
||||
route idRoute
|
||||
compile $ do
|
||||
let yearItems =
|
||||
[ Item (fromFilePath ("year-" ++ yr))
|
||||
(yr, length (Map.findWithDefault [] yr yearMap))
|
||||
| yr <- years
|
||||
]
|
||||
yrCtx =
|
||||
field "year" (return . fst . itemBody)
|
||||
<> field "year-url" (\i -> return $ "/photography/by-year/"
|
||||
++ fst (itemBody i) ++ "/")
|
||||
<> field "year-count"
|
||||
(return . show . snd . itemBody)
|
||||
ctx =
|
||||
listField "years" yrCtx (return yearItems)
|
||||
<> constField "title" "Photography by year"
|
||||
<> constField "photography" "true"
|
||||
<> siteCtx
|
||||
makeItem ""
|
||||
>>= loadAndApplyTemplate
|
||||
"templates/photography-by-year-index.html" ctx
|
||||
>>= loadAndApplyTemplate "templates/default.html" ctx
|
||||
>>= relativizeUrls
|
||||
|
||||
-- | @/photography/by-year/<year>/@ — list of photos captured that year.
|
||||
photographyByYearPageRule :: String -> [Identifier] -> Rules ()
|
||||
photographyByYearPageRule yr idents =
|
||||
create [fromFilePath ("photography/by-year/" ++ yr ++ "/index.html")] $ do
|
||||
route idRoute
|
||||
compile $ do
|
||||
photos <- recentFirstByDisplay
|
||||
=<< mapM (\i -> load i :: Compiler (Item String)) idents
|
||||
let ctx =
|
||||
listField "photos" photographyCtx (return photos)
|
||||
<> constField "title" ("Photography · " ++ yr)
|
||||
<> constField "year" yr
|
||||
<> constField "photography" "true"
|
||||
<> constField "list-page" "true"
|
||||
<> siteCtx
|
||||
makeItem ""
|
||||
>>= loadAndApplyTemplate
|
||||
"templates/photography-by-year.html" ctx
|
||||
>>= loadAndApplyTemplate "templates/default.html" ctx
|
||||
>>= relativizeUrls
|
||||
|
||||
-- ---------------------------------------------------------------------------
|
||||
-- Contact sheet (Phase 5)
|
||||
-- ---------------------------------------------------------------------------
|
||||
|
||||
-- | @/photography/contact-sheet/@ — alternate view of every photo in
|
||||
-- a film-strip aesthetic: thin white-bordered frames, frame numbers
|
||||
-- in the corner, slightly grainy backdrop. Distinct from the main
|
||||
-- grid views; deep cut rather than primary surface.
|
||||
--
|
||||
-- Sort order: chronological by display date (asc). The contact-sheet
|
||||
-- convention reads top-to-bottom in capture order — a roll of film,
|
||||
-- not a recency feed. Each frame's index doubles as its frame
|
||||
-- number. The CSS handles the frame numbering via a CSS counter so
|
||||
-- we don't have to thread the index through the template.
|
||||
photographyContactSheetRule :: Rules ()
|
||||
photographyContactSheetRule =
|
||||
create ["photography/contact-sheet/index.html"] $ do
|
||||
route idRoute
|
||||
compile $ do
|
||||
-- Reverse the recent-first sort to get oldest-first
|
||||
-- (capture chronology), matching the contact-sheet
|
||||
-- convention.
|
||||
photos <- reverse <$> (recentFirstByDisplay
|
||||
=<< loadAll (P.allPhotoEntries .&&. hasNoVersion)
|
||||
:: Compiler [Item String])
|
||||
let ctx =
|
||||
listField "photos" photographyCtx (return photos)
|
||||
<> constField "title" "Contact sheet · Photography"
|
||||
<> constField "photography" "true"
|
||||
<> constField "portal" "true"
|
||||
<> siteCtx
|
||||
makeItem ""
|
||||
>>= loadAndApplyTemplate
|
||||
"templates/photography-contact-sheet.html" ctx
|
||||
>>= loadAndApplyTemplate "templates/default.html" ctx
|
||||
>>= relativizeUrls
|
||||
|
||||
|
|
@ -0,0 +1,195 @@
|
|||
{-# LANGUAGE GHC2021 #-}
|
||||
{-# LANGUAGE OverloadedStrings #-}
|
||||
-- | Similar-links field: injects a "Related" list into essay/page contexts.
|
||||
--
|
||||
-- @data/similar-links.json@ is produced by @tools/embed.py@ at build time
|
||||
-- (called from the Makefile after pagefind, before sign). It is a plain
|
||||
-- JSON object mapping root-relative URL paths to lists of similar pages:
|
||||
--
|
||||
-- { "/essays/my-essay/": [{"url": "...", "title": "...", "score": 0.87}] }
|
||||
--
|
||||
-- This module loads that file with dependency tracking (so pages recompile
|
||||
-- when embeddings change) and provides @similarLinksField@, which resolves
|
||||
-- to an HTML list for the current page's URL.
|
||||
--
|
||||
-- If the file is absent (e.g. @.venv@ not set up, or first build) the field
|
||||
-- returns @noResult@ — the @$if(similar-links)$@ guard in the template is
|
||||
-- false and no "Related" section is rendered.
|
||||
module SimilarLinks (similarLinksField) where
|
||||
|
||||
import Data.Maybe (fromMaybe)
|
||||
import qualified Data.ByteString as BS
|
||||
import qualified Data.Map.Strict as Map
|
||||
import Data.Map.Strict (Map)
|
||||
import qualified Data.Text as T
|
||||
import qualified Data.Text.Encoding as TE
|
||||
import qualified Data.Text.Encoding.Error as TE
|
||||
import qualified Data.Aeson as Aeson
|
||||
import Hakyll
|
||||
|
||||
-- ---------------------------------------------------------------------------
|
||||
-- JSON schema
|
||||
-- ---------------------------------------------------------------------------
|
||||
|
||||
data SimilarEntry = SimilarEntry
|
||||
{ seUrl :: String
|
||||
, seTitle :: String
|
||||
, seScore :: Double
|
||||
} deriving (Show)
|
||||
|
||||
instance Aeson.FromJSON SimilarEntry where
|
||||
parseJSON = Aeson.withObject "SimilarEntry" $ \o ->
|
||||
SimilarEntry
|
||||
<$> o Aeson..: "url"
|
||||
<*> o Aeson..: "title"
|
||||
<*> o Aeson..: "score"
|
||||
|
||||
-- ---------------------------------------------------------------------------
|
||||
-- Context field
|
||||
-- ---------------------------------------------------------------------------
|
||||
|
||||
-- | Maximum entries rendered in the "Related" block. The on-disk JSON may
|
||||
-- contain more (embed.py's TOP_N); 'similarLinksField' caps the list
|
||||
-- (@take maxSimilar@) before rendering.
|
||||
maxSimilar :: Int
|
||||
maxSimilar = 3
|
||||
|
||||
-- | Provides @$similar-links$@ (HTML list) and @$has-similar-links$@
|
||||
-- (boolean flag for template guards).
|
||||
-- Returns @noResult@ when the JSON file is absent, unparseable, or the
|
||||
-- current page has no similar entries.
|
||||
--
|
||||
-- Note on normalisation: 'tools/embed.py' emits map keys using the live
|
||||
-- site URL (e.g. @/essays/foo.html@ for a flat page, @/essays/foo/@ for a
|
||||
-- directory-index page), while Hakyll's route gives @essays/foo.html@.
|
||||
-- 'normaliseUrl' collapses both forms to a canonical stem, and we apply
|
||||
-- it to every JSON key on load so the lookup cannot miss.
|
||||
similarLinksField :: Context String
|
||||
similarLinksField = field "similar-links" $ \item -> do
|
||||
-- Load with dependency tracking — pages recompile when the JSON changes.
|
||||
slItem <- load (fromFilePath "data/similar-links.json") :: Compiler (Item String)
|
||||
case Aeson.decodeStrict (TE.encodeUtf8 (T.pack (itemBody slItem)))
|
||||
:: Maybe (Map T.Text [SimilarEntry]) of
|
||||
Nothing -> fail "similar-links: could not parse data/similar-links.json"
|
||||
Just rawMap -> do
|
||||
mRoute <- getRoute (itemIdentifier item)
|
||||
case mRoute of
|
||||
Nothing -> fail "similar-links: item has no route"
|
||||
Just r ->
|
||||
let normMap = Map.mapKeys (T.pack . normaliseUrl . T.unpack) rawMap
|
||||
key = T.pack (normaliseUrl ("/" ++ r))
|
||||
entries = take maxSimilar (fromMaybe [] (Map.lookup key normMap))
|
||||
in if null entries
|
||||
then fail "no similar links"
|
||||
else return (renderSimilarLinks entries)
|
||||
|
||||
-- ---------------------------------------------------------------------------
|
||||
-- URL normalisation (mirrors embed.py's URL derivation)
|
||||
-- ---------------------------------------------------------------------------
|
||||
|
||||
normaliseUrl :: String -> String
|
||||
normaliseUrl url =
|
||||
let t = T.pack url
|
||||
-- strip query + fragment
|
||||
t1 = fst (T.breakOn "?" (fst (T.breakOn "#" t)))
|
||||
-- ensure leading slash
|
||||
t2 = if T.isPrefixOf "/" t1 then t1 else "/" `T.append` t1
|
||||
-- strip trailing index.html → keep the directory slash
|
||||
t3 = fromMaybe t2 (T.stripSuffix "index.html" t2)
|
||||
-- strip bare .html extension only for non-index pages
|
||||
t4 = fromMaybe t3 (T.stripSuffix ".html" t3)
|
||||
in percentDecode (T.unpack t4)
|
||||
|
||||
-- | Percent-decode @%XX@ escapes (UTF-8) so percent-encoded paths
|
||||
-- collide with their decoded form on map lookup. Mirrors
|
||||
-- 'Backlinks.percentDecode' (and 'Backlinks.normaliseUrl' now applies
|
||||
-- the same strip-@index.html@-then-@.html@ normalisation as this
|
||||
-- module); the duplication keeps the two modules dependency-free of
|
||||
-- each other.
|
||||
percentDecode :: String -> String
|
||||
percentDecode = T.unpack . TE.decodeUtf8With TE.lenientDecode . BS.pack . go
|
||||
where
|
||||
go [] = []
|
||||
go ('%':a:b:rest)
|
||||
| Just hi <- hexDigit a
|
||||
, Just lo <- hexDigit b
|
||||
= fromIntegral (hi * 16 + lo) : go rest
|
||||
go (c:rest) = fromIntegral (fromEnum c) : go rest
|
||||
|
||||
hexDigit c
|
||||
| c >= '0' && c <= '9' = Just (fromEnum c - fromEnum '0')
|
||||
| c >= 'a' && c <= 'f' = Just (fromEnum c - fromEnum 'a' + 10)
|
||||
| c >= 'A' && c <= 'F' = Just (fromEnum c - fromEnum 'A' + 10)
|
||||
| otherwise = Nothing
|
||||
|
||||
-- | Percent-encode a string for use as a URI query value: RFC 3986
|
||||
-- unreserved characters pass through; everything else — including @&@,
|
||||
-- @?@, @#@, spaces, and non-ASCII text via its UTF-8 bytes — becomes
|
||||
-- @%XX@. Hand-rolled (the moral equivalent of network-uri's
|
||||
-- @escapeURIString isUnreserved@) because network-uri is not otherwise
|
||||
-- a dependency. The output is also HTML-attribute-safe: it contains
|
||||
-- only unreserved characters and @%XX@ escapes.
|
||||
percentEncode :: String -> String
|
||||
percentEncode = concatMap enc . BS.unpack . TE.encodeUtf8 . T.pack
|
||||
where
|
||||
enc b
|
||||
| unreserved b = [toEnum (fromIntegral b)]
|
||||
| otherwise = ['%', hexDigit (b `div` 16), hexDigit (b `mod` 16)]
|
||||
unreserved b =
|
||||
let c = toEnum (fromIntegral b) :: Char
|
||||
in (c >= 'A' && c <= 'Z') || (c >= 'a' && c <= 'z')
|
||||
|| (c >= '0' && c <= '9') || c `elem` ("-._~" :: String)
|
||||
hexDigit n = "0123456789ABCDEF" !! fromIntegral n
|
||||
|
||||
-- ---------------------------------------------------------------------------
|
||||
-- HTML rendering
|
||||
-- ---------------------------------------------------------------------------
|
||||
|
||||
-- | Render the Related block. Each anchor gets:
|
||||
-- * @class="similar-link"@ — whitelist for popups.js so the default
|
||||
-- footer-exclusion does not fire (content preview on hover).
|
||||
-- * @data-link-icon@ / @data-link-icon-type@ — page or document icon,
|
||||
-- rendered via the existing a[data-link-icon] mask-image system in
|
||||
-- typography.css.
|
||||
-- * For PDFs: @class="pdf-link"@ + @data-pdf-src@ + href rewritten to
|
||||
-- the PDF.js viewer, matching the rest of the site. The pdfContent
|
||||
-- provider in popups.js binds on @.pdf-link[data-pdf-src]@.
|
||||
renderSimilarLinks :: [SimilarEntry] -> String
|
||||
renderSimilarLinks entries =
|
||||
"<ul class=\"similar-links-list\">\n"
|
||||
++ concatMap renderOne entries
|
||||
++ "</ul>"
|
||||
where
|
||||
renderOne se
|
||||
| isPdfUrl (seUrl se) = renderPdf se
|
||||
| otherwise = renderPage se
|
||||
|
||||
renderPage se =
|
||||
"<li class=\"similar-links-item\">"
|
||||
++ "<a class=\"similar-link\""
|
||||
++ " href=\"" ++ escapeHtml (seUrl se) ++ "\""
|
||||
++ " data-link-icon=\"internal\" data-link-icon-type=\"svg\">"
|
||||
++ escapeHtml (seTitle se)
|
||||
++ "</a></li>\n"
|
||||
|
||||
renderPdf se =
|
||||
-- The PDF path becomes the @file=@ query value, so it must be
|
||||
-- percent-encoded (HTML escaping alone leaves @&@/@?@/@#@/spaces
|
||||
-- free to break the query). A @#page=N@ fragment stays a fragment
|
||||
-- of the viewer URL itself — PDF.js reads it from location.hash.
|
||||
let raw = seUrl se
|
||||
(path, frag) = break (== '#') raw
|
||||
viewerUrl = "/pdfjs/web/viewer.html?file="
|
||||
++ percentEncode path ++ escapeHtml frag
|
||||
in "<li class=\"similar-links-item\">"
|
||||
++ "<a class=\"similar-link pdf-link\""
|
||||
++ " href=\"" ++ viewerUrl ++ "\""
|
||||
++ " data-pdf-src=\"" ++ escapeHtml raw ++ "\""
|
||||
++ " data-link-icon=\"document\" data-link-icon-type=\"svg\">"
|
||||
++ escapeHtml (seTitle se)
|
||||
++ "</a></li>\n"
|
||||
|
||||
isPdfUrl u =
|
||||
let lower = T.toLower (T.pack u)
|
||||
(path, _) = T.break (== '#') lower
|
||||
in ".pdf" `T.isSuffixOf` path
|
||||
File diff suppressed because it is too large
Load Diff
|
|
@ -0,0 +1,361 @@
|
|||
{-# LANGUAGE GHC2021 #-}
|
||||
{-# LANGUAGE OverloadedStrings #-}
|
||||
-- | Stability auto-calculation, last-reviewed derivation, and version history.
|
||||
--
|
||||
-- For each content page:
|
||||
-- * If the page's source path appears in @IGNORE.txt@, the stability and
|
||||
-- last-reviewed fields fall back to the frontmatter values.
|
||||
-- * Otherwise, @git log --follow@ is used. Stability is derived from
|
||||
-- commit count + age; last-reviewed is the most-recent commit date.
|
||||
--
|
||||
-- Version history (@$version-history$@):
|
||||
-- * Prioritises frontmatter @history:@ list (date + note pairs).
|
||||
-- * Falls back to the raw git log dates (date-only, no message).
|
||||
-- * Falls back to nothing (template shows created/modified dates instead).
|
||||
--
|
||||
-- @IGNORE.txt@ is cleared by the build target in the Makefile after
|
||||
-- every successful build, so pins are one-shot.
|
||||
module Stability
|
||||
( stabilityField
|
||||
, resolveStability
|
||||
, lastReviewedField
|
||||
, lastReviewedIsoField
|
||||
, versionHistoryField
|
||||
, versionHistoryPrimaryField
|
||||
, versionHistoryRestField
|
||||
, versionHistoryRangeField
|
||||
, versionHistoryRangeStartField
|
||||
, versionHistoryRangeEndField
|
||||
, versionHistoryCommitsField
|
||||
) where
|
||||
|
||||
import Control.Exception (catch, IOException)
|
||||
import Data.Aeson (Value (..))
|
||||
import qualified Data.Aeson.KeyMap as KM
|
||||
import qualified Data.Vector as V
|
||||
import Data.List (sortBy)
|
||||
import Data.Maybe (catMaybes, fromMaybe, listToMaybe)
|
||||
import Data.Ord (comparing, Down (..))
|
||||
import Data.Time.Calendar (Day, diffDays)
|
||||
import Data.Time.Clock (getCurrentTime, utctDay)
|
||||
import Data.Time.Format (parseTimeM, formatTime, defaultTimeLocale)
|
||||
import qualified Data.Text as T
|
||||
import qualified Data.Text.IO as TIO
|
||||
import System.Exit (ExitCode (..))
|
||||
import System.IO (hPutStrLn, stderr)
|
||||
import System.Process (readProcessWithExitCode)
|
||||
import Hakyll
|
||||
|
||||
-- ---------------------------------------------------------------------------
|
||||
-- IGNORE.txt
|
||||
-- ---------------------------------------------------------------------------
|
||||
|
||||
-- | Read @IGNORE.txt@ (paths relative to project root, one per line).
|
||||
-- Returns an empty list when the file is absent or empty.
|
||||
--
|
||||
-- Uses strict text IO so the file handle is released immediately rather
|
||||
-- than left dangling on the lazy spine of 'readFile'.
|
||||
readIgnore :: IO [FilePath]
|
||||
readIgnore =
|
||||
(filter (not . null) . map T.unpack . T.lines <$> TIO.readFile "IGNORE.txt")
|
||||
`catch` \(_ :: IOException) -> return []
|
||||
|
||||
-- ---------------------------------------------------------------------------
|
||||
-- Git helpers
|
||||
-- ---------------------------------------------------------------------------
|
||||
|
||||
-- | Return commit dates (ISO "YYYY-MM-DD", newest-first) for @fp@.
|
||||
--
|
||||
-- Logs git's stderr to the build's stderr when present so the author
|
||||
-- isn't left in the dark when a file isn't tracked yet (the warning
|
||||
-- otherwise vanishes silently).
|
||||
gitDates :: FilePath -> IO [String]
|
||||
gitDates fp = do
|
||||
(ec, out, err) <- readProcessWithExitCode
|
||||
"git" ["log", "--follow", "--format=%ad", "--date=short", "--", fp] ""
|
||||
case ec of
|
||||
ExitFailure _ -> do
|
||||
let msg = if null err then "git log failed" else err
|
||||
hPutStrLn stderr $ "[Stability] " ++ fp ++ ": " ++ msg
|
||||
return []
|
||||
ExitSuccess -> do
|
||||
case err of
|
||||
"" -> return ()
|
||||
_ -> hPutStrLn stderr $ "[Stability] " ++ fp ++ ": " ++ err
|
||||
return $ filter (not . null) (lines out)
|
||||
|
||||
-- | Parse an ISO "YYYY-MM-DD" string to a 'Day'.
|
||||
parseIso :: String -> Maybe Day
|
||||
parseIso = parseTimeM True defaultTimeLocale "%Y-%m-%d"
|
||||
|
||||
-- | Derive stability label from commit dates (newest-first), judged as
|
||||
-- of @today@.
|
||||
--
|
||||
-- Thresholds (commit count + age in days since first commit):
|
||||
--
|
||||
-- * @volatile@ — solo commit OR less than two weeks old.
|
||||
-- * @revising@ — under six commits AND under three months old.
|
||||
-- * @fairly stable@ — under sixteen commits OR under one year old.
|
||||
-- * @stable@ — under thirty-one commits OR under two years old.
|
||||
-- * @established@ — anything beyond.
|
||||
--
|
||||
-- These cliffs are deliberately conservative: a fast burst of commits
|
||||
-- early in a piece's life looks volatile until enough time has passed
|
||||
-- to demonstrate it has settled. Age is measured from the first commit
|
||||
-- to /today/, not to the most recent commit — a piece written in a
|
||||
-- one-week burst must be able to stabilise as quiet time accumulates.
|
||||
stabilityFromDates :: Day -> [String] -> String
|
||||
stabilityFromDates _ [] = "volatile"
|
||||
stabilityFromDates today dates =
|
||||
classify (length dates) ageDays
|
||||
where
|
||||
-- 'last' is safe: the [] case is handled above.
|
||||
ageDays = case parseIso (last dates) of
|
||||
Just firstDay -> fromIntegral (diffDays today firstDay)
|
||||
Nothing -> 0
|
||||
classify n age
|
||||
| n <= 1 || age < volatileAge = "volatile"
|
||||
| n <= 5 && age < revisingAge = "revising"
|
||||
| n <= 15 || age < fairlyStableAge = "fairly stable"
|
||||
| n <= 30 || age < stableAge = "stable"
|
||||
| otherwise = "established"
|
||||
|
||||
volatileAge, revisingAge, fairlyStableAge, stableAge :: Int
|
||||
volatileAge = 14
|
||||
revisingAge = 90
|
||||
fairlyStableAge = 365
|
||||
stableAge = 730
|
||||
|
||||
-- | Format an ISO date as "%-d %B %Y" (e.g. "16 March 2026").
|
||||
fmtIso :: String -> String
|
||||
fmtIso s = case parseIso s of
|
||||
Nothing -> s
|
||||
Just day -> formatTime defaultTimeLocale "%-d %B %Y" (day :: Day)
|
||||
|
||||
-- ---------------------------------------------------------------------------
|
||||
-- Stability and last-reviewed context fields
|
||||
-- ---------------------------------------------------------------------------
|
||||
|
||||
-- | Resolve the stability label for an item — frontmatter override
|
||||
-- when the source path is pinned via @IGNORE.txt@, else the heuristic
|
||||
-- run over @git log --follow@ on the source path.
|
||||
--
|
||||
-- Used by 'stabilityField' (which exposes the label as a context field)
|
||||
-- and by Marks.hs (which feeds the label into the epistemic figure's
|
||||
-- outer-ring tick count).
|
||||
resolveStability :: Item a -> Compiler String
|
||||
resolveStability item = do
|
||||
let srcPath = toFilePath (itemIdentifier item)
|
||||
meta <- getMetadata (itemIdentifier item)
|
||||
unsafeCompiler $ do
|
||||
ignored <- readIgnore
|
||||
if srcPath `elem` ignored
|
||||
then return $ fromMaybe "volatile" (lookupString "stability" meta)
|
||||
else do
|
||||
today <- utctDay <$> getCurrentTime
|
||||
stabilityFromDates today <$> gitDates srcPath
|
||||
|
||||
-- | Context field @$stability$@.
|
||||
-- Always resolves to a label; prefers frontmatter when the file is pinned.
|
||||
stabilityField :: Context String
|
||||
stabilityField = field "stability" resolveStability
|
||||
|
||||
-- | Context field @$last-reviewed$@.
|
||||
-- Returns the formatted date of the most-recent commit, or @noResult@ when
|
||||
-- unavailable (making @$if(last-reviewed)$@ false in templates).
|
||||
lastReviewedField :: Context String
|
||||
lastReviewedField = field "last-reviewed" $ \item -> do
|
||||
let srcPath = toFilePath (itemIdentifier item)
|
||||
meta <- getMetadata (itemIdentifier item)
|
||||
mDate <- unsafeCompiler $ do
|
||||
ignored <- readIgnore
|
||||
if srcPath `elem` ignored
|
||||
-- Frontmatter convention is ISO; format it like the git
|
||||
-- branch so pinned pages don't render a raw "2026-05-01".
|
||||
then return $ fmtIso <$> lookupString "last-reviewed" meta
|
||||
else fmap fmtIso . listToMaybe <$> gitDates srcPath
|
||||
case mDate of
|
||||
Nothing -> fail "no last-reviewed"
|
||||
Just d -> return d
|
||||
|
||||
-- | Raw-ISO companion to @$last-reviewed$@ — for hover-popup
|
||||
-- @data-date-start@ attribute. Falls back to the frontmatter value for
|
||||
-- pinned files (which is expected to already be ISO, the same convention
|
||||
-- used by 'lastReviewedField' before it applied 'fmtIso').
|
||||
lastReviewedIsoField :: Context String
|
||||
lastReviewedIsoField = field "last-reviewed-iso" $ \item -> do
|
||||
let srcPath = toFilePath (itemIdentifier item)
|
||||
meta <- getMetadata (itemIdentifier item)
|
||||
mIso <- unsafeCompiler $ do
|
||||
ignored <- readIgnore
|
||||
if srcPath `elem` ignored
|
||||
then return $ lookupString "last-reviewed" meta
|
||||
else listToMaybe <$> gitDates srcPath
|
||||
case mIso of
|
||||
Nothing -> fail "no last-reviewed ISO"
|
||||
Just d -> return d
|
||||
|
||||
-- ---------------------------------------------------------------------------
|
||||
-- Version history
|
||||
-- ---------------------------------------------------------------------------
|
||||
|
||||
data VHEntry = VHEntry
|
||||
{ vhDate :: String -- human-readable, e.g. "12 April 2026"
|
||||
, vhDateIso :: String -- raw ISO, e.g. "2026-04-12"
|
||||
, vhMessage :: Maybe String -- Nothing for git-log-only entries
|
||||
}
|
||||
|
||||
-- | Parse the optional frontmatter @history:@ list.
|
||||
-- Each item must have @date:@ and @note:@ keys.
|
||||
parseFmHistory :: Metadata -> [VHEntry]
|
||||
parseFmHistory meta =
|
||||
case KM.lookup "history" meta of
|
||||
Just (Array v) -> catMaybes (map parseOne (V.toList v))
|
||||
_ -> []
|
||||
where
|
||||
parseOne (Object o) =
|
||||
case getString =<< KM.lookup "date" o of
|
||||
Nothing -> Nothing
|
||||
Just d -> Just $ VHEntry (fmtIso d) d (getString =<< KM.lookup "note" o)
|
||||
parseOne _ = Nothing
|
||||
|
||||
getString (String t) = Just (T.unpack t)
|
||||
getString _ = Nothing
|
||||
|
||||
-- | Get git log for a file as version history entries (date-only, no message).
|
||||
gitLogHistory :: FilePath -> IO [VHEntry]
|
||||
gitLogHistory fp = map (\d -> VHEntry (fmtIso d) d Nothing) <$> gitDates fp
|
||||
|
||||
-- | Maximum entries shown by default in the version-history footer block.
|
||||
-- The remainder is revealed via a <details>/<summary> expand affordance,
|
||||
-- matching the cap on the RELATED column.
|
||||
versionHistoryHeadCount :: Int
|
||||
versionHistoryHeadCount = 3
|
||||
|
||||
-- | Load version-history entries for an item.
|
||||
-- Priority: frontmatter @history:@ list → git log dates → empty.
|
||||
--
|
||||
-- Entries are sorted newest-first by ISO date regardless of authored
|
||||
-- order: every consumer (primary/rest split, range fields) assumes the
|
||||
-- head is the newest entry, and the @history:@ list may be authored in
|
||||
-- either direction. Git dates already arrive newest-first; the sort is
|
||||
-- idempotent there.
|
||||
loadVersionHistory :: Item a -> Compiler [VHEntry]
|
||||
loadVersionHistory item = do
|
||||
let srcPath = toFilePath (itemIdentifier item)
|
||||
meta <- getMetadata (itemIdentifier item)
|
||||
let newestFirst = sortBy (comparing (Down . vhDateIso))
|
||||
fmEntries = newestFirst (parseFmHistory meta)
|
||||
if not (null fmEntries)
|
||||
then return fmEntries
|
||||
else unsafeCompiler (newestFirst <$> gitLogHistory srcPath)
|
||||
|
||||
-- | Wrap a list of 'VHEntry' as Hakyll Items with unique paths so the
|
||||
-- list field works correctly inside @$for$@.
|
||||
vhItems :: String -> [VHEntry] -> [Item VHEntry]
|
||||
vhItems tag =
|
||||
zipWith (\i e -> Item (fromFilePath (tag ++ "-" ++ show (i :: Int))) e)
|
||||
[1..]
|
||||
|
||||
-- | Shared sub-context for version-history entries: @$vh-date$@,
|
||||
-- @$vh-date-iso$@ (raw ISO for hover popups), and (optionally) @$vh-message$@.
|
||||
vhEntryCtx :: Context VHEntry
|
||||
vhEntryCtx =
|
||||
field "vh-date" (return . vhDate . itemBody)
|
||||
<> field "vh-date-iso" (return . vhDateIso . itemBody)
|
||||
<> field "vh-message" (\i -> case vhMessage (itemBody i) of
|
||||
Nothing -> fail "no message"
|
||||
Just m -> return m)
|
||||
|
||||
-- | Context list field @$version-history$@ — full list, kept for callers
|
||||
-- (e.g. feeds, stats) that want every entry in one pass.
|
||||
versionHistoryField :: Context String
|
||||
versionHistoryField =
|
||||
listFieldWith "version-history" vhEntryCtx $ \item -> do
|
||||
entries <- loadVersionHistory item
|
||||
if null entries
|
||||
then fail "no version history"
|
||||
else return (vhItems "vh" entries)
|
||||
|
||||
-- | @$version-history-primary$@ — first 'versionHistoryHeadCount' entries,
|
||||
-- rendered outside the expand affordance.
|
||||
versionHistoryPrimaryField :: Context String
|
||||
versionHistoryPrimaryField =
|
||||
listFieldWith "version-history-primary" vhEntryCtx $ \item -> do
|
||||
entries <- loadVersionHistory item
|
||||
let primary = take versionHistoryHeadCount entries
|
||||
if null primary
|
||||
then fail "no version history"
|
||||
else return (vhItems "vh-p" primary)
|
||||
|
||||
-- | @$version-history-rest$@ — overflow entries (count > head cap), which
|
||||
-- the template wraps in a <details> block. Fails (noResult) when the total
|
||||
-- fits inside the head cap, so @$if(version-history-rest)$@ collapses
|
||||
-- cleanly.
|
||||
versionHistoryRestField :: Context String
|
||||
versionHistoryRestField =
|
||||
listFieldWith "version-history-rest" vhEntryCtx $ \item -> do
|
||||
entries <- loadVersionHistory item
|
||||
let rest = drop versionHistoryHeadCount entries
|
||||
if null rest
|
||||
then fail "no overflow"
|
||||
else return (vhItems "vh-r" rest)
|
||||
|
||||
-- | @$version-history-range$@ — formatted span between the oldest and
|
||||
-- newest entry. A single-date history renders as that date alone; a
|
||||
-- multi-date history renders as "OLDEST \x2013 NEWEST" (en-dash).
|
||||
-- Fails when no history is available so @$if(version-history-range)$@
|
||||
-- in the template falls back to a literal label.
|
||||
--
|
||||
-- Dates in the underlying VHEntry list are already pre-formatted
|
||||
-- ("12 April 2026") by 'parseFmHistory' / 'gitLogHistory'.
|
||||
versionHistoryRangeField :: Context String
|
||||
versionHistoryRangeField = field "version-history-range" $ \item -> do
|
||||
entries <- loadVersionHistory item
|
||||
case entries of
|
||||
[] -> fail "no version-history range"
|
||||
[one] -> return (vhDate one)
|
||||
es@(newest:_) ->
|
||||
let oldest = last es -- safe: es is non-empty by pattern
|
||||
newD = vhDate newest
|
||||
oldD = vhDate oldest
|
||||
in if newD == oldD
|
||||
then return newD
|
||||
else return (oldD ++ " \x2013 " ++ newD)
|
||||
|
||||
-- | Raw-ISO start date (oldest entry) for hover-popup machine use.
|
||||
versionHistoryRangeStartField :: Context String
|
||||
versionHistoryRangeStartField =
|
||||
field "version-history-range-start" $ \item -> do
|
||||
entries <- loadVersionHistory item
|
||||
case entries of
|
||||
[] -> fail "no version-history start"
|
||||
_ -> return (vhDateIso (last entries))
|
||||
|
||||
-- | Raw-ISO end date (newest entry) for hover-popup machine use.
|
||||
-- Only resolves when the range spans more than one calendar day — single-day
|
||||
-- histories don't need an end attribute on the popup trigger.
|
||||
versionHistoryRangeEndField :: Context String
|
||||
versionHistoryRangeEndField =
|
||||
field "version-history-range-end" $ \item -> do
|
||||
entries <- loadVersionHistory item
|
||||
case entries of
|
||||
[] -> fail "no version-history end"
|
||||
[_] -> fail "single-day history — no end"
|
||||
es@(newest:_) ->
|
||||
let oldest = last es -- safe: es is non-empty by pattern
|
||||
in if vhDateIso newest == vhDateIso oldest
|
||||
then fail "single-day history — no end"
|
||||
else return (vhDateIso newest)
|
||||
|
||||
-- | Commit count — used by the frontmatter popup to surface the *density*
|
||||
-- of attention the piece has received. Deliberately only wired into the
|
||||
-- metadata-strip date link, not the aftermatter list (where it would be
|
||||
-- redundant next to the enumeration of entries).
|
||||
versionHistoryCommitsField :: Context String
|
||||
versionHistoryCommitsField =
|
||||
field "version-history-commits" $ \item -> do
|
||||
entries <- loadVersionHistory item
|
||||
case entries of
|
||||
[] -> fail "no commits"
|
||||
_ -> return (show (length entries))
|
||||
File diff suppressed because it is too large
Load Diff
|
|
@ -0,0 +1,425 @@
|
|||
{-# LANGUAGE GHC2021 #-}
|
||||
{-# LANGUAGE OverloadedStrings #-}
|
||||
-- | Hierarchical tag system.
|
||||
--
|
||||
-- Tags are slash-separated strings in YAML frontmatter:
|
||||
-- tags: [research/mathematics, nonfiction/essays, typography]
|
||||
--
|
||||
-- "research/mathematics" expands to ["research", "research/mathematics"]
|
||||
-- so /research/ aggregates everything tagged with any research/* sub-tag.
|
||||
--
|
||||
-- Pages live at /<tag>/index.html — no /tags/ namespace:
|
||||
-- research → /research/
|
||||
-- research/mathematics → /research/mathematics/
|
||||
-- typography → /typography/
|
||||
--
|
||||
-- Optional sidecar files at @content/tag-meta/<tag-path>.md@ supply
|
||||
-- a per-tag @tooltip:@ (frontmatter) and prose intro (body). When
|
||||
-- a sidecar exists, the tag page exposes @$portal-tooltip$@ and
|
||||
-- @$portal-intro$@; when it is absent, both fields are noResult
|
||||
-- and the corresponding @$if$@ blocks render nothing.
|
||||
module Tags
|
||||
( buildAllTags
|
||||
, applyTagRules
|
||||
, tagPaginationThreshold
|
||||
, tagPageSize
|
||||
, sidecarIdentifier
|
||||
, portalIntroField
|
||||
, portalTooltipField
|
||||
, seeAlsoContext
|
||||
) where
|
||||
|
||||
import Data.Char (isSpace)
|
||||
import Data.List (intercalate, isPrefixOf, nub, sort, sortBy)
|
||||
import Data.Maybe (fromMaybe, isNothing, maybeToList)
|
||||
import Data.Ord (comparing)
|
||||
import Data.Set (Set)
|
||||
import qualified Data.Set as Set
|
||||
import Data.Time.Clock (UTCTime)
|
||||
import Data.Time.Format (defaultTimeLocale, parseTimeM)
|
||||
import Hakyll
|
||||
import Patterns (tagIndexable)
|
||||
import Contexts (Revision (..), abstractField, contentKindField,
|
||||
getRevisions, recentFirstByDisplay, revisionDateFields,
|
||||
siteCtx, tagLinksFieldExcludingScope)
|
||||
|
||||
|
||||
-- ---------------------------------------------------------------------------
|
||||
-- Pagination policy
|
||||
-- ---------------------------------------------------------------------------
|
||||
|
||||
-- | Maximum number of items for which a tag index ships its full list and
|
||||
-- relies purely on the client-side 25/50/100/All toggle (matching
|
||||
-- @\/new.html@). Tags above this threshold fall back to server-side
|
||||
-- pagination at 'tagPageSize' per page; the count toggle then operates
|
||||
-- within the current page only.
|
||||
tagPaginationThreshold :: Int
|
||||
tagPaginationThreshold = 150
|
||||
|
||||
-- | Page size used for server-side pagination on tag pages that exceed
|
||||
-- 'tagPaginationThreshold'.
|
||||
tagPageSize :: Int
|
||||
tagPageSize = 100
|
||||
|
||||
|
||||
-- ---------------------------------------------------------------------------
|
||||
-- Hierarchy expansion
|
||||
-- ---------------------------------------------------------------------------
|
||||
|
||||
wordsBy :: (Char -> Bool) -> String -> [String]
|
||||
wordsBy p s = case dropWhile p s of
|
||||
"" -> []
|
||||
s' -> w : wordsBy p rest
|
||||
where (w, rest) = break p s'
|
||||
|
||||
-- | "research/mathematics" → ["research", "research/mathematics"]
|
||||
-- "a/b/c" → ["a", "a/b", "a/b/c"]
|
||||
-- "typography" → ["typography"]
|
||||
expandTag :: String -> [String]
|
||||
expandTag t =
|
||||
let segs = wordsBy (== '/') t
|
||||
in [ intercalate "/" (take n segs) | n <- [1 .. length segs] ]
|
||||
|
||||
-- | Top-level tags that own a section URL outside the tag system, and
|
||||
-- therefore must NOT be created as tag pages — doing so would
|
||||
-- collide with a section landing route. Hakyll does not error on
|
||||
-- duplicate routes (one item silently overwrites the other), so an
|
||||
-- essay tagged e.g. @music@ would otherwise clobber
|
||||
-- @music/index.html@. The set therefore lists every namespace that
|
||||
-- owns a @<name>/index.html@ route, not just the tags currently in
|
||||
-- use: @photography@ (every photo's @tags:@ list begins with it, per
|
||||
-- the section convention) plus the other section landings and
|
||||
-- generated index namespaces.
|
||||
--
|
||||
-- Sub-tags (@photography/landscape@, @photography/film@, …) are
|
||||
-- unaffected; they keep their tag pages because no section landing
|
||||
-- claims those URLs.
|
||||
sectionOwnedTopLevelTags :: [String]
|
||||
sectionOwnedTopLevelTags =
|
||||
[ "photography", "poetry", "fiction", "music", "essays", "blog"
|
||||
, "cv", "archive", "authors", "bibliography"
|
||||
]
|
||||
|
||||
-- | All expanded tags for an item (reads the "tags" metadata field).
|
||||
-- Filters out any 'sectionOwnedTopLevelTags' to prevent route
|
||||
-- collisions with section landings.
|
||||
getExpandedTags :: MonadMetadata m => Identifier -> m [String]
|
||||
getExpandedTags ident =
|
||||
filter (`notElem` sectionOwnedTopLevelTags) . nub . concatMap expandTag
|
||||
<$> getTags ident
|
||||
|
||||
|
||||
-- ---------------------------------------------------------------------------
|
||||
-- Identifiers and URLs
|
||||
-- ---------------------------------------------------------------------------
|
||||
|
||||
tagFilePath :: String -> FilePath
|
||||
tagFilePath tag = tag ++ "/index.html"
|
||||
|
||||
tagIdentifier :: String -> Identifier
|
||||
tagIdentifier = fromFilePath . tagFilePath
|
||||
|
||||
-- | Identifier of the optional sidecar for a given tag.
|
||||
-- "nonfiction" → content/tag-meta/nonfiction.md
|
||||
-- "nonfiction/philosophy" → content/tag-meta/nonfiction/philosophy.md
|
||||
sidecarIdentifier :: String -> Identifier
|
||||
sidecarIdentifier tag = fromFilePath ("content/tag-meta/" ++ tag ++ ".md")
|
||||
|
||||
|
||||
-- ---------------------------------------------------------------------------
|
||||
-- Building the Tags index
|
||||
-- ---------------------------------------------------------------------------
|
||||
|
||||
-- | Scan all essays and blog posts and build the Tags index.
|
||||
buildAllTags :: Rules Tags
|
||||
buildAllTags =
|
||||
buildTagsWith getExpandedTags tagIndexable tagIdentifier
|
||||
|
||||
|
||||
-- ---------------------------------------------------------------------------
|
||||
-- Sidecar fields
|
||||
-- ---------------------------------------------------------------------------
|
||||
|
||||
-- | Field exposing a sidecar's rendered HTML body as @$portal-intro$@.
|
||||
-- Fails with 'noResult' when the sidecar body is empty or whitespace-only,
|
||||
-- so @$if(portal-intro)$@ is false and the render site emits nothing.
|
||||
--
|
||||
-- Takes a function that yields the sidecar identifier from the current
|
||||
-- item — this lets tag pages bind the sidecar statically at rule time
|
||||
-- (@const sidecarId@) while the home-page portal listField derives it
|
||||
-- per-item from the item body.
|
||||
portalIntroField :: (Item a -> Identifier) -> Context a
|
||||
portalIntroField getSidecarId = field "portal-intro" $ \item -> do
|
||||
let sidecarId = getSidecarId item
|
||||
html <- itemBody <$> loadSnapshot sidecarId "body"
|
||||
if all isSpace html
|
||||
then noResult "sidecar body is empty"
|
||||
else return html
|
||||
|
||||
-- | Field exposing a sidecar's @tooltip:@ frontmatter value as
|
||||
-- @$portal-tooltip$@. Fails with 'noResult' when the key is absent
|
||||
-- or the value is empty / whitespace-only. Accepts a per-item
|
||||
-- identifier resolver for the same reason as 'portalIntroField'.
|
||||
portalTooltipField :: (Item a -> Identifier) -> Context a
|
||||
portalTooltipField getSidecarId = field "portal-tooltip" $ \item -> do
|
||||
let sidecarId = getSidecarId item
|
||||
meta <- getMetadata sidecarId
|
||||
case fmap trim (lookupString "tooltip" meta) of
|
||||
Just t | not (null t) -> return t
|
||||
_ -> noResult "no tooltip"
|
||||
|
||||
-- ---------------------------------------------------------------------------
|
||||
-- See Also: parent / sibling / child computation
|
||||
-- ---------------------------------------------------------------------------
|
||||
|
||||
-- | Direct parent of a tag path. @Nothing@ for top-level tags (portals).
|
||||
-- "nonfiction/philosophy" → Just "nonfiction"
|
||||
-- "nonfiction" → Nothing
|
||||
-- "a/b/c" → Just "a/b"
|
||||
parentOf :: String -> Maybe String
|
||||
parentOf t = case wordsBy (== '/') t of
|
||||
[] -> Nothing
|
||||
[_] -> Nothing
|
||||
segs -> Just (intercalate "/" (init segs)) -- 'init' safe: 2+ segments
|
||||
|
||||
-- | Number of @/@ characters in a tag path (i.e., depth - 1).
|
||||
slashCount :: String -> Int
|
||||
slashCount = length . filter (== '/')
|
||||
|
||||
-- | Parent / siblings / children of a scope tag, each filtered to tags that
|
||||
-- appear in 'tagsMap' (i.e., have at least one item). Parent is returned
|
||||
-- unconditionally — if it isn't in @tagsMap@ the See Also still links it,
|
||||
-- since a parent with no direct items but some descendant items is still
|
||||
-- a navigable aggregation page.
|
||||
--
|
||||
-- Sibling portals (scope is top-level) render in @portalOrder@. Sibling
|
||||
-- subcategories (scope has a parent) render alphabetically. Children
|
||||
-- always render alphabetically.
|
||||
seeAlsoGroups :: [String] -- ^ canonical portal tag order
|
||||
-> Tags -- ^ all tags for has-items filter
|
||||
-> String -- ^ current scope
|
||||
-> (Maybe String, [String], [String])
|
||||
seeAlsoGroups portalOrder tags scope =
|
||||
let tKeys = map fst (tagsMap tags)
|
||||
mParent = parentOf scope
|
||||
|
||||
sibs = case mParent of
|
||||
Nothing ->
|
||||
-- Scope is a portal. Siblings: other portals in tagsMap,
|
||||
-- emitted in portalOrder.
|
||||
[ p | p <- portalOrder, p /= scope, p `elem` tKeys ]
|
||||
Just parent ->
|
||||
-- Scope is a subcategory. Siblings: other direct children
|
||||
-- of the parent, alphabetical.
|
||||
sort [ s | s <- tKeys
|
||||
, parentOf s == Just parent
|
||||
, s /= scope
|
||||
]
|
||||
|
||||
kids = sort
|
||||
[ c | c <- tKeys
|
||||
, (scope ++ "/") `isPrefixOf` c
|
||||
, slashCount c == slashCount scope + 1
|
||||
]
|
||||
in (mParent, sibs, kids)
|
||||
|
||||
|
||||
-- ---------------------------------------------------------------------------
|
||||
-- See Also: rendering into a Context
|
||||
-- ---------------------------------------------------------------------------
|
||||
|
||||
-- | Display name for a tag path. Portals get their capitalized form from
|
||||
-- @portalPairs@ (e.g., "Research"); subcategories display their raw tag
|
||||
-- path (e.g., "nonfiction/philosophy").
|
||||
displayNameFor :: [(String, String)] -> String -> String
|
||||
displayNameFor portalPairs t =
|
||||
fromMaybe t (lookup t (map (\(d, tg) -> (tg, d)) portalPairs))
|
||||
|
||||
-- | Item-level context for See Also entries. Body is a tag path string.
|
||||
seeAlsoItemCtx :: [(String, String)] -> Context String
|
||||
seeAlsoItemCtx portalPairs =
|
||||
field "see-also-name" (\i -> return (displayNameFor portalPairs (itemBody i)))
|
||||
<> field "see-also-url" (\i -> return $ "/" ++ itemBody i ++ "/")
|
||||
<> portalTooltipField (sidecarIdentifier . itemBody)
|
||||
|
||||
-- | Full See Also context contribution for a scope: three listFields
|
||||
-- (@see-also-parent@ at most one entry, @see-also-siblings@,
|
||||
-- @see-also-children@) and a @has-see-also@ gate that fails when all
|
||||
-- three are empty so the template's @$if(has-see-also)$@ suppresses
|
||||
-- the entire @<nav>@ wrapper.
|
||||
seeAlsoContext :: [(String, String)] -> Tags -> String -> Context String
|
||||
seeAlsoContext portalPairs tags scope =
|
||||
listField "see-also-parent" itemCtx (return (toItems (maybeToList mParent)))
|
||||
<> listField "see-also-siblings" itemCtx (return (toItems sibs))
|
||||
<> listField "see-also-children" itemCtx (return (toItems kids))
|
||||
<> field "has-see-also" (\_ ->
|
||||
if isNothing mParent && null sibs && null kids
|
||||
then noResult "no see-also entries"
|
||||
else return "true")
|
||||
where
|
||||
(mParent, sibs, kids) = seeAlsoGroups (map snd portalPairs) tags scope
|
||||
itemCtx = seeAlsoItemCtx portalPairs
|
||||
toItems = map (Item (fromFilePath ""))
|
||||
|
||||
|
||||
-- | Context contribution for the current tag's sidecar, if one exists,
|
||||
-- and eager registration of the snapshot dependency.
|
||||
--
|
||||
-- When a sidecar exists, the body snapshot is loaded unconditionally
|
||||
-- (and discarded) so Hakyll's dependency tracker sees the edge
|
||||
-- /tag page → sidecar body/ on every compile — even when the
|
||||
-- rendered @$if(portal-intro)$@ gate is false because the body is
|
||||
-- empty. Without this, the first build after populating a previously
|
||||
-- empty sidecar would not re-render the tag page (the lazy field
|
||||
-- load inside 'portalIntroField' never fires while the gate is
|
||||
-- false, so the dep is never established).
|
||||
--
|
||||
-- Tags with no sidecar take the @mempty@ branch and register no
|
||||
-- dependency, which is correct — there is nothing to depend on.
|
||||
sidecarContext :: Set Identifier -> String -> Compiler (Context String)
|
||||
sidecarContext sidecarSet tag
|
||||
| sidecarId `Set.member` sidecarSet = do
|
||||
_ <- loadSnapshot sidecarId "body" :: Compiler (Item String)
|
||||
return ( portalIntroField (const sidecarId)
|
||||
<> portalTooltipField (const sidecarId))
|
||||
| otherwise = return mempty
|
||||
where
|
||||
sidecarId = sidecarIdentifier tag
|
||||
|
||||
|
||||
-- ---------------------------------------------------------------------------
|
||||
-- Tag index page rules
|
||||
-- ---------------------------------------------------------------------------
|
||||
|
||||
-- | Item-level context used inside @$for(items)$@ on tag index pages.
|
||||
-- Provides the fields consumed by @templates/partials/item-card.html@
|
||||
-- (@$item-kind$@, @$date-iso$@, @$date-created$@, @$abstract$@,
|
||||
-- @$item-tags$@) with tag-ribbon suppression scoped to the current tag.
|
||||
--
|
||||
-- Composes 'siteCtx' (not bare 'defaultContext') so per-item fields
|
||||
-- the card partial gates on — notably @$has-monogram$@ — fire here
|
||||
-- the same way they do on /new.html and the library.
|
||||
tagItemCtx :: String -> Context String
|
||||
tagItemCtx scope =
|
||||
contentKindField
|
||||
<> dateField "date-created" "%-d %B %Y"
|
||||
<> dateField "date" "%-d %B %Y"
|
||||
<> revisionDateFields
|
||||
<> tagLinksFieldExcludingScope "item-tags" scope
|
||||
<> abstractField
|
||||
<> siteCtx
|
||||
|
||||
-- | Page identifier for a tag index page.
|
||||
-- Page 1 → <tag>/index.html
|
||||
-- Page N → <tag>/page/N/index.html
|
||||
tagPageId :: String -> PageNumber -> Identifier
|
||||
tagPageId tag 1 = fromFilePath $ tag ++ "/index.html"
|
||||
tagPageId tag n = fromFilePath $ tag ++ "/page/" ++ show n ++ "/index.html"
|
||||
|
||||
-- | Generate index pages for every tag. Tags with at most
|
||||
-- 'tagPaginationThreshold' items render a single page with the full
|
||||
-- list and rely on the client-side count toggle; larger tags fall
|
||||
-- back to server-side pagination at 'tagPageSize' per page.
|
||||
--
|
||||
-- Each tag's context is augmented with its sidecar, if present, so
|
||||
-- the template can render a @$portal-intro$@ section and (later)
|
||||
-- a See Also block keyed on @$portal-tooltip$@.
|
||||
--
|
||||
-- @baseCtx@ should be @siteCtx@ (passed in to avoid a circular import).
|
||||
applyTagRules :: Tags -> [(String, String)] -> Context String -> Rules ()
|
||||
applyTagRules tags portalPairs baseCtx = do
|
||||
-- Hakyll's @**/*@ glob needs a subdirectory level, so the flat and
|
||||
-- nested sidecar paths each need their own pattern. Keep this list
|
||||
-- in sync with the matching rule in Site.rules.
|
||||
sidecarIds <- getMatches ("content/tag-meta/*.md" .||. "content/tag-meta/**/*.md")
|
||||
let sidecarSet = Set.fromList sidecarIds
|
||||
tagsRules tags $ \tag pat -> do
|
||||
let itemCount = length (fromMaybe [] (lookup tag (tagsMap tags)))
|
||||
saCtx = seeAlsoContext portalPairs tags tag
|
||||
if itemCount <= tagPaginationThreshold
|
||||
then clientPaginatedRule tag pat sidecarSet saCtx baseCtx
|
||||
else serverPaginatedRule tag pat sidecarSet saCtx baseCtx
|
||||
|
||||
-- | Single-page tag index: the count toggle runs client-side against
|
||||
-- the full list. No server-side pagination, no @page/N@ URLs.
|
||||
clientPaginatedRule :: String
|
||||
-> Pattern
|
||||
-> Set Identifier
|
||||
-> Context String -- ^ See Also contribution
|
||||
-> Context String -- ^ base (siteCtx)
|
||||
-> Rules ()
|
||||
clientPaginatedRule tag pat sidecarSet saCtx baseCtx = do
|
||||
route idRoute
|
||||
compile $ do
|
||||
scCtx <- sidecarContext sidecarSet tag
|
||||
items <- recentFirstByDisplay =<< loadAll (pat .&&. hasNoVersion)
|
||||
let ctx = listField "items" (tagItemCtx tag) (return items)
|
||||
<> constField "tag" tag
|
||||
<> constField "title" tag
|
||||
<> constField "list-page" "true"
|
||||
<> saCtx
|
||||
<> scCtx
|
||||
<> baseCtx
|
||||
makeItem ""
|
||||
>>= loadAndApplyTemplate "templates/tag-index.html" ctx
|
||||
>>= loadAndApplyTemplate "templates/default.html" ctx
|
||||
>>= relativizeUrls
|
||||
|
||||
-- | Display date of an identifier: the most-recent @revised:@ entry's
|
||||
-- date when present and parseable, else the creation date. Mirrors
|
||||
-- the (unexported) @itemDisplayUTC@ behind 'Contexts.recentFirstByDisplay',
|
||||
-- but needs only 'MonadMetadata' — the paginate grouper runs in
|
||||
-- 'Rules' over bare 'Identifier's, where no 'Item's exist yet.
|
||||
identifierDisplayUTC :: (MonadMetadata m, MonadFail m)
|
||||
=> Identifier -> m UTCTime
|
||||
identifierDisplayUTC ident = do
|
||||
meta <- getMetadata ident
|
||||
case getRevisions meta of
|
||||
(r:_) | Just utc <- (parseTimeM True defaultTimeLocale "%Y-%m-%d"
|
||||
(revisionDateISO r) :: Maybe UTCTime)
|
||||
-> return utc
|
||||
_ -> getItemUTC defaultTimeLocale ident
|
||||
|
||||
-- | Partition identifiers into pages of @n@, most recent first by
|
||||
-- /display/ date — the same revision-aware key
|
||||
-- 'recentFirstByDisplay' sorts by within each rendered page — so
|
||||
-- cross-page ordering is monotone. With creation-date partitioning
|
||||
-- (plain @sortRecentFirst@), a recently revised old item stayed on a
|
||||
-- late page but jumped to its top; now it migrates to the early page
|
||||
-- where its displayed date says it belongs.
|
||||
sortAndGroupByDisplayAt :: (MonadMetadata m, MonadFail m)
|
||||
=> Int -> [Identifier] -> m [[Identifier]]
|
||||
sortAndGroupByDisplayAt n ids = do
|
||||
keyed <- mapM (\i -> (,) <$> identifierDisplayUTC i <*> pure i) ids
|
||||
return $ paginateEvery n $ map snd $ sortBy (flip (comparing fst)) keyed
|
||||
|
||||
-- | Server-side pagination at 'tagPageSize' per page. Previous/next
|
||||
-- navigation renders via @templates/partials/paginate-nav.html@;
|
||||
-- the count toggle operates within the current page only. Pages are
|
||||
-- partitioned and sorted by the same display-date key (see
|
||||
-- 'sortAndGroupByDisplayAt').
|
||||
serverPaginatedRule :: String
|
||||
-> Pattern
|
||||
-> Set Identifier
|
||||
-> Context String -- ^ See Also contribution
|
||||
-> Context String -- ^ base (siteCtx)
|
||||
-> Rules ()
|
||||
serverPaginatedRule tag pat sidecarSet saCtx baseCtx = do
|
||||
paginate <- buildPaginateWith (sortAndGroupByDisplayAt tagPageSize) pat (tagPageId tag)
|
||||
paginateRules paginate $ \pageNum pat' -> do
|
||||
route idRoute
|
||||
compile $ do
|
||||
scCtx <- sidecarContext sidecarSet tag
|
||||
items <- recentFirstByDisplay =<< loadAll (pat' .&&. hasNoVersion)
|
||||
let ctx = listField "items" (tagItemCtx tag) (return items)
|
||||
<> paginateContext paginate pageNum
|
||||
<> constField "tag" tag
|
||||
<> constField "title" tag
|
||||
<> constField "list-page" "true"
|
||||
<> saCtx
|
||||
<> scCtx
|
||||
<> baseCtx
|
||||
makeItem ""
|
||||
>>= loadAndApplyTemplate "templates/tag-index.html" ctx
|
||||
>>= loadAndApplyTemplate "templates/default.html" ctx
|
||||
>>= relativizeUrls
|
||||
|
|
@ -0,0 +1,83 @@
|
|||
{-# LANGUAGE GHC2021 #-}
|
||||
{-# LANGUAGE OverloadedStrings #-}
|
||||
-- | Shared utilities used across the build system.
|
||||
--
|
||||
-- The HTML escapers (one for 'String', one for 'Text') live here so that
|
||||
-- every filter, context, and renderer goes through the same definition.
|
||||
-- The expansion order matters: @&@ MUST be replaced first, otherwise the
|
||||
-- @&@ injected by other rules gets re-escaped to @&amp;@. The
|
||||
-- pure-character-by-character implementation used here avoids that hazard
|
||||
-- entirely (each character is mapped exactly once).
|
||||
module Utils
|
||||
( wordCount
|
||||
, readingTime
|
||||
, escapeHtml
|
||||
, escapeHtmlText
|
||||
, trim
|
||||
, authorSlugify
|
||||
, authorNameOf
|
||||
) where
|
||||
|
||||
import Data.Char (isAlphaNum, isSpace, toLower)
|
||||
import Data.List (dropWhileEnd)
|
||||
import qualified Data.Text as T
|
||||
|
||||
-- | Count the number of words in a string (split on whitespace).
|
||||
wordCount :: String -> Int
|
||||
wordCount = length . words
|
||||
|
||||
-- | Estimate reading time in minutes (assumes 200 words per minute).
|
||||
-- Rounds up — 399 words is 2 minutes, not 1. Minimum is 1 minute.
|
||||
readingTime :: String -> Int
|
||||
readingTime s = max 1 ((wordCount s + 199) `div` 200)
|
||||
|
||||
-- | Escape HTML special characters: @&@, @<@, @>@, @\"@, @\'@.
|
||||
--
|
||||
-- Safe for use in attribute values and text content. The order of the
|
||||
-- @case@ branches is irrelevant — each input character maps to exactly
|
||||
-- one output sequence.
|
||||
escapeHtml :: String -> String
|
||||
escapeHtml = concatMap escChar
|
||||
where
|
||||
escChar '&' = "&"
|
||||
escChar '<' = "<"
|
||||
escChar '>' = ">"
|
||||
escChar '"' = """
|
||||
escChar '\'' = "'"
|
||||
escChar c = [c]
|
||||
|
||||
-- | 'Text' counterpart of 'escapeHtml'.
|
||||
escapeHtmlText :: T.Text -> T.Text
|
||||
escapeHtmlText = T.concatMap escChar
|
||||
where
|
||||
escChar '&' = "&"
|
||||
escChar '<' = "<"
|
||||
escChar '>' = ">"
|
||||
escChar '"' = """
|
||||
escChar '\'' = "'"
|
||||
escChar c = T.singleton c
|
||||
|
||||
-- | Strip leading and trailing whitespace.
|
||||
trim :: String -> String
|
||||
trim = dropWhileEnd isSpace . dropWhile isSpace
|
||||
|
||||
-- | Lowercase a string, drop everything that isn't alphanumeric or
|
||||
-- space, then replace each space with a hyphen. Note that a run of
|
||||
-- spaces therefore becomes a run of hyphens (@"A B" → "a--b"@) —
|
||||
-- deliberately left as-is, since every slug on the site is generated
|
||||
-- by this one function and collapsing runs now would move existing
|
||||
-- author URLs.
|
||||
--
|
||||
-- Used for author URL slugs (e.g. @"Levi Neuwirth" → "levi-neuwirth"@).
|
||||
-- Centralised here so 'Authors' and 'Contexts' cannot drift on Unicode
|
||||
-- edge cases.
|
||||
authorSlugify :: String -> String
|
||||
authorSlugify = map (\c -> if c == ' ' then '-' else c)
|
||||
. filter (\c -> isAlphaNum c || c == ' ')
|
||||
. map toLower
|
||||
|
||||
-- | Extract the author name from a "Name | url" frontmatter entry.
|
||||
-- The URL portion is dropped (it's no longer used by the author system,
|
||||
-- which routes everything through @/authors/{slug}/@).
|
||||
authorNameOf :: String -> String
|
||||
authorNameOf s = trim (takeWhile (/= '|') s)
|
||||
|
|
@ -0,0 +1,667 @@
|
|||
{-# LANGUAGE GHC2021 #-}
|
||||
{-# LANGUAGE OverloadedStrings #-}
|
||||
-- | Vita page: renders education, publications, presentations, and
|
||||
-- experience for @/about.html@ from @yaml-source/data/*.yml@ — the same
|
||||
-- files that drive the CV and résumé PDFs via the Jinja/xelatex pipeline.
|
||||
--
|
||||
-- The point is single-sourcing. Before this module the vita page carried a
|
||||
-- hand-typed copy of all four sections, and it had drifted from the PDFs in
|
||||
-- roughly fifteen places (stale roles, a superseded line count, a talk still
|
||||
-- listed as forthcoming after it was given). Anything rendered here is read
|
||||
-- from the same YAML the PDFs are built from, so the two cannot disagree.
|
||||
--
|
||||
-- Sections the site already owns are deliberately absent: in-progress work
|
||||
-- belongs to @/current@ (data\/now.yaml, see "Now"), the engineering index to
|
||||
-- @/cv/projects/@, and the personal narrative to @/me/@. This module renders
|
||||
-- only what nothing else did.
|
||||
--
|
||||
-- The YAML is LaTeX-flavoured, because its first consumer is xelatex. Values
|
||||
-- may contain @\\textbf{}@, @\\href{}{}@, @$\\times$@, @~@, @--@ and friends,
|
||||
-- so every value goes through 'latexToHtml' on the way out. See that function
|
||||
-- for the full supported set and why escaping happens before conversion.
|
||||
module Vita
|
||||
( vitaCtx
|
||||
, projectsCtx
|
||||
) where
|
||||
|
||||
import Data.Aeson (FromJSON (..), Object, Value (..), withObject, (.:), (.:?), (.!=))
|
||||
import Data.Aeson.Types (Parser, typeMismatch)
|
||||
import Data.Char (toLower)
|
||||
import Data.List (intercalate, isPrefixOf, sortOn)
|
||||
import Data.Maybe (mapMaybe)
|
||||
import Data.Scientific (isInteger, toRealFloat)
|
||||
import qualified Data.Aeson.Key as K
|
||||
import qualified Data.Text as T
|
||||
import qualified Data.Text.Encoding as TE
|
||||
import qualified Data.Yaml as Y
|
||||
import Hakyll hiding (escapeHtml)
|
||||
import Contexts (siteCtx)
|
||||
import Utils (escapeHtml)
|
||||
|
||||
-- ---------------------------------------------------------------------------
|
||||
-- Loose scalars
|
||||
-- ---------------------------------------------------------------------------
|
||||
|
||||
-- | A YAML scalar that may be written as either a string or a number for the
|
||||
-- same logical field. @publications.yml@ has both @year: 2026@ (parsed as a
|
||||
-- number) and @year: "2026--2027"@ (a string); @experience.yml@ has
|
||||
-- @start: "2025"@ next to @start: July 2026@. Accept either and normalise
|
||||
-- to 'String' rather than forcing the YAML to be quoted consistently — the
|
||||
-- PDF pipeline does not care, and this module should not make it care.
|
||||
newtype Loose = Loose { unLoose :: String }
|
||||
|
||||
instance FromJSON Loose where
|
||||
parseJSON (String t) = pure (Loose (T.unpack t))
|
||||
parseJSON (Number n)
|
||||
| isInteger n = pure (Loose (show (truncate (toRealFloat n :: Double) :: Integer)))
|
||||
| otherwise = pure (Loose (show (toRealFloat n :: Double)))
|
||||
parseJSON (Bool b) = pure (Loose (if b then "true" else "false"))
|
||||
parseJSON v = typeMismatch "string or number" v
|
||||
|
||||
reqLoose :: Object -> String -> Parser String
|
||||
reqLoose o k = unLoose <$> o .: K.fromString k
|
||||
|
||||
optLoose :: Object -> String -> Parser (Maybe String)
|
||||
optLoose o k = fmap unLoose <$> o .:? K.fromString k
|
||||
|
||||
-- | Whether an entry appears on this page.
|
||||
--
|
||||
-- The YAML has carried two visibility axes since it drove two documents:
|
||||
-- @cv_visible@ and @resume_visible@, so the CV and the résumé can disagree
|
||||
-- about an entry without duplicating it. Generating the vita page from the
|
||||
-- same data added a third surface, and folding it into @cv_visible@ would
|
||||
-- have silently collapsed a distinction the file already knew how to make
|
||||
-- — an entry can be worth keeping on a document handed to a reader while
|
||||
-- being wrong for a page that is crawled.
|
||||
--
|
||||
-- @web_visible@ therefore defaults to @cv_visible@: existing entries behave
|
||||
-- exactly as before, and the axis only exists where someone sets it.
|
||||
webVisible :: Object -> Parser Bool
|
||||
webVisible o = do
|
||||
cv <- o .:? "cv_visible" .!= True
|
||||
o .:? "web_visible" .!= cv
|
||||
|
||||
-- ---------------------------------------------------------------------------
|
||||
-- Entry types
|
||||
-- ---------------------------------------------------------------------------
|
||||
|
||||
data Link = Link
|
||||
{ lkLabel :: String
|
||||
, lkHref :: String
|
||||
}
|
||||
|
||||
instance FromJSON Link where
|
||||
parseJSON = withObject "Link" $ \o -> Link
|
||||
<$> o .: "label"
|
||||
<*> o .: "href"
|
||||
|
||||
data Edu = Edu
|
||||
{ edInstitution :: String
|
||||
, edLocation :: Maybe String
|
||||
, edDegree :: String
|
||||
, edStart :: String
|
||||
, edEnd :: Maybe String
|
||||
, edGpa :: Maybe String
|
||||
, edNotes :: Maybe String
|
||||
, edWeb :: Bool
|
||||
}
|
||||
|
||||
instance FromJSON Edu where
|
||||
parseJSON = withObject "Edu" $ \o -> Edu
|
||||
<$> o .: "institution"
|
||||
<*> optLoose o "location"
|
||||
<*> reqLoose o "degree"
|
||||
<*> reqLoose o "start"
|
||||
<*> optLoose o "end"
|
||||
<*> optLoose o "gpa"
|
||||
<*> optLoose o "notes_cv"
|
||||
<*> webVisible o
|
||||
|
||||
newtype EduDoc = EduDoc { unEduDoc :: [Edu] }
|
||||
|
||||
instance FromJSON EduDoc where
|
||||
parseJSON = withObject "EduDoc" $ \o -> EduDoc <$> o .: "education"
|
||||
|
||||
data Pub = Pub
|
||||
{ pbAuthors :: String
|
||||
, pbTitle :: Maybe String
|
||||
, pbVenue :: String
|
||||
, pbYear :: String
|
||||
, pbMonth :: Maybe String
|
||||
, pbTarget :: Maybe String
|
||||
, pbLinks :: [Link]
|
||||
, pbNote :: Maybe String
|
||||
, pbWeb :: Bool
|
||||
}
|
||||
|
||||
instance FromJSON Pub where
|
||||
parseJSON = withObject "Pub" $ \o -> Pub
|
||||
<$> reqLoose o "authors"
|
||||
<*> optLoose o "title"
|
||||
<*> reqLoose o "venue"
|
||||
<*> reqLoose o "year"
|
||||
<*> optLoose o "month"
|
||||
<*> optLoose o "target"
|
||||
<*> o .:? "links" .!= []
|
||||
<*> optLoose o "equal_contrib_note"
|
||||
<*> webVisible o
|
||||
|
||||
newtype PubDoc = PubDoc { unPubDoc :: [Pub] }
|
||||
|
||||
instance FromJSON PubDoc where
|
||||
parseJSON = withObject "PubDoc" $ \o -> PubDoc <$> o .: "publications"
|
||||
|
||||
data Pres = Pres
|
||||
{ prAuthors :: String
|
||||
, prTitle :: String
|
||||
, prVenue :: String
|
||||
, prKind :: Maybe String
|
||||
, prYear :: String
|
||||
, prMonth :: Maybe String
|
||||
, prStatus :: Maybe String
|
||||
, prWeb :: Bool
|
||||
}
|
||||
|
||||
instance FromJSON Pres where
|
||||
parseJSON = withObject "Pres" $ \o -> Pres
|
||||
<$> reqLoose o "authors"
|
||||
<*> reqLoose o "title"
|
||||
<*> reqLoose o "venue"
|
||||
<*> optLoose o "kind"
|
||||
<*> reqLoose o "year"
|
||||
<*> optLoose o "month"
|
||||
<*> optLoose o "status"
|
||||
<*> webVisible o
|
||||
|
||||
newtype PresDoc = PresDoc { unPresDoc :: [Pres] }
|
||||
|
||||
instance FromJSON PresDoc where
|
||||
parseJSON = withObject "PresDoc" $ \o -> PresDoc <$> o .: "presentations"
|
||||
|
||||
data Exp = Exp
|
||||
{ exOrg :: String
|
||||
, exRole :: Maybe String
|
||||
, exLocation :: Maybe String
|
||||
, exLocUrl :: Maybe String
|
||||
, exStart :: String
|
||||
, exEnd :: Maybe String
|
||||
, exSection :: Maybe String
|
||||
, exOrder :: Int
|
||||
, exPreamble :: Maybe String
|
||||
, exBullets :: [String]
|
||||
, exWeb :: Bool
|
||||
}
|
||||
|
||||
instance FromJSON Exp where
|
||||
parseJSON = withObject "Exp" $ \o -> Exp
|
||||
<$> reqLoose o "organization"
|
||||
<*> optLoose o "role"
|
||||
<*> optLoose o "location"
|
||||
<*> optLoose o "location_url"
|
||||
<*> reqLoose o "start"
|
||||
<*> optLoose o "end"
|
||||
<*> optLoose o "cv_section"
|
||||
<*> o .:? "cv_order" .!= 99
|
||||
<*> optLoose o "cv_preamble"
|
||||
<*> o .:? "bullets" .!= []
|
||||
<*> webVisible o
|
||||
|
||||
newtype ExpDoc = ExpDoc { unExpDoc :: [Exp] }
|
||||
|
||||
instance FromJSON ExpDoc where
|
||||
parseJSON = withObject "ExpDoc" $ \o -> ExpDoc <$> o .: "experience"
|
||||
|
||||
data Proj = Proj
|
||||
{ pjName :: String
|
||||
, pjGroup :: String
|
||||
, pjEssay :: Maybe String
|
||||
, pjStart :: String
|
||||
, pjEnd :: Maybe String
|
||||
, pjDescription :: String
|
||||
, pjLinks :: [Link]
|
||||
, pjWeb :: Bool
|
||||
}
|
||||
|
||||
instance FromJSON Proj where
|
||||
parseJSON = withObject "Proj" $ \o -> Proj
|
||||
<$> reqLoose o "name"
|
||||
<*> o .:? "group" .!= "Projects"
|
||||
<*> optLoose o "essay"
|
||||
<*> reqLoose o "start"
|
||||
<*> optLoose o "end"
|
||||
<*> reqLoose o "description"
|
||||
<*> o .:? "links" .!= []
|
||||
<*> webVisible o
|
||||
|
||||
newtype ProjDoc = ProjDoc { unProjDoc :: [Proj] }
|
||||
|
||||
instance FromJSON ProjDoc where
|
||||
parseJSON = withObject "ProjDoc" $ \o -> ProjDoc <$> o .: "projects"
|
||||
|
||||
-- | @personal.yml@ also carries a @display@ string per link (the value the
|
||||
-- CV prints in full, since paper cannot be clicked). It is deliberately
|
||||
-- not read here — see 'renderContact'.
|
||||
data ProfileLink = ProfileLink
|
||||
{ plLabel :: String
|
||||
, plHref :: String
|
||||
, plWeb :: Bool
|
||||
}
|
||||
|
||||
instance FromJSON ProfileLink where
|
||||
parseJSON = withObject "ProfileLink" $ \o -> ProfileLink
|
||||
<$> reqLoose o "label"
|
||||
<*> reqLoose o "href"
|
||||
<*> webVisible o
|
||||
|
||||
-- | Contact details from @personal.yml@. The phone number is deliberately
|
||||
-- not parsed: it is printed on the CV PDF, which is a document handed to
|
||||
-- a chosen reader, whereas this page is crawled. Nothing here should hand
|
||||
-- a scraper a phone number it did not already have to go looking for.
|
||||
data Person = Person
|
||||
{ pnEmail :: String
|
||||
, pnLinks :: [ProfileLink]
|
||||
}
|
||||
|
||||
instance FromJSON Person where
|
||||
parseJSON = withObject "Person" $ \o -> Person
|
||||
<$> reqLoose o "email"
|
||||
<*> o .:? "links" .!= []
|
||||
|
||||
-- ---------------------------------------------------------------------------
|
||||
-- LaTeX → HTML
|
||||
-- ---------------------------------------------------------------------------
|
||||
|
||||
-- | Convert the LaTeX subset that actually appears in the CV YAML into HTML.
|
||||
--
|
||||
-- Callers must escape HTML /before/ calling this, never after: this
|
||||
-- function emits real tags, so escaping afterwards would turn them into
|
||||
-- visible @<strong>@. Escaping first is safe because none of the
|
||||
-- LaTeX constructs contain @<@, @>@ or @&@ — and an @&@ inside an
|
||||
-- @\\href@ URL becomes @&@, which is what an HTML attribute wants
|
||||
-- anyway.
|
||||
--
|
||||
-- The supported set is deliberately closed and matches what the YAML
|
||||
-- contains today (@\\textbf@, @\\textit@, @\\texttt@, @\\href@,
|
||||
-- @$\\times$@, @$\\delta$@, @\\#@, @{,}@, @~@, @--@, @---@). An unhandled
|
||||
-- command passes through verbatim and is therefore visible on the page —
|
||||
-- the intended failure mode, since a silently swallowed @\\emph@ would
|
||||
-- drop its argument's text.
|
||||
latexToHtml :: String -> String
|
||||
latexToHtml =
|
||||
substAll "---" "—"
|
||||
. substAll "--" "–"
|
||||
. substAll "$\\times$" "×"
|
||||
. substAll "$\\delta$" "δ"
|
||||
. substAll "$\\rightarrow$" "→"
|
||||
-- Approximation, not a non-breaking space. Bare `~` is LaTeX's nbsp, so
|
||||
-- "~10 crates" silently renders as "( 10 crates" and loses the "about".
|
||||
. substAll "$\\sim$" "~"
|
||||
. substAll "\\#" "#"
|
||||
. substAll "{,}" ","
|
||||
. substAll "~" " "
|
||||
. rewriteCmd2 "href" (\u t -> "<a href=\"" ++ u ++ "\">" ++ t ++ "</a>")
|
||||
. rewriteCmd1 "textbf" (\x -> "<strong>" ++ x ++ "</strong>")
|
||||
. rewriteCmd1 "textit" (\x -> "<em>" ++ x ++ "</em>")
|
||||
. rewriteCmd1 "texttt" (\x -> "<code>" ++ x ++ "</code>")
|
||||
|
||||
-- | Escape, then convert. The one-step form every renderer should use.
|
||||
tex :: String -> String
|
||||
tex = latexToHtml . escapeHtml
|
||||
|
||||
substAll :: String -> String -> String -> String
|
||||
substAll _ _ [] = []
|
||||
substAll pat rep s@(c:cs)
|
||||
| pat `isPrefixOf` s = rep ++ substAll pat rep (drop (length pat) s)
|
||||
| otherwise = c : substAll pat rep cs
|
||||
|
||||
-- | Split a leading @{...}@ group, tracking brace depth so nested groups
|
||||
-- survive. Returns the group's contents and whatever follows it.
|
||||
takeGroup :: String -> Maybe (String, String)
|
||||
takeGroup ('{':rest) = go (0 :: Int) "" rest
|
||||
where
|
||||
go _ _ [] = Nothing
|
||||
go d acc ('}':cs)
|
||||
| d == 0 = Just (reverse acc, cs)
|
||||
| otherwise = go (d - 1) ('}':acc) cs
|
||||
go d acc ('{':cs) = go (d + 1) ('{':acc) cs
|
||||
go d acc (c:cs) = go d (c:acc) cs
|
||||
takeGroup _ = Nothing
|
||||
|
||||
-- | Rewrite every @\\cmd{arg}@ with a function of its argument.
|
||||
rewriteCmd1 :: String -> (String -> String) -> String -> String
|
||||
rewriteCmd1 name f = go
|
||||
where
|
||||
marker = '\\' : name
|
||||
go [] = []
|
||||
go s@(c:cs)
|
||||
| marker `isPrefixOf` s
|
||||
, Just (arg, rest) <- takeGroup (drop (length marker) s)
|
||||
= f (go arg) ++ go rest
|
||||
| otherwise = c : go cs
|
||||
|
||||
-- | Rewrite every @\\cmd{a}{b}@ with a function of both arguments.
|
||||
rewriteCmd2 :: String -> (String -> String -> String) -> String -> String
|
||||
rewriteCmd2 name f = go
|
||||
where
|
||||
marker = '\\' : name
|
||||
go [] = []
|
||||
go s@(c:cs)
|
||||
| marker `isPrefixOf` s
|
||||
, Just (a, rest1) <- takeGroup (drop (length marker) s)
|
||||
, Just (b, rest2) <- takeGroup rest1
|
||||
= f a (go b) ++ go rest2
|
||||
| otherwise = c : go cs
|
||||
|
||||
-- ---------------------------------------------------------------------------
|
||||
-- Shared rendering pieces
|
||||
-- ---------------------------------------------------------------------------
|
||||
|
||||
-- | @start – end@, or just @start@ when the entry has no end.
|
||||
dateRange :: String -> Maybe String -> String
|
||||
dateRange s me = tex s ++ maybe "" (\e -> " – " ++ tex e) me
|
||||
|
||||
-- | Title and date on one baseline, title left and date flush right — the
|
||||
-- layout item-card.css already establishes for every other list on the
|
||||
-- site. An earlier version stacked the date on its own line below the
|
||||
-- title, which read as a stray indented fragment: @#markdownBody p + p@
|
||||
-- applies the essay prose indent of 1.5em to consecutive paragraphs, and a
|
||||
-- date is not prose.
|
||||
--
|
||||
-- Dates stay in a @span@ rather than a @time@ element on purpose. Half of
|
||||
-- them are "Fall 2024", "expected 2028", "Present" — no valid @datetime@
|
||||
-- value exists for those, and a @time@ without one is worse than no @time@.
|
||||
headerRow :: String -> String -> String
|
||||
headerRow titleHtml dates = concat
|
||||
[ "<div class=\"item-card-header\">"
|
||||
, "<h3 class=\"vita-entry-title\">", titleHtml, "</h3>"
|
||||
, "<span class=\"item-card-date\">", dates, "</span>"
|
||||
, "</div>"
|
||||
]
|
||||
|
||||
-- | The line under the header: role or degree, then whatever secondary facts
|
||||
-- the entry carries (location, GPA), middot-separated in a quieter ink so
|
||||
-- the role still leads.
|
||||
subLine :: Maybe String -> [String] -> String
|
||||
subLine mrole extras
|
||||
| null parts = ""
|
||||
| otherwise = "<p class=\"vita-role\">" ++ lead ++ trailing ++ "</p>"
|
||||
where
|
||||
parts = maybe [] (pure . tex) mrole ++ extras
|
||||
lead = head parts
|
||||
rest = tail parts
|
||||
trailing
|
||||
| null rest = ""
|
||||
| otherwise = "<span class=\"vita-quiet\"> · "
|
||||
++ intercalate " · " rest
|
||||
++ "</span>"
|
||||
|
||||
-- | A location, linked when the entry gives a URL for it.
|
||||
locationPart :: Maybe String -> Maybe String -> [String]
|
||||
locationPart Nothing _ = []
|
||||
locationPart (Just loc) murl = pure $ case murl of
|
||||
Just u -> "<a class=\"vita-location\" href=\"" ++ escapeHtml u ++ "\">" ++ tex loc ++ "</a>"
|
||||
Nothing -> tex loc
|
||||
|
||||
-- | Link chips. The affordance the print CV cannot offer: every artifact one
|
||||
-- click away, rather than a bracketed label the reader has to retype.
|
||||
renderLinks :: [Link] -> String
|
||||
renderLinks [] = ""
|
||||
renderLinks ls = concat
|
||||
[ "<p class=\"vita-links\">"
|
||||
, concatMap one ls
|
||||
, "</p>"
|
||||
]
|
||||
where
|
||||
one l = concat
|
||||
[ "<a class=\"vita-chip\" href=\"", escapeHtml (lkHref l), "\">"
|
||||
, tex (lkLabel l)
|
||||
, "</a>"
|
||||
]
|
||||
|
||||
renderBullets :: [String] -> String
|
||||
renderBullets [] = ""
|
||||
renderBullets bs = concat
|
||||
[ "<ul class=\"vita-bullets\">"
|
||||
, concatMap (\b -> "<li>" ++ tex b ++ "</li>") bs
|
||||
, "</ul>"
|
||||
]
|
||||
|
||||
section :: String -> String -> String -> String
|
||||
section slug heading inner = concat
|
||||
[ "<section class=\"vita-section library-section\" id=\"", slug, "\">"
|
||||
, "<h2 class=\"vita-section-heading\">", heading, "</h2>"
|
||||
, inner
|
||||
, "</section>"
|
||||
]
|
||||
|
||||
-- ---------------------------------------------------------------------------
|
||||
-- Section renderers
|
||||
-- ---------------------------------------------------------------------------
|
||||
|
||||
renderEducation :: [Edu] -> String
|
||||
renderEducation es
|
||||
| null visible = ""
|
||||
| otherwise = section "education" "Education" $ concat
|
||||
[ "<ul class=\"item-card-list vita-list\">"
|
||||
, concatMap one visible
|
||||
, "</ul>"
|
||||
]
|
||||
where
|
||||
visible = filter edWeb es
|
||||
one e = concat
|
||||
[ "<li class=\"item-card vita-card\">"
|
||||
, "<div class=\"item-card-main\">"
|
||||
, headerRow (tex (edInstitution e)) (dateRange (edStart e) (edEnd e))
|
||||
, subLine (Just (edDegree e))
|
||||
( maybe [] (\g -> ["GPA " ++ tex g]) (edGpa e)
|
||||
++ locationPart (edLocation e) Nothing
|
||||
)
|
||||
, maybe "" (\n -> "<p class=\"vita-note\">" ++ tex n ++ "</p>") (edNotes e)
|
||||
, "</div>"
|
||||
, "</li>"
|
||||
]
|
||||
|
||||
renderPublications :: [Pub] -> String
|
||||
renderPublications ps
|
||||
| null visible = ""
|
||||
| otherwise = section "publications" "Publications and Preprints" $ concat
|
||||
[ "<ul class=\"item-card-list vita-list\">"
|
||||
, concatMap one visible
|
||||
, "</ul>"
|
||||
, footnote
|
||||
]
|
||||
where
|
||||
visible = filter pbWeb ps
|
||||
-- The dagger legend lives on whichever entry declares it, but reads as a
|
||||
-- section-level note, so it is rendered once at the foot of the list.
|
||||
footnote = case mapMaybe pbNote visible of
|
||||
(n:_) -> "<p class=\"vita-footnote\">" ++ tex n ++ "</p>"
|
||||
[] -> ""
|
||||
dateOf p = tex (pbYear p) ++ maybe "" (\m -> ", " ++ tex m) (pbMonth p)
|
||||
one p = concat
|
||||
[ "<li class=\"item-card vita-card\">"
|
||||
, "<div class=\"item-card-main\">"
|
||||
, case pbTitle p of
|
||||
-- Entries without a title (work in preparation) put the label in
|
||||
-- the authors field; the CV template makes the same distinction.
|
||||
Nothing -> concat
|
||||
[ headerRow (tex (pbAuthors p)) (dateOf p)
|
||||
, "<p class=\"vita-venue\">", tex (pbVenue p)
|
||||
, maybe "" (\t -> " " ++ tex t) (pbTarget p)
|
||||
, "</p>"
|
||||
]
|
||||
Just t -> concat
|
||||
[ headerRow (tex t) (dateOf p)
|
||||
, "<p class=\"vita-authors\">", tex (pbAuthors p), "</p>"
|
||||
, "<p class=\"vita-venue\">", tex (pbVenue p), "</p>"
|
||||
]
|
||||
, renderLinks (pbLinks p)
|
||||
, "</div>"
|
||||
, "</li>"
|
||||
]
|
||||
|
||||
renderPresentations :: [Pres] -> String
|
||||
renderPresentations ps
|
||||
| null visible = ""
|
||||
| otherwise = section "presentations" "Presentations" $ concat
|
||||
[ "<ul class=\"item-card-list vita-list\">"
|
||||
, concatMap one visible
|
||||
, "</ul>"
|
||||
]
|
||||
where
|
||||
visible = filter prWeb ps
|
||||
dateOf p = maybe "" (\m -> tex m ++ " ") (prMonth p) ++ tex (prYear p)
|
||||
one p = concat
|
||||
[ "<li class=\"item-card vita-card\">"
|
||||
, "<div class=\"item-card-main\">"
|
||||
, headerRow (tex (prTitle p)) (dateOf p)
|
||||
, "<p class=\"vita-authors\">", tex (prAuthors p), "</p>"
|
||||
, "<p class=\"vita-venue\">"
|
||||
, maybe "" (\k -> tex k ++ ", ") (prKind p)
|
||||
, tex (prVenue p)
|
||||
, maybe "" (\st -> "<span class=\"vita-status\">" ++ tex st ++ "</span>") (prStatus p)
|
||||
, "</p>"
|
||||
, "</div>"
|
||||
, "</li>"
|
||||
]
|
||||
|
||||
-- | Experience keeps the CV's research/industry split — that division is
|
||||
-- editorial, lives in @cv_section@, and a homepage that merged the two
|
||||
-- would say something the PDF does not.
|
||||
renderExperience :: [Exp] -> String
|
||||
renderExperience xs = research ++ industry
|
||||
where
|
||||
visible = sortOn exOrder (filter exWeb xs)
|
||||
isRes e = exSection e == Just "research"
|
||||
research = group "experience-research" "Research Experience" (filter isRes visible)
|
||||
industry = group "experience-industry" "Industry Experience" (filter (not . isRes) visible)
|
||||
group slug heading es
|
||||
| null es = ""
|
||||
| otherwise = section slug heading $ concat
|
||||
[ "<ul class=\"item-card-list vita-list\">"
|
||||
, concatMap one es
|
||||
, "</ul>"
|
||||
]
|
||||
one e = concat
|
||||
[ "<li class=\"item-card vita-card\">"
|
||||
, "<div class=\"item-card-main\">"
|
||||
, headerRow (tex (exOrg e)) (dateRange (exStart e) (exEnd e))
|
||||
, subLine (exRole e) (locationPart (exLocation e) (exLocUrl e))
|
||||
, maybe "" (\p -> "<p class=\"vita-note\">" ++ tex p ++ "</p>") (exPreamble e)
|
||||
, renderBullets (exBullets e)
|
||||
, "</div>"
|
||||
, "</li>"
|
||||
]
|
||||
|
||||
-- | The @/cv/projects/@ index. Groups render in first-appearance order, the
|
||||
-- same convention "Now" uses for its sections — reordering the YAML
|
||||
-- reorders the page and no separate ordering key is needed.
|
||||
--
|
||||
-- Entry titles link to the project's essay where one exists. The essays are
|
||||
-- deliberately not generated: a writeup is an informal presentation of a
|
||||
-- project, not a record of it, and belongs in the same voice as the rest of
|
||||
-- the essays.
|
||||
renderProjects :: [Proj] -> String
|
||||
renderProjects ps = concatMap one (groupOrder visible)
|
||||
where
|
||||
visible = filter pjWeb ps
|
||||
groupOrder = foldl (\acc g -> if g `elem` acc then acc else acc ++ [g]) []
|
||||
. map pjGroup
|
||||
-- "Machine Learning & Deployed" → "machine-learning-deployed".
|
||||
slugify s = case foldr step [] s of
|
||||
('-':rest) -> rest
|
||||
cleaned -> cleaned
|
||||
where
|
||||
step c acc
|
||||
| c `elem` (['a'..'z'] ++ ['0'..'9']) = c : acc
|
||||
| c `elem` ['A'..'Z'] = toLower c : acc
|
||||
| null acc || head acc == '-' = acc
|
||||
| otherwise = '-' : acc
|
||||
one g = section ("projects-" ++ slugify g) (escapeHtml g) $ concat
|
||||
[ "<ul class=\"item-card-list vita-list\">"
|
||||
, concatMap entry (filter ((== g) . pjGroup) visible)
|
||||
, "</ul>"
|
||||
]
|
||||
entry p = concat
|
||||
[ "<li class=\"item-card vita-card\">"
|
||||
, "<div class=\"item-card-main\">"
|
||||
, headerRow
|
||||
(case pjEssay p of
|
||||
Just u -> "<a href=\"" ++ escapeHtml u ++ "\">" ++ tex (pjName p) ++ "</a>"
|
||||
Nothing -> tex (pjName p))
|
||||
(dateRange (pjStart p) (pjEnd p))
|
||||
, "<p class=\"vita-note\">", tex (pjDescription p), "</p>"
|
||||
, renderLinks (pjLinks p)
|
||||
, "</div>"
|
||||
, "</li>"
|
||||
]
|
||||
|
||||
renderContact :: Person -> String
|
||||
renderContact p = section "contact" "Contact" $ concat
|
||||
[ "<p class=\"vita-links\">"
|
||||
, "<a class=\"vita-chip\" href=\"mailto:", escapeHtml (pnEmail p), "\">"
|
||||
, escapeHtml (pnEmail p)
|
||||
, "</a>"
|
||||
, concatMap one (filter plWeb (pnLinks p))
|
||||
, "</p>"
|
||||
]
|
||||
where
|
||||
-- Chips carry the label, not personal.yml's `display` value. The CV
|
||||
-- prints "ORCID: 0009-0002-0162-3587" because a printed page cannot be
|
||||
-- clicked; a chip reading "0009-0002-0162-3587" alone identifies
|
||||
-- nothing, and "github.com/levineuwirth" is a URL doing a label's job.
|
||||
one l = concat
|
||||
[ "<a class=\"vita-chip\" href=\"", escapeHtml (plHref l), "\">"
|
||||
, tex (plLabel l)
|
||||
, "</a>"
|
||||
]
|
||||
|
||||
-- ---------------------------------------------------------------------------
|
||||
-- Load
|
||||
-- ---------------------------------------------------------------------------
|
||||
|
||||
-- | Same UTF-8 round-trip as "Now": Hakyll hands back a 'String' of Unicode
|
||||
-- codepoints and the yaml library wants a UTF-8 'ByteString'.
|
||||
-- 'Data.ByteString.Char8.pack' would truncate every 'Char' to 8 bits and
|
||||
-- silently mangle the em-dashes and daggers this data is full of.
|
||||
loadYaml :: FromJSON a => FilePath -> Compiler a
|
||||
loadYaml path = do
|
||||
raw <- load (fromFilePath path) :: Compiler (Item String)
|
||||
case Y.decodeEither' (TE.encodeUtf8 (T.pack (itemBody raw))) of
|
||||
Left err -> fail (path ++ ": " ++ show err)
|
||||
Right doc -> return doc
|
||||
|
||||
-- | Render a section, or drop the field entirely when it comes out empty so
|
||||
-- the template's @$if(...)$@ guards behave.
|
||||
sectionField :: String -> Compiler String -> Context String
|
||||
sectionField name gen = field name $ \_ -> do
|
||||
html <- gen
|
||||
if null html then noResult (name ++ ": empty") else return html
|
||||
|
||||
-- ---------------------------------------------------------------------------
|
||||
-- Context
|
||||
-- ---------------------------------------------------------------------------
|
||||
|
||||
vitaCtx :: Context String
|
||||
vitaCtx =
|
||||
constField "vita" "true"
|
||||
<> sectionField "vita-education-html"
|
||||
(renderEducation . unEduDoc <$> loadYaml "yaml-source/data/education.yml")
|
||||
<> sectionField "vita-publications-html"
|
||||
(renderPublications . unPubDoc <$> loadYaml "yaml-source/data/publications.yml")
|
||||
<> sectionField "vita-presentations-html"
|
||||
(renderPresentations . unPresDoc <$> loadYaml "yaml-source/data/presentations.yml")
|
||||
<> sectionField "vita-experience-html"
|
||||
(renderExperience . unExpDoc <$> loadYaml "yaml-source/data/experience.yml")
|
||||
<> sectionField "vita-contact-html"
|
||||
(renderContact <$> loadYaml "yaml-source/data/personal.yml")
|
||||
<> siteCtx
|
||||
|
||||
-- | The @/cv/projects/@ index. Reuses the vita flag so it picks up the same
|
||||
-- stylesheets and reads as the same kind of surface.
|
||||
projectsCtx :: Context String
|
||||
projectsCtx =
|
||||
constField "vita" "true"
|
||||
<> sectionField "vita-projects-html"
|
||||
(renderProjects . unProjDoc <$> loadYaml "yaml-source/data/projects.yml")
|
||||
<> siteCtx
|
||||
|
|
@ -0,0 +1,8 @@
|
|||
packages: .
|
||||
|
||||
with-compiler: ghc-9.6.6
|
||||
|
||||
-- Optimise the build program itself. -O1 is sufficient and much faster
|
||||
-- to compile than -O2.
|
||||
program-options
|
||||
ghc-options: -O1
|
||||
|
|
@ -0,0 +1,317 @@
|
|||
active-repositories: hackage.haskell.org:merge
|
||||
constraints: any.Cabal ==3.10.3.0,
|
||||
any.Cabal-syntax ==3.10.3.0,
|
||||
any.Glob ==0.10.2,
|
||||
any.HUnit ==1.6.2.0,
|
||||
any.JuicyPixels ==3.3.9,
|
||||
JuicyPixels -mmap,
|
||||
any.OneTuple ==0.4.3,
|
||||
OneTuple +base-ge-4-15 +base-ge-4-16,
|
||||
any.Only ==0.1,
|
||||
any.QuickCheck ==2.18.0.0,
|
||||
QuickCheck -old-random +templatehaskell,
|
||||
any.StateVar ==1.2.2,
|
||||
any.aeson ==2.2.5.0,
|
||||
aeson +ordered-keymap,
|
||||
any.aeson-pretty ==0.8.11,
|
||||
aeson-pretty -lib-only,
|
||||
any.alex ==3.5.4.2,
|
||||
any.ansi-terminal ==1.1.5,
|
||||
ansi-terminal -example,
|
||||
any.ansi-terminal-types ==1.1.3,
|
||||
any.appar ==0.1.8,
|
||||
any.array ==0.5.6.0,
|
||||
any.asn1-encoding ==0.9.6,
|
||||
any.asn1-parse ==0.9.5,
|
||||
any.asn1-types ==0.3.4,
|
||||
any.assoc ==1.1.1,
|
||||
assoc -tagged,
|
||||
any.async ==2.2.6,
|
||||
async -bench -debug-auto-label,
|
||||
any.attoparsec ==0.14.4,
|
||||
attoparsec -developer,
|
||||
any.attoparsec-aeson ==2.2.2.0,
|
||||
any.auto-update ==0.2.6,
|
||||
any.base ==4.18.2.1,
|
||||
any.base-compat ==0.15.0,
|
||||
any.base-orphans ==0.9.4,
|
||||
any.base16-bytestring ==1.0.2.0,
|
||||
any.base64-bytestring ==1.2.1.0,
|
||||
any.basement ==0.0.16,
|
||||
any.bifunctors ==5.6.3,
|
||||
bifunctors +tagged,
|
||||
any.binary ==0.8.9.1,
|
||||
any.bitvec ==1.1.6.0,
|
||||
bitvec +simd,
|
||||
any.blaze-builder ==0.4.4.1,
|
||||
any.blaze-html ==0.9.2.0,
|
||||
any.blaze-markup ==0.8.3.0,
|
||||
any.bsb-http-chunked ==0.0.0.4,
|
||||
any.byteorder ==1.0.4,
|
||||
any.bytestring ==0.11.5.3,
|
||||
any.cabal-doctest ==1.0.12,
|
||||
any.call-stack ==0.4.0,
|
||||
any.case-insensitive ==1.2.1.0,
|
||||
any.cassava ==0.5.4.1,
|
||||
any.cborg ==0.2.10.0,
|
||||
cborg +optimize-gmp,
|
||||
any.cereal ==0.5.8.3,
|
||||
cereal -bytestring-builder,
|
||||
any.character-ps ==0.1,
|
||||
any.citeproc ==0.8.1.3,
|
||||
citeproc -executable -icu,
|
||||
any.cmdargs ==0.10.22,
|
||||
cmdargs +quotation -testprog,
|
||||
any.colour ==2.3.7,
|
||||
any.commonmark ==0.2.6.1,
|
||||
any.commonmark-extensions ==0.2.7.1,
|
||||
any.commonmark-pandoc ==0.2.3,
|
||||
any.comonad ==5.0.10,
|
||||
comonad +containers +distributive +indexed-traversable,
|
||||
any.conduit ==1.3.6.1,
|
||||
any.conduit-extra ==1.3.8,
|
||||
any.containers ==0.6.7,
|
||||
any.contravariant ==1.5.6,
|
||||
contravariant +statevar,
|
||||
any.cookie ==0.5.1,
|
||||
any.cryptohash-md5 ==0.11.101.0,
|
||||
any.crypton ==1.0.6,
|
||||
crypton -check_alignment +integer-gmp -old_toolchain_inliner +support_aesni +support_deepseq +support_pclmuldq +support_rdrand -support_sse +use_target_attributes,
|
||||
any.crypton-connection ==0.4.5,
|
||||
any.crypton-socks ==0.6.2,
|
||||
crypton-socks -example +network-3-0-0-0,
|
||||
any.crypton-x509 ==1.7.7,
|
||||
any.crypton-x509-store ==1.6.14,
|
||||
any.crypton-x509-system ==1.6.8,
|
||||
any.crypton-x509-validation ==1.6.14,
|
||||
any.data-default ==0.7.1.3,
|
||||
any.data-default-class ==0.1.2.2,
|
||||
any.data-default-instances-containers ==0.1.0.3,
|
||||
any.data-default-instances-dlist ==0.0.1.2,
|
||||
any.data-default-instances-old-locale ==0.0.1.2,
|
||||
any.data-fix ==0.3.4,
|
||||
any.deepseq ==1.4.8.1,
|
||||
any.digest ==0.0.2.1,
|
||||
digest -have_arm64_crc32c -have_builtin_prefetch -have_mm_prefetch -have_sse42 -have_strong_getauxval -have_weak_getauxval +pkg-config,
|
||||
any.directory ==1.3.8.5,
|
||||
any.distributive ==0.6.3,
|
||||
distributive +tagged,
|
||||
any.djot ==0.1.4.1,
|
||||
any.dlist ==1.0,
|
||||
dlist -werror,
|
||||
any.doclayout ==0.5.0.3,
|
||||
any.doctemplates ==0.11.0.1,
|
||||
any.easy-file ==0.2.5,
|
||||
any.ech-config ==0.0.1,
|
||||
ech-config -devel,
|
||||
any.emojis ==0.1.5,
|
||||
any.exceptions ==0.10.7,
|
||||
any.fast-logger ==3.2.6,
|
||||
any.file-embed ==0.0.16.0,
|
||||
any.filepath ==1.4.300.1,
|
||||
any.fsnotify ==0.4.4.0,
|
||||
any.ghc-bignum ==1.3,
|
||||
any.ghc-boot-th ==9.6.6,
|
||||
any.ghc-prim ==0.10.0,
|
||||
any.gridtables ==0.1.1.0,
|
||||
any.haddock-library ==1.11.0,
|
||||
any.hakyll ==4.16.7.1,
|
||||
hakyll -buildwebsite +checkexternal +previewserver +usepandoc +watchserver,
|
||||
any.half ==0.3.3,
|
||||
any.happy ==2.2,
|
||||
any.happy-lib ==2.2,
|
||||
any.hashable ==1.5.1.0,
|
||||
hashable -arch-native -random-initial-seed,
|
||||
any.haskell-lexer ==1.2.1,
|
||||
any.haskell-src-exts ==1.24.0,
|
||||
any.haskell-src-meta ==0.8.16,
|
||||
any.hinotify ==0.4.2,
|
||||
any.hourglass ==0.2.12,
|
||||
any.hpke ==0.0.0,
|
||||
any.hsc2hs ==0.68.10,
|
||||
hsc2hs -in-ghc-tree,
|
||||
any.http-client ==0.7.19,
|
||||
http-client +network-uri,
|
||||
any.http-client-tls ==0.3.6.4,
|
||||
any.http-conduit ==2.3.9.1,
|
||||
http-conduit +aeson,
|
||||
any.http-date ==0.0.11,
|
||||
any.http-semantics ==0.4.1,
|
||||
any.http-types ==0.12.5,
|
||||
any.http2 ==5.4.3,
|
||||
http2 -devel -h2spec,
|
||||
any.indexed-traversable ==0.1.5,
|
||||
indexed-traversable +base-ge-4-18,
|
||||
any.indexed-traversable-instances ==0.1.2.1,
|
||||
any.integer-conversion ==0.1.1,
|
||||
any.integer-gmp ==1.1,
|
||||
any.integer-logarithms ==1.0.5,
|
||||
integer-logarithms -check-bounds +integer-gmp,
|
||||
any.iproute ==1.7.15,
|
||||
any.ipynb ==0.2,
|
||||
any.jira-wiki-markup ==1.5.1,
|
||||
any.libyaml ==0.1.4,
|
||||
libyaml -no-unicode -system-libyaml,
|
||||
any.libyaml-clib ==0.2.5,
|
||||
any.lifted-base ==0.2.3.12,
|
||||
any.lrucache ==1.2.0.1,
|
||||
any.memory ==0.18.0,
|
||||
memory +support_bytestring +support_deepseq,
|
||||
any.mime-types ==0.1.2.2,
|
||||
any.monad-control ==1.0.3.1,
|
||||
any.monad-logger ==0.3.42,
|
||||
monad-logger +template_haskell,
|
||||
any.monad-loops ==0.4.3,
|
||||
monad-loops +base4,
|
||||
any.mono-traversable ==1.0.21.0,
|
||||
any.mtl ==2.3.1,
|
||||
any.mtl-compat ==0.2.2,
|
||||
mtl-compat -two-point-one -two-point-two,
|
||||
any.network ==3.2.8.0,
|
||||
network -devel,
|
||||
any.network-byte-order ==0.1.8,
|
||||
any.network-control ==0.1.7,
|
||||
any.network-uri ==2.6.4.2,
|
||||
any.old-locale ==1.0.0.7,
|
||||
any.old-time ==1.1.1.0,
|
||||
any.optparse-applicative ==0.19.0.0,
|
||||
optparse-applicative +process,
|
||||
any.ordered-containers ==0.2.4,
|
||||
any.os-string ==2.0.11,
|
||||
any.pandoc ==3.6.4,
|
||||
pandoc -embed_data_files,
|
||||
any.pandoc-types ==1.23.1.2,
|
||||
any.parsec ==3.1.16.1,
|
||||
any.pem ==0.2.4,
|
||||
any.pretty ==1.1.3.6,
|
||||
any.pretty-show ==1.10,
|
||||
any.prettyprinter ==1.7.2,
|
||||
prettyprinter -buildreadme +text,
|
||||
any.prettyprinter-ansi-terminal ==1.1.4,
|
||||
prettyprinter-ansi-terminal +text,
|
||||
any.primitive ==0.9.1.0,
|
||||
any.process ==1.6.19.0,
|
||||
any.psqueues ==0.2.8.3,
|
||||
any.random ==1.3.1,
|
||||
any.recv ==0.1.1,
|
||||
any.regex-base ==0.94.0.3,
|
||||
any.regex-tdfa ==1.3.2.5,
|
||||
regex-tdfa +doctest -force-o2,
|
||||
any.resourcet ==1.3.0,
|
||||
any.retry ==0.9.3.1,
|
||||
retry -lib-werror,
|
||||
any.rts ==1.0.2,
|
||||
any.safe ==0.3.21,
|
||||
any.safe-exceptions ==0.1.7.4,
|
||||
any.scientific ==0.3.8.1,
|
||||
scientific -integer-simple,
|
||||
any.semialign ==1.4,
|
||||
semialign +semigroupoids,
|
||||
any.semigroupoids ==6.0.2,
|
||||
semigroupoids +comonad +containers +contravariant +tagged +unordered-containers,
|
||||
any.serialise ==0.2.6.1,
|
||||
serialise +newtime15,
|
||||
any.simple-sendfile ==0.2.32,
|
||||
simple-sendfile +allow-bsd -fallback,
|
||||
any.skylighting ==0.14.7,
|
||||
skylighting -executable,
|
||||
any.skylighting-core ==0.14.7,
|
||||
skylighting-core -executable,
|
||||
any.skylighting-format-ansi ==0.1,
|
||||
any.skylighting-format-blaze-html ==0.1.2.1,
|
||||
any.skylighting-format-context ==0.1.0.2,
|
||||
any.skylighting-format-latex ==0.1,
|
||||
any.skylighting-format-typst ==0.1,
|
||||
any.split ==0.2.5,
|
||||
any.splitmix ==0.1.3.2,
|
||||
splitmix -optimised-mixer,
|
||||
any.stm ==2.5.1.0,
|
||||
any.stm-chans ==3.0.0.11,
|
||||
any.streaming-commons ==0.2.3.1,
|
||||
streaming-commons -use-bytestring-builder,
|
||||
any.strict ==0.5.1,
|
||||
any.string-interpolate ==0.3.4.0,
|
||||
string-interpolate -bytestring-builder -extended-benchmarks -text-builder,
|
||||
any.syb ==0.7.4,
|
||||
any.tagged ==0.8.10,
|
||||
tagged +deepseq +template-haskell,
|
||||
any.tagsoup ==0.14.8,
|
||||
any.tasty ==1.5.4,
|
||||
tasty +unix,
|
||||
any.template-haskell ==2.20.0.0,
|
||||
any.temporary ==1.3,
|
||||
any.texmath ==0.12.10.1,
|
||||
texmath -executable -server,
|
||||
any.text ==2.0.2,
|
||||
any.text-conversions ==0.3.1.1,
|
||||
any.text-iso8601 ==0.1.1.1,
|
||||
any.text-short ==0.1.6.1,
|
||||
text-short -asserts,
|
||||
any.th-abstraction ==0.7.2.0,
|
||||
any.th-compat ==0.1.7,
|
||||
any.th-expand-syns ==0.4.12.0,
|
||||
any.th-lift ==0.8.7,
|
||||
any.th-lift-instances ==0.1.20,
|
||||
any.th-orphans ==0.13.17,
|
||||
any.th-reify-many ==0.1.10,
|
||||
any.these ==1.2.1,
|
||||
any.time ==1.12.2,
|
||||
any.time-compat ==1.9.9,
|
||||
any.time-locale-compat ==0.1.1.5,
|
||||
time-locale-compat -old-locale,
|
||||
any.time-manager ==0.3.2,
|
||||
any.tls ==2.1.14,
|
||||
tls -devel,
|
||||
any.toml-parser ==2.0.2.0,
|
||||
any.transformers ==0.6.1.0,
|
||||
any.transformers-base ==0.4.6.1,
|
||||
transformers-base +orphaninstances,
|
||||
any.transformers-compat ==0.7.2,
|
||||
transformers-compat -five +five-three -four +generic-deriving +mtl -three -two,
|
||||
any.typed-process ==0.2.13.0,
|
||||
any.typst ==0.7,
|
||||
typst -executable,
|
||||
any.typst-symbols ==0.1.7,
|
||||
any.unicode-collation ==0.1.3.7,
|
||||
unicode-collation -doctests -executable -icu-benchmark,
|
||||
any.unicode-data ==0.8.0,
|
||||
unicode-data -dev-has-icu,
|
||||
any.unicode-transforms ==0.4.0.1,
|
||||
unicode-transforms -bench-show -dev -has-icu -has-llvm -use-gauge,
|
||||
any.uniplate ==1.6.13,
|
||||
any.unix ==2.8.4.0,
|
||||
any.unix-compat ==0.7.4.1,
|
||||
any.unix-time ==0.4.17,
|
||||
any.unliftio ==0.2.25.1,
|
||||
any.unliftio-core ==0.2.1.0,
|
||||
any.unordered-containers ==0.2.21,
|
||||
unordered-containers -debug,
|
||||
any.utf8-string ==1.0.2,
|
||||
any.uuid-types ==1.0.6.1,
|
||||
any.vault ==0.3.2.0,
|
||||
vault +useghc,
|
||||
any.vector ==0.13.2.0,
|
||||
vector +boundschecks -internalchecks -unsafechecks -wall,
|
||||
any.vector-algorithms ==0.9.1.0,
|
||||
vector-algorithms +bench +boundschecks -internalchecks -llvm -unsafechecks,
|
||||
any.vector-stream ==0.1.0.1,
|
||||
any.wai ==3.2.4,
|
||||
any.wai-app-static ==3.2.1,
|
||||
wai-app-static -print,
|
||||
any.wai-extra ==3.1.18,
|
||||
wai-extra -build-example,
|
||||
any.wai-logger ==2.5.0,
|
||||
any.warp ==3.4.14,
|
||||
warp +allow-sendfilefd +include-warp-version -network-bytestring -warp-debug +x509,
|
||||
any.witherable ==0.5,
|
||||
any.word8 ==0.1.3,
|
||||
any.xml ==1.3.14,
|
||||
any.xml-conduit ==1.10.1.0,
|
||||
any.xml-types ==0.3.8,
|
||||
any.yaml ==0.11.11.2,
|
||||
yaml +no-examples +no-exe,
|
||||
any.zip-archive ==0.4.3.2,
|
||||
zip-archive -executable,
|
||||
any.zlib ==0.7.1.1,
|
||||
zlib -bundled-c-zlib +non-blocking-ffi +pkg-config
|
||||
index-state: hackage.haskell.org 2026-07-31T23:07:19Z
|
||||
|
|
@ -0,0 +1,25 @@
|
|||
---
|
||||
title: Levi Neuwirth — Vita
|
||||
tags: meta
|
||||
---
|
||||
|
||||
The formal record. For a less formal, more detailed introduction to who I am, see [[Me]]; for what I am actively working on this month, see [Current](/current.html).
|
||||
|
||||
## Documents
|
||||
|
||||
- **[Curriculum Vitae (PDF)](/cv.pdf)** — the complete record, including grants, affiliations, languages, and technical skills.
|
||||
- **[Resume (PDF)](/resume.pdf)** — one page, engineering-facing.
|
||||
- **[Project index](/cv/projects/)** — engineering artifacts in depth, with links to writeups and source.
|
||||
|
||||
The sections below are generated from the same data as the two PDFs, so they cannot fall out of step with them.
|
||||
|
||||
## Research Interests
|
||||
|
||||
My work clusters into four threads:
|
||||
|
||||
- **AI safety and applied AI** — zero-knowledge proofs for cryptographic verification of large language models, as a MARS V fellow with the [Cambridge AI Safety Hub](https://caish.org/mars) (mentored by James Petrie, Future of Life Institute); a Magic: The Gathering reinforcement-learning project; and reasoning, evaluation, and red-teaming research contracts.
|
||||
- **Mathematics** — graph theory, number theory, and theoretical computer science. Public results so far are graph-theory-centered: static coverage and persistence in tree-ball geometry ([preprint](/essays/branch-based-local-capture-in-tree-balls/)), and the annealed critical window for growing-radius domination in random regular graphs ([preprint](/essays/near-critical-growing-radius-domination.html)).
|
||||
- **Computer systems and high-performance computing** — the Weenix kernel, a TCP/IP networking stack from scratch in Go, and micro-architectural performance work (SIMD, hardware counters via PAPI, RAPL energy, cross-ISA ports across AVX2 / ARM NEON-SVE / RISC-V V) on Brown's OSCAR HPC cluster.
|
||||
- **Machine learning** — order-invariant ICD-10-CM embeddings (under review at *JAMIA*, deployed calculator), the NeuroPose 3D-kinematics system in Liqi Shu's lab at Brown Neurology, and ongoing research engineering at [NeuroAI](https://neuroai.health). Undergraduate work has been clinically focused; graduate study broadens the scope.
|
||||
|
||||
Computer vision and security thread through all four but do not stand on their own.
|
||||
|
|
@ -0,0 +1,81 @@
|
|||
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 280 280" role="img" aria-labelledby="mark-title-build">
|
||||
<title id="mark-title-build">A vertical compilation pipeline rendered as a small DAG, with a clock-face fragment in the upper corner</title>
|
||||
<desc>A frontispiece mark for the Build telemetry page. Three source nodes at top funnel through a filter stage, narrow to a single canonical AST, then expand to four output artifacts at the bottom. A small arc with a single tick mark in the upper-left of the inner field is a clock-face fragment, an unobtrusive nod to the build-timing aspect of the page.</desc>
|
||||
|
||||
<circle cx="140" cy="140" r="128" stroke="currentColor" stroke-width="0.6" fill="none"/>
|
||||
|
||||
<g stroke="currentColor" fill="none" stroke-linecap="round" stroke-linejoin="round">
|
||||
<path d="M 50 70 A 10 10 0 0 1 60 60" stroke-width="0.7"/>
|
||||
<line x1="50" y1="65" x2="52" y2="65" stroke-width="0.6"/>
|
||||
<line x1="56" y1="60" x2="56" y2="62" stroke-width="0.6"/>
|
||||
</g>
|
||||
|
||||
<g stroke="currentColor" fill="none" stroke-linecap="round" stroke-linejoin="round">
|
||||
<line x1="92" y1="80" x2="120" y2="116" stroke-width="0.7"/>
|
||||
<line x1="140" y1="80" x2="140" y2="116" stroke-width="0.7"/>
|
||||
<line x1="188" y1="80" x2="160" y2="116" stroke-width="0.7"/>
|
||||
|
||||
<line x1="120" y1="124" x2="140" y2="148" stroke-width="0.7"/>
|
||||
<line x1="140" y1="124" x2="140" y2="148" stroke-width="0.7"/>
|
||||
<line x1="160" y1="124" x2="140" y2="148" stroke-width="0.7"/>
|
||||
|
||||
<line x1="140" y1="160" x2="140" y2="180" stroke-width="1.0"/>
|
||||
|
||||
<line x1="140" y1="190" x2="80" y2="222" stroke-width="0.6"/>
|
||||
<line x1="140" y1="190" x2="115" y2="222" stroke-width="0.6"/>
|
||||
<line x1="140" y1="190" x2="165" y2="222" stroke-width="0.6"/>
|
||||
<line x1="140" y1="190" x2="200" y2="222" stroke-width="0.6"/>
|
||||
</g>
|
||||
|
||||
<g stroke="currentColor" fill="none" stroke-linecap="round" stroke-linejoin="miter">
|
||||
<rect x="84" y="72" width="16" height="14" stroke-width="0.9"/>
|
||||
<rect x="132" y="72" width="16" height="14" stroke-width="0.9"/>
|
||||
<rect x="180" y="72" width="16" height="14" stroke-width="0.9"/>
|
||||
|
||||
<line x1="87" y1="78" x2="97" y2="78" stroke-width="0.4" opacity="0.7"/>
|
||||
<line x1="87" y1="82" x2="93" y2="82" stroke-width="0.4" opacity="0.7"/>
|
||||
<line x1="135" y1="78" x2="145" y2="78" stroke-width="0.4" opacity="0.7"/>
|
||||
<line x1="135" y1="82" x2="142" y2="82" stroke-width="0.4" opacity="0.7"/>
|
||||
<line x1="183" y1="78" x2="193" y2="78" stroke-width="0.4" opacity="0.7"/>
|
||||
<line x1="183" y1="82" x2="190" y2="82" stroke-width="0.4" opacity="0.7"/>
|
||||
</g>
|
||||
|
||||
<g stroke="currentColor" fill="none" stroke-linejoin="miter" stroke-linecap="round">
|
||||
<line x1="106" y1="120" x2="174" y2="120" stroke-width="0.4" opacity="0.55"/>
|
||||
<line x1="120" y1="120" x2="120" y2="124" stroke-width="0.7"/>
|
||||
<line x1="140" y1="120" x2="140" y2="124" stroke-width="0.7"/>
|
||||
<line x1="160" y1="120" x2="160" y2="124" stroke-width="0.7"/>
|
||||
</g>
|
||||
|
||||
<g fill="currentColor" stroke="none">
|
||||
<circle cx="120" cy="120" r="1.6"/>
|
||||
<circle cx="140" cy="120" r="1.6"/>
|
||||
<circle cx="160" cy="120" r="1.6"/>
|
||||
</g>
|
||||
|
||||
<circle cx="140" cy="154" r="6" stroke="currentColor" stroke-width="1.2" fill="none"/>
|
||||
<circle cx="140" cy="154" r="2" fill="currentColor" stroke="none"/>
|
||||
|
||||
<g stroke="currentColor" fill="none" stroke-linejoin="miter" stroke-linecap="butt">
|
||||
<rect x="72" y="222" width="16" height="14" stroke-width="0.7"/>
|
||||
<rect x="107" y="222" width="16" height="14" stroke-width="0.7"/>
|
||||
<rect x="157" y="222" width="16" height="14" stroke-width="0.7"/>
|
||||
<rect x="192" y="222" width="16" height="14" stroke-width="0.7"/>
|
||||
|
||||
<line x1="80" y1="222" x2="80" y2="220" stroke-width="0.6"/>
|
||||
<line x1="115" y1="222" x2="115" y2="220" stroke-width="0.6"/>
|
||||
<line x1="165" y1="222" x2="165" y2="220" stroke-width="0.6"/>
|
||||
<line x1="200" y1="222" x2="200" y2="220" stroke-width="0.6"/>
|
||||
</g>
|
||||
|
||||
<g stroke="currentColor" fill="none" stroke-width="0.4" opacity="0.7" stroke-linecap="round">
|
||||
<line x1="75" y1="227" x2="85" y2="227"/>
|
||||
<line x1="75" y1="231" x2="83" y2="231"/>
|
||||
<line x1="110" y1="227" x2="120" y2="227"/>
|
||||
<line x1="110" y1="231" x2="118" y2="231"/>
|
||||
<line x1="160" y1="227" x2="170" y2="227"/>
|
||||
<line x1="160" y1="231" x2="168" y2="231"/>
|
||||
<line x1="195" y1="227" x2="205" y2="227"/>
|
||||
<line x1="195" y1="231" x2="203" y2="231"/>
|
||||
</g>
|
||||
</svg>
|
||||
|
After Width: | Height: | Size: 4.3 KiB |
|
|
@ -0,0 +1,91 @@
|
|||
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 280 280" role="img" aria-labelledby="mark-title-colophon">
|
||||
<title id="mark-title-colophon">A printer's device — a small ordered tree of typesetting at center, ringed by gathering marks indicating the book is mid-set</title>
|
||||
<desc>A frontispiece mark for the Colophon — drawn in the lineage of Aldine, Plantin, and Elzevir printer's marks. The central tree is the AST of one document being typeset; the outer ring of small marks indicates pages still being gathered, the document not yet bound.</desc>
|
||||
|
||||
<circle cx="140" cy="140" r="128" stroke="currentColor" stroke-width="0.6" fill="none"/>
|
||||
<circle cx="140" cy="140" r="120" stroke="currentColor" stroke-width="0.4" fill="none" opacity="0.55"/>
|
||||
|
||||
<g stroke="currentColor" fill="none" stroke-linecap="round" stroke-linejoin="round">
|
||||
|
||||
<line x1="140" y1="78" x2="140" y2="98" stroke-width="1.2"/>
|
||||
|
||||
<line x1="140" y1="98" x2="108" y2="118" stroke-width="1.0"/>
|
||||
<line x1="140" y1="98" x2="172" y2="118" stroke-width="1.0"/>
|
||||
|
||||
<line x1="108" y1="118" x2="92" y2="142" stroke-width="0.8"/>
|
||||
<line x1="108" y1="118" x2="118" y2="148" stroke-width="0.8"/>
|
||||
<line x1="172" y1="118" x2="162" y2="142" stroke-width="0.8"/>
|
||||
<line x1="172" y1="118" x2="184" y2="148" stroke-width="0.8"/>
|
||||
|
||||
<line x1="92" y1="142" x2="84" y2="166" stroke-width="0.6"/>
|
||||
<line x1="92" y1="142" x2="100" y2="170" stroke-width="0.6"/>
|
||||
<line x1="118" y1="148" x2="112" y2="172" stroke-width="0.6"/>
|
||||
<line x1="118" y1="148" x2="125" y2="174" stroke-width="0.6"/>
|
||||
|
||||
<line x1="162" y1="142" x2="156" y2="170" stroke-width="0.6"/>
|
||||
<line x1="162" y1="142" x2="168" y2="172" stroke-width="0.6"/>
|
||||
<line x1="184" y1="148" x2="178" y2="174" stroke-width="0.6"/>
|
||||
<line x1="184" y1="148" x2="192" y2="166" stroke-width="0.6"/>
|
||||
</g>
|
||||
|
||||
<g fill="currentColor" stroke="none">
|
||||
<circle cx="140" cy="78" r="2.0"/>
|
||||
<circle cx="140" cy="98" r="1.4"/>
|
||||
|
||||
<circle cx="108" cy="118" r="1.2"/>
|
||||
<circle cx="172" cy="118" r="1.2"/>
|
||||
|
||||
<circle cx="92" cy="142" r="1.0"/>
|
||||
<circle cx="118" cy="148" r="1.0"/>
|
||||
<circle cx="162" cy="142" r="1.0"/>
|
||||
<circle cx="184" cy="148" r="1.0"/>
|
||||
|
||||
<circle cx="84" cy="166" r="0.7"/>
|
||||
<circle cx="100" cy="170" r="0.7"/>
|
||||
<circle cx="112" cy="172" r="0.7"/>
|
||||
<circle cx="125" cy="174" r="0.7"/>
|
||||
<circle cx="156" cy="170" r="0.7"/>
|
||||
<circle cx="168" cy="172" r="0.7"/>
|
||||
<circle cx="178" cy="174" r="0.7"/>
|
||||
<circle cx="192" cy="166" r="0.7"/>
|
||||
</g>
|
||||
|
||||
<g stroke="currentColor" fill="none" stroke-linecap="round" stroke-width="0.6">
|
||||
<line x1="140" y1="190" x2="140" y2="208"/>
|
||||
<line x1="135" y1="200" x2="145" y2="200"/>
|
||||
<line x1="137" y1="204" x2="143" y2="204"/>
|
||||
</g>
|
||||
|
||||
<g stroke="currentColor" fill="none" stroke-linecap="round" stroke-width="0.7">
|
||||
<line x1="140" y1="38" x2="140" y2="44"/>
|
||||
|
||||
<line x1="172" y1="44" x2="174" y2="50"/>
|
||||
<line x1="200" y1="62" x2="204" y2="66"/>
|
||||
|
||||
<line x1="220" y1="92" x2="226" y2="94"/>
|
||||
<line x1="234" y1="125" x2="240" y2="125"/>
|
||||
<line x1="234" y1="155" x2="240" y2="155"/>
|
||||
|
||||
<line x1="226" y1="186" x2="220" y2="188"/>
|
||||
|
||||
<line x1="204" y1="214" x2="200" y2="218"/>
|
||||
<line x1="174" y1="230" x2="172" y2="236"/>
|
||||
|
||||
<line x1="106" y1="230" x2="108" y2="236"/>
|
||||
<line x1="76" y1="214" x2="80" y2="218"/>
|
||||
|
||||
<line x1="54" y1="186" x2="60" y2="188"/>
|
||||
|
||||
<line x1="40" y1="155" x2="46" y2="155"/>
|
||||
<line x1="40" y1="125" x2="46" y2="125"/>
|
||||
|
||||
<line x1="60" y1="92" x2="54" y2="94"/>
|
||||
<line x1="80" y1="62" x2="76" y2="66"/>
|
||||
<line x1="108" y1="44" x2="106" y2="50"/>
|
||||
</g>
|
||||
|
||||
<g fill="currentColor" stroke="none">
|
||||
<circle cx="124" cy="225" r="1.2"/>
|
||||
<circle cx="156" cy="225" r="1.2"/>
|
||||
</g>
|
||||
</svg>
|
||||
|
After Width: | Height: | Size: 3.7 KiB |
|
|
@ -0,0 +1,174 @@
|
|||
---
|
||||
title: Colophon
|
||||
date: 2026-03-21
|
||||
status: "Durable"
|
||||
confidence: 93
|
||||
tags: [meta]
|
||||
abstract: On the design, tools, and philosophy of this site — and by extension, its author.
|
||||
---
|
||||
|
||||
::: dropcap
|
||||
A personal website is not a publication. It is a position. A publication presents work
|
||||
in a finalized, immutable state, and carries with it some sort of declaration - "this is my most polished and prized work!";
|
||||
a position is something you inhabit, argue from, and continuously
|
||||
revise in public. This page explains the design decisions forming my broader **position** and why they took the form they did.
|
||||
|
||||
What follows is a colophon in the grand old sense: a printer's note at the end of the book,
|
||||
recording how it was made, who made it, etc. The difference: here, the
|
||||
printer and the author are the same person, and the process of making is itself not only *a* form
|
||||
of argument, but *the only* form of argument permitted.
|
||||
:::
|
||||
|
||||
---
|
||||
|
||||
## Typography
|
||||
|
||||
::: dropcap
|
||||
You are reading this sentence in SPECTRAL, which is not only a font with particular personal importance to me, but also an exceedingly pleasing font to read. [OpenType]{.smallcaps} features — `smcp`, `onum`, `liga`, `calt` are used throughout the website, which necessitates our self-hosting setup.^[Google Fonts strips OpenType features during
|
||||
subsetting for bandwidth. Self-hosting with `pyftsubset` preserves `smcp` (small
|
||||
capitals), `onum` (old-style figures), `liga` (common ligatures), and the full optical
|
||||
size range. The difference is visible: old-style figures sit on the baseline rather
|
||||
than hanging above it, small capitals are drawn to match the x-height rather than being
|
||||
shrunken full caps, and ligatures prevent collisions in letter pairs like *fi* and *fl*.]
|
||||
:::
|
||||
|
||||
The UI and headers are Fira Sans. Variation is good, and moreover, humanist sans are rather ubiquitous (we have Frutiger to thank for this fact!) - perhaps I am making some type of statement by not choosing one of the more corporation variations of it, like the dreaded Calibri and Tahoma you might recognize from Microslop (formerly known as *Microsoft*) products. Code uses Jetbrains Mono, which is simply the font that I use within my editor. Code should look like code, simple as that.
|
||||
|
||||
The monochrome palette is an application of restraint grounded in my studies of Tools for Thought.
|
||||
Color is often used to do work that typography
|
||||
should do, such as demonstrating hierarchy, creating emphasis, etc. When those
|
||||
functions are handled by weight, size, and spacing instead^[Color and saturation/hue are actually well known to be less effective than other means of distinguishment. I refer you to Tufte's *The Visual Display of Quantitative Information* for more.], color becomes available for the things it cannot be substituted for — and on a site with no data visualizations
|
||||
requiring color encoding, those things turn out to be very few.^[The one exception is
|
||||
the heatmap on the statistics page, which uses a four-step greyscale scale. Even there,
|
||||
the encoding is luminance rather than hue.]
|
||||
---
|
||||
|
||||
## The Build
|
||||
|
||||
This is a [Static Website](https://en.wikipedia.org/wiki/Static_web_page). For the purposes of this website, though the content is highly dynamic and iterated upon, the medium of expression is rather stable. There are numerous advantages to using a static webpage, many of which are focused at the Hetzner box from which this webpage is served. I use [Hakyll](https://jaspervdj.be/hakyll/) for reasons of performance, extensibility, and, of course given the underlying language Haskell, elegance! I had been wanting to do a project in Haskell ever since I took my undergraduate programming languages course,^[The programming languages course at Brown, somewhat infamous, is taught entirely with Racket, which is essentially a dialect of Lisp. The course itself is extremely focused within the functional paradigm as far as implementations go. I am aware that Racket itself, curiously, has some means by which static webpages can be built - the course infrastructure was produced this way, and this website has in a few places taken minor inspiration from it!] and Hakyll was more extensible and thus suitable than the alternative I was looking at most strongly, Hugo (in Go, a language with which I am intimately familiar). The philosophy of a static website is that the website is a program and the content is the source code^[Source code here **chiefly distinct** from mere markup language, like HTML.]. The step of compilation present in Haskell, which is outlined below, means that what you have here received in your browser is not merely a runtime rendering decision, but rather a deterministic artifact. By this step of compilation, the [Markdown](https://en.wikipedia.org/wiki/Markdown) in which I write these webpages is transformed to exactly what you currently see.
|
||||
|
||||
The [AST](https://en.wikipedia.org/wiki/Abstract_syntax_tree) we use is heavily customized and modified. The chain is roughly markdown -> pandoc -> citations -> wikilinks -> preprocessing -> sidenotes -> smallcaps and dropcaps -> links -> images -> math. Mathematics with LaTeX requires a second pass and is rendered at build-time with KaTeX - no math rendering occurs in your browser. Samples from music are displayed as SVGs, generally typeset with Lilypond through some helper scripts I wrote to automate the process.
|
||||
|
||||
Not all content on this site is markdown. A handful of pages — the [[Library]], the [[Commonplace]], and [[Current]] — are driven instead by YAML files under `data/`, rendered through dedicated Haskell modules that parse the schema, apply a sort or filter discipline particular to the surface, and emit the rendered HTML directly into a template. The split is deliberate: prose belongs in markdown, where the discipline is rhetorical; curated lists belong in YAML, where the discipline is editorial. The Current page, for instance, ladders entries by status (*in-review* → *revising* → *drafting* → *building*) before falling back to recency, and stamps each entry with its own "last updated" date — features that would be tedious to maintain in markdown but trivial to express in a small declarative schema.
|
||||
|
||||
The semantic search model is a particularly intriguing aspect of the website. The model used is self-hosted, with weights served from the same origin. There are NO external API calls when you use this, in contrast to just about every other similar feature on other websites. This is essential for the privacy model that this site strives to achieve - see **Design Decisions** for more.
|
||||
|
||||
A full accounting of what this build process has actually produced is available at the [[Build]] page. It is generated automatically at each compile: corpus word counts, length distributions, tag frequencies, link analysis, epistemic coverage, repository metrics, and build timing — all computed from the same source tree that produces the content. Think of it as the build system reporting on itself.
|
||||
|
||||
{{build}}
|
||||
|
||||
---
|
||||
|
||||
## The Computing Environment
|
||||
|
||||
::: dropcap
|
||||
I am, like many passionate nerds within the realm of computing, obsessive over my technological choices. They are the subject of constant critique, review, revision, etc. I believe in the value of putting deep thought into the systems that one interacts with, rather than accepting the first showing of convenience and going with the flow. A system interacted with is an experience moreso than a mere tool.
|
||||
:::
|
||||
|
||||
### Desktop
|
||||
|
||||
My primary desktop is a rig I built myself, running **Gentoo Linux** with **Hyprland** and a custom shell, *Levshell*, implemented with [Quickshell](https://github.com/quickshell-mirror/quickshell). The reasons for Gentoo are worth stating explicitly, since the choice is frequently met with bewilderment:
|
||||
|
||||
- Compiling software from source delivers measurable performance increases — not marginal ones.
|
||||
- It provides fine-grained control over software configuration via [USE]{.smallcaps} flags, linking options, and the like. This matters for a machine tuned to a high degree of specificity.
|
||||
- It is, in my experience, the best-maintained Linux distribution I have ever used.
|
||||
- The community is phenomenal.
|
||||
|
||||
I have strong hardware preferences to match; [AMD]{.smallcaps} hardware is used in favor of Intel and NVIDIA. I have used at least 2 distinct chips from each manufacturer and consistently find that, as far as x86-64 is concerned, AMD is the clear winner. As for my preference for AMD gpus, not only do I quite disagree with the business direction of NVIDIA, but the VRAM offerings are simply superior from AMD.
|
||||
|
||||
### Laptop
|
||||
|
||||
For mobile computing, I use a [P]{.smallcaps}-series ThinkPad running **Arch Linux** — the same Hyprland environment, the same *Levshell*, the same muscle memory down to the config file. Gentoo is impractical on battery-constrained hardware, as compilation times are simply too long and require active power connection. Arch is a sound alternative, configurable enough to pass. Portage is better than pacman in my opinion, but pacman is still far better than horrid package managers like apt.
|
||||
|
||||
### Editor
|
||||
|
||||
Everything on this site — every word of prose, every line of Haskell, every [CSS]{.smallcaps} rule — was written in **Emacs**. I have used most of the major editors, and consistently experiment with others, but have yet to find any which come close to the power of Emacs. I intend to complete a project called "Pmacs," which will introduce much moderner parallelization, among other features, to Emacs. This is a project I intend to tackle in the Summer of 2026 at high intensity.
|
||||
|
||||
### Privacy-First Computing
|
||||
|
||||
My email and [VPN]{.smallcaps} are self-hosted; I use Thunderbird as a client for the former. For browsing I use [LibreWolf](https://librewolf.net/), not Firefox: the Chromium monopoly and Mozilla's evident incompetence at browser development are, to me, equally concerning developments, and LibreWolf is the most coherent response to both. My phone runs [GrapheneOS](https://grapheneos.org/) — the only reasonably secure and private option for a mobile device, and one whose restrictions are, frankly, a feature rather than a limitation.
|
||||
|
||||
The principle underlying all of these choices is the same one underlying the site's **No Tracking** policy: privacy is an architectural decision, not a settings toggle. Bolting on privacy after the fact, whether in a browser or on a website, is not privacy — it is the appearance of privacy.
|
||||
|
||||
---
|
||||
|
||||
## [AI]{.smallcaps} and This Site
|
||||
|
||||
::: dropcap
|
||||
I will never use AI to write, whether for my personal communications with anyone or for pieces on this website. I take this extremely seriously - writing is religious in severity to me. The writing on this website is wholly human and wholly my own, to the extent that any writing can be.
|
||||
:::
|
||||
|
||||
Much of the code that comprises the build system of this website was created in collaboration with AI. Rather than "vibe coding" proper, this was the result of an intensive engineering process where AI and I were equals in collaboration. Notably, all of the major architectural choices, design decisions, idiosyncracies, and elements of the tech stack were chosen entirely by me, and AI systems were only used to automate production of some (but not all) of the code that was required.
|
||||
|
||||
The commit history, of course, is available for you to view and licensed accordingly - see **No Tracking** for more.
|
||||
|
||||
---
|
||||
|
||||
## Design Decisions
|
||||
|
||||
### Sidenotes
|
||||
|
||||
The sidenotes are provided by a JavaScript file that was forked from the website of Gwern Branwen and authored by
|
||||
Said Achmiz; I have simplified the script to fit the needs of this website and made some minor modifications.
|
||||
|
||||
|
||||
### No Tracking
|
||||
|
||||
The site has no analytics, no visit counters, no fingerprinting, and no third-party
|
||||
scripts.^[This is enforced at the nginx level via [CSP]{.smallcaps} headers, not just
|
||||
by omission. The Content Security Policy prevents any script not explicitly whitelisted
|
||||
from executing. The whitelist is short.] The Hetzner VPS that provides this content runs
|
||||
only open source software, and my machines use *almost exclusively*^[It is nearly impossible to run an entirely free system, but in approximation, it is actually wonderfully easy.] the same. The code is licensed under MIT and hosted
|
||||
on a [self-hosted Forgejo instance](https://git.levineuwirth.org/neuwirth/levineuwirth.org) at this domain, with a [GitHub mirror](https://github.com/levineuwirth/levineuwirth.org); you are welcome
|
||||
to inspect it, fork it, or, more broadly, do whatever you please with it.
|
||||
|
||||
### Living Documents
|
||||
|
||||
The dominant convention of academic and professional publication is that a document, once released, is finished. It carries an implicit claim: *this is what I think, full stop.*^[This is particularly problematic in academia, where there is a long tradition of researchers whose work was eventually disproven taking an extreme defensive stance, usually rooted in [confirmation bias](https://en.wikipedia.org/wiki/Confirmation_bias).] I find this convention dishonest in proportion to how seldom it is actually true. Thinking is continuous; positions shift; evidence accumulates; people change their minds and rarely say so in public. This site operates under a different premise, one that I strive to operate all of my life under.
|
||||
|
||||
Every essay and post on this site carries an **epistemic footer** — a structured block that reports my current relationship to the work. The footer only appears when a `status` field is set in the document's frontmatter; standalone pages and very short items omit it.
|
||||
|
||||
The vocabulary below is genre-general but reads differently across genres. For a personal essay, *confidence* reflects credence in a thesis — "I might change my mind." For an empirical research paper, it reflects expected generalization — "this would replicate." For formal mathematics, it reflects credence in proof correctness, with a special value `proved` available for theorems with complete proofs (where any numeric value would be false precision). *Evidence* reads analogously: the strength of arguments and supporting writing in essays, the empirical base in research, the structure of the proof in mathematics. The fields are the same; the interpretive frame shifts with the work.
|
||||
|
||||
The full set of fields:
|
||||
|
||||
- **Status** — a controlled vocabulary describing where the work stands: *Draft*, *Working model*, *Durable*, *Refined*, *Superseded*, or *Deprecated*. A document marked *Working model* is not just unfinished — it is a position I currently hold but would not stake much on. A document marked *Durable* is something I expect to hold up. *Superseded* means I wrote a better version; *Deprecated* means I no longer endorse it.
|
||||
|
||||
- **Confidence** — an integer from 0–100, representing my credence in the central thesis. Not false precision: a rough honest assessment is more useful than no assessment at all. When a `confidence-history` list is present, a trend arrow (↑ ↓ →) is derived automatically from the last two entries — so you can see not just *what* I think but whether I am growing more or less confident over time.
|
||||
|
||||
- **Importance** — how much I think this matters, on a 1–5 dot scale (●●●○○). Useful for orienting a reader who has limited time.
|
||||
|
||||
- **Evidence** — how well-evidenced the claims are, on the same 1–5 scale. An essay with high importance and low evidence is a speculative position and should be read accordingly.
|
||||
|
||||
- **Trust score** — a single 0–100 integer derived automatically from confidence (weighted 60%) and evidence (weighted 40%, with the 1–5 scale rescaled so that evidence=1 contributes zero and evidence=5 contributes the full 40 points). It is deliberately *narrow*: it answers "how much should you trust the central claim?" and nothing else. It says nothing about how broadly the work matters, how novel it is, or how useful it is in practice — those are separate axes (see below) that are deliberately *not* folded into a composite, so a high trust score on a personal essay cannot be misread as "world-shaking." Following Gwern's lead, the orientations are presented in parallel rather than blended into a single index. The score is not entered manually and lives only in the epistemic footer.
|
||||
|
||||
- **Scope**, **Novelty**, **Practicality** — orientation fields shown as their own rows in the epistemic footer alongside confidence, importance, and evidence. *Scope* ranges from *personal* to *civilizational*; *novelty* from *conventional* to *innovative*; *practicality* from *abstract* to *exceptional*. These are not ratings — they are orientations, and they intentionally do not feed the trust score.
|
||||
|
||||
- **Peer status** — the *external* review state, distinct from `status` (which is my internal position). Values: *unreviewed* (default), *under review*, *peer reviewed*, *published*, *retracted*. A piece can be *Durable* (I expect it to hold up) and *unreviewed* (the world hasn't checked yet) at the same time; the two axes are deliberately factored. A *retracted* piece renders with the field name struck through and the outer ring of the epistemic figure crossed out.
|
||||
|
||||
- **Result shape** — the shape of the central claim: *positive* (argues something works), *negative* (argues something does not), *mixed* (both, as in a double-pincer barrier paper), *comparative* (compares approaches), or *descriptive* (describes without arguing for or against). Encoded as a small glyph beside the trust score on the epistemic figure. Adds nothing to the compact row.
|
||||
|
||||
- **Stability** — auto-computed at every build from `git log --follow`. The heuristic: very new or barely-touched documents are *volatile*; actively-revised documents are *revising*; older documents with more commits settle into *fairly stable*, *stable*, or *established*. This requires no manual maintenance — the build reads the repository history and makes the inference.
|
||||
|
||||
The version history block, directly above the epistemic footer, uses a three-tier fallback: authored `history:` notes when they exist (written by me when the git log alone would not convey what changed), then the raw git log, then the `date:` frontmatter field as a creation record. `make build` auto-commits any changed content files before the Hakyll compilation runs, so the git log is always current.
|
||||
|
||||
The [[Current]] page extends this premise from essays to ongoing work. Every research project listed there carries its own `updated:` timestamp and a status drawn from a controlled vocabulary — *in-review*, *revising*, *drafting*, *building*, *early-stage*, *paused* — and the page itself wears a masthead "last updated" date as its thesis. The page has no epistemic footer because it isn't an argument; it is, rather, the closest thing this site has to a publication, and yet by design it is the part of the site most committed to being out of date the moment you finish reading it.
|
||||
|
||||
The point of all this is simple: when you read something on this site, you should know what kind of claim I am making. The date a document was last modified is not decorative. A 40% confidence rating is not self-deprecation. The system is an attempt to make explicit something that most writing leaves implicit — where the author actually stands.
|
||||
|
||||
---
|
||||
|
||||
## Influences
|
||||
|
||||
The amount of influences on this website is immense, and cannot be detailed in the fullest extent. Every other webpage that I have visited, whether beautiful or pitiful, has evoked some type of reaction or response in me, and that response has played some role, even if minute, in the design of this website. I can point to Tufte's influence on many of my design choices, and for the introduction to Tufte, I am thankful to CSCI1377 at Brown. I am thankful to the many other courses I took in my undergrad that influenced how I interact or ideologically view visualizations, networks, systems, etc.
|
||||
|
||||
The tradition of the personal website is one that is built on a sense of community and interaction. I am thankful to everyone else who has a personal website and shares their content with the world. I am also particularly greatful to the open source and broader open culture movements, who have given me and the world so much. This website would not exist without you - and I wouldn't be the person I am without your influence - what a role model!
|
||||
|
||||
---
|
||||
|
||||
## The Future
|
||||
|
||||
This site is unfinished. Several portals have no content yet. The annotated bibliography
|
||||
is sparse. I am in the progress of migrating content, so stay tuned!
|
||||
|
||||
The colophon itself is a living document. When the site changes substantially, this page will change with it. The git repository on Forgejo (hosted on the git subdomain here) should always be considered to take precedence.
|
||||
|
|
@ -0,0 +1,5 @@
|
|||
---
|
||||
title: Commonplace
|
||||
---
|
||||
|
||||
That which I wished to capture follows.
|
||||
|
|
@ -0,0 +1,5 @@
|
|||
---
|
||||
title: Current
|
||||
---
|
||||
|
||||
A working index of what I am building, writing, and thinking through right now — kept current rather than comprehensive. The page is rebuilt whenever an entry moves; the stamp above the first section is the canonical mark of how fresh the picture is.
|
||||
|
|
@ -0,0 +1,9 @@
|
|||
---
|
||||
title: Projects
|
||||
tags: meta
|
||||
portal: true
|
||||
---
|
||||
|
||||
Index of engineering artifacts, generated from the same data as the project sections of the [CV](/cv.pdf) and [résumé](/resume.pdf). Systems depth is the primary axis; self-directed tools and deployed machine-learning work follow.
|
||||
|
||||
Where a project has a writeup, its title links to it. Those are essays rather than records — informal, longer, and written in their own voice.
|
||||
|
|
@ -0,0 +1,60 @@
|
|||
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 280 280" role="img" aria-labelledby="mark-title-asymmetric-forgetting-a2">
|
||||
<title id="mark-title-asymmetric-forgetting-a2">A vertical chain of links that thins and fades as it rises, beside a tall ladder-scaffold that stays uniform in weight throughout, with three inward arrows touching it and one empty rung-stub extending toward the chain</title>
|
||||
<desc>A frontispiece mark for "Asymmetric Forgetting." Both figures rise from a heavy horizontal baseline (the moment of instruction). On the left, a vertical chain of links thins to hairline as it rises — the procedure, decaying without reactivation. On the right, a ladder-scaffold of rungs between two rails stays uniform in weight throughout — the concept, persistent. Three inward arrows touch the ladder from outside the figure at irregular heights — the world's analogues reaching in to refresh the schema. One empty rung extends from the ladder toward the chain side, capped with a small open circle: the docking slot where a procedure can re-attach when re-acquired. The chain has no such reach back; the absence is the point.</desc>
|
||||
|
||||
<circle cx="140" cy="140" r="128" stroke="currentColor" stroke-width="0.6" fill="none"/>
|
||||
|
||||
<line x1="40" y1="220" x2="240" y2="220" stroke="currentColor" stroke-width="1.4" stroke-linecap="round"/>
|
||||
|
||||
<g stroke="currentColor" fill="none" stroke-linecap="round" stroke-linejoin="round">
|
||||
<ellipse cx="72" cy="210" rx="8" ry="5" stroke-width="1.3"/>
|
||||
<ellipse cx="72" cy="197" rx="5" ry="8" stroke-width="1.3"/>
|
||||
|
||||
<ellipse cx="72" cy="182" rx="8" ry="5" stroke-width="1.1"/>
|
||||
<ellipse cx="72" cy="169" rx="5" ry="8" stroke-width="1.1"/>
|
||||
|
||||
<ellipse cx="72" cy="154" rx="8" ry="5" stroke-width="0.85"/>
|
||||
<ellipse cx="72" cy="141" rx="5" ry="8" stroke-width="0.85"/>
|
||||
|
||||
<ellipse cx="72" cy="126" rx="8" ry="5" stroke-width="0.55"/>
|
||||
<ellipse cx="72" cy="113" rx="5" ry="8" stroke-width="0.45"/>
|
||||
|
||||
<ellipse cx="72" cy="98" rx="8" ry="5" stroke-width="0.3" opacity="0.5"/>
|
||||
<ellipse cx="72" cy="86" rx="5" ry="8" stroke-width="0.25" opacity="0.3"/>
|
||||
|
||||
<ellipse cx="72" cy="72" rx="7" ry="4" stroke-width="0.2" opacity="0.15"/>
|
||||
</g>
|
||||
|
||||
<g stroke="currentColor" fill="none" stroke-linecap="round" stroke-linejoin="round" stroke-width="1.3">
|
||||
<line x1="180" y1="220" x2="180" y2="72"/>
|
||||
<line x1="222" y1="220" x2="222" y2="72"/>
|
||||
|
||||
<line x1="180" y1="206" x2="222" y2="206"/>
|
||||
<line x1="180" y1="192" x2="222" y2="192"/>
|
||||
<line x1="180" y1="178" x2="222" y2="178"/>
|
||||
<line x1="180" y1="164" x2="222" y2="164"/>
|
||||
<line x1="180" y1="150" x2="222" y2="150"/>
|
||||
<line x1="180" y1="136" x2="222" y2="136"/>
|
||||
<line x1="180" y1="122" x2="222" y2="122"/>
|
||||
<line x1="180" y1="108" x2="222" y2="108"/>
|
||||
<line x1="180" y1="94" x2="222" y2="94"/>
|
||||
<line x1="180" y1="80" x2="222" y2="80"/>
|
||||
</g>
|
||||
|
||||
<g stroke="currentColor" fill="none" stroke-linecap="round" stroke-linejoin="miter" stroke-width="0.9">
|
||||
<line x1="158" y1="150" x2="180" y2="150"/>
|
||||
<circle cx="155" cy="150" r="2.2" stroke-width="0.9" fill="none"/>
|
||||
</g>
|
||||
|
||||
<g stroke="currentColor" fill="none" stroke-linecap="round" stroke-linejoin="miter" stroke-width="0.8" opacity="0.85">
|
||||
<line x1="244" y1="94" x2="226" y2="94"/>
|
||||
<path d="M 230 91 L 226 94 L 230 97"/>
|
||||
|
||||
<line x1="244" y1="178" x2="226" y2="178"/>
|
||||
<path d="M 230 175 L 226 178 L 230 181"/>
|
||||
|
||||
<line x1="244" y1="206" x2="226" y2="206"/>
|
||||
<path d="M 230 203 L 226 206 L 230 209"/>
|
||||
</g>
|
||||
|
||||
</svg>
|
||||
|
After Width: | Height: | Size: 3.5 KiB |
|
|
@ -0,0 +1,81 @@
|
|||
---
|
||||
title: "Asymmetric Forgetting"
|
||||
date: 2026-05-26
|
||||
abstract: >
|
||||
Curricula in mathematics and the sciences optimize for procedural fluency — the half of what they teach that decays once the student stops being a student. What survives twenty years on is conceptual residue, generated only as an accidental byproduct of the curriculum's intended work. The asymmetry compounds across generations of teachers and produces a population unable to do the work that civic life requires of it.
|
||||
tags:
|
||||
- education
|
||||
- nonfiction
|
||||
- nonfiction/philosophy
|
||||
- political
|
||||
authors:
|
||||
- "Levi Neuwirth | /me.html"
|
||||
|
||||
status: "Working model"
|
||||
confidence: 80
|
||||
importance: 4
|
||||
evidence: 2
|
||||
scope: broad
|
||||
novelty: moderate
|
||||
practicality: moderate
|
||||
confidence-history:
|
||||
- 80
|
||||
---
|
||||
|
||||
If you ask the prototypical adult who took AP Calculus in high school what a "derivative" is, you'll generally get a half-decent answer, assuming they were reasonably engaged in class. You might hear that it's a slope, a rate of change, a measurement of how fast something moves. You'll get something actionable. If you ask the same adult to compute an elementary [derivative](https://en.wikipedia.org/wiki/Derivative) in front of you, they'll almost certainly fail, even if they aren't too far removed from their time in class. This is not a failure of their education by any means. In some ways, it is the only meaningful success that said education currently has.
|
||||
|
||||
Curricula can attempt to instill two things in a student. The first is procedural fluency: the ability to perform the steps of some algorithm on demand, to execute a technique, to perform a computation. We tend to think of this as the most important. But the second thing is more durable, even if less outwardly crisp: an epistemic residue, a working sense of what the concept being learned about fundamentally is, what kinds of claims it can support and what would refute it, and where it lies within the bigger picture of knowledge that the student has accumulated over time. In the United States, curricula in mathematics, the sciences, and the limited curricula that exist in computation are designed and assessed essentially exclusively against the first. This is in direct contrast to what survives in the graduates of these curricula ten, twenty, fifty years later — almost exclusively the second thing, delivered almost entirely by serendipitous accident, in the gaps between the procedures the Curriculum was actually intended to teach.
|
||||
|
||||
Concepts and epistemic residues persist over time, while procedures without regular reactivation do not. The cognitive infrastructure that our Curricula provide adults, the backbone by which they are intended to lead their lives and function in an increasingly technological society, bears almost no resemblance to the infrastructure which the curriculum was ostensibly optimized for. Worse, this asymmetric forgetting is not bidirectional in its effects. An adult who retained the concept can quickly re-acquire the related procedures on demand. The adult who, cramming for their examinations, once learned the procedure but knew nothing of the concept will be unable to run the reverse. There is nothing here that the procedure could have left behind, for the scaffolding to which it naturally should've attached was never built. Our curricula optimize for the thing that doesn't last, at the chief expense of the thing that does, and the thing that does is the thing from which all else follows.
|
||||
|
||||
## The Mechanism
|
||||
|
||||
The asymmetrical forgetting rests on a distinction older than the cognitive science vocabulary widely adopted to describe it. Procedural knowledge and conceptual knowledge are not stored, retrieved, nor reactivated in the same way, and they consequently do not decay in the same way.
|
||||
|
||||
Procedural knowledge is inherently sequential. To compute a derivative is to execute a series of moves in a particular order, conditioned only on the form of the input. To balance a chemical equation, to write a for-loop in a syntax that has not been recently used, or to manually run long division for the first time since fourth grade — these are all chains of steps whose only durable representation in the brain is the chain itself. Such chains inherently require reactivation to persist. The adult who has not balanced an equation in fifteen years[^1] has not forgotten because they were incapable or because they were poorly taught; they have forgotten because the sequential chain hasn't fired in fifteen years, and chains that are not run inevitably decay. This is not controversial. It is the easy direction, the half of the asymmetry that the curriculum tacitly acknowledges through myriad practice problems and lifeless examinations. What the curriculum fails to acknowledge is the expiration date: the day the student stops being a student.
|
||||
|
||||
[^1]: Or, sadly, the recent college graduate who hasn't done it in a mere six years... embarrassing, I know!
|
||||
|
||||
Conceptual knowledge is structured fundamentally differently. A concept is not a sequence but a schema. This allows for integration into the rest of what one understands about the world they inhabit. The adult who has retained the basic concept of the derivative has retained it because the schema gets reactivated incidentally, for the ordinary course of living provides suitable analogues[^2]. The schema is constantly reactivated; every news article mentioning acceleration, every casual remark about curves of stocks becoming steeper, every passing thought about how quickly something is growing. The procedural skill has no such ambient reactivation. There is nothing in adult civic life that can incidentally re-run the steps of the [quotient rule](https://en.wikipedia.org/wiki/Quotient_rule). The schema persists by virtue of the world's analogues consistently reaching in and touching it; the procedure decays because it is left devoid of interaction that the curriculum once served to forcibly provide.
|
||||
|
||||
[^2]: I would go further to hypothesize that any reasonable schema will with probability ~1 be reactivated, for [everything is correlated](https://gwern.net/everything).
|
||||
|
||||
The final component of this mechanic is the most severe. Consider two adults, one who has retained the concept and lost the procedure, and the other who has improbably (but for the sake of argument) retained the procedure without ever having the concept. We will continue with our example of differentiation. The first adult, on encountering a problem that requires a derivative, will find the procedure from a reference, say a web search, and re-acquire it in minutes. The concept's scaffolding has built a place to store the procedure when it returns. The second adult, faced with an identical problem, cannot recognize that it requires a derivative in the first place, for they never had such a scaffolding; the schema that would enable them to notice is markedly absent. Even if the procedure is entirely intact, it has nowhere to attach to, no occasion on which to be deployed. A procedure cannot magically summon a concept that was never built.
|
||||
|
||||
From this final component follows the weight of the asymmetry as a design constraint. If both of these aspects of cognitive infrastructure were equally durable, or even equally recoverable in reduction, the question of which to prioritize would be a matter of taste, pedagogical convenience, and moderate pretentiousness. This is, of course, not the case. The conceptual component is the substrate within which the procedural becomes meaningful, the only component surviving long enough to matter for the adult life the retired student will lead. A curriculum that optimizes for students who can blindly execute procedures they will lose in five years has produced essentially nothing of lasting value. It has produced nothing more than a cacophonous credential that lacks any semblance of underlying understanding.
|
||||
|
||||
## Substantiation
|
||||
|
||||
Where do I land amidst all of this, and why do I care? I offer my own retention audit of sorts not as proof of the mechanism, but as a demonstration that it is at least observable in lived form. If nothing else, perhaps you, dear reader, can run such an audit on yourself and see if the results are the same.
|
||||
|
||||
I attended a rural public school in upstate New York. By every metric that the system has conceived, I was a success by the time of my graduation. I was in the top ten of my class, I had straight As on my state examinations in mathematics and the sciences, I had all 4s and 5s on my AP examinations (including some which were not even offered by my institution), and graduated with the highest honors possible in my district, heading outbound to an Ivy League university. The audit forces reframing: of what was taught to me with procedural intent, what has survived, and in what form?
|
||||
|
||||
What do I remember of [stoichiometry](https://en.wikipedia.org/wiki/Stoichiometry)? I got a perfect 100 on my chemistry examination in high school, converting between grams and molecules, running limiting-reagent problems. I was clearly good at it at the time, and yet I remember nothing of how to balance even a trivial equation without re-deriving it slowly from first principles. I have not computed a mole quantity since my examination in 2020. What I retain from my Chemistry experience, other than the fact that providing troublemaking high school students with Bunsen Burners is outright objectionable at best, is that chemical reactions are quantitative, that matter is conserved across them, and the relationships between reactants and products are precise. I retained all of the concepts despite the curriculum such that when I went on to take advanced courses in mathematics and physics at Brown, the scaffolding was laid; I could connect what I learned to form a bigger picture. The procedure has decayed because I have not balanced an equation since I was sixteen, but the structural concept has survived.
|
||||
|
||||
What of AP Biology, another course of which I earned a perfect score on the final examination? I will not bore the readers with another long-winded description. The procedures that I once mastered, say those for [Punnett Squares](https://en.wikipedia.org/wiki/Punnett_square), have long left me. Yet the concepts are strong enough that in the years since, I have been able to do research work that is strictly integrated with medicine and the life sciences. When I need a procedure from these fields that are outside of my expertise, I have the conceptual scaffolding to place what I derive into.
|
||||
|
||||
The pattern is identical across every subject that I can examine. What was taught with mere procedural intent has decayed, while what has survived is the epistemic and conceptual residues, the schemas. The success of my education, by my own retention audit, is a success that the curriculum was not optimizing for. It is not, therefore, a success that can be attributed to some quality of that curriculum or quality of the public institution at which I studied. It is the pattern of a curriculum that failed at what it tried to do and, accidentally, in the failure, left behind for me the only thing of value.
|
||||
|
||||
I do not believe that my case is unusual, and I invite you to perform such an audit on yourself if you feel open to it. Ask yourself: of any procedural unit that you remember being drilled on for exams, *what survives now?* Is it the procedure that was assessed or the concept that was incidentally surfaced alongside it? The mechanism predicts what your audit will find, and by virtue of my results, I'd place my money on the same prediction.
|
||||
|
||||
## Implication
|
||||
|
||||
So far we have been concerned with the individual. The individual graduate, twenty years on, retains the concept and loses the procedure by failure of the curriculum. At this scale, the consequence is little more than regrettable in a bounded way. The individual is poorer for the loss, and perhaps there has been some time squandered away by the system, but the promise of the residue may remain, providing enough to build on if they ever so choose to put in a bit of effort. One can see why we might just shrug our shoulders and keep on walking right past this, for people will muddle through, and the people who do care can refresh and derive for themselves.
|
||||
|
||||
The true consequence of this fact lives at the scale of the population. The population that emerges from the widespread adoption and delivery of such a curriculum is the population that has to operate the society all graduates inhabit. Let us consider, then, what the mechanism predicts at such a scale. The median adult, twenty years on, has neither working procedural fluency nor the robust conceptual scaffolding that would enable subsequent procedural acquisition. The shards of residue that they do retain are little more than incidental, surfaced in the gaps between the procedures that the curriculum tried to teach, never deliberately developed, never assessed, and never built into the structure that the curriculum optimized for. It is thin where it should be thick, accidental where it should be load bearing.
|
||||
|
||||
This population is the one that we then ask to do the work that civic life requires. We ask this population to evaluate claims made by scientific institutions during a pandemic. We ask them to vote on the regulation of technologies they have never been taught to reason about, to navigate algorithmic and financial systems whose underlying structure and principles they have no schema for, to distinguish a credible statistical claim from a contrived and misleading one. The asymmetric forgetting mechanism predicts, and with high accuracy, that the population we have is widely unable to do these tasks at the level that is implicitly required. This is not due to incapability, for even a thin conceptual residue would provide meaningful opportunities, but it is rather because the curriculum is optimized for the wrong part of what it could instill and leave behind in the long term. We live inside the aggregate consequence of this fact.
|
||||
|
||||
This is made worse by the fact that it compounds. Each generation of teachers is drawn from a population produced by the curriculum of the previous generation. My rural hometown teachers had themselves passed through a curriculum optimized for procedural fluency. Whatever conceptual residue they retained was that from which their teaching stemmed. The conceptual depth that asymmetric forgetting calls for simply cannot be requested of teachers who were themselves never taught to that depth. It is not a reasonable expectation, and is thus not a personal failure of the teachers. The current teacher workforce that cannot provide a concept-first curriculum is rather the predictable product of a curriculum that did not build concepts deeply. Those who are interested in the concepts must find it within themselves to search further in the current setting. The path from where we are to a system that optimizes for the concepts runs through the teachers and their own educations, and that path is inevitably long. There is no magic solution that will resolve this in six months, nor within a glorious five year plan. Recognizing this is not a counsel of despair, but rather a counsel of patience, a refusal to mistake or conflate the difficulty of the path for evidence against the destination.
|
||||
|
||||
I have deliberately chosen not to describe what a concept-first curriculum would look like. This is a separate piece of work that is owed its own diligent treatment, and I refuse to attempt to distill and collapse it into the closing of this one. What follows from the asymmetry is the subject of work to be done and essays yet to be written, by myself and by others.
|
||||
|
||||
## Coda
|
||||
|
||||
The adult who remembers what a derivative is at the conceptual level has been given something. The curriculum that gave it to them did so by accident, in the margins between what it was actually optimized to deliver, and at the cost of everything else that it could've deliberately built into the same student. The residue is real and is genuinely the only thing that survived. It is also catastrophically less than what twelve years of schooling could have left behind, if only the system had known what it was for.
|
||||
|
||||
The asymmetry is not subtle and it is not new. We have been running this experiment on every cohort of American students for as long as "American students" have existed. The results have now been replicated millions of times. Those results *are* the population that we now have. A population that retains the wrong half of what it was taught as a thin accident, subsequently tasked with operating a society whose questions and demands require the half that is absent. The curriculum may have optimized for what would not last, but what actually lasted is the accidental byproduct. We ask that byproduct to do the work, but the work is too large for such an accident to bear.
|
||||
|
||||
We can choose different. The asymmetry shows us exactly what to reach for: the epistemic residue that persists, the schemas that the world's analogues continually reach in to refresh, and the conceptual scaffolding to which future learning can attach. We are choosing against this. We have been choosing against it for a very long time. The cost of that choice is borne not by the student who took the exams but by the adults they became, the society those adults now have to navigate without the infrastructure their schooling could have built.
|
||||
|
||||
The most valuable thing a curriculum can give a student is what the student will still have twenty years down the line. We are giving them everything else.
|
||||
|
|
@ -0,0 +1,429 @@
|
|||
---
|
||||
title: "Ball-Occupation Certificates under Coarse Graph Projections"
|
||||
subtitle: "Degree Reduction, Square-Root Hard Families, and Toroidal Barriers"
|
||||
date: 2026-07-27
|
||||
abstract: >
|
||||
We isolate an abstract strategy-transfer principle for Cops and Robber: a
|
||||
coarse graph projection with bounded fibers and bounded distance distortion
|
||||
lets cops occupy a lifted macro-ball before the robber escapes, giving cop
|
||||
number O(sqrt N) up to polylogarithmic factors. Applied to the
|
||||
Hosseini-Mohar-Gonzalez Hermosillo de la Maza degree-reduction
|
||||
construction, this shows the known hard family for Meyniel's conjecture
|
||||
already meets the square-root exponent, sharper than the usual notation
|
||||
suggests. A counting argument then proves a sharp limit on the strategy
|
||||
class itself: Cartesian tori of cycles have bounded doubling and constant
|
||||
cop number but linear occupation cost at every radius, so weak expansion
|
||||
alone cannot certify a universal robustness theorem.
|
||||
tags:
|
||||
- research
|
||||
- research/mathematics
|
||||
- research/graph-theory
|
||||
authors:
|
||||
- "Levi Neuwirth | /me.html"
|
||||
affiliation:
|
||||
- "Brown University | https://www.brown.edu"
|
||||
bibliography: data/ball-occupation-paper.bib
|
||||
preprint: /papers/ball-occupation-paper.pdf
|
||||
no-collapse: true
|
||||
status: "Durable"
|
||||
confidence: proved
|
||||
evidence: 5
|
||||
peer-status: unreviewed
|
||||
result-shape: mixed
|
||||
---
|
||||
|
||||
# Introduction and main conclusions
|
||||
|
||||
The multi-cop version of Cops and Robber was developed by Aigner and Fromme, who proved that three cops suffice on every planar graph [@AignerFromme]. Meyniel's conjecture asks whether every connected $n$-vertex graph has cop number $O(\sqrt n)$. The current best universal upper bound remains $$\frac{n}{2^{(1-o(1))\sqrt{\log_2 n}}},$$ proved independently by Lu–Peng and Scott–Sudakov [@LuPeng; @ScottSudakov]. Bose–Esperet–Hodor–Joret–Micek–Rambaud recently extended the same scale of bound from graph order to vertex-cover number [@BoseEsperetHodorJoretMicekRambaud]. Expansion is one of the principal settings in which polynomial savings are known: Bradshaw–Hosseini–Mohar–Stacho obtain weak Meyniel bounds from bounded-degree expansion restricted to sublinear set scales [@BradshawHosseiniMoharStacho], while Clow's withdrawn preprint developed a closely related structural program connecting failure of weak Meyniel to high-cop expanding examples [@Clow].
|
||||
|
||||
The motivation here is the effect of bounded-degree replacement gadgets on pursuit. The degree-reduction construction of Hosseini–Mohar–Gonzalez Hermosillo de la Maza (HMGHM) preserves lower bounds on cop number and produces subcubic graphs with cop number $M^{1/2-o(1)}$ [@HMG]. A natural converse question is whether a useful upper strategy on the base graph survives the replacement tower.
|
||||
|
||||
An arbitrary winning strategy does not lift transparently. Moving one step in the quotient may require a squad dispersed through a cloud to reorganize while the robber continues moving. The successful object is narrower and more stable: an *occupation certificate* that assigns distinct cops to all vertices of a region before the robber can leave it. Distance stretching slows both deployment and escape, and a bounded normalized additive error leaves a strict timing margin.
|
||||
|
||||
The first result is therefore stated for an abstract projection, rather than for the HMGHM gadget. The gadget enters only later, through an exact metric calculation. The resulting upper bound is stronger quantitatively than the notation $M^{1/2+o(1)}$ suggests: it is $\sqrt M$ times a polylogarithmic factor. By contrast, the available lower bound approaches the square-root exponent at the triple-logarithmic rate displayed in the abstract. These two facts should not be conflated merely because both can be written $M^{1/2+o(1)}$.
|
||||
|
||||
The final result marks the boundary of the mechanism. A one-shot occupation certificate needs polynomial ball amplification between radii $R$ and $2R$. Polynomially weak expansion alone does not supply this. For every fixed $k$, the Cartesian tori $C_L^{\square k}$ have bounded metric doubling, exact cop number $k+1$, and linear one-shot occupation cost at every radius. Taking $k>1/\delta$ puts these examples inside every window $h(G)\ge |G|^{-\delta}$. A separate cubic replacement retains the barrier at $\delta=1/2$. The obstacle in the universal problem is therefore not degree reduction itself; it is the need for adaptive reuse over many weak-growth layers.
|
||||
|
||||
# Scale-adaptive cores and the robustness window
|
||||
|
||||
For a connected graph $J$, write $$h(J)=\min_{\varnothing\ne A\subseteq V(J),\ |A|\le |J|/2}
|
||||
\frac{|\partial_J A|}{|A|}.$$ The following elementary reduction explains why polynomially weak expansion is the relevant robustness window for the universal problem.
|
||||
|
||||
::: {#prop-adaptive-core .exhibit .exhibit--proposition data-exhibit-type="proposition" data-exhibit-name="Proposition 1 (Scale-adaptive induced core)"}
|
||||
**Proposition 1** (Scale-adaptive induced core). *Let $J$ be a connected graph of order $n$, and let $\eta(1)\ge\cdots\ge\eta(n)\ge0$. Then $J$ contains a connected induced subgraph $K$, of order $m$, such that $$\boxed{h(K)\ge\eta(m)}
|
||||
\qquad\text{and}\qquad
|
||||
\boxed{c(J)\le c(K)+\sum_{j=m+1}^{n}\eta(j).}$$*
|
||||
:::
|
||||
|
||||
::: proof
|
||||
*Proof.* Maintain the connected induced region $J_i$ containing the robber. Whenever $|J_i|=m_i$ and $h(J_i)<\eta(m_i)$, choose $A_i\subseteq V(J_i)$ with $0<|A_i|\le m_i/2$ and $|\partial_{J_i}A_i|<\eta(m_i)|A_i|$, and occupy its boundary. The robber is then confined to one component $J_{i+1}$ of $J_i-\partial_{J_i}A_i$. Whether that component lies inside $A_i$ or outside it, one has $$|A_i|\le m_i-m_{i+1}.$$ Consequently the separator costs at most $$\eta(m_i)(m_i-m_{i+1})
|
||||
\le
|
||||
\sum_{j=m_{i+1}+1}^{m_i}\eta(j).$$ These integer intervals are disjoint along the robber's nested component chain. The process terminates at a connected induced $K$ with $h(K)\ge\eta(|K|)$, and the separator costs telescope to the displayed sum. ◻
|
||||
:::
|
||||
|
||||
Taking $\eta(j)=j^{-a}$ shows that a polynomial cop-number saving on subcubic graphs with $h(K)\ge |K|^{-a}$ would imply a weak form of Meyniel for arbitrary graphs after the bounded-degree transfer of Hosseini–Mohar–Gonzalez Hermosillo de la Maza [@HMG]. Bradshaw–Hosseini–Mohar–Stacho already treat constant expansion restricted to sublinear set scales [@BradshawHosseiniMoharStacho]; the unresolved axis in this reduction is expansion that itself shrinks polynomially.
|
||||
|
||||
# Coarse occupation projections
|
||||
|
||||
::: {#def-projection .exhibit .exhibit--definition data-exhibit-type="definition" data-exhibit-name="Definition 2 (Coarse occupation projection)"}
|
||||
**Definition 2** (Coarse occupation projection). Let $G$ and $H$ be connected graphs. A surjection $\pi:V(H)\to V(G)$ is a $(\lambda,P)$-occupation projection if, writing $F_v=\pi^{-1}(v)$,
|
||||
|
||||
1. $|F_v|\le P$ for every $v\in V(G)$;
|
||||
|
||||
2. for distinct $u,v\in V(G)$ and arbitrary $x\in F_u$, $y\in F_v$, $$\lambda(\operatorname{dist}_G(u,v)-1)+1
|
||||
\le
|
||||
\operatorname{dist}_H(x,y)
|
||||
\le
|
||||
\lambda(\operatorname{dist}_G(u,v)+2)-2;$$
|
||||
|
||||
3. $\operatorname{diam}_H(F_v)\le2(\lambda-1)$ for every $v\in V(G)$.
|
||||
:::
|
||||
|
||||
The particular constants in [Definition 2](#def-projection) are chosen because they are exact for the HMGHM tower. The proof below only needs a bounded additive slack after division by $\lambda$ and a strict gap between deployment and escape deadlines.
|
||||
|
||||
For $U\subseteq V(G)$, let $B_G(U,r)$ be its closed radius-$r$ neighborhood.
|
||||
|
||||
::: {#thm-abstract-transfer .exhibit .exhibit--theorem data-exhibit-type="theorem" data-exhibit-name="Theorem 3 (Abstract macro-ball occupation transfer)"}
|
||||
**Theorem 3** (Abstract macro-ball occupation transfer). *Let $G$ have $N$ vertices. Fix $d\ge2$, $R\ge2$, and constants $a,A_0>0$. Assume $$\sqrt N\le d^{R-2}<d\sqrt N,
|
||||
\qquad
|
||||
d^3\le\sqrt N,$$ and $$\begin{aligned}
|
||||
|B_G(U,R-2)|
|
||||
&\ge a\min\{|U|d^{R-2},N\}
|
||||
&&\text{for every }U\subseteq V(G),\\
|
||||
|B_G(v,R)|
|
||||
&\le A_0d^R
|
||||
&&\text{for every }v\in V(G).
|
||||
\end{aligned}$$ If $H$ admits a $(\lambda,P)$-occupation projection onto $G$, then $$\boxed{
|
||||
c(H)
|
||||
\le
|
||||
C(a,A_0)P\bigl(d^3+\log(ePN)\bigr)\sqrt N.
|
||||
}$$ The displayed number of cops captures the robber within at most $\lambda R$ cop moves. In particular, the cop-number bound is independent of the scale factor $\lambda$; tower depth affects capture time but not the required bank size.*
|
||||
:::
|
||||
|
||||
::: proof
|
||||
*Proof.* Put $$\Theta=d^3+\log(ePN),
|
||||
\qquad
|
||||
\mu=\frac{AP\Theta}{\sqrt N},$$ where $A$ is a sufficiently large constant depending only on $a,A_0$. Choose one canonical vertex $z^*\in F_z$ in every fiber. At $z^*$ place an independent Poisson number of cop tokens of mean $\mu$.
|
||||
|
||||
For a possible robber fiber $F_v$, set $$X_v=\pi^{-1}(B_G(v,R)).$$ By the upper-growth hypothesis and the displayed scale conditions above, $$Q_v:=|X_v|
|
||||
\le PA_0d^R
|
||||
<A_0Pd^3\sqrt N
|
||||
\le A_0P\Theta\sqrt N.$$ We prove simultaneously for every $v$ that the sampled tokens can be matched distinctly to all vertices of $X_v$, with every assigned token based over a base vertex within distance $R-2$ of its target fiber.
|
||||
|
||||
Let $S\subseteq X_v$, $|S|=s$, and put $U=\pi(S)$. Since every fiber has at most $P$ targets, $|U|\ge s/P$. Every token based over $Z=B_G(U,R-2)$ is adjacent in the assignment graph to at least one target in $S$.
|
||||
|
||||
If $|U|d^{R-2}<N$, then the number of available tokens is Poisson with mean at least $$\mu a|U|d^{R-2}\ge Aa\Theta s.$$ For a Poisson variable $Y$ of mean $\Lambda\ge Aa\Theta s$, $$\Pr(Y<s)\le e^{-\Lambda}\left(\frac{e\Lambda}{s}\right)^s.$$ After increasing $A$, this is at most $(ePN)^{-4s}$. There are at most $N\binom{PN}{s}$ choices of a root and a target subset of size $s$, so summing over $s\ge1$ gives $o(1)$.
|
||||
|
||||
If $|U|d^{R-2}\ge N$, then the available-token mean is at least $$\mu aN=AaP\Theta\sqrt N
|
||||
\ge\frac{Aa}{A_0}Q_v$$ using the bound on $Q_v$ above. Taking $A$ large and applying the same Poisson bound shows, after a union bound over at most $N2^{Q_v}$ target subsets, that no saturated Hall condition fails with probability $1-o(1)$. Here every saturated target set has $$s\ge |U|\ge\frac{N}{d^{R-2}}>\frac{\sqrt N}{d}\ge N^{1/3},$$ so the polynomial prefactors are negligible.
|
||||
|
||||
Hall's condition therefore holds simultaneously for all macro-balls with probability $1-o(1)$. The total number of sampled tokens is at most $2AP\Theta\sqrt N$ with probability $1-o(1)$, so a deterministic placement of the claimed size exists.
|
||||
|
||||
It remains to compare deadlines. A target $y\in X_v$ lies over some $u\in B_G(v,R)$. Its assigned cop begins over $z$ with $\operatorname{dist}_G(z,u)\le R-2$. If $z\ne u$, the upper distortion bound gives travel time at most $$\lambda((R-2)+2)-2=\lambda R-2.$$ If $z=u$, the fiber-diameter bound gives at most $2(\lambda-1)\le\lambda R-2$, since $R\ge2$.
|
||||
|
||||
To leave $X_v$, the robber must enter a fiber over a base vertex at distance at least $R+1$ from $v$. The lower distortion bound makes this require at least $$\lambda((R+1)-1)+1=\lambda R+1$$ steps. Every vertex of $X_v$ is occupied first, and the cop assigned to the robber's current vertex captures her. ◻
|
||||
:::
|
||||
|
||||
::: {#rem-pw-uniform .exhibit .exhibit--remark data-exhibit-type="remark" data-exhibit-name="Remark 4 (The quantifier needed from the random base)"}
|
||||
*Remark 4* (The quantifier needed from the random base). The use of the lower-growth hypothesis above is graph-uniform, not a per-source-set probability statement. In the dense theorem of Prałat and Wormald, condition (i) of their deterministic Theorem 3.1 is explicitly quantified over *every* source set and radius. Their Theorem 3.4 proves that a single $G(N,p)$ satisfies those hypotheses asymptotically almost surely; its proof unions over the bad source sets and concludes that the growth estimate holds simultaneously for all sets and radii [@PralatWormald]. Thus the external input has the quantifier order required by [Theorem 3](#thm-abstract-transfer).
|
||||
:::
|
||||
|
||||
# The HMGHM replacement tower
|
||||
|
||||
For a vertex of degree $r$, the HMGHM replacement has one external port for every incident edge. The ports are partitioned into nearly equal classes, and for each pair of classes there is an internal vertex adjacent to every port in the two classes [@HMG].
|
||||
|
||||
::: {#lem-portgeometry .exhibit .exhibit--lemma data-exhibit-type="lemma" data-exhibit-name="Lemma 5 (One-round port geometry)"}
|
||||
**Lemma 5** (One-round port geometry). *For every HMGHM replacement cloud of degree at least two:*
|
||||
|
||||
1. *distinct ports are nonadjacent and have distance exactly two;*
|
||||
|
||||
2. *every cloud vertex is within distance at most three of every specified port;*
|
||||
|
||||
3. *the cloud diameter is at most four.*
|
||||
:::
|
||||
|
||||
::: proof
|
||||
*Proof.* Two ports in different classes share the internal vertex associated with their class pair. Two ports in the same class share any internal vertex associated with that class and another nonempty class. Since ports are mutually nonadjacent, their distance is exactly two.
|
||||
|
||||
An internal vertex is adjacent to every port in either of two classes. If a specified port lies in neither class, travel to a port in one of the two classes, then through the internal vertex corresponding to that class and the specified port's class, and finally to the specified port. This takes three steps. The diameter bound follows by routing arbitrary endpoints through a specified port. ◻
|
||||
:::
|
||||
|
||||
Let $$G=G_0,G_1,\ldots,G_k=H$$ be an iterated HMGHM tower, and let $\pi:V(H)\to V(G)$ map every final vertex to its original ancestor. Put $$F_v=\pi^{-1}(v),
|
||||
\qquad
|
||||
P=\max_v|F_v|,
|
||||
\qquad
|
||||
\lambda=3^k.$$
|
||||
|
||||
::: {#thm-metric .exhibit .exhibit--theorem data-exhibit-type="theorem" data-exhibit-name="Theorem 6 (Exact normalized distortion)"}
|
||||
**Theorem 6** (Exact normalized distortion). *The ancestry projection is a $(3^k,P)$-occupation projection. Explicitly, for distinct base vertices $u,v$, $r=\operatorname{dist}_G(u,v)$, and arbitrary $x\in F_u$, $y\in F_v$, $$\boxed{
|
||||
3^k(r-1)+1
|
||||
\le
|
||||
\operatorname{dist}_H(x,y)
|
||||
\le
|
||||
3^k(r+2)-2,
|
||||
}$$ and $$\boxed{\operatorname{diam}_H(F_v)\le2(3^k-1).}$$*
|
||||
:::
|
||||
|
||||
::: proof
|
||||
*Proof.* For one round, a shortest path between distinct clouds uses $e\ge r$ external edges. Between consecutive external edges it enters and leaves an intermediate cloud through distinct ports: otherwise it immediately traverses one external edge back. By [Lemma 5](#lem-portgeometry), each intermediate port change costs at least two internal edges, and hence $$\operatorname{dist}_{G_1}(x,y)\ge e+2(e-1)\ge3r-2.$$ For the upper bound, follow a base geodesic. Reaching the first prescribed port costs at most three, each intermediate port change costs two, the external edges cost $r$, and reaching the final endpoint costs at most three. Thus $$\operatorname{dist}_{G_1}(x,y)\le3+r+2(r-1)+3=3r+4.$$ The one-round fiber diameter is at most four.
|
||||
|
||||
The lower and upper affine recurrences are $$L_j(r)=3L_{j-1}(r)-2,
|
||||
\qquad
|
||||
U_j(r)=3U_{j-1}(r)+4,$$ with $L_0(r)=U_0(r)=r$. Solving gives $$L_k(r)=3^k(r-1)+1,
|
||||
\qquad
|
||||
U_k(r)=3^k(r+2)-2.$$ The diameter recurrence $D_j\le3D_{j-1}+4$, $D_0=0$, gives $D_k\le2(3^k-1)$. ◻
|
||||
:::
|
||||
|
||||
The feature that matters is not the number of rounds but the normalized additive error: after division by $3^k$, it remains two quotient layers. By [Theorem 3](#thm-abstract-transfer), any other graph projection with the same three properties inherits the same occupation-certificate transfer.
|
||||
|
||||
# Expansion retention under port-cloud replacement
|
||||
|
||||
::: {#prop-port-exp .exhibit .exhibit--proposition data-exhibit-type="proposition" data-exhibit-name="Proposition 7 (Expansion under connected port replacement)"}
|
||||
**Proposition 7** (Expansion under connected port replacement). *Let $H$ be obtained from a base graph $G$ by replacing every vertex by a connected cloud of order at most $L_0$, with distinct external ports for the incident base edges. If $\iota(G)$ is the edge-isoperimetric constant of $G$, then $$\boxed{
|
||||
\iota(H)
|
||||
\ge
|
||||
\frac{1}{2L_0}
|
||||
\min\left\{1,\frac{\iota(G)}{L_0}\right\}.
|
||||
}$$*
|
||||
:::
|
||||
|
||||
::: proof
|
||||
*Proof.* Let $S\subseteq V(H)$ with $0<|S|\le|H|/2$. In each cloud, classify the majority side and let $\mathcal M$ be the total number of minority vertices. Since every partially cut cloud is connected and has at most $L_0$ vertices, its internal cut contributes at least one edge, so the total internal contribution is at least $\mathcal M/L_0$.
|
||||
|
||||
Let $U$ be the set of base vertices whose clouds have majority in $S$, and put $E=e_G(U,V(G)\setminus U)$. A base cut edge can fail to cross the lifted cut only if one of its two ports is a minority vertex. Distinct base edges use distinct ports, so at most $\mathcal M$ of the $E$ external cut edges fail. Hence $$e_H(S,V(H)\setminus S)
|
||||
\ge
|
||||
\frac{\mathcal M}{L_0}+\max\{0,E-\mathcal M\}
|
||||
\ge
|
||||
\frac{E+\mathcal M}{2L_0}.$$ Apply the same majority accounting to $S$ or its complement, according as $|U|\le|G|/2$ or not, to obtain $$\min\{|U|,|V(G)\setminus U|\}
|
||||
\ge
|
||||
\frac{|S|-\mathcal M}{L_0}.$$ Thus $$E\ge\frac{\iota(G)}{L_0}(|S|-\mathcal M),$$ and consequently $$E+\mathcal M
|
||||
\ge
|
||||
\min\left\{1,\frac{\iota(G)}{L_0}\right\}|S|.$$ Combining the inequalities proves the proposition. ◻
|
||||
:::
|
||||
|
||||
The HMGHM cloud-size recurrence turns the one-round estimate into a subpolynomial-loss statement in the polylogarithmic-degree regime. If $D$ is the initial maximum degree, $L_i$ is the largest cloud order in round $i$, and $k$ rounds are used, HMGHM prove $$\prod_{i=0}^{k-1}L_i
|
||||
\le C D^2(\log D)^{\log_2(11/5)},
|
||||
\qquad
|
||||
2^k=O(\log D).$$ Iterating [Proposition 7](#prop-port-exp) therefore gives the following.
|
||||
|
||||
::: {#cor-exp-retention .exhibit .exhibit--corollary data-exhibit-type="corollary" data-exhibit-name="Corollary 8 (Expansion retained by HMGHM reduction)"}
|
||||
**Corollary 8** (Expansion retained by HMGHM reduction). *Let $H$ be the final subcubic graph obtained from a connected graph $G$ of maximum degree $D\ge4$. Then $$\boxed{
|
||||
h(H)
|
||||
\ge
|
||||
\frac{\iota(G)}{C D^4(\log D)^{\kappa}},
|
||||
\qquad
|
||||
\kappa=1+2\log_2(11/5)<3.28.
|
||||
}$$ In particular, if $D=|G|^{o(1)}$ and $\iota(G)=|G|^{-o(1)}$, then $|H|=|G|^{1+o(1)}$ and $h(H)=|H|^{-o(1)}$.*
|
||||
:::
|
||||
|
||||
::: proof
|
||||
*Proof.* At every round $\iota(G_i)\le D_i\le L_i$, so [Proposition 7](#prop-port-exp) gives $$\iota(G_{i+1})\ge\frac{\iota(G_i)}{2L_i^2}.$$ Thus $$\iota(H)
|
||||
\ge
|
||||
\frac{\iota(G)}{2^k(\prod_iL_i)^2}
|
||||
\ge
|
||||
\frac{\iota(G)}{C D^4(\log D)^\kappa}.$$ Since $H$ has maximum degree at most three, its vertex expansion is at least one third of its edge expansion. The order statement follows from the same cloud-product bound. ◻
|
||||
:::
|
||||
|
||||
::: {#rem-replacement-products .exhibit .exhibit--remark data-exhibit-type="remark" data-exhibit-name="Remark 9 (Relation to replacement products)"}
|
||||
*Remark 9* (Relation to replacement products). The regular replacement-product literature proves stronger spectral conclusions under much stronger hypotheses on the clouds; see, for example, Reingold–Vadhan–Wigderson [@ReingoldVadhanWigderson]. [Proposition 7](#prop-port-exp) allows arbitrary connected, nonuniform clouds and consequently gives only a crude isoperimetric estimate. No novelty claim is made here beyond this precise form without a fuller graph-substitution review.
|
||||
:::
|
||||
|
||||
# A polylogarithmically tight square-root family
|
||||
|
||||
Take $$d=(\log N)^4,
|
||||
\qquad
|
||||
p=\frac{d}{N-1},
|
||||
\qquad
|
||||
G\sim G(N,p).$$ Iterate the HMGHM replacement until the graph $H$ is subcubic, and write $M=|H|$.
|
||||
|
||||
With high probability, $\Delta(G)\le2d$. By [Remark 4](#rem-pw-uniform), the dense Prałat–Wormald theorem supplies the uniform lower growth needed above; in the volume range used here it also supplies the matching upper growth [@PralatWormald]. Choose $R$ minimally so that $d^{R-2}\ge\sqrt N$. Since $d$ is polylogarithmic, the scale conditions of [Theorem 3](#thm-abstract-transfer) hold.
|
||||
|
||||
HMGHM give, both globally and along one ancestry fiber, $$P\le C d^2(\log d)^{1.14},
|
||||
\qquad
|
||||
N\le M\le PN.$$ Their shadow strategy gives $c(H)\ge c(G)$, and the random-graph lower bound of Bollobás–Kun–Leader used in their argument [@BollobasKunLeader] yields $$c(G)
|
||||
\ge
|
||||
d^{-2}N^{\frac12-\frac{9}{2\log\log d}}.$$ The abstract transfer theorem gives $$c(H)
|
||||
\le
|
||||
CP\bigl(d^3+\log(ePN)\bigr)\sqrt N
|
||||
\le
|
||||
\sqrt M\,(\log M)^{20+o(1)}.$$ Using $d=(\log N)^4$ and $M=N(\log N)^{O(1)}$ in the lower bound gives the following more informative formulation.
|
||||
|
||||
::: {#thm-hardfamily .exhibit .exhibit--theorem data-exhibit-type="theorem" data-exhibit-name="Theorem 10 (Quantitative HMGHM hard family)"}
|
||||
**Theorem 10** (Quantitative HMGHM hard family). *There is a sequence of connected subcubic graphs $H$, of order $M\to\infty$, for which $$\boxed{
|
||||
M^{\frac12-\frac{9+o(1)}{2\log\log\log M}}
|
||||
\le
|
||||
c(H)
|
||||
\le
|
||||
\sqrt M\,(\log M)^{20+o(1)}.
|
||||
}$$ The upper bound is $\sqrt M$ times a polylogarithmic factor. The lower exponent tends to $1/2$ only at a triple-logarithmic rate.*
|
||||
:::
|
||||
|
||||
The base edge expansion is $\Omega(d)$ with high probability. Since $d=\operatorname{polylog}N$, [Corollary 8](#cor-exp-retention) gives $$h(H)\ge (\log M)^{-O(1)}=M^{-o(1)}.$$ This is exactly the degree regime in which the retention factor is informative; for polynomial initial degree the crude $D^4$ loss can be vacuous.
|
||||
|
||||
::: {#cor-weakexpander .exhibit .exhibit--corollary data-exhibit-type="corollary" data-exhibit-name="Corollary 11 (Square-root weak-expander family)"}
|
||||
**Corollary 11** (Square-root weak-expander family). *There are connected subcubic graphs satisfying $$h(H)\ge M^{-o(1)}
|
||||
\qquad\text{and}\qquad
|
||||
c(H)=M^{1/2+o(1)}.$$ More precisely, they obey the two-sided bounds of [Theorem 10](#thm-hardfamily).*
|
||||
:::
|
||||
|
||||
::: {#rem-robustness-endpoint .exhibit .exhibit--remark data-exhibit-type="remark" data-exhibit-name="Remark 12 (What is forced, and what is achieved)"}
|
||||
*Remark 12* (What is forced, and what is achieved). By [Proposition 1](#prop-adaptive-core), polynomially weak expansion is a natural robustness window for weak Meyniel. The HMGHM lower bound alone already forces every proposed estimate $$c(J)\le C\phi^{-p}|J|^{1-\varepsilon+o(1)}
|
||||
\qquad(h(J)\ge\phi=|J|^{-o(1)})$$ to have $\varepsilon\le1/2$. That restriction predates the upper transfer proved here. The new conclusion is that the known stressing family itself achieves the endpoint order $\sqrt M$ up to polylogarithmic factors, so this family is not an obstruction to a Meyniel-strength theorem on polynomially weak subcubic expanders.
|
||||
:::
|
||||
|
||||
# Why chase strategies need not transfer
|
||||
|
||||
The abstract theorem deliberately transfers a strategy class, not arbitrary cop number. The smallest example explains the distinction. For a degree-two vertex, one HMGHM cloud is a three-vertex path. Replacing every vertex of $C_3$ therefore produces $C_9$. But $$c(C_3)=1,
|
||||
\qquad
|
||||
c(C_9)=2.$$ The one-cop win on $C_3$ is a direct chase/dismantling phenomenon. The subdivision-like stretching destroys it. By contrast, an occupation certificate is synchronized to a deadline: the replacement tower stretches the cops' travel and the robber's escape by the same factor, and the bounded normalized additive slack preserves a strict margin. The examples $K_4$ and the diamond graph exhibit the same one-round increase, so the issue is structural rather than peculiar to one cycle.
|
||||
|
||||
# A one-shot occupation barrier
|
||||
|
||||
::: {#def-occ .exhibit .exhibit--definition data-exhibit-type="definition" data-exhibit-name="Definition 13 (Universal one-shot occupation number)"}
|
||||
**Definition 13** (Universal one-shot occupation number). For a connected graph $G$ and integer $R\ge0$, let $\operatorname{Occ}_R(G)$ be the minimum size of a finite set $X$ of distinct cop tokens, equipped with a position map $p:X\to V(G)$, such that for every $v\in V(G)$ there is an injection $$f_v:B_G(v,R)\longrightarrow X$$ with $$\operatorname{dist}_G(u,p(f_v(u)))\le R
|
||||
\qquad\text{for every }u\in B_G(v,R).$$ Different tokens may have the same initial position. After learning the robber's starting vertex, the common prepositioned bank can occupy her entire radius-$R$ ball within $R$ moves.
|
||||
:::
|
||||
|
||||
::: {#rem-target-deadline .exhibit .exhibit--remark data-exhibit-type="remark" data-exhibit-name="Remark 14 (Why the target and deadline are natural)"}
|
||||
*Remark 14* (Why the target and deadline are natural). If the robber starts at $v$, she needs at least $R+1$ robber moves to leave $B_G(v,R)$. Occupying that whole ball within $R$ cop moves is therefore the canonical one-shot certificate: every vertex she could still occupy is filled before her first possible escape. The parameter $\operatorname{Occ}_R$ measures this specific strategy class, not ordinary cop number.
|
||||
:::
|
||||
|
||||
::: {#thm-occ-lower .exhibit .exhibit--theorem data-exhibit-type="theorem" data-exhibit-name="Theorem 15 (Counting barrier)"}
|
||||
**Theorem 15** (Counting barrier). *Every connected graph satisfies $$\boxed{
|
||||
\operatorname{Occ}_R(G)
|
||||
\ge
|
||||
\frac{\sum_{v\in V(G)}|B_G(v,R)|}
|
||||
{\max_{x\in V(G)}|B_G(x,2R)|}.
|
||||
}$$ In particular, if $G$ is vertex-transitive, then $$\boxed{
|
||||
\operatorname{Occ}_R(G)
|
||||
\ge
|
||||
|V(G)|\frac{|B_G(o,R)|}{|B_G(o,2R)|}.
|
||||
}$$*
|
||||
:::
|
||||
|
||||
::: proof
|
||||
*Proof.* Fix a feasible multiset $X$. For each possible robber start $v$, every cop token used by the injection $f_v$ lies in $B_G(v,2R)$, by the triangle inequality. Hence at least $|B_G(v,R)|$ tokens of $X$ lie in $B_G(v,2R)$. Summing over $v$, the number of incident pairs $(v,x)$ with $x\in X\cap B_G(v,2R)$ is at least $\sum_v|B_G(v,R)|$.
|
||||
|
||||
A fixed token based at $x$ is counted only for starts $v\in B_G(x,2R)$, at most $\max_y|B_G(y,2R)|$ times. Therefore $$|X|\max_y|B_G(y,2R)|
|
||||
\ge
|
||||
\sum_v|B_G(v,R)|,$$ which proves the claim. ◻
|
||||
:::
|
||||
|
||||
The theorem identifies the exact growth ratio demanded by one-shot occupation. A polynomial saving from the trivial $|V(G)|$ bound requires polynomial amplification from radius $R$ to radius $2R$.
|
||||
|
||||
We now give subcubic witnesses showing that polynomially weak expansion does not imply such amplification.
|
||||
|
||||
::: {#def-qt .exhibit .exhibit--definition data-exhibit-type="definition" data-exhibit-name="Definition 16 (The cubic truncated torus)"}
|
||||
**Definition 16** (The cubic truncated torus). For $L\ge5$, let $Q_L$ have vertex set $$(\mathbb Z/L\mathbb Z)^2\times\mathbb Z/4\mathbb Z.$$ Inside each fiber $(x,y)\times\mathbb Z/4\mathbb Z$, join the four vertices in a cycle. Add the external edges $$(x,y,0)(x,y+1,2)
|
||||
\qquad\text{and}\qquad
|
||||
(x,y,1)(x+1,y,3)$$ for every $(x,y)$. Equivalently, $(x,y,2)$ receives its external edge from $(x,y-1,0)$, and $(x,y,3)$ receives its external edge from $(x-1,y,1)$. Thus every vertex has two internal cycle neighbors and one external neighbor, and $Q_L$ is the four-cycle port replacement of the square torus $C_L\square C_L$.
|
||||
:::
|
||||
|
||||
Every vertex of $Q_L$ has degree three, and the construction embeds on the torus by replacing each base vertex inside a small disk. Its order is $4L^2$.
|
||||
|
||||
::: {#lem-doubling .exhibit .exhibit--lemma data-exhibit-type="lemma" data-exhibit-name="Lemma 17 (Vertex transitivity and explicit doubling of Q_L)"}
|
||||
**Lemma 17** (Vertex transitivity and explicit doubling of $Q_L$). *The graph $Q_L$ is vertex-transitive and, for every vertex $x$ and radius $R\ge0$, $$|B_{Q_L}(x,2R)|\le 5500\,|B_{Q_L}(x,R)|.$$*
|
||||
:::
|
||||
|
||||
::: proof
|
||||
*Proof.* Translations in the first two coordinates are automorphisms. The map $$\rho(x,y,i)=(y,-x,i+1)$$ (with coordinates interpreted cyclically) preserves internal cycle edges and interchanges the two external edge directions. Translations together with $\rho$ act transitively.
|
||||
|
||||
Let $T_L=C_L\square C_L$ and project $(x,y,i)$ to $(x,y)$. Projection does not increase distance, so $$|B_{Q_L}(x,2R)|\le4|B_{T_L}(\pi x,2R)|
|
||||
\le4\min\{L,4R+1\}^2.$$ Conversely, from an arbitrary cloud vertex one can enter the required port in at most two internal moves and then lift each base step using at most three moves. Hence, with $r=\lfloor(R-2)/3\rfloor$ for $R\ge2$, $$|B_{Q_L}(x,R)|\ge |B_{T_L}(\pi x,r)|.$$ The coordinate box of cyclic radius $\lfloor r/2\rfloor$ lies inside the $\ell_1$ ball, so $$|B_{T_L}(\pi x,r)|
|
||||
\ge \min\{L,2\lfloor r/2\rfloor+1\}^2.$$ For $R<10$, the upper bound is at most $4\cdot37^2<5500$ and the denominator is at least one. For $R\ge10$, one has $r\ge R/6$ and $2\lfloor r/2\rfloor+1\ge r$, whence the ratio is at most $4\cdot30^2<5500$. This proves the displayed constant. ◻
|
||||
:::
|
||||
|
||||
::: {#thm-cubic-barrier .exhibit .exhibit--theorem data-exhibit-type="theorem" data-exhibit-name="Theorem 18 (Cubic one-shot barrier)"}
|
||||
**Theorem 18** (Cubic one-shot barrier). *For $L\ge5$, the connected cubic graphs $Q_L$, with $M=|Q_L|=4L^2$, satisfy $$\boxed{
|
||||
h(Q_L)=\Theta(M^{-1/2}),
|
||||
\qquad
|
||||
c(Q_L)\le3,
|
||||
\qquad
|
||||
\operatorname{Occ}_R(Q_L)\ge \frac{M}{5500}
|
||||
\quad\text{for every }R\ge0.
|
||||
}$$*
|
||||
:::
|
||||
|
||||
::: proof
|
||||
*Proof.* The square torus has edge and vertex expansion $\Theta(1/L)$ by the discrete torus isoperimetric inequality [@BollobasLeader]. Applying [Proposition 7](#prop-port-exp) with cloud size four gives the matching lower bound for $Q_L$; lifting a coordinate slab of width $\lfloor L/2\rfloor$ gives the upper bound for every $L$. Since $Q_L$ is cubic, edge and vertex expansion differ by at most a constant factor. Thus $h(Q_L)=\Theta(1/L)=\Theta(M^{-1/2})$.
|
||||
|
||||
The graph $Q_L$ is toroidal, and every toroidal graph has cop number at most three [@Lehner]. Vertex transitivity, [Theorem 15](#thm-occ-lower), and [Lemma 17](#lem-doubling) give $$\operatorname{Occ}_R(Q_L)
|
||||
\ge
|
||||
M\frac{|B_{Q_L}(x,R)|}{|B_{Q_L}(x,2R)|}
|
||||
\ge \frac{M}{5500}.$$ The finite audit suggests that the optimal asymptotic constant is $1/4$, but that sharpening is not needed here. ◻
|
||||
:::
|
||||
|
||||
## The barrier throughout every polynomial expansion window
|
||||
|
||||
For integers $k\ge2$ and $L\ge4$, write $$T_{L,k}=\underbrace{C_L\square\cdots\square C_L}_{k\text{ factors}}.$$ Its order is $m=L^k$, its degree is $2k$, and its metric is the cyclic $\ell_1$ metric.
|
||||
|
||||
::: {#lem-ktorus-doubling .exhibit .exhibit--lemma data-exhibit-type="lemma" data-exhibit-name="Lemma 19 (Uniform doubling of Cartesian tori)"}
|
||||
**Lemma 19** (Uniform doubling of Cartesian tori). *For every fixed $k\ge2$, every $L\ge4$, every vertex $x$, and every radius $R\ge0$, $$|B_{T_{L,k}}(x,2R)|\le (5k)^k|B_{T_{L,k}}(x,R)|.$$*
|
||||
:::
|
||||
|
||||
::: proof
|
||||
*Proof.* Every coordinate of a point in $B(x,2R)$ has cyclic distance at most $2R$, so $$|B(x,2R)|\le \min\{L,4R+1\}^k.$$ The coordinate box in which every coordinate has cyclic distance at most $\lfloor R/k\rfloor$ lies in $B(x,R)$, and therefore $$|B(x,R)|\ge\min\{L,2\lfloor R/k\rfloor+1\}^k.$$ If $R<k$, the ratio is at most $(4k+1)^k$. If $R\ge k$, then $2\lfloor R/k\rfloor+1\ge R/k$ and $4R+1\le5R$; taking the minima with $L$ does not increase their ratio beyond $5k$. The claim follows. ◻
|
||||
:::
|
||||
|
||||
::: {#thm-full-window .exhibit .exhibit--theorem data-exhibit-type="theorem" data-exhibit-name="Theorem 20 (Full-window toroidal barrier)"}
|
||||
**Theorem 20** (Full-window toroidal barrier). *For every fixed $k\ge2$, the graphs $T_{L,k}$ satisfy $$\boxed{
|
||||
h(T_{L,k})=\Theta_k(m^{-1/k}),
|
||||
\qquad
|
||||
c(T_{L,k})=k+1,
|
||||
\qquad
|
||||
\operatorname{Occ}_R(T_{L,k})\ge (5k)^{-k}m
|
||||
\quad\text{for every }R\ge0.
|
||||
}$$ Consequently, for every $\delta>0$ there is a constant-degree graph family with $$h(G)\ge |G|^{-\delta},
|
||||
\qquad
|
||||
c(G)=O_\delta(1),
|
||||
\qquad
|
||||
\operatorname{Occ}_R(G)=\Omega_\delta(|G|)
|
||||
\quad\text{for every radius }R.$$*
|
||||
:::
|
||||
|
||||
::: proof
|
||||
*Proof.* The discrete-torus edge-isoperimetric inequality gives order $1/L$ [@BollobasLeader]. Since $T_{L,k}$ has degree $2k$, edge boundary and external vertex boundary differ by at most the fixed factor $2k$; a coordinate slab of width $\lfloor L/2\rfloor$ supplies the matching upper bound. Hence $h(T_{L,k})=\Theta_k(1/L)=\Theta_k(m^{-1/k})$. Neufeld and Nowakowski proved that a Cartesian product of $k$ cycles, each of length at least four, has cop number exactly $k+1$ [@NeufeldNowakowski]. Since the torus is vertex-transitive, [Theorem 15](#thm-occ-lower) and [Lemma 19](#lem-ktorus-doubling) give the occupation lower bound.
|
||||
|
||||
Given $\delta>0$, choose $k=\max\{2,\lfloor1/\delta\rfloor+1\}$. Then $1/k<\delta$, so for sufficiently large $m$ the expansion lower bound $h(T_{L,k})\ge m^{-\delta}$ holds after absorbing the fixed $k$-dependent constant. ◻
|
||||
:::
|
||||
|
||||
::: {#rem-sharp-metric-constant .exhibit .exhibit--remark data-exhibit-type="remark" data-exhibit-name="Remark 21 (The sharp metric constant)"}
|
||||
*Remark 21* (The sharp metric constant). For fixed $k$, choose radii $1\ll R\ll L$. Lattice-point asymptotics for the $\ell_1$ ball give $$\frac{|B_{T_{L,k}}(x,2R)|}{|B_{T_{L,k}}(x,R)|}=2^k+o(1).$$ Thus no uniform doubling constant below $2^k$ is possible, and the counting bound of [Theorem 15](#thm-occ-lower) approaches the natural fraction $2^{-k}m$ on these local radii. The explicit constant $(5k)^{-k}$ is chosen only for a short all-radii proof.
|
||||
:::
|
||||
|
||||
::: {#cor-no-exp-occ .exhibit .exhibit--corollary data-exhibit-type="corollary" data-exhibit-name="Corollary 22 (No expansion-only one-shot theorem)"}
|
||||
**Corollary 22** (No expansion-only one-shot theorem). *For every $\delta>0$, there is no implication of the form $$h(G)\ge |G|^{-\delta}
|
||||
\quad\Longrightarrow\quad
|
||||
\operatorname{Occ}_R(G)\le |G|^{1-\varepsilon}
|
||||
\text{ for some radius $R$}$$ with any fixed $\varepsilon>0$, even when the maximum degree is bounded by a constant depending only on $\delta$.*
|
||||
:::
|
||||
|
||||
::: proof
|
||||
*Proof.* Take the family from [Theorem 20](#thm-full-window). It lies in the prescribed expansion window, while $\operatorname{Occ}_R(G)=\Omega_\delta(|G|)$ for every $R$. ◻
|
||||
:::
|
||||
|
||||
::: {#rem-architectural-meaning .exhibit .exhibit--remark data-exhibit-type="remark" data-exhibit-name="Remark 23 (Architectural meaning)"}
|
||||
*Remark 23* (Architectural meaning). The logical obstruction is not that tori are difficult pursuit instances; they are not. Rather, [Theorem 15](#thm-occ-lower) makes every vertex-transitive bounded-doubling graph expensive for one-shot occupation, and bounded doubling is compatible with every polynomial weak-expansion window by [Theorem 20](#thm-full-window). The tori certify that the hypothesis class contains such graphs while coordinate-wise shadowing still uses exactly $k+1$ cops. Therefore ball amplification is the wrong invariant for adaptive pursuit, not merely a poor description of one particular easy family. The cubic family $Q_L$ records that the same separation already occurs in maximum degree three at exponent $1/2$.
|
||||
:::
|
||||
|
||||
# Outlook
|
||||
|
||||
For vertex expansion $h(G)\ge\phi$, iterating the elementary growth factor $1+\phi$ reaches global scale after $O(\phi^{-1}\log |G|)$ layers. The same iteration gives the standard diameter bound of that order; these are two forms of the same calculation, not independent evidence. The strategic consequence is that, when $\phi=|G|^{-a}$, an amplification-based pursuit scheme must operate over a full-traversal timescale $\Theta(|G|^a\log|G|)$.
|
||||
|
||||
The present paper separates three phenomena:
|
||||
|
||||
1. bounded normalized metric distortion preserves a strong occupation certificate through degree reduction;
|
||||
|
||||
2. the HMGHM stressing family itself meets the square-root endpoint up to polylogarithmic factors;
|
||||
|
||||
3. one-shot occupation is nevertheless incapable of proving a universal robustness theorem throughout any polynomial weak-expansion window, even on bounded-degree graphs with constant cop number; a cubic instance already appears at exponent $1/2$.
|
||||
|
||||
The remaining universal question is therefore an adaptive one. On the tori, ball growth carries essentially no information about pursuit cost; product structure instead supports coordinate-wise shadowing. What geometric or combinatorial quantity replaces product coordinates on a general polynomially weak expander? Equivalently, can a capacitated, correlated, or deferred witness system reuse the same cop resources over polynomially many weak-growth layers, or must every such one-traversal certificate incur polynomial congestion?
|
||||
|
||||
# Acknowledgments
|
||||
|
||||
The author is grateful to Anthony Clow, Peter Bradshaw, Bojan Mohar, and Florian Lehner for work and perspectives that helped shape the questions addressed here. Additional acknowledgments will be added in a later version. The author welcomes corrections concerning priority, related graph-substitution inequalities, and the scope of the occupation framework.
|
||||
|
||||
# Audit and reproducibility
|
||||
|
||||
The metric inequalities were independently tested on HMGHM towers rebuilt from the published gadget description, including structured base graphs not used in the original audit. The Hall inequalities and timing margins were checked numerically, and exact small replacement games were solved by retrograde analysis. The toroidal barrier audit computes exact ball profiles of $C_L^{\square k}$ by convolving cyclic distance distributions, verifies the $(5k)^k$ doubling bound for $k=2,3,4,5$, constructs $Q_L$, checks cubicity, connectivity, and the displayed rotation automorphism, and evaluates the counting lower bound at every radius. No theorem depends on the computations.
|
||||
Binary file not shown.
|
After Width: | Height: | Size: 183 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 631 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 370 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 345 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 85 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 88 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 52 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 80 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 207 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 232 KiB |
|
|
@ -0,0 +1,369 @@
|
|||
---
|
||||
title: "Beyond Comorbidity Indices"
|
||||
date: 2026-04-09
|
||||
abstract: >
|
||||
A deep learning model using ICD-10-CM diagnosis codes with a permutation-invariant Deep Sets aggregator improved 30-day unplanned readmission (AUC 0.7496 vs 0.6553 for CCI) and 30-day postdischarge in-hospital mortality (AUC 0.8557 vs 0.7844 for age-adjusted CCI) compared with Charlson and Elixhauser comorbidity-index benchmarks in a national claims database of over 113 million adult hospitalizations.
|
||||
tags:
|
||||
- research
|
||||
- research/machine-learning
|
||||
authors:
|
||||
- "Levi Neuwirth | /me.html"
|
||||
- "Liqi Shu"
|
||||
- "Xilin Wang"
|
||||
- "Henry Zheng"
|
||||
affiliation:
|
||||
- "Department of Neurology, Warren Alpert Medical School, Brown University"
|
||||
- "Department of Computer Science, Brown University | https://cs.brown.edu"
|
||||
- "Department of Mathematics, Brown University | https://mathematics.brown.edu/"
|
||||
- "Department of Computer Science, Northeastern University"
|
||||
status: "Durable"
|
||||
confidence: 80
|
||||
importance: 3
|
||||
evidence: 5
|
||||
scope: average
|
||||
novelty: moderate
|
||||
practicality: moderate
|
||||
peer-status: under-review
|
||||
result-shape: comparative
|
||||
bibliography: data/bci-paper.bib
|
||||
repository: "https://git.levineuwirth.org/neuwirth/beyond_comorbidity_indices"
|
||||
summary: |
|
||||
**Question.** Among adult hospitalizations in a national claims database, does a deep learning model using ICD-10-CM diagnosis codes improve prediction of 30-day unplanned readmission and 30-day postdischarge in-hospital mortality compared with benchmark models based on Charlson and Elixhauser comorbidity indices?
|
||||
|
||||
**Findings.** In this cohort study of 3,226,831 temporally held-out discharges, the ICD-10-CM--based model showed better discrimination than benchmark comorbidity-index models for both outcomes.
|
||||
|
||||
**Meaning.** Using the full set of discharge diagnosis codes may improve short-term claims-based outcome prediction beyond summary comorbidity indices.
|
||||
history:
|
||||
- date: "2026-03-28"
|
||||
note: Preprint auto-formatted for levineuwirth.org
|
||||
---
|
||||
|
||||
## Introduction (Background and Significance)
|
||||
|
||||
::: dropcap
|
||||
Accurate prediction and risk adjustment for short-term clinical outcomes, such as 30-day mortality and readmission, are critical for enhancing healthcare research quality, allowing fair assessment of healthcare outcomes and quality metrics [@cms_hrrp]. Most claims-based risk adjustment continues to rely on comorbidity indices such as the Charlson Comorbidity Index (CCI) and Elixhauser Comorbidity Index (ECI), which map diagnosis codes to a limited set of conditions [@charlson1987; @elixhauser1998]. While these indices are interpretable and widely deployed, they inevitably discard granularity and may miss clinically meaningful comorbidity patterns and interactions among diagnoses.
|
||||
:::
|
||||
|
||||
Recent machine-learning approaches increasingly use a set of ICD-10-CM diagnosis codes and have demonstrated improved prediction for a range of outcomes [@deschepper2020; @lelay2022; @qiao2022]. However, many approaches simplify or truncate ICD codes, aggregate diagnosis lists in ways that depend on code order, or are trained and evaluated in settings where coding practices differ across sites---each of which can limit generalizability across settings. In addition, many claims-based studies focus on in-hospital mortality and do not evaluate postdischarge mortality among outcomes relevant at the time of discharge [@qiao2022; @davis2022; @harerimana2021; @matsui2022; @nguyen2017].
|
||||
|
||||
In this study, we developed and temporally validated a claims-based deep learning model using ICD-10-CM diagnosis codes to predict 30-day unplanned readmission and 30-day postdischarge mortality in the Nationwide Readmissions Database. We compared its performance with benchmark models based on the Charlson and Elixhauser comorbidity indices, which are widely used for claims-based risk adjustment but were not originally designed for these specific outcomes. We also evaluated the model with different architectural design and examined diagnosis-level contributions to model predictions.
|
||||
|
||||
## Materials and Methods
|
||||
|
||||
### Study Design, Data Source, and Oversight
|
||||
|
||||
We conducted a retrospective cohort study using the Healthcare Cost and Utilization Project (HCUP) Nationwide Readmissions Database (NRD), 2016--2022. Adult discharges from 2016 through 2020 were used for model development, and a later temporally separated cohort from 2021 through 2022 was reserved for temporal validation. Discharges in December of each year were excluded to allow complete 30-day follow-up within the same calendar year.
|
||||
|
||||
Use of the NRD was governed by the HCUP data use agreement. Because the NRD contains deidentified data, the institutional review board determined the study was not human participants research and that informed consent was not required.
|
||||
|
||||
### Cohort Definition
|
||||
|
||||
We included hospitalizations for patients aged 18 years or older with a valid patient linkage identifier within each calendar year. For both the readmission and mortality analyses, index hospitalizations ending in in-hospital death were excluded because patients were not at risk for postdischarge outcomes. In-hospital death during the index hospitalization was examined in a prespecified secondary mortality analysis (eResults 1).
|
||||
|
||||
### Outcomes
|
||||
|
||||
The coprimary outcomes were (1) 30-day unplanned readmission and (2) 30-day postdischarge in-hospital mortality (hereafter, postdischarge mortality). Readmissions were classified as unplanned if they were coded as nonelective admissions in the HCUP database. Postdischarge mortality was defined as inpatient death occurring during a subsequent hospitalization within 30 days after discharge. Deaths outside the hospital are not captured in the NRD.
|
||||
|
||||
### Predictors
|
||||
|
||||
For each index hospitalization, we used up to 40 ICD-10-CM diagnosis codes (principal and secondary) and patient-level covariates (age, sex, primary payer, and ZIP-code median income quartile). Age was standardized, and categorical variables were represented using one-hot encoding. Analyses were restricted to records with nonmissing outcome ascertainment and complete covariates.
|
||||
|
||||
### Comparator Models
|
||||
|
||||
For benchmarking, we computed the Elixhauser Comorbidity Index (ECI) and Charlson Comorbidity Index (CCI) for each index hospitalization and treated each index as a continuous risk score [@charlson1987; @elixhauser1998]. The ECI identifies 30+ distinct conditions from administrative data, serving as a critical tool for risk adjustment in studies evaluating in-hospital mortality and short-term readmissions [@elixhauser1998; @ahrq_elixhauser; @fernando2019; @quan2005]. The CCI consolidates up to 19 comorbid conditions into a weighted numeric score, including variants that adjust for age, primarily predicting long-term mortality and readmissions [@charlson1987; @fernando2019; @quan2005; @deyo1992; @quan2011]. The ECI was computed using an ICD-10-CM--adapted AHRQ approach that identifies chronic comorbidities primarily from secondary diagnoses [@quan2005]. The CCI was computed using ICD-10-CM mappings to 17 comorbidity categories; both raw CCI and age-adjusted CCI were evaluated [@stagg2006]. These benchmark models used the index score alone as the predictor; age, sex, primary payer, and ZIP-code income quartile were not added separately. Discrimination and threshold-dependent classification metrics were derived directly from the score distributions, with operating thresholds selected on the validation set and then applied unchanged to the temporal test evaluation subsample.
|
||||
|
||||
### Model Architecture
|
||||
|
||||
We developed a deep learning framework that embeds each patient's diagnosis list along with demographic and socioeconomic information to predict the outcome. Each ICD-10-CM code was mapped into a dense vector representation through a learned numerical transformation. To obtain a single representation for each patient while avoiding reliance on diagnosis ordering, we used an aggregation approach that does not depend on code order:
|
||||
|
||||
$$f(x) = \rho\!\left(\sum_{x \in X} \phi(x)\right)$$
|
||||
|
||||
where $X$ denotes the set of embedded diagnosis vectors. Functions $\phi$ and $\rho$ were implemented as multilayer perceptrons with ReLU activations [@zaheer2017].
|
||||
|
||||
Demographic and socioeconomic variables were processed via a separate 2-layer multilayer perceptron. The resulting vector was concatenated with the aggregated diagnosis representation and passed through fully connected layers with ReLU activations and dropout regularization. A sigmoid output layer finally produced a predicted probability for each outcome.
|
||||
|
||||
### Model Development and Temporal Validation
|
||||
|
||||
Data from 2016--2020 were split into training (90%) and validation (10%) sets. For each outcome, models were trained to minimize binary cross-entropy loss. To address class imbalance, majority-class downsampling was applied during training (see Supplementary eMethods 1). Because majority-class downsampling altered the effective outcome prevalence in the training data, predicted probabilities were corrected using the original training-set prevalence before reporting calibration, temporal-test probabilities, and web-calculator outputs [@pozzolo2015]. This deterministic correction affects probability scaling but not rank-based discrimination. Hyperparameters (embedding dimension, Deep Sets depth/width, demographic tower width, predictor multilayer perceptron configuration, and dropout rate) were tuned using random search; the configuration with best validation AUROC (with recall-weighted metrics used as secondary criteria) was selected (see Supplementary eTable 1).
|
||||
|
||||
Temporal validation was based on eligible 2021--2022 discharges. To support computational feasibility while preserve outcome prevalence, primary performance evaluation was conducted in a prespecified stratified random subsample of the eligible 2021--2022 temporal test cohort, with 10% of outcome-positive and 10% of outcome-negative discharges sampled for each outcome (Supplementary eMethods 2). Models were implemented in Python using TensorFlow [@abadi2016].
|
||||
|
||||
### Performance Metrics
|
||||
|
||||
For threshold-dependent metrics, binary classification thresholds were selected on the validation set by maximizing the Youden index (sensitivity + specificity − 1) and then applied unchanged to the temporal test evaluation subsample for each model [@youden1950].
|
||||
|
||||
Because outcomes were imbalanced, we emphasized discrimination and precision--recall performance. Primary metrics included AUROC with 95% confidence intervals (CIs), average precision, precision, recall, $F_1$ score, and $F_2$ score (placing greater weight on recall).
|
||||
|
||||
### Statistical Analysis
|
||||
|
||||
AUROCs and their 95% CIs were estimated using DeLong's nonparametric method [@delong1988]. Pairwise comparisons in AUROC between the embedding model and each comorbidity-index comparator were performed using DeLong tests for correlated ROC curves [@delong1988]. Resulting $P$ values are unadjusted and interpreted alongside effect sizes and 95% CIs.
|
||||
|
||||
We conducted prespecified ablation analyses to estimate the incremental contribution of key model components, including addition of transformer blocks, replacement of the order-invariant Deep Sets aggregator with a permutation-variant flattening comparator, and removal of demographic and socioeconomic inputs; details are provided in Supplementary eMethods 4.
|
||||
|
||||
### Model Interpretation
|
||||
|
||||
We used Integrated Gradients (IG) to estimate code-level contributions to model predictions for each outcome [@sundararajan2017; @placido2023]. Attribution values were summarized at the ICD-10-CM code level, with positive values indicating higher predicted risk and negative values indicating lower predicted risk. To reduce instability from rare codes, ranked summaries were restricted to codes with at least 50 occurrences in the temporal test evaluation subsample. Additional implementation details are provided in Supplementary eMethods 3.
|
||||
|
||||
### Web Application
|
||||
|
||||
A public, read-only web calculator accepts discharge diagnosis lists and returns risk estimates with code-level explanations; inputs are not stored. The tool is intended for research and demonstration purposes rather than clinical decision-making. The calculator is available at: <https://levineuwirth.github.io/icd_embeddings/>. Implementation details are provided in Supplementary eFigure 4.
|
||||
|
||||
## Results
|
||||
|
||||
### Cohort size and event prevalence
|
||||
|
||||
The NRD included 80,217,696 discharges in 2016--2020 for model development and 33,322,761 discharges in 2021--2022 for temporal testing (Figure 1). After application of outcome-specific eligibility criteria, the validation cohort included 7,828,015 discharges. Primary performance evaluation was conducted in a prespecified stratified random subsample of 3,226,831 discharges from the eligible 2021--2022 temporal test cohort, and this evaluation subsample was not downsampled. For model training, majority-class downsampling was applied to address class imbalance, yielding analytic training samples of 17,200,994 discharges for the readmission analysis and 544,138 discharges for the postdischarge mortality analysis. In the temporal test evaluation subsample, 30-day unplanned readmission occurred in 362,696 discharges (11.2%), and 30-day postdischarge in-hospital mortality occurred in 13,071 discharges (0.4%).
|
||||
|
||||
### Model Performance
|
||||
|
||||
Detailed performance metrics for the ICD-10-CM--based model and benchmark comorbidity-index models are shown in Table 1. In the temporal test evaluation subsample, the ICD-10-CM--based model showed higher discrimination than comparator models for both 30-day unplanned readmission and 30-day postdischarge mortality (Figure 2). For readmission, the AUROC was 0.750 (95% CI, 0.749--0.750) for the ICD-10-CM--based model, compared with 0.655 (95% CI, 0.654--0.656) for the CCI model, 0.644 (95% CI, 0.644--0.645) for the age-adjusted CCI model, and 0.636 (95% CI, 0.635--0.637) for the ECI model. For postdischarge mortality, the AUROC was 0.856 (95% CI, 0.853--0.858) for the ICD-10-CM--based model, compared with 0.784 (95% CI, 0.781--0.787) for the best-performing comparator, the age-adjusted CCI model; the AUROC for the ECI model was 0.641 (95% CI, 0.636--0.647). DeLong tests comparing the ICD-10-CM--based model with each comorbidity-index comparator were significant for all pairwise comparisons ($P < .001$). Calibration curves showed overall agreement between predicted and observed risk for both outcomes, with greater deviation at higher predicted-risk ranges; this deviation was less pronounced for postdischarge mortality than for readmission (Figure 3).
|
||||
|
||||
At the prespecified threshold selected on the validation set, the ICD-10-CM--based model showed higher recall-weighted performance than comparator models, with $F_2$ scores of 0.485 vs 0.407 for 30-day readmission and 0.053 vs 0.048 for postdischarge mortality. Threshold-dependent metrics, including precision, recall, and specificity, are shown in Table 1. Because the classification threshold was selected using the Youden index, these metrics reflect a balance of sensitivity and specificity rather than optimization for a specific clinical use case. For readmission, these gains were accompanied by only modest precision, consistent with the difficulty of predicting this heterogeneous outcome.
|
||||
|
||||
In the prespecified secondary analysis expanding mortality to include in-hospital death during the index hospitalization, the ICD-10-CM--based model achieved an AUROC of 0.965 (95% CI, 0.965--0.966), exceeding that of the best-performing comparator model (age-adjusted CCI: AUROC, 0.750 [95% CI, 0.749--0.751]) (eTable 2).
|
||||
|
||||
### Ablation Studies
|
||||
|
||||
We evaluated a set of prespecified model variants that removed or augmented architectural components (eg, ICD-only inputs and insertion of transformer blocks) to estimate the incremental contribution of each element. We also compared the order-invariant aggregation approach with an order-dependent flattening-based aggregator to quantify any performance tradeoff attributable to enforcing invariance. In covariate ablation, removing demographic and socioeconomic inputs (age, sex, payer, and ZIP-income quartile) caused modest attenuation in performance (readmission AUROC, 0.750 vs 0.748; postdischarge mortality AUROC, 0.856 vs 0.848; similar $F_2$ scores), suggesting that diagnosis patterns captured most, but not all, of the predictive signal. Implementation details are provided in Supplementary eMethods 4; results are summarized in eTable 3.
|
||||
|
||||
### Feature Importance
|
||||
|
||||
ICD-10-CM codes with the 10 highest positive and negative contributions to both prediction outcomes are shown in Figure 4. For 30-day readmission, acute myeloblastic leukemia, in relapse (C9202), had the greatest positive contribution, whereas encounter for care and examination of mother immediately after delivery (Z390) had the greatest negative contribution. For 30-day postdischarge mortality, C9202 also had the greatest positive contribution, whereas assault by unspecified sharp object, initial encounter (X999XXA) had the greatest negative contribution. The most influential diagnosis codes for 30-day mortality prediction including inpatient death are shown in Supplementary eFigure 2.
|
||||
|
||||
## Discussion
|
||||
|
||||
In this national claims-based cohort study, a deep learning model using the full set of discharge diagnosis codes showed better discrimination than benchmark models based on Charlson and Elixhauser comorbidity indices for both 30-day unplanned readmission and 30-day postdischarge in-hospital mortality. The performance gain was larger for postdischarge mortality than for readmission. Performance remained favorable in a later, temporally separated NRD cohort, supporting robustness across subsequent years of the same database [@davis2020; @collins2024]. At the same time, these comparisons should be interpreted as benchmarking against widely used summary comorbidity approaches rather than as head-to-head comparisons with models purpose-built for these exact outcomes.
|
||||
|
||||
### Comparison with prior work
|
||||
|
||||
This pattern is consistent with the structure of the compared methods. Charlson and Elixhauser indices compress diagnosis information into a limited set of predefined conditions and were designed primarily for broad case-mix adjustment rather than high-resolution outcome prediction [@charlson1987; @elixhauser1998]. By contrast, the present model learns from the full diagnosis-code set and can represent co-occurrence patterns that are not captured by summary indices [@morgan2019; @beam2018]. Unlike many prior deep-learning approaches that depend on richer electronic health record inputs and site-specific preprocessing, this framework was designed for portability within claims-based settings by using routinely available diagnosis, demographic, and payer-related variables [@rajkomar2018]. This design also preserves the broader diagnostic context of each hospitalization rather than reducing diagnoses to fixed summary weights as ECI and CCI.
|
||||
|
||||
### Interpretability
|
||||
|
||||
Interpretability in this setting should not be viewed as an afterthought to an otherwise opaque model. Using Integrated Gradients, the model provided code-level attributions that were generally clinically plausible and helped explain why predicted risk increased or decreased for a given patient. For example, diagnoses associated with high treatment burden or advanced systemic illness, such as relapsed acute myeloid leukemia and alcoholic cirrhosis with ascites, tended to increase predicted risk, whereas postpartum encounters and some assault-related injuries tended to decrease it. These findings suggest that learning-based models can yield clinically meaningful information rather than functioning only as "black boxes," even when they are more flexible than traditional summary indices [@sundararajan2017; @placido2023; @rudin2019].
|
||||
|
||||
One plausible explanation for the performance gap between the present model and the Charlson and Elixhauser indices is that the prognostic contribution of a diagnosis is not fixed across patients. Summary comorbidity indices assign prespecified, static weights to diagnosis groups, effectively assuming that a given condition contributes similarly regardless of the broader diagnostic context. By contrast, in the present model, the contribution of a diagnosis could vary according to the full set of co-occurring diagnoses, which is more consistent with how risk is often understood clinically. We did not directly test this mechanism, so it should be interpreted as a hypothesis supported by the attribution patterns rather than as a proven explanation for the observed performance differences. Nevertheless, this dynamic view of diagnosis contribution may help explain why retaining the full diagnosis-code context improved prediction beyond summary comorbidity scores. As with other attribution methods, these explanations improve transparency but do not establish causality.
|
||||
|
||||
### Clinical and policy implications
|
||||
|
||||
These findings have two potential implications. First, in discharge-facing workflows, a claims-compatible model could be evaluated in read-only settings to identify patients who may warrant closer follow-up, medication reconciliation, or transitional-care outreach [@hansen2011; @coleman2006]. Second, in research and quality measurement, more granular use of diagnosis data may improve outcome prediction when summary comorbidity indices underrepresent diagnostic complexity [@joynt2013; @desai2016; @zuckerman2017]. This tool is intended for clinical prioritization and equitable quality measurement, not for coverage denial or utilization gatekeeping [@obermeyer2019].
|
||||
|
||||
Because demographic and socioeconomic factors are known to influence postdischarge outcomes, we assessed their incremental contribution beyond diagnosis patterns using ablation [@kind2014; @joynt2011]. Removing age, sex, payer, and neighborhood income produced minimal changes in performance, suggesting that much of the predictive signal available to this model was already captured by diagnosis patterns. This finding should not be interpreted to mean that demographic or socioeconomic factors are unimportant. Rather, within this claims-based framework, coded diagnoses may already capture part of the risk signal associated with demographic and socioeconomic differences, whether through differences in disease burden, comorbidity clustering, or patterns of healthcare use.
|
||||
|
||||
The public read-only calculator is intended for research and demonstration rather than clinical deployment. Future work should focus on external validation, prospective evaluation in read-only workflows, monitoring for coding and case-mix drift, and recalibration when needed [@davis2020; @collins2024; @collins2015].
|
||||
|
||||
### Limitations
|
||||
|
||||
This study has several limitations. First, the NRD captures deaths only during inpatient encounters; therefore, the mortality outcome reflects postdischarge in-hospital mortality rather than all-cause 30-day mortality. Second, claims data are subject to coding error and variation and do not directly capture functional status, physiologic severity, or many social risk factors. Third, although temporal validation in later NRD years reduces optimism, it is not a substitute for external validation, and performance may differ in other health systems or data sources with different coding practices, case mix, and discharge workflows. Fourth, this study evaluated predictive performance rather than downstream improvement in confounding control, hospital profiling, or other risk-adjustment applications; thus, better discrimination does not by itself establish superior risk adjustment, and because the model was trained specifically for 30-day unplanned readmission and postdischarge in-hospital mortality, performance may not generalize to other outcomes without separate validation. Fifth, Charlson and Elixhauser indices were included as benchmark comparators because of their widespread use in claims-based analyses, but they were not originally developed for these specific outcomes; accordingly, these comparisons should be interpreted as benchmarking rather than definitive head-to-head testing. Finally, attribution methods may improve transparency but do not establish causality [@rudin2019].
|
||||
|
||||
## Conclusions
|
||||
|
||||
A deep learning model using ICD-10-CM diagnosis codes improved prediction of 30-day unplanned readmission and 30-day postdischarge mortality compared with Charlson and Elixhauser comorbidity-index models in the Nationwide Readmissions Database. Prospective validation, drift monitoring, and attention to intended use will be essential before implementation for clinical decision support or policy applications.
|
||||
|
||||
## Data Sharing Statement
|
||||
|
||||
The study used de-identified data from the Healthcare Cost and Utilization Project (HCUP) Nationwide Readmissions Database under a data-use agreement. Data are available from HCUP to qualified researchers.
|
||||
|
||||
## Code Availability
|
||||
|
||||
The analytic code (including the non-elective readmission implementation, model training, and evaluation) will be made publicly available at publication in a GitHub repository (<https://github.com/Rice-wxl/icd-10-embedding>), with a versioned release tag/commit to support reproducibility. HCUP NRD data cannot be shared by the authors under the data-use agreement.
|
||||
|
||||
## Conflict of Interest Disclosures
|
||||
|
||||
The authors report no conflicts of interest related to this work.
|
||||
|
||||
::: aftermatter
|
||||
|
||||
## Tables
|
||||
|
||||
### Table 1: Performance metrics for the ICD model vs. CCI and ECI {#table-1}
|
||||
|
||||
Performance metrics for the ICD model vs. CCI and ECI for 30-day readmission (a) and 30-day postdischarge mortality (b) in the temporal test evaluation subsample.
|
||||
|
||||
**(a) 30-day readmission**
|
||||
|
||||
| Methods | AUC-ROC | Accuracy | Precision | Recall | $F_1$ | $F_2$ |
|
||||
|:-------------------------------|:-----------------------------|---------:|-----------:|-----------:|-----------:|-----------:|
|
||||
| ICD Model (Threshold: 0.5022) | **0.7496** [0.7488, 0.7504] | 0.5892 | **0.1881** | **0.8006** | **0.3046** | **0.4848** |
|
||||
| CCI | 0.6553 [0.6544, 0.6562] | **0.6962** | 0.1844 | 0.4973 | 0.2690 | 0.3713 |
|
||||
| CCI Age-Adjusted | 0.6444 [0.6435, 0.6453] | 0.6479 | 0.1673 | 0.5360 | 0.2550 | 0.3720 |
|
||||
| ECI | 0.6363 [0.6353, 0.6372] | 0.5708 | 0.1598 | 0.6622 | 0.2575 | 0.4066 |
|
||||
|
||||
**(b) 30-day postdischarge mortality**
|
||||
|
||||
| Methods | AUC-ROC | Accuracy | Precision | Recall | $F_1$ | $F_2$ |
|
||||
|:-------------------------------|:-----------------------------|-----------:|-----------:|-----------:|-----------:|-----------:|
|
||||
| ICD Model (Threshold: 0.4644) | **0.8557** [0.8532, 0.8581] | 0.6848 | **0.0111** | **0.8756** | **0.0220** | **0.0530** |
|
||||
| CCI | 0.7621 [0.7585, 0.7657] | 0.6987 | 0.0093 | 0.6963 | 0.0184 | 0.0442 |
|
||||
| CCI Age-Adjusted | 0.7844 [0.7813, 0.7874] | 0.7352 | 0.0102 | 0.6700 | 0.0201 | 0.0480 |
|
||||
| ECI | 0.6414 [0.6358, 0.6469] | **0.7763** | 0.0089 | 0.4915 | 0.0175 | 0.0415 |
|
||||
|
||||
*Primary performance evaluation used a prespecified stratified random subsample of eligible 2021--2022 discharges. Classification thresholds were selected on the validation set by maximizing the Youden index and then applied unchanged to the temporal test evaluation subsample.*
|
||||
|
||||
## Figures
|
||||
|
||||
::: {.annotation .annotation--static}
|
||||
**Figure 1 --- [placeholder]** Flow chart of discharge records in NRD for training, validation, and testing cohorts. Primary performance evaluation used a prespecified stratified random subsample of eligible 2021--2022 discharges. This temporal-test evaluation subsample was not downsampled. Classification thresholds were selected on the validation set by maximizing the Youden index and then applied unchanged to the temporal test subsample.
|
||||
:::
|
||||
|
||||
**Figure 2.** Receiver operating characteristic (ROC) curves and area under the curve (AUC) for 30-day readmission and postdischarge mortality in the temporal test evaluation subsample. Each curve depicts the trade-off between sensitivity and specificity across different thresholds.
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
**Figure 3.** Calibration curves on temporal test evaluation subsample for 30-day readmission and postdischarge mortality.
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
**Figure 4.** Top influential ICD codes for model prediction. Mean Integrated Gradients attribution per occurrence (positive values indicate higher predicted risk; negative values indicate lower predicted risk).
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
## Supplement
|
||||
|
||||
### eMethods
|
||||
|
||||
1. **Training and Hyperparameter Tuning**
|
||||
2. **Temporal Test Set Construction and Threshold Selection**
|
||||
3. **Integrated Gradients for Code-Level Attribution**
|
||||
4. **Ablation Analyses**
|
||||
1. Transformer Blocks
|
||||
2. Deep Sets vs Permutation-Variant Flattening Comparator
|
||||
3. Demographic and Socioeconomic Inputs
|
||||
|
||||
### eTables
|
||||
|
||||
1. **Hyperparameter Configurations Used in Models**
|
||||
2. **Performance Comparison for 30-Day Mortality Including Index-Hospital Death**
|
||||
3. **Ablation Study Results**
|
||||
- (A) Addition of Transformer Blocks
|
||||
- (B) Replacement of Deep Sets With Flattening Comparator
|
||||
- (C) Removal of Demographic and Socioeconomic Inputs (ICD-Only)
|
||||
|
||||
### eFigures
|
||||
|
||||
1. **Permutation-invariant ICD Embedding Model**
|
||||
2. **Top ICD-10-CM Codes by Integrated Gradients for 30-Day Mortality Including Index-Hospital Death**
|
||||
3. **Calibration Reliability Plots for Temporal Test Set Predictions**
|
||||
- 30-Day Readmission
|
||||
- 30-Day Postdischarge Mortality
|
||||
4. **Web Calculator Interface Examples**
|
||||
|
||||
### eMethods 1. Training and Hyperparameter Tuning
|
||||
|
||||
Models were trained using batch size 128 for 10 epochs with the Adam optimizer at a learning rate of 2e-5. Early stopping was applied with patience of 2 epochs, retaining the checkpoint with the best validation performance. To address outcome imbalance, we randomly downsampled majority-class encounters in the training set to achieve a target case-control ratio of 1:1 (validation data and temporal test evaluation subsample were not downsampled). Because majority-class downsampling changes the effective outcome prevalence and can bias predicted probabilities, we performed adjustment with the original downsampling ratio of the training set and use the readjusted model outputs in all metrics/plots and for the web calculator. Random seeds for the train/validation split, downsampling, and model initialization were set to ensure reproducibility.
|
||||
|
||||
Hyperparameters were tuned via random search (32 trials per outcome). eTable 1 reports the selected configurations. Hyperparameter definitions: $d_{\text{embed}}$ (ICD embedding dimension); $d_{\text{hidden}}$ (Deep Sets hidden dimension); $r_{\text{deepset}} \times d_{\text{hidden}}$ (Deep Sets output dimension); $n_{\text{encode}}$ and $n_{\text{decode}}$ (numbers of Deep Sets encoding/decoding layers); $d_{\text{demo}}$ (first-layer width of the demographic/socioeconomic MLP; second layer set to $d_{\text{demo}}/2$); $d_{\text{mlp}}$ (first-layer width of the predictor MLP, halving each layer to a minimum of 32 over 4 layers); $r_{\text{dropout}}$ (dropout rate). Model selection prioritized validation AUROC; recall-weighted metrics (including $F_2$) were used as secondary criteria.
|
||||
|
||||
### eMethods 2. Temporal Test Set Construction and Threshold Selection
|
||||
|
||||
Temporal testing used eligible 2021--2022 data. To enable computationally feasible evaluation while preserving outcome prevalence, we created the temporal test evaluation subsample by stratified random sampling of the combined eligible 2021--2022 cohort, sampling 10% of outcome-positive and 10% of outcome-negative discharges for each outcome. This yielded approximately 3.2 million discharges for primary performance evaluation. Interpretability analyses used the same temporal test evaluation subsample.
|
||||
|
||||
Binary classification thresholds for each model were selected on the validation set by maximizing the Youden index (sensitivity + specificity − 1) and then applied unchanged to the temporal test evaluation subsample.
|
||||
|
||||
### eMethods 3. Integrated Gradients for Code-Level Attribution
|
||||
|
||||
Integrated Gradients (IG) was used to quantify code-level influence on model predictions. The baseline input was defined as a neutral input corresponding to an empty diagnosis list (ie, no diagnosis codes). The straight-line interpolation path from baseline to the observed input was discretized into 32 steps. At each step, gradients of the model logit were computed with respect to diagnosis embeddings; gradients were accumulated across steps and summed across embedding dimensions to yield a scalar attribution per code occurrence. Attribution values retained sign, with positive values indicating higher predicted risk and negative values indicating lower predicted risk. To reduce instability from rare codes, ICD-10-CM codes with fewer than 50 total occurrences in the temporal test evaluation subsample were excluded from ranked summaries. For each outcome, we reported the 10 codes with the largest mean positive attributions and the 10 codes with the largest mean negative attributions.
|
||||
|
||||
### eMethods 4. Ablation Analyses
|
||||
|
||||
#### eMethods 4.1 Transformer blocks
|
||||
|
||||
To evaluate whether attention-based contextualization improves performance, we added three multi-head transformer blocks operating over individual ICD embeddings. Each block followed a standard transformer design with multi-head attention, residual connections, normalization, and feed-forward sublayers. We used 3 attention heads, dropout 0.3, embedding dimension $d_{\text{embed}}$, and feed-forward dimension $4 \times d_{\text{embed}}$.
|
||||
|
||||
Comparative results are shown in eTable 3, Panel A. Across both outcomes, transformer blocks did not materially improve AUROC and $F_2$ score relative to the base model. Reported $P$ values for AUROC differences were $P < .001$ for readmission and $P = 0.57$ for postdischarge mortality.
|
||||
|
||||
#### eMethods 4.2 Deep Sets vs permutation-variant flattening comparator
|
||||
|
||||
To test whether permutation invariance via Deep Sets reduced predictive performance, we compared the base model with a permutation-variant alternative: a flattening layer that converts the 2-dimensional ICD embedding matrix into a 1-dimensional vector, followed by two MLP layers with $d_{\text{hidden}}$ and $r_{\text{deepset}} \times d_{\text{hidden}}$ units to mirror the Deep Sets hidden/output sizes.
|
||||
|
||||
Results are shown in eTable 3, Panel B. The base Deep Sets models outperformed the flattening comparators on AUROC and $F_2$ score. Reported $P$ values for AUROC differences were $P < .001$ for readmission and $P = 0.014$ for postdischarge mortality.
|
||||
|
||||
#### eMethods 4.3 Demographic and socioeconomic inputs
|
||||
|
||||
To evaluate the incremental value of non-diagnosis covariates, we removed the 2-layer demographic/socioeconomic MLP and trained ICD-only variants.
|
||||
|
||||
Results are shown in eTable 3, Panel C. ICD-only variants had slightly worse AUROC and $F_2$ score. Reported $P$ values for AUROC differences were $P = 0.020$ for readmission and $P < .001$ for postdischarge mortality.
|
||||
|
||||
### eTable 1: Hyperparameter configurations used in models
|
||||
|
||||
| Outcome | $d_{\text{embed}}$ | $d_{\text{hidden}}$ | $r_{\text{deepset}}$ | $n_{\text{encode}}$ | $n_{\text{decode}}$ | $d_{\text{demo}}$ | $d_{\text{mlp}}$ | $r_{\text{dropout}}$ |
|
||||
|:--------------------------------------------------|---:|----:|----:|---:|---:|---:|----:|----:|
|
||||
| 30-day readmission | 32 | 416 | 0.5 | 1 | 3 | 64 | 480 | 0.1 |
|
||||
| 30-day postdischarge mortality | 64 | 320 | 0.6 | 2 | 1 | 64 | 384 | 0.1 |
|
||||
| 30-day mortality including index-hospital death | 64 | 416 | 0.8 | 3 | 3 | 64 | 448 | 0.4 |
|
||||
|
||||
*The model configurations are determined through hyperparameter tuning for each outcome variable, respectively. All models share the same batch size and learning rate.*
|
||||
|
||||
### eTable 2. Performance comparison for 30-day mortality including index-hospital death
|
||||
|
||||
| Methods | AUC-ROC | Precision | Recall | $F_1$ | $F_2$ |
|
||||
|:-----------------|:-------------------------|----------:|-------:|-------:|-------:|
|
||||
| ICD Model | 0.9651 [0.9647, 0.9656] | 0.2107 | 0.9165 | 0.3427 | 0.5489 |
|
||||
| CCI | 0.7217 [0.7203, 0.7231] | 0.0663 | 0.6270 | 0.1200 | 0.2331 |
|
||||
| CCI Age-Adjusted | 0.7501 [0.7489, 0.7513] | 0.0724 | 0.6043 | 0.1294 | 0.2448 |
|
||||
| ECI | 0.6158 [0.6139, 0.6177] | 0.0637 | 0.4427 | 0.1114 | 0.2022 |
|
||||
|
||||
### eTable 3. Ablation study results
|
||||
|
||||
**(A) Addition of transformer blocks**
|
||||
|
||||
| Outcome Variable | Model Variant | AUC-ROC | AUC-ROC CI | Precision | Recall | $F_1$ | $F_2$ |
|
||||
|:-------------------------------|:---------------------|--------:|:-----------------|----------:|-------:|-------:|-------:|
|
||||
| 30-day readmission | Full Model | 0.7496 | [0.7488, 0.7504] | 0.1881 | 0.8006 | 0.3046 | 0.4848 |
|
||||
| | 3 Transformer Blocks | 0.7472 | [0.7464, 0.7479] | 0.1974 | 0.7565 | 0.3131 | 0.4829 |
|
||||
| 30-day postdischarge mortality | Full Model | 0.8557 | [0.8532, 0.8581] | 0.0111 | 0.8756 | 0.0220 | 0.0530 |
|
||||
| | 3 Transformer Blocks | 0.8547 | [0.8523, 0.8572] | 0.0114 | 0.8662 | 0.0225 | 0.0542 |
|
||||
|
||||
**(B) Replacement of Deep Sets with flattening comparator**
|
||||
|
||||
| Outcome Variable | Model Variant | AUC-ROC | AUC-ROC CI | Precision | Recall | $F_1$ | $F_2$ |
|
||||
|:-------------------------------|:----------------|--------:|:-----------------|----------:|-------:|-------:|-------:|
|
||||
| 30-day readmission | Full Model | 0.7496 | [0.7488, 0.7504] | 0.1881 | 0.8006 | 0.3046 | 0.4848 |
|
||||
| | Without DeepSet | 0.7474 | [0.7466, 0.7482] | 0.1933 | 0.7724 | 0.3092 | 0.4829 |
|
||||
| 30-day postdischarge mortality | Full Model | 0.8557 | [0.8532, 0.8581] | 0.0111 | 0.8756 | 0.0220 | 0.0530 |
|
||||
| | Without DeepSet | 0.8513 | [0.8488, 0.8538] | 0.0108 | 0.8777 | 0.0214 | 0.0515 |
|
||||
|
||||
**(C) Removal of demographic and socioeconomic inputs (ICD-only)**
|
||||
|
||||
| Outcome Variable | Model Variant | AUC-ROC | AUC-ROC CI | Precision | Recall | $F_1$ | $F_2$ |
|
||||
|:-------------------------------|:----------------|--------:|:-----------------|----------:|-------:|-------:|-------:|
|
||||
| 30-day readmission | Full Model | 0.7496 | [0.7488, 0.7504] | 0.1881 | 0.8006 | 0.3046 | 0.4848 |
|
||||
| | ICD Inputs Only | 0.7483 | [0.7475, 0.7490] | 0.1907 | 0.7868 | 0.3070 | 0.4842 |
|
||||
| 30-day postdischarge mortality | Full Model | 0.8557 | [0.8532, 0.8581] | 0.0111 | 0.8756 | 0.0220 | 0.0530 |
|
||||
| | ICD Inputs Only | 0.8483 | [0.8457, 0.8509] | 0.0110 | 0.8627 | 0.0218 | 0.0525 |
|
||||
|
||||
### eFigure 1: Permutation-invariant ICD Embedding Model
|
||||
|
||||
|
||||
|
||||
### eFigure 2: Top ICD-10-CM codes by Integrated Gradients for 30-day mortality including index-hospital death
|
||||
|
||||
|
||||
|
||||
### eFigure 3. Calibration reliability plots for temporal test evaluation subsample predictions
|
||||
|
||||
*(A) 30-day unplanned readmission and (B) 30-day postdischarge mortality.*
|
||||
|
||||
::: {.annotation .annotation--static}
|
||||
**eFigure 3 --- [placeholder]** Calibration reliability plots for temporal test evaluation subsample predictions.
|
||||
:::
|
||||
|
||||
### eFigure 4. Web Calculator Interface Examples
|
||||
|
||||
*(A) With demographics and (B) Without demographics.*
|
||||
|
||||
**(A)**
|
||||
|
||||
|
||||
|
||||
**(B)**
|
||||
|
||||
|
||||
|
||||
:::
|
||||
|
|
@ -0,0 +1,56 @@
|
|||
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 280 280" role="img" aria-labelledby="mark-title-comorbidity">
|
||||
<title id="mark-title-comorbidity">A small grid of nodes resolving into a tighter ROC-style curve, with a baseline arc beneath</title>
|
||||
<desc>A frontispiece mark for "Beyond Comorbidity Indices" — many diagnosis codes aggregated to a discriminating risk score.</desc>
|
||||
|
||||
<circle cx="140" cy="140" r="128" stroke="currentColor" stroke-width="0.6" fill="none"/>
|
||||
|
||||
<g fill="currentColor" stroke="none">
|
||||
<circle cx="65" cy="65" r="1.1"/>
|
||||
<circle cx="80" cy="55" r="1.1"/>
|
||||
<circle cx="95" cy="68" r="1.1"/>
|
||||
<circle cx="105" cy="55" r="1.1"/>
|
||||
<circle cx="65" cy="80" r="1.1"/>
|
||||
<circle cx="78" cy="78" r="1.1"/>
|
||||
<circle cx="92" cy="82" r="1.1"/>
|
||||
<circle cx="108" cy="80" r="1.1"/>
|
||||
<circle cx="68" cy="95" r="1.1"/>
|
||||
<circle cx="82" cy="98" r="1.1"/>
|
||||
<circle cx="98" cy="92" r="1.1"/>
|
||||
<circle cx="110" cy="98" r="1.1"/>
|
||||
<circle cx="62" cy="110" r="1.1"/>
|
||||
<circle cx="78" cy="112" r="1.1"/>
|
||||
<circle cx="92" cy="108" r="1.1"/>
|
||||
<circle cx="105" cy="115" r="1.1"/>
|
||||
</g>
|
||||
|
||||
<g stroke="currentColor" fill="none" stroke-linecap="round" stroke-width="0.4" opacity="0.55">
|
||||
<line x1="65" y1="65" x2="138" y2="142"/>
|
||||
<line x1="80" y1="55" x2="138" y2="142"/>
|
||||
<line x1="95" y1="68" x2="138" y2="142"/>
|
||||
<line x1="105" y1="55" x2="138" y2="142"/>
|
||||
<line x1="78" y1="78" x2="138" y2="142"/>
|
||||
<line x1="92" y1="82" x2="138" y2="142"/>
|
||||
<line x1="98" y1="92" x2="138" y2="142"/>
|
||||
<line x1="82" y1="98" x2="138" y2="142"/>
|
||||
<line x1="105" y1="115" x2="138" y2="142"/>
|
||||
<line x1="92" y1="108" x2="138" y2="142"/>
|
||||
</g>
|
||||
|
||||
<g stroke="currentColor" fill="none" stroke-linecap="round" stroke-linejoin="round">
|
||||
<line x1="138" y1="225" x2="245" y2="225" stroke-width="0.8"/>
|
||||
<line x1="138" y1="225" x2="138" y2="118" stroke-width="0.8"/>
|
||||
|
||||
<line x1="138" y1="225" x2="245" y2="118" stroke-width="0.4" stroke-dasharray="2 3"/>
|
||||
|
||||
<path d="M 138 225 Q 152 175 175 158 Q 200 142 230 128 L 245 122" stroke-width="1.6"/>
|
||||
|
||||
<path d="M 138 225 Q 165 200 195 185 Q 225 170 245 162" stroke-width="0.8" stroke-dasharray="3 3"/>
|
||||
</g>
|
||||
|
||||
<g stroke="currentColor" fill="none" stroke-linecap="round" stroke-width="0.5">
|
||||
<line x1="138" y1="225" x2="138" y2="229"/>
|
||||
<line x1="138" y1="225" x2="134" y2="225"/>
|
||||
<line x1="245" y1="225" x2="245" y2="229"/>
|
||||
<line x1="138" y1="118" x2="134" y2="118"/>
|
||||
</g>
|
||||
</svg>
|
||||
|
After Width: | Height: | Size: 2.5 KiB |
|
|
@ -0,0 +1,377 @@
|
|||
---
|
||||
title: "Branch-Tube Persistence and Static Coverage in Tree-Ball Geometry"
|
||||
date: 2026-07-21
|
||||
abstract: >
|
||||
We analyze exhaustive static coverage by path tubes indexed by length-$t$
|
||||
nonbacktracking robber paths from $v_0$ in a finite-horizon local chase on a
|
||||
$d$-regular graph. An endpoint-sensitive geodesic lemma shows that, among
|
||||
possibly infinite $d$-regular graphs, radius $R+t$ is sharp for arbitrary
|
||||
pairs in $B_R(v_0)\times B_t(v_0)$, while synchronized witnesses along a
|
||||
prescribed path require only a radius-$R$ tree-ball. If every tube is
|
||||
occupied, each surviving round along every path in this class ends in
|
||||
capture, blockage, or branch-load support on at least two branches. The
|
||||
tubes partition the outer ball, so deterministic coverage has minimum cost
|
||||
$N_t=d(d-1)^{t-1}$; conditional on a specified root, i.i.d. uniform
|
||||
coverage has threshold $\Theta(N_t\log N_t)$ for fixed $d$. An augmented
|
||||
prefix-depth profile that retains the complete shallow configuration still
|
||||
need not determine later support. The result is local and root-dependent:
|
||||
it treats neither arbitrary robber walks nor a robber-independent cop
|
||||
strategy, and it gives no cop-number bound.
|
||||
tags:
|
||||
- research
|
||||
- research/mathematics
|
||||
- research/graph-theory
|
||||
authors:
|
||||
- "Levi Neuwirth | /me.html"
|
||||
affiliation:
|
||||
- "Brown University | https://www.brown.edu"
|
||||
bibliography: data/branch-capture-paper.bib
|
||||
preprint: /papers/branch-capture-paper.pdf
|
||||
no-collapse: true
|
||||
status: "Durable"
|
||||
confidence: proved
|
||||
evidence: 5
|
||||
peer-status: unreviewed
|
||||
result-shape: mixed
|
||||
further-reading:
|
||||
- Quilliot
|
||||
- NowakowskiWinkler
|
||||
- AignerFromme
|
||||
- Frankl
|
||||
- LuPeng
|
||||
- ScottSudakov
|
||||
- FriezeKrivelevichLoh
|
||||
- PralatWormald
|
||||
- BradshawHosseiniMoharStacho
|
||||
- HMG
|
||||
---
|
||||
|
||||
# Introduction
|
||||
|
||||
## Motivation
|
||||
|
||||
The cops-and-robbers game, introduced independently in its one-cop form by Quilliot and by Nowakowski–Winkler and developed in its multiple-cop form by Aigner–Fromme [@Quilliot; @NowakowskiWinkler; @AignerFromme], is a pursuit-evasion game on a graph in which $k$ cops attempt to capture one robber. The minimum such $k$ is the *cop number* $c(G)$. *Meyniel's conjecture*, attributed to Henri Meyniel and appearing in the early literature in Frankl's work [@Frankl], asserts that $c(G)=O(\sqrt n)$ for every connected graph on $n$ vertices and remains a central open problem in the area. The best general upper bound, $$c(G) \leq \frac{n}{2^{(1-o(1))\sqrt{\log_2 n}}},$$ was proved independently by Lu–Peng, Scott–Sudakov, and Frieze–Krivelevich–Loh [@LuPeng; @ScottSudakov; @FriezeKrivelevichLoh].
|
||||
|
||||
Prałat–Wormald's random-graph argument combines an initial random cop placement with assignments made after the robber's start is revealed: in different regimes, cop teams fully occupy a neighborhood or densely cover a sphere [@PralatWormald]. Tree-like geometry enters the literature in several other ways. Aigner–Fromme show that one moving cop can guard a fixed isometric path [@AignerFromme]. Frankl established a high-girth lower bound [@Frankl], and Bradshaw–Hosseini–Mohar–Stacho refine that line through a branch-weight argument in which unique local geodesics let the robber choose a sufficiently lightly controlled forward branch [@BradshawHosseiniMoharStacho]. Related bounded-degree reductions provide broader extremal context [@HMG]. These mechanisms motivate a path-conditioned local question: whether cops near the robber can coordinate their distance-decreasing moves along every prescribed length-$t$ nonbacktracking path starting at the robber's initial vertex.
|
||||
|
||||
The present paper instead isolates one rigid, root-dependent certificate: every length-$t$ nonbacktracking path from $v_0$ indexes an occupied descendant tube. Paths here index a static partition of the outer ball rather than a route guarded by one moving cop or an adaptively selected escape branch. We determine this certificate's guarantee and cost. Arbitrary robber walks, including stationary moves and reversals, are not analyzed, and exact tree geometry is not asserted to be necessary.
|
||||
|
||||
The uniform sampling below is conditional on the fixed root $v_0$: the root is specified before cop positions are sampled from $B_R(v_0)$. This measures the local cost of the certificate, not a legal initial placement in the standard game, where cops choose positions before the robber chooses its start. A global strategy would need comparable coverage simultaneously or adaptively for an unknown robber position. Accordingly, the results neither construct a robber-independent global strategy nor improve the cop-number bound.
|
||||
|
||||
## Setup and endpoint-sensitive tree-ball geodesics
|
||||
|
||||
Graphs are simple and undirected. The local results allow finite or infinite graphs unless finiteness is stated explicitly; discussion of the cop number and all $n$-vertex asymptotics concerns finite connected graphs. Throughout, $G$ denotes a $d$-regular graph with $d\geq3$, and $v_0\in V(G)$ is the robber's initial position in the rooted local experiment. All radii and time horizons are nonnegative integers, and $t,r\geq1$ whenever length-$t$ or order-$r$ objects are used. All unadorned logarithms are natural. We write $$B_R(v)=\{x\in V(G):\operatorname{dist}(x,v)\leq R\},
|
||||
\qquad
|
||||
S_j(v)=\{x\in V(G):\operatorname{dist}(x,v)=j\}.$$
|
||||
|
||||
::: {#def-tree-ball .exhibit .exhibit--definition data-exhibit-type="definition" data-exhibit-name="Definition 1 (Tree-ball)"}
|
||||
**Definition 1** (Tree-ball). The ball $B_R(v_0)$ is a *tree-ball* if the induced subgraph $G[B_R(v_0)]$ is a tree.
|
||||
:::
|
||||
|
||||
This is a local condition at the specified center. The global condition $\operatorname{girth}(G)>2R+1$ implies that every radius-$R$ ball is a tree-ball. Conversely, if every radius-$R$ ball is a tree-ball, then $\operatorname{girth}(G)>2R+1$.
|
||||
|
||||
An induced tree-ball does not control shortest paths between arbitrary pairs of its vertices: a competing path may leave the ball and re-enter. The required radius depends on the endpoint depths and on the length of their path inside the tree-ball.
|
||||
|
||||
::: {#lem-buffer .exhibit .exhibit--lemma data-exhibit-type="lemma" data-exhibit-name="Lemma 2 (Endpoint-sensitive tree-ball geodesics)"}
|
||||
**Lemma 2** (Endpoint-sensitive tree-ball geodesics). *Suppose $B_q(v_0)$ is a tree-ball, let $x,y\in B_q(v_0)$, and let $L$ be the length of the $x$–$y$ path in $G[B_q(v_0)]$. If $$\operatorname{dist}(x,v_0)+\operatorname{dist}(y,v_0)+L\leq 2q,$$ then every ambient $x$–$y$ geodesic lies in $B_q(v_0)$. Consequently, that geodesic is unique and equals the $x$–$y$ path in $G[B_q(v_0)]$.*
|
||||
:::
|
||||
|
||||
::: proof
|
||||
*Proof.* Let $P$ be an ambient $x$–$y$ geodesic of length $\ell$. The path in $G[B_q(v_0)]$ gives $\ell\leq L$. For $z\in P$, write $h=\operatorname{dist}_P(x,z)$. Then $$\operatorname{dist}(z,v_0)
|
||||
\leq \min\{\operatorname{dist}(x,v_0)+h,\ \operatorname{dist}(y,v_0)+\ell-h\}
|
||||
\leq \frac{\operatorname{dist}(x,v_0)+\operatorname{dist}(y,v_0)+\ell}{2}
|
||||
\leq q.$$ Thus $P\subseteq B_q(v_0)$. Since the induced graph on this ball is a tree, $P$ is its unique $x$–$y$ path. □
|
||||
:::
|
||||
|
||||
::: {#rem-uniform-buffers .exhibit .exhibit--remark data-exhibit-type="remark" data-exhibit-name="Remark 3 (Uniform buffers versus chase endpoints)"}
|
||||
**Remark 3** (Uniform buffers versus chase endpoints). For arbitrary $x\in B_R(v_0)$ and $y\in B_t(v_0)$, the walk from $x$ to $y$ through $v_0$ has length at most $R+t$. If $B_{R+t}(v_0)$ is a tree-ball, the path between $x$ and $y$ in the rooted tree $G[B_{R+t}(v_0)]$ is no longer than this walk, so Lemma [2](#lem-buffer) applies. For $R,t\geq1$, this uniform radius cannot in general be reduced. Put $L=R+t$, begin with the rooted $d$-regular tree truncated at depth $L-1$, and choose vertices $x$ and $y$ at depths $R$ and $t$ in distinct root branches. Choose descendants $a$ of $x$ and $b$ of $y$ in $S_{L-1}(v_0)$, allowing equality, add a new vertex $z$, and join $z$ to $a$ and $b$.
|
||||
|
||||
Complete the graph explicitly: for each remaining degree deficit at a vertex of $S_{L-1}(v_0)\cup\{z\}$, attach by one edge a separate infinite rooted $(d-1)$-ary tree. Every vertex then has degree $d$, and the resulting graph is infinite, simple, and connected. The induced ball $B_{L-1}(v_0)$ is still the original tree. The path from $x$ to $y$ through $v_0$ has length $L$, while the path through $z$ has length $$\operatorname{dist}(x,a)+2+\operatorname{dist}(b,y)=(t-1)+2+(R-1)=L.$$ Each attached infinite tree meets the displayed core at only one vertex, so it creates no additional route between $x$ and $y$. Thus the two displayed paths are distinct geodesics. This proves sharpness at radius $R+t-1$ among possibly infinite $d$-regular graphs.
|
||||
|
||||
The endpoint pairs used along a prescribed length-$t$ nonbacktracking path are more restricted. Their rooted-tree path and endpoint depths satisfy Lemma [2](#lem-buffer) with $q=R$, so the persistence result requires no tree-ball beyond the cop-placement radius. This does not assert that the radius-$R$ tree-ball hypothesis is necessary. Controlled cyclic geometry is not addressed here, and no cyclic extension is proved.
|
||||
:::
|
||||
|
||||
Whenever $B_q(v_0)$ is a tree-ball, we root $G[B_q(v_0)]$ at $v_0$. Every vertex other than $v_0$ has a unique parent, and every vertex of depth less than $q$ has $d-1$ children.
|
||||
|
||||
::: {#def-cone .exhibit .exhibit--definition data-exhibit-type="definition" data-exhibit-name="Definition 4 (Geodesic cone)"}
|
||||
**Definition 4** (Geodesic cone). Suppose $B_r(v)$ is a tree-ball and $u\in N(v)$. The *geodesic cone* through $u$ at radius $r$ is $$C_u(v,r):=\{x\in B_r(v)\setminus\{v\}:\text{the path from $x$ to $v$ in $B_r(v)$ has penultimate vertex $u$}\}.$$
|
||||
:::
|
||||
|
||||
::: {#def-tube .exhibit .exhibit--definition data-exhibit-type="definition" data-exhibit-name="Definition 5 (Path tube)"}
|
||||
**Definition 5** (Path tube). Let $R\geq t$, suppose $B_R(v_0)$ is a tree-ball, and let $$\sigma=(v_0,v_1,\ldots,v_t)$$ be a length-$t$ nonbacktracking path starting at $v_0$: consecutive vertices are adjacent and $v_{i+1}\neq v_{i-1}$ whenever the latter condition is defined. The *length-$t$ tube* associated with $\sigma$ is $$T_\sigma(R):=\{x\in B_R(v_0):\text{the rooted path from $x$ to $v_0$ contains $v_t,v_{t-1},\ldots,v_1$ in order}\}.$$ Equivalently, $T_\sigma(R)$ is the set of descendants of $v_t$ lying in $B_R(v_0)$.
|
||||
:::
|
||||
|
||||
## Game conventions
|
||||
|
||||
A *cop configuration* $X$ is a finite multiset of vertices, represented by its finitely supported multiplicity function $X(v)\in\mathbb Z_{\geq0}$. For $A\subseteq V(G)$, set $$X(A):=\sum_{v\in A}X(v),
|
||||
\qquad
|
||||
|X|:=X(V(G)),
|
||||
\qquad
|
||||
\mathop{\mathrm{supp}}(X):=\{v:X(v)>0\}.$$ Thus counts of cops include multiplicity, several cops may occupy one vertex, and $A$ is occupied exactly when $X(A)>0$. We write $X=[x_1,\ldots,x_k]$ for a multiset of cop positions, with repeated entries allowed. Independent uniform sampling is with replacement and produces the multiset of sampled positions.
|
||||
|
||||
Capture occurs whenever a cop and the robber occupy the same vertex, including before the first cop move; once capture occurs, no further move is made. Conditional on no capture, the cops move first in each round, simultaneously, and each cop moves to a neighboring vertex minimizing its distance to the robber's current vertex. Ties may be broken arbitrarily. Some formulations also allow cops to pass; the results remain valid there by having the selected witnesses make the prescribed distance-decreasing moves. Under the tree-ball hypotheses below, Lemma [2](#lem-buffer) shows that each witness selected in our proofs has a unique distance-decreasing move.
|
||||
|
||||
We analyze only length-$t$ nonbacktracking robber paths starting at $v_0$. In the full game a robber walk may reverse an edge or, under common conventions, remain stationary. Neither behavior is covered here. Fix a prescribed path $$\tau=(v_0,v_1,\ldots,v_t).$$ It survives through round $0$ precisely when no initial capture occurs. If it has survived through round $s-1$, then the robber is at $v_{s-1}$ when round $s$ begins. After the cops move toward $v_{s-1}$, the robber is captured if a cop occupies $v_{s-1}$. Otherwise the prescribed move $v_{s-1}\to v_s$ is *legal* if no cop occupies $v_s$ after the cop move. The path survives through round $s$ if no capture or blockage has occurred through that move.
|
||||
|
||||
For a robber position $v$ and a neighbor $u\in N(v)$, an individual cop at $c\neq v$ whose $c$–$v$ geodesic is unique *contributes to branch $u$ at $v$* if that geodesic has penultimate vertex $u$. The *branch-load support* at $v$ is the set of branches receiving at least one contributing cop; its size counts branches, not cops.
|
||||
|
||||
## Main results and scope
|
||||
|
||||
Although $R+t$ is the sharp uniform radius for arbitrary endpoint pairs in $B_R(v_0)\times B_t(v_0)$, the synchronized pairs arising along a prescribed length-$t$ nonbacktracking path satisfy the endpoint-sensitive criterion already at radius $R$. Accordingly, the persistence theorem assumes only that the cop-placement ball $B_R(v_0)$ is a tree-ball.
|
||||
|
||||
Exact rooted-tree counts identify first-level cones as the $t=1$ path tubes and show that the length-$t$ tubes partition the outer ball. Conditional on the prescribed path continuing, a witness's initial depth determines a potential capture or blockage time within the horizon or certifies descendant pressure throughout it. This yields a persistence theorem uniform over all length-$t$ nonbacktracking paths starting at $v_0$, and only over that class.
|
||||
|
||||
The next results quantify the price of exhaustive static coverage. Exactly $$N_t=d(d-1)^{t-1}$$ cops are necessary and sufficient to occupy every tube at a fixed root. Along fixed-degree sequences with $t\to\infty$, conditional uniform sampling has a threshold of order $N_t\log N_t$; its leading factor depends explicitly on $R-t$ and tends to $1$ when $R-t\to\infty$.
|
||||
|
||||
Finally, Section [5](#sec-profile-loss) records information lost by finite-order tube profiles. Even after retaining every shallow cop multiplicity, such a profile need not determine branch support after $r+1$ rounds. This exact nondeterminacy does not rule out one-sided certificates that reject ambiguous profile fibers.
|
||||
|
||||
# Cone and Tube Counts
|
||||
|
||||
::: {#lem-shell .exhibit .exhibit--lemma data-exhibit-type="lemma" data-exhibit-name="Lemma 6 (Sharp shell and cone counts)"}
|
||||
**Lemma 6** (Sharp shell and cone counts). *Let $G$ be $d$-regular with $d\geq3$, and suppose $B_r(v)$ is a tree-ball. Then for every $u\in N(v)$ and $1\leq j\leq r$, $$|C_u(v,r)\cap S_j(v)|=(d-1)^{j-1},
|
||||
\qquad
|
||||
|S_j(v)|=d(d-1)^{j-1}.$$ Hence $$|C_u(v,r)|=\sum_{j=1}^r(d-1)^{j-1}
|
||||
=\frac{(d-1)^r-1}{d-2},$$ and $$|B_r(v)|=1+d\frac{(d-1)^r-1}{d-2}.$$*
|
||||
:::
|
||||
|
||||
::: proof
|
||||
*Proof.* The branch rooted at $u$ is a $(d-1)$-ary rooted tree of depth $r-1$. Its level $j-1$ contains $(d-1)^{j-1}$ vertices, giving the first formula. Summing over the $d$ branches gives the shell count, and the remaining identities follow by summing the geometric series. □
|
||||
:::
|
||||
|
||||
For $t=1$, geodesic cones are exactly the path tubes. We now count the general length-$t$ objects that drive the persistence argument.
|
||||
|
||||
::: {#lem-tube-partition .exhibit .exhibit--lemma data-exhibit-type="lemma" data-exhibit-name="Lemma 7 (Tube count and partition)"}
|
||||
**Lemma 7** (Tube count and partition). *Let $R\geq t$, and suppose $B_R(v_0)$ is a tree-ball. For every length-$t$ nonbacktracking path $\sigma$ starting at $v_0$, $$|T_\sigma(R)|
|
||||
=\sum_{j=t}^R(d-1)^{j-t}
|
||||
=\frac{(d-1)^{R-t+1}-1}{d-2}.$$ The length-$t$ tubes are pairwise disjoint and form a partition $$B_R(v_0)\setminus B_{t-1}(v_0)
|
||||
=\bigsqcup_{\sigma}T_\sigma(R),$$ where $\sigma$ ranges over all length-$t$ nonbacktracking paths from $v_0$. Their number is $$N_t=d(d-1)^{t-1}.$$*
|
||||
:::
|
||||
|
||||
::: proof
|
||||
*Proof.* At total depth $j\geq t$, the descendants of the terminal vertex $v_t$ form the depth-$(j-t)$ level of a $(d-1)$-ary rooted tree and hence contribute $(d-1)^{j-t}$ vertices. Summing gives the tube size.
|
||||
|
||||
Every vertex of depth at least $t$ has a unique rooted prefix of length $t$, so it lies in exactly one tube. The first edge of such a prefix has $d$ choices, and every later edge has $d-1$ choices, giving $N_t=d(d-1)^{t-1}$. □
|
||||
:::
|
||||
|
||||
For a fixed rooted tree-ball $(G,v_0,R)$, after the root has been specified, let $C$ be uniform on $B_R(v_0)$. The probability that $C$ lies in a prescribed length-$t$ tube is $$q_{t,R}:=\Pr(C\in T_\sigma(R)\mid G,v_0,R)
|
||||
=\frac{|T_\sigma(R)|}{|B_R(v_0)|}.$$ The partition gives the exact identity $$
|
||||
N_tq_{t,R}
|
||||
=1-\frac{|B_{t-1}(v_0)|}{|B_R(v_0)|}.$$ Put $h=R-t$ and $b=d-1$. Lemmas [6](#lem-shell) and [7](#lem-tube-partition) give $$q_{t,R}=\frac{b^{h+1}-1}{(b+1)b^{t+h}-2}$$ and $$\frac{1}{N_tq_{t,R}}
|
||||
=\frac{b^{h+1}-2/((b+1)b^{t-1})}{b^{h+1}-1}
|
||||
=\frac{1}{1-b^{-(h+1)}}+O(b^{-t}),$$ uniformly for $h\geq0$. Thus $q_{t,R}=\Theta(N_t^{-1})$ uniformly over $R\geq t$, but its leading constant depends on $R-t$ when that difference remains bounded.
|
||||
|
||||
# Path-Tube Persistence
|
||||
|
||||
## The interception schedule
|
||||
|
||||
::: {#lem-interception .exhibit .exhibit--lemma data-exhibit-type="lemma" data-exhibit-name="Lemma 8 (Interception schedule along a tube)"}
|
||||
**Lemma 8** (Interception schedule along a tube). *Let $R\geq t$, suppose $B_R(v_0)$ is a tree-ball, and let $$\sigma=(v_0,v_1,\ldots,v_t)$$ be a length-$t$ nonbacktracking path starting at $v_0$. Let a cop start at $c\in T_\sigma(R)$ at depth $j=\operatorname{dist}(c,v_0)$. Suppose the robber follows $\sigma$ for as long as the path survives. For every $s\in\{1,\ldots,t\}$ such that the path has survived through round $s-1$, after the cop move in round $s$:*
|
||||
|
||||
1. *if $j=2s-1$, the cop reaches $v_{s-1}$ and captures the robber;*
|
||||
|
||||
2. *if $j=2s$, the cop reaches $v_s$ and blocks the intended move;*
|
||||
|
||||
3. *if $j\geq2s+1$, the cop remains a strict descendant of $v_s$.*
|
||||
|
||||
*The alternative $j\leq2s-2$ cannot occur: in that case the path was already captured or blocked by the end of round $s-1$.*
|
||||
:::
|
||||
|
||||
::: proof
|
||||
*Proof.* Set $s_*=\lceil j/2\rceil$. For every reached round $k\leq\min\{s_*,t\}$, an induction shows that before the cop move the cop is a descendant of $v_{k-1}$ and has depth $j-k+1$. This is true at $k=1$. At round $k$, the endpoint-depth sum for the cop and $v_{k-1}$ is $j$, while their rooted-path length is $j-2k+2$. Hence the left side of Lemma [2](#lem-buffer)'s inequality, with $q=R$, is $$j+(j-2k+2)=2(j-k+1)\leq2j\leq2R.$$ The rooted path is therefore the unique ambient geodesic, and the cop moves one step toward $v_0$, to depth $j-k$. If $k<s_*$, then $j-k\geq k+1$, so the cop remains a strict descendant of $v_k$, proving the induction step.
|
||||
|
||||
At $k=s_*$, if $j$ is odd then $j=2s_*-1$ and the cop reaches $v_{s_*-1}$; if $j$ is even then $j=2s_*$ and the cop reaches $v_{s_*}$. This proves the three cases. If $j\leq2s-2$, then $s_*\leq s-1$, so the scheduled capture or blockage prevents survival through round $s-1$. □
|
||||
:::
|
||||
|
||||
::: {#rem-odd-even-depths .exhibit .exhibit--remark data-exhibit-type="remark" data-exhibit-name="Remark 9 (Odd and even initial depths as capture and blockage times)"}
|
||||
**Remark 9**. Odd and even initial depths encode potential capture and blockage times, respectively. If the corresponding time is at most $t$ and the prescribed play reaches it, the event occurs then. If $\lceil j/2\rceil>t$, that time lies beyond the horizon and the cop remains descendant pressure throughout the first $t$ reached rounds.
|
||||
:::
|
||||
|
||||
## Persistence along nonbacktracking paths
|
||||
|
||||
::: {#thm-persistence .exhibit .exhibit--theorem data-exhibit-type="theorem" data-exhibit-name="Theorem 10 (Nonbacktracking path-tube persistence)"}
|
||||
**Theorem 10** (Nonbacktracking path-tube persistence). *Let $d\geq3$ and $1\leq t\leq R$, suppose $B_R(v_0)$ is a tree-ball, and let $X_0$ be any initial cop configuration satisfying $$X_0(T_\sigma(R))\geq1$$ for every length-$t$ tube. Cops outside $B_R(v_0)$ are permitted; the proof ignores them unless they cause capture, blockage, or additional branch support. If $X_0(\{v_0\})>0$, the robber is captured before any move. Otherwise, let $$\tau=(v_0,v_1,\ldots,v_t)$$ be any length-$t$ nonbacktracking path starting at $v_0$. At every round $s\in\{1,\ldots,t\}$ for which the robber has followed $\tau$ through round $s-1$, at least one of the following occurs during round $s$:*
|
||||
|
||||
1. *on the cops' move, a cop captures the robber at $v_{s-1}$;*
|
||||
|
||||
2. *after the cops' move, the robber is not captured, but a cop occupies $v_s$, so the intended move is illegal;*
|
||||
|
||||
3. *the move to $v_s$ is legal, and after that move the branch-load support at $v_s$ has size at least $2$.*
|
||||
:::
|
||||
|
||||
::: proof
|
||||
*Proof.* Assume there was no initial capture, and fix a round $s$ reached along $\tau$. If a cop reaches $v_{s-1}$, then (a) holds. If no cop captures the robber but a cop occupies $v_s$, then (b) holds. Assume instead that the move to $v_s$ is legal and produce two branch witnesses.
|
||||
|
||||
*Descendant witness.* Choose a cop $c^\downarrow$ from $T_\tau(R)$ and let $j_\downarrow$ be its initial depth. Survival through round $s-1$ rules out $j_\downarrow\leq2s-2$, while the failure of alternatives (a) and (b) rules out $j_\downarrow\in\{2s-1,2s\}$. Hence $j_\downarrow\geq2s+1$, and Lemma [8](#lem-interception) implies that after the round-$s$ cop move it is a strict descendant of $v_s$, at depth $j_\downarrow-s$. Its rooted path to $v_s$ has length $j_\downarrow-2s$, so the endpoint-depth sum plus this length is $$(j_\downarrow-s)+s+(j_\downarrow-2s)
|
||||
=2(j_\downarrow-s)\leq2R.$$ Lemma [2](#lem-buffer), with $q=R$, makes this the unique ambient geodesic. Thus the cop contributes to a child branch of $v_s$.
|
||||
|
||||
*Parent witness.* Choose $u_0\in N(v_0)\setminus\{v_1\}$, extend $(v_0,u_0)$ to a length-$t$ nonbacktracking path $\sigma'$, and choose a cop $c^\uparrow\in T_{\sigma'}(R)$ of initial depth $j$. Then $R\geq j\geq t\geq s$. We prove by induction on $k\in\{1,\ldots,s\}$ that before its move in round $k$, the cop has depth $j-k+1$ and lies in the root branch through $u_0$. This is immediate for $k=1$. At round $k$, the rooted path from the cop to $v_{k-1}$ passes through $v_0$; both its length and the sum of the endpoint depths equal $$(j-k+1)+(k-1)=j.$$ Lemma [2](#lem-buffer), with $q=R$, therefore forces one step toward $v_0$. If $k<s$, then $j-k\geq t-s+1\geq1$, so the cop remains in the $u_0$-branch and the induction continues.
|
||||
|
||||
After the move in round $s$, the cop has depth $j-s$. It either remains in the $u_0$-branch or, when $j=s$, is at $v_0$. Its rooted path to $v_s$ enters through $v_{s-1}$, and both the path length and endpoint-depth sum equal $(j-s)+s=j$. Lemma [2](#lem-buffer) again makes this the unique ambient geodesic, so the cop contributes to the parent branch of $v_s$. If $j=s=1$, the cop instead captures at $v_0$, a case already excluded.
|
||||
|
||||
The two witnesses contribute to distinct branches, proving (c). □
|
||||
:::
|
||||
|
||||
## The one-round case
|
||||
|
||||
::: {#cor-one-round .exhibit .exhibit--corollary data-exhibit-type="corollary" data-exhibit-name="Corollary 11 (One-round persistence)"}
|
||||
**Corollary 11** (One-round persistence). *Let $R\geq1$, suppose $B_R(v_0)$ is a tree-ball, and let $X_0$ be a cop configuration satisfying $$X_0(C_u(v_0,R))\geq1
|
||||
\qquad\text{for every }u\in N(v_0).$$ For every proposed neighboring first move $v_0\to w$, the robber is captured at $v_0$ (initially or on the first cop move), blocked at $w$, or, after a legal move to $w$, faces branch-load support of size at least two.*
|
||||
:::
|
||||
|
||||
::: proof
|
||||
*Proof.* Initial capture gives the first alternative. Otherwise apply Theorem [10](#thm-persistence) with $t=1$. □
|
||||
:::
|
||||
|
||||
# The Cost of Exhaustive Tube Coverage
|
||||
|
||||
Theorem [10](#thm-persistence) uses a strong static hypothesis: every length-$t$ nonbacktracking path starting at $v_0$ indexes an occupied tube. We now quantify the exact deterministic cost and the conditional random-sampling threshold of this local certificate. These costs are not cop-number bounds and do not establish a robber-independent global placement.
|
||||
|
||||
::: {#prop-deterministic-cost .exhibit .exhibit--proposition data-exhibit-type="proposition" data-exhibit-name="Proposition 12 (Deterministic coverage cost)"}
|
||||
**Proposition 12** (Deterministic coverage cost). *Suppose $B_R(v_0)$ is a tree-ball with $R\geq t$, and let $X$ be any cop configuration. Then $X$ occupies every length-$t$ tube if and only if $$X(T_\sigma(R))\geq1$$ for every member of the family of $N_t=d(d-1)^{t-1}$ pairwise disjoint tubes. Consequently:*
|
||||
|
||||
1. *every such configuration satisfies $|X|\geq N_t$, counting cops with multiplicity;*
|
||||
|
||||
2. *equality suffices for this occupancy property, by choosing one cop position from each tube.*
|
||||
:::
|
||||
|
||||
::: proof
|
||||
*Proof.* This is immediate from Lemma [7](#lem-tube-partition). □
|
||||
:::
|
||||
|
||||
::: {#thm-sampling-threshold .exhibit .exhibit--theorem data-exhibit-type="theorem" data-exhibit-name="Theorem 13 (Conditional uniform-sampling threshold)"}
|
||||
**Theorem 13** (Conditional uniform-sampling threshold). *Fix $d\geq3$. For each positive integer $t$, let $G_t$ be a $d$-regular graph with distinguished vertex $v_{0,t}$, and let $R_t\geq t$ be such that $B_{R_t}^{G_t}(v_{0,t})$ is a tree-ball. Let $\mathcal P_t$ be the set of length-$t$ nonbacktracking paths from $v_{0,t}$, and let $T_{\sigma,t}$ be the tube of $\sigma\in\mathcal P_t$ in that ball. Set $$N_t:=|\mathcal P_t|=d(d-1)^{t-1},
|
||||
\qquad
|
||||
q_t:=\frac{|T_{\sigma,t}|}{|B_{R_t}^{G_t}(v_{0,t})|}.$$ For each $t$, after the rooted ball has been fixed, sample $m_t$ positions independently and uniformly with replacement from $B_{R_t}^{G_t}(v_{0,t})$. Let $\mathcal C_t$ be the event that every $T_{\sigma,t}$ contains at least one sample. For every fixed $\varepsilon$ with $0<\varepsilon<1$:*
|
||||
|
||||
1. *if $$m_t\geq(1+\varepsilon)\frac{\log N_t}{q_t},$$ then $\Pr(\mathcal C_t)\to1$;*
|
||||
|
||||
2. *if $$m_t\leq(1-\varepsilon)\frac{\log N_t}{q_t},$$ then $\Pr(\mathcal C_t)\to0$.*
|
||||
|
||||
*Writing $h_t=R_t-t$, one has, uniformly over $h_t\geq0$, $$\frac{\log N_t}{q_t}
|
||||
=\left(\frac{1}{1-(d-1)^{-(h_t+1)}}+o(1)\right)N_t\log N_t.$$ If $h_t=k$ eventually, the leading factor relative to $N_t\log N_t$ is $[1-(d-1)^{-(k+1)}]^{-1}$; if $h_t\to\infty$, it tends to $1$. Uniformly over $R_t\geq t$, $$\frac{\log N_t}{q_t}
|
||||
=\Theta(N_t\log N_t)
|
||||
=\Theta\bigl((d-1)^{t-1}t\bigr).$$*
|
||||
:::
|
||||
|
||||
::: proof
|
||||
*Proof.* For a fixed $\sigma\in\mathcal P_t$, the probability that $T_{\sigma,t}$ receives no sample is $(1-q_t)^{m_t}$. Hence $$\Pr(\mathcal C_t^{\mathsf c})
|
||||
\leq N_t(1-q_t)^{m_t}
|
||||
\leq N_te^{-m_tq_t}.$$ Under (i), this is at most $N_t^{-\varepsilon}\to0$.
|
||||
|
||||
For (ii), let $$Z_t:=\sum_{\sigma\in\mathcal P_t}I_{\sigma,t},
|
||||
\qquad
|
||||
I_{\sigma,t}:=\mathbf 1_{\{T_{\sigma,t}\text{ receives no sample}\}}.$$ Then $\mathbb E Z_t=N_t(1-q_t)^{m_t}$. For distinct $\sigma,\tau\in\mathcal P_t$, disjointness gives $$\mathbb E[I_{\sigma,t}I_{\tau,t}]
|
||||
=(1-2q_t)^{m_t}
|
||||
\leq(1-q_t)^{2m_t}
|
||||
=\mathbb E I_{\sigma,t}\,\mathbb E I_{\tau,t}.$$ Thus $$\operatorname{Var}(Z_t)
|
||||
\leq\sum_{\sigma\in\mathcal P_t}\operatorname{Var}(I_{\sigma,t})
|
||||
\leq\mathbb E Z_t.$$ Since $q_t=\Theta(N_t^{-1})$, one has $q_t\to0$, and under (ii), $$\mathbb E Z_t
|
||||
\geq N_t(1-q_t)^{(1-\varepsilon)(\log N_t)/q_t}
|
||||
=N_t^{\varepsilon+o(1)}\longrightarrow\infty.$$ Chebyshev's inequality therefore yields $$\Pr(\mathcal C_t)
|
||||
\leq\frac{\operatorname{Var}(Z_t)}{(\mathbb E Z_t)^2}
|
||||
\leq\frac1{\mathbb E Z_t}
|
||||
\longrightarrow0.$$ Finally, the exact calculation preceding the theorem, with $R=R_t$, gives $$\frac1{N_tq_t}
|
||||
=\frac1{1-(d-1)^{-(h_t+1)}}+O((d-1)^{-t})$$ uniformly for $h_t\geq0$, proving the remaining assertions. □
|
||||
:::
|
||||
|
||||
::: {#cor-polylog-horizon .exhibit .exhibit--corollary data-exhibit-type="corollary" data-exhibit-name="Corollary 14 (Finite horizon from polylogarithmic conditional sampling)"}
|
||||
**Corollary 14** (Finite horizon from polylogarithmic conditional sampling). *Fix $d\geq3$. Let $(G_n)$ be a sequence of $n$-vertex $d$-regular graphs with distinguished vertices $v_{0,n}$ and integers $1\leq t_n\leq R_n$ such that $B_{R_n}^{G_n}(v_{0,n})$ is a tree-ball. After each rooted ball is fixed, sample $m_n$ positions independently and uniformly with replacement from it, and let $\mathcal C_n$ be the event that every length-$t_n$ tube rooted at $v_{0,n}$ is occupied.*
|
||||
|
||||
*Call $(m_n)$ *polylogarithmic* if $m_n=O((\log n)^C)$ for some fixed $C>0$. If $(m_n)$ is polylogarithmic and $\Pr(\mathcal C_n)\to1$, then $t_n=O(\log\log n)$. If $t_n=\Theta(\log n)$ and $\Pr(\mathcal C_n)\to1$, then $$m_n=\Omega(N_{t_n}\log N_{t_n}),
|
||||
\qquad
|
||||
N_{t_n}=d(d-1)^{t_n-1};$$ in particular, $m_n\geq n^c$ eventually for some $c>0$.*
|
||||
:::
|
||||
|
||||
::: proof
|
||||
*Proof.* If $t_n\neq O(\log\log n)$, pass to a subsequence on which $t_n/\log\log n\to\infty$, and then to a further subsequence on which $t_n$ is strictly increasing. Apply the proof of Theorem [13](#thm-sampling-threshold) along this subsequence, with $t$ replaced by $t_n$. Now $t_n\to\infty$ and $N_{t_n}=(\log n)^{\omega(1)}$. Because the threshold is $\Theta(N_{t_n}\log N_{t_n})$ uniformly in $R_n\geq t_n$, polylogarithmic $m_n$ satisfies $$m_n\leq\frac12\frac{\log N_{t_n}}{q_n}$$ eventually on this subsequence, where $q_n$ is the common tube mass. Part (ii) of Theorem [13](#thm-sampling-threshold), with $\varepsilon=1/2$, then gives $\Pr(\mathcal C_n)\to0$, a contradiction. Hence $t_n=O(\log\log n)$.
|
||||
|
||||
If $t_n=\Theta(\log n)$ and $m_n=o(N_{t_n}\log N_{t_n})$ along a subsequence, pass if necessary to a further subsequence with strictly increasing horizons and apply the same subsequence argument. This contradicts $\Pr(\mathcal C_n)\to1$. Thus $m_n=\Omega(N_{t_n}\log N_{t_n})$. Since $N_{t_n}=n^{\Omega(1)}$, this is at least $n^c$ eventually for some $c>0$. □
|
||||
:::
|
||||
|
||||
::: {#rem-partial-coverage .exhibit .exhibit--remark data-exhibit-type="remark" data-exhibit-name="Remark 15 (Partial coverage is cheaper)"}
|
||||
**Remark 15** (Partial coverage is cheaper). At one fixed, known root, the expected fraction of length-$t$ tubes occupied by $m$ independent uniform samples is $$1-(1-q_{t,R})^m.$$ Since $q_{t,R}=\Theta(N_t^{-1})$, $m=\Theta(N_t)$ samples occupy a fixed positive fraction of the tubes in expectation, while $m=\omega(N_t)$ occupies a $1-o(1)$ fraction in expectation. Under uniform sampling, reducing the expected number of empty tubes to $O(1)$ requires $\Theta(N_t\log N_t)$ samples; the same order is required to leave at most $O(1)$ tubes empty with probability tending to one. These are occupancy statements only: they give neither an adversarial guarantee over all paths from that root nor a root-independent placement, and they do not show that partial coverage supports an adaptive chase.
|
||||
:::
|
||||
|
||||
# Information Loss in Tube Profiles {#sec-profile-loss}
|
||||
|
||||
Fix integers $Q\geq r\geq1$ and $k\geq0$, and suppose $B_Q(v_0)$ is a tree-ball. Let $\mathcal P_r(v_0)$ be the set of length-$r$ nonbacktracking paths starting at $v_0$, and let $$\mathcal C_{Q,k}(v_0)
|
||||
:=\{X: X\text{ is a $k$-cop multiset with }\mathop{\mathrm{supp}}(X)\subseteq B_Q(v_0)\}.$$ For $X\in\mathcal C_{Q,k}(v_0)$, $\sigma\in\mathcal P_r(v_0)$, and $r\leq j\leq Q$, set $$N_X(\sigma,j)
|
||||
:=\sum_{\substack{x\in S_j(v_0)\\
|
||||
\text{the rooted path from $v_0$ to $x$ begins with }\sigma}}X(x).$$ The *augmented order-$r$ tube profile* is $$\Pi_{r;Q,k}(X)
|
||||
:=\left(
|
||||
\bigl(X(x)\bigr)_{x\in B_{r-1}(v_0)},
|
||||
\bigl(N_X(\sigma,j)\bigr)_{\substack{\sigma\in\mathcal P_r(v_0)\\ r\leq j\leq Q}}
|
||||
\right).$$ It records the complete configuration through depth $r-1$ and the exact depth histogram in every length-$r$ tube; the $j=r$ coordinates also recover the multiplicities at depth $r$. It forgets only how cops at a fixed greater depth split among descendants below the terminal vertex of a length-$r$ prefix. For an admissible profile $P$, define its fixed-universe fiber by $$\mathcal F_{r;Q,k}(P)
|
||||
:=\{X\in\mathcal C_{Q,k}(v_0):\Pi_{r;Q,k}(X)=P\}.$$
|
||||
|
||||
::: {#prop-profile-nondeterminacy .exhibit .exhibit--proposition data-exhibit-type="proposition" data-exhibit-name="Proposition 16 (Augmented tube profiles do not determine later support)"}
|
||||
**Proposition 16** (Augmented tube profiles do not determine later support). *Fix $d\geq3$ and $r\geq1$, put $Q=2r+3$, and suppose $B_Q(v_0)$ is a tree-ball. There exist $X,Y\in\mathcal C_{Q,2}(v_0)$ and a length-$(r+1)$ nonbacktracking path $\tau=(v_0,\ldots,v_{r+1})$ such that $$\Pi_{r;Q,2}(X)=\Pi_{r;Q,2}(Y).$$ The path survives all $r+1$ rounds from both configurations, with all relevant cop moves unique, but immediately after the robber's legal move to $v_{r+1}$ the branch-load supports in $X$ and $Y$ have sizes $2$ and $1$, respectively. Hence, for this fixed path $\tau$, the predicate that the branch-load support after $r+1$ safe rounds has size at least $2$ does not factor through $\Pi_{r;Q,2}$.*
|
||||
:::
|
||||
|
||||
::: proof
|
||||
*Proof.* Choose $\tau$ and two distinct children $a,b$ of $v_{r+1}$. Choose descendants $x_a$ of $a$ and $x_b$ of $b$ at total depth $Q=2r+3$ from $v_0$, and choose two distinct descendants $y_a,y_a'$ of $a$ at the same total depth. Such choices exist because the vertices at descendant-distance $r+1$ below $a$ number $(d-1)^{r+1}\geq2$. Define $$X=[x_a,x_b],
|
||||
\qquad
|
||||
Y=[y_a,y_a'].$$ Both configurations have zero multiplicity on $B_{r-1}(v_0)$, and their only nonzero tube-depth coordinate is the cell indexed by $((v_0,\ldots,v_r),Q)$, where both have value $2$. Thus their augmented profiles agree.
|
||||
|
||||
Every robber position on $\tau$ is an ancestor of every cop in the construction. Immediately before the cop move in round $s$, each cop has total depth $Q-s+1$, while the robber is at $v_{s-1}$. Their rooted path has length $Q-2s+2$, and $$(Q-s+1)+(s-1)+(Q-2s+2)=2Q-2s+2\leq2Q.$$ Lemma [2](#lem-buffer), with $q=Q$, therefore makes this path the unique ambient geodesic and forces one step upward. After that move, the cop's distances to $v_{s-1}$ and $v_s$ are $$2r+4-2s\geq2
|
||||
\qquad\text{and}\qquad
|
||||
2r+3-2s\geq1$$ for $1\leq s\leq r+1$. Thus no capture or blockage occurs. After $r+1$ cop moves, the configurations are $[a,b]$ and $[a,a]$, so the branch-load support sizes after the robber moves to $v_{r+1}$ are $2$ and $1$. □
|
||||
:::
|
||||
|
||||
For $r=1$, the profile includes the multiplicity at $v_0$ as well as the depth histogram in every first-level tube, yet it still does not determine the two-round outcome. Proposition [16](#prop-profile-nondeterminacy) identifies one ambiguous fiber, not a barrier to every one-sided certificate: a sound certificate may reject that fiber.
|
||||
|
||||
# Limitations and Further Directions {#sec-open}
|
||||
|
||||
The present arguments do not address five natural directions; no claim of novelty is made for the questions themselves.
|
||||
|
||||
## Arbitrary robber walks
|
||||
|
||||
Theorem [10](#thm-persistence) treats only length-$t$ nonbacktracking paths starting at $v_0$. Stationary moves and reversals destroy the monotone depth evolution used by the interception schedule.
|
||||
|
||||
::: question
|
||||
**Question 17**. *Can a static or adaptive local certificate give an analogue of Theorem [10](#thm-persistence) for arbitrary length-$t$ robber walks from $v_0$, including stationary moves and reversals? How do repeated vertices and reversed edges change the required witnesses and coverage cost?*
|
||||
:::
|
||||
|
||||
## Partial and adaptive coverage
|
||||
|
||||
Remark [15](#rem-partial-coverage) separates partial from exhaustive conditional coverage at one fixed root. Along a prescribed nonbacktracking path, the persistence proof uses its occupied tube for the descendant witness and a tube with a different first edge for the parent witness; exhaustive coverage supplies both uniformly. A large expected covered fraction alone gives no persistence or adaptive-chase guarantee.
|
||||
|
||||
::: question
|
||||
**Question 18**. *For a fixed horizon $t$, is there a condition strictly weaker than occupancy of every length-$t$ tube that guarantees capture or a quantified decrease in the number of compatible future nonbacktracking continuations? Can it be updated after each move by reusing witnesses among tubes with a common shorter prefix?*
|
||||
:::
|
||||
|
||||
## Uniformly favorable augmented-profile fibers
|
||||
|
||||
Fix integers $Q\geq r+1\geq2$ and $k\geq1$, suppose $B_Q(v_0)$ is a tree-ball, and fix a length-$(r+1)$ nonbacktracking path $\tau=(v_0,\ldots,v_{r+1})$. The graph, root, radius, cop number, allowed support region, and allowed distance-minimizing tie-breaks are thereby fixed.
|
||||
|
||||
::: question
|
||||
**Question 19**. *For which admissible profiles $P$ do both of the following hold?*
|
||||
|
||||
1. *Some $X\in\mathcal F_{r;Q,k}(P)$ and some allowed sequence of cop moves let $\tau$ survive through round $r+1$.*
|
||||
|
||||
2. *For every $X\in\mathcal F_{r;Q,k}(P)$ and every allowed sequence of cop moves, the robber is captured or blocked by round $r+1$, or $\tau$ survives and its final branch-load support has size at least $2$.*
|
||||
:::
|
||||
|
||||
## Global overlap of tube systems
|
||||
|
||||
Fix a graph $G$, integers $R\geq t\geq1$, a set $A$ of admissible roots whose radius-$R$ balls are tree-balls, and a root-independent set $W$ of allowed cop locations. For $v\in A$ and a length-$t$ nonbacktracking path $\sigma$ from $v$, write $T_\sigma^v(R)$ for its rooted tube. Define the global tube hypergraph $$\mathcal H_{R,t}(G,A;W)
|
||||
:=\left(W,\{T_\sigma^v(R)\cap W:v\in A,\ \sigma\text{ is a length-$t$ nonbacktracking path from }v\}\right).$$ A root-independent set of cop positions supplies the exhaustive tube certificate simultaneously for every $v\in A$ exactly when it is a transversal of $\mathcal H_{R,t}$. This is the local-to-global gap absent from the conditional sampling theorem; multiplicity at an already selected vertex does not improve coverage.
|
||||
|
||||
::: question
|
||||
**Question 20**. *How do girth, expansion, and nonbacktracking path counts bound the transversal number, fractional transversal number, and codegrees of $\mathcal H_{R,t}$? Can these estimates produce a root-independent placement with controlled cost?*
|
||||
:::
|
||||
|
||||
## Beyond exact tree-balls
|
||||
|
||||
The persistence proof uses only the unique geodesics of synchronized witnesses inside $B_R(v_0)$. With cycles, rooted branches would have to give way to a shortest-path directed acyclic graph in which paths may split or merge. No persistence theorem or coverage bound is proved in that setting.
|
||||
|
||||
::: question
|
||||
**Question 21**. *For a radius-$R$ ball obtained from a tree by adding one edge, can geodesic tubes and branch-load support be replaced by notions for which an analogue of Theorem [10](#thm-persistence) holds? More generally, how do the answer and the minimum exhaustive-coverage cost depend on bounded tree excess?*
|
||||
:::
|
||||
|
||||
# Conclusion
|
||||
|
||||
We analyzed a local certificate indexed by length-$t$ nonbacktracking robber paths starting at a fixed root $v_0$ in radius-$R$ tree-ball geometry. The larger radius $R+t$ is sharp, among possibly infinite $d$-regular graphs, for uniform control of arbitrary endpoint pairs, but the synchronized chase witnesses require only $B_R(v_0)$. The length-$t$ tubes partition the outer ball into $N_t=d(d-1)^{t-1}$ parts, and occupying every tube gives the stated capture, blockage, or two-branch-support alternative along every prescribed path in this class.
|
||||
|
||||
At one fixed root the deterministic occupancy cost is $N_t$. Along fixed-degree rooted sequences with $t\to\infty$, the conditional uniform-sampling threshold is $$\left(\frac{1}{1-(d-1)^{-(h_t+1)}}+o(1)\right)N_t\log N_t,
|
||||
\qquad h_t=R_t-t.$$ Because the sampling distribution is chosen after the root, it is not a legal standard-game initial placement and gives no cop-number bound. The arbitrary-walk, partial/adaptive, global, augmented-profile, and cyclic directions above are not a
|
||||
|
|
@ -0,0 +1,51 @@
|
|||
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 280 280" role="img" aria-labelledby="mark-title-branch-based">
|
||||
<title id="mark-title-branch-based">Two cones meeting at a vertex on a regular tree, with a depth-budget bracket beneath</title>
|
||||
<desc>A frontispiece mark for "Branch-Tube Persistence and Static Coverage in Tree-Ball Geometry".</desc>
|
||||
|
||||
<circle cx="140" cy="140" r="128" stroke="currentColor" stroke-width="0.6" fill="none"/>
|
||||
|
||||
<g stroke="currentColor" fill="none" stroke-linecap="round" stroke-linejoin="round">
|
||||
|
||||
<line x1="140" y1="78" x2="140" y2="118" stroke-width="1.2"/>
|
||||
|
||||
<line x1="140" y1="118" x2="105" y2="148" stroke-width="1.0"/>
|
||||
<line x1="140" y1="118" x2="175" y2="148" stroke-width="1.0"/>
|
||||
|
||||
<line x1="105" y1="148" x2="85" y2="178" stroke-width="0.8"/>
|
||||
<line x1="105" y1="148" x2="125" y2="178" stroke-width="0.8"/>
|
||||
<line x1="175" y1="148" x2="155" y2="178" stroke-width="0.8"/>
|
||||
<line x1="175" y1="148" x2="195" y2="178" stroke-width="0.8"/>
|
||||
|
||||
<line x1="85" y1="178" x2="73" y2="200" stroke-width="0.5"/>
|
||||
<line x1="85" y1="178" x2="97" y2="200" stroke-width="0.5"/>
|
||||
<line x1="125" y1="178" x2="113" y2="200" stroke-width="0.5"/>
|
||||
<line x1="125" y1="178" x2="137" y2="200" stroke-width="0.5"/>
|
||||
<line x1="155" y1="178" x2="143" y2="200" stroke-width="0.5"/>
|
||||
<line x1="155" y1="178" x2="167" y2="200" stroke-width="0.5"/>
|
||||
<line x1="195" y1="178" x2="183" y2="200" stroke-width="0.5"/>
|
||||
<line x1="195" y1="178" x2="207" y2="200" stroke-width="0.5"/>
|
||||
</g>
|
||||
|
||||
<g fill="currentColor" stroke="none">
|
||||
<circle cx="140" cy="78" r="2.4"/>
|
||||
<circle cx="140" cy="118" r="1.8"/>
|
||||
<circle cx="105" cy="148" r="1.6"/>
|
||||
<circle cx="175" cy="148" r="1.6"/>
|
||||
<circle cx="85" cy="178" r="1.2"/>
|
||||
<circle cx="125" cy="178" r="1.2"/>
|
||||
<circle cx="155" cy="178" r="1.2"/>
|
||||
<circle cx="195" cy="178" r="1.2"/>
|
||||
</g>
|
||||
|
||||
<g stroke="currentColor" fill="none" stroke-linecap="round" stroke-width="0.6">
|
||||
<path d="M 105 148 Q 140 168 175 148"/>
|
||||
<path d="M 85 178 Q 195 215 195 178"/>
|
||||
</g>
|
||||
|
||||
<g stroke="currentColor" fill="none" stroke-linecap="round" stroke-linejoin="round">
|
||||
<line x1="58" y1="226" x2="222" y2="226" stroke-width="0.8"/>
|
||||
<line x1="58" y1="226" x2="58" y2="220" stroke-width="0.8"/>
|
||||
<line x1="222" y1="226" x2="222" y2="220" stroke-width="0.8"/>
|
||||
<line x1="140" y1="226" x2="140" y2="232" stroke-width="0.8"/>
|
||||
</g>
|
||||
</svg>
|
||||
|
After Width: | Height: | Size: 2.4 KiB |
|
|
@ -0,0 +1,22 @@
|
|||
---
|
||||
title: "Empirical Musings on Spaced Repetition for Learning Beyond Memorization"
|
||||
date: 2026-03-26
|
||||
abstract: >
|
||||
A compendium of years of informal, empirical experiments aimed at extending the efficacy of Anki beyond rote memorization to more intricate levels of learning.
|
||||
tags:
|
||||
- nonfiction
|
||||
- research
|
||||
- research/informal
|
||||
- miscellany
|
||||
authors:
|
||||
- "Levi Neuwirth | /me.html"
|
||||
status: "Draft"
|
||||
confidence: 65
|
||||
importance: 3
|
||||
evidence: 2
|
||||
scope: broad
|
||||
novelty: idiosyncratic
|
||||
practicality: high
|
||||
confidence-history:
|
||||
- 65
|
||||
---
|
||||
|
|
@ -0,0 +1,65 @@
|
|||
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 280 280" role="img" aria-labelledby="mark-title-growing-radius">
|
||||
<title id="mark-title-growing-radius">The same radius-2 ball with a soft circumscribing arc, a reverse-shooting orbit of growing dots leading from a terminal wall at right back toward the root, and a printer's signature stub beneath</title>
|
||||
<desc>A frontispiece mark for "From Path Tubes to a Near-Critical Domination Bound" — the narrative companion to the preprint. The central figure is the same radius-2 ball in the 3-regular tree used by the preprint mark, so the two marks read as a matched pair. To the right of the ball, a small vertical bar marks the terminal parameter; from there, a sequence of dots grows leftward-and-downward toward the root, each successively larger than the last, encoding the reverse-shooting orbit that reconstructs the stationary point from a terminal parameter. Beneath the ball, a small printer's signature stub with an open circle: the piece's whole point is auditable code as part of the mathematics, and the stub is the visual echo of the SHA-256 signing hash the paper insists on. The mark says: here is the ball, here is how it was found, here is the artifact of the process.</desc>
|
||||
|
||||
<circle cx="140" cy="140" r="128" stroke="currentColor" stroke-width="0.6" fill="none"/>
|
||||
|
||||
<g stroke="currentColor" fill="none" stroke-linecap="round" stroke-linejoin="round">
|
||||
<line x1="140" y1="180" x2="102" y2="160" stroke-width="1.0"/>
|
||||
<line x1="140" y1="180" x2="140" y2="154" stroke-width="1.0"/>
|
||||
<line x1="140" y1="180" x2="178" y2="160" stroke-width="1.0"/>
|
||||
|
||||
<line x1="102" y1="160" x2="82" y2="134" stroke-width="0.8"/>
|
||||
<line x1="102" y1="160" x2="112" y2="130" stroke-width="0.8"/>
|
||||
<line x1="140" y1="154" x2="126" y2="124" stroke-width="0.8"/>
|
||||
<line x1="140" y1="154" x2="154" y2="124" stroke-width="0.8"/>
|
||||
<line x1="178" y1="160" x2="168" y2="130" stroke-width="0.8"/>
|
||||
<line x1="178" y1="160" x2="198" y2="134" stroke-width="0.8"/>
|
||||
</g>
|
||||
|
||||
<g fill="currentColor" stroke="none">
|
||||
<circle cx="140" cy="180" r="2.4"/>
|
||||
|
||||
<circle cx="102" cy="160" r="1.7"/>
|
||||
<circle cx="140" cy="154" r="1.7"/>
|
||||
<circle cx="178" cy="160" r="1.7"/>
|
||||
|
||||
<circle cx="82" cy="134" r="1.3"/>
|
||||
<circle cx="112" cy="114" r="1.3"/>
|
||||
<circle cx="126" cy="124" r="1.3"/>
|
||||
<circle cx="154" cy="124" r="1.3"/>
|
||||
<circle cx="168" cy="114" r="1.3"/>
|
||||
<circle cx="198" cy="134" r="1.3"/>
|
||||
</g>
|
||||
|
||||
<path
|
||||
d="M 70 136 Q 70 112 90 102 Q 120 92 140 92 Q 160 92 190 102 Q 210 112 210 136"
|
||||
stroke="currentColor" stroke-width="0.5" fill="none" stroke-linecap="round"
|
||||
stroke-dasharray="3 2" opacity="0.55"/>
|
||||
|
||||
<line x1="238" y1="60" x2="238" y2="100" stroke="currentColor" stroke-width="1.2" stroke-linecap="round"/>
|
||||
|
||||
<g fill="currentColor" stroke="none">
|
||||
<circle cx="234" cy="76" r="0.7"/>
|
||||
<circle cx="224" cy="82" r="0.9"/>
|
||||
<circle cx="212" cy="90" r="1.1"/>
|
||||
<circle cx="198" cy="100" r="1.3"/>
|
||||
<circle cx="182" cy="112" r="1.5"/>
|
||||
</g>
|
||||
|
||||
<g stroke="currentColor" fill="none" stroke-linecap="round" stroke-width="0.4" opacity="0.6">
|
||||
<path d="M 232 78 L 226 80"/>
|
||||
<path d="M 222 84 L 214 88"/>
|
||||
<path d="M 210 92 L 200 98"/>
|
||||
<path d="M 196 102 L 184 110"/>
|
||||
</g>
|
||||
|
||||
<g stroke="currentColor" fill="none" stroke-linecap="round" stroke-linejoin="round">
|
||||
<line x1="140" y1="220" x2="140" y2="234" stroke-width="0.8"/>
|
||||
<line x1="132" y1="234" x2="148" y2="234" stroke-width="0.9"/>
|
||||
<line x1="134" y1="238" x2="146" y2="238" stroke-width="0.7"/>
|
||||
</g>
|
||||
|
||||
<circle cx="140" cy="216" r="2.2" stroke="currentColor" stroke-width="0.8" fill="none"/>
|
||||
|
||||
</svg>
|
||||
|
After Width: | Height: | Size: 3.6 KiB |
|
|
@ -0,0 +1,932 @@
|
|||
---
|
||||
title: "From Path Tubes to a Near-Critical Domination Bound"
|
||||
date: 2026-07-22
|
||||
abstract: >
|
||||
A first-person, code-heavy companion to the preprint *Near-Critical
|
||||
First-Moment Lower Bounds for Growing-Radius Domination in Random Regular
|
||||
Graphs*. Rather than reproducing its theorem-and-proof form, this page
|
||||
traces where the problem came from — a cops-and-robbers hypergraph question
|
||||
that collapsed into a domination bound — why the answer carries an
|
||||
unnecessary coupon-collector logarithm, and how a chain of computational
|
||||
detours (a failed concavity conjecture, a catastrophic cancellation, an
|
||||
independent audit that caught a stale constant) repeatedly redirected the
|
||||
proof before it reached its final shape.
|
||||
tags:
|
||||
- research
|
||||
- research/mathematics
|
||||
- research/graph-theory
|
||||
- tech/python
|
||||
authors:
|
||||
- "Levi Neuwirth | /me.html"
|
||||
preprint: /papers/growing-radius-domination-paper.pdf
|
||||
status: "Durable"
|
||||
confidence: proved
|
||||
evidence: 5
|
||||
peer-status: unreviewed
|
||||
result-shape: mixed
|
||||
---
|
||||
|
||||
This page is a companion to the preprint [**The Annealed Critical Window for Growing-Radius Domination in Random Regular Graphs**](/papers/growing-radius-domination-paper.pdf) — [the full theorem-and-proof form is here](/essays/near-critical-growing-radius-domination.html). A complete, runnable version — [`growing-radius-domination-demo.py`](/papers/growing-radius-domination-demo.py) — ships alongside this page, together with [the CSV of diagnostic output](/papers/growing-radius-domination-demo-output.csv) it produces.
|
||||
|
||||
The main theorem is the following. Fix a degree $d\ge 3$ and let
|
||||
|
||||
$$
|
||||
B_h=1+d\frac{(d-1)^h-1}{d-2}
|
||||
$$
|
||||
|
||||
be the number of vertices in a radius-$h$ ball of the infinite $d$-regular tree. For a graph $G$, write $\gamma_h(G)$ for the minimum size of a set whose distance-$h$ neighborhoods cover every vertex. If $G_{n,d}$ is a uniformly random simple $d$-regular graph and $h=h(n)\to\infty$, then for every $W_h\to\infty$ for which the displayed coordinate stays a positive fraction of $\log B_h$ and the type-counting error stays asymptotically below the coupon term,
|
||||
|
||||
::: {.exhibit .exhibit--equation}
|
||||
$$
|
||||
\gamma_h(G_{n,d})\ge
|
||||
\frac{n}{B_h}
|
||||
\left(
|
||||
\log B_h-2\log\log B_h-W_h
|
||||
\right)
|
||||
$$
|
||||
:::
|
||||
|
||||
with high probability.
|
||||
|
||||
The $-W_h$ is deliberate, and it is the honest part of the statement. Letting $W_h$ diverge arbitrarily slowly walks the bound arbitrarily close to the predicted bounded critical window without ever resolving that window. The theorem reaches to within a diverging additive whisker of the transition and stops. At the comparison scale $B_h\asymp\sqrt n$, with such a slowly growing $W_h$, the lower bound has order
|
||||
|
||||
$$
|
||||
\sqrt n\log n,
|
||||
$$
|
||||
|
||||
not merely $\sqrt n$. That single extra logarithm is QUITE LITERALLY the entire reason this project exists, and it is why the whole thing began, improbably, inside a cops-and-robbers calculation. Before I return to pursuit-evasion, the domination problem deserves to be seen on its own terms.
|
||||
|
||||
---
|
||||
|
||||
## 1. The first calculation: how large is a ball?
|
||||
|
||||
The geometry starts with a one-line function. I mean that literally.
|
||||
|
||||
```python
|
||||
import mpmath as mp
|
||||
|
||||
|
||||
def tree_ball_volume(d: int, h: int) -> mp.mpf:
|
||||
"""Radius-h volume of the infinite d-regular tree."""
|
||||
if d < 3:
|
||||
raise ValueError("d must be at least 3")
|
||||
if h < 0:
|
||||
raise ValueError("h must be nonnegative")
|
||||
b = d - 1
|
||||
return mp.mpf(1) + d * (mp.power(b, h) - 1) / (d - 2)
|
||||
```
|
||||
|
||||
If every chosen vertex covers at most $B_h$ vertices, pure volume counting gives
|
||||
|
||||
$$
|
||||
\gamma_h(G)\ge \frac{n}{B_h}.
|
||||
$$
|
||||
|
||||
That bound is occasionally attainable, on graphs organized precisely enough to make it tight — perfect covering codes are the cleanest examples of what such organization looks like. A typical random graph is not organized. Its neighborhoods overlap wastefully, and covering the last few stubborn vertices costs extra. The question is how much extra, and the coupon heuristic gives the first honest guess.
|
||||
|
||||
For a random selected set of density $\alpha$, a particular radius-$h$ ball is missed with probability roughly
|
||||
|
||||
$$
|
||||
(1-\alpha)^{B_h}\approx e^{-\alpha B_h}.
|
||||
$$
|
||||
|
||||
There are about $\exp(nH(\alpha))$ sets of density $\alpha$, with $H$ the binary entropy. Balancing the entropy of choosing the set against the probability that it happens to cover everything predicts the coordinate
|
||||
|
||||
$$
|
||||
\alpha B_h
|
||||
\approx
|
||||
\log B_h-2\log\log B_h.
|
||||
$$
|
||||
|
||||
The helper below computes a point just below that predicted transition.
|
||||
|
||||
```python
|
||||
|
||||
def near_critical_coordinate(
|
||||
d: int,
|
||||
h: int,
|
||||
W: mp.mpf | None = None,
|
||||
) -> mp.mpf:
|
||||
"""Return C = log B_h - 2 log log B_h - W."""
|
||||
B = tree_ball_volume(d, h)
|
||||
L = mp.log(B)
|
||||
if W is None:
|
||||
W = mp.log(L)
|
||||
return L - 2 * mp.log(L) - W
|
||||
```
|
||||
|
||||
The theorem says this heuristic scale survives optimization over the selected set, and that survival is the whole difficulty. A minimum dominating set is not sampled independently of anything. It sees the graph and gets to arrange itself around the overlaps, and there is no a priori reason a clever arrangement could not beat the coupon prediction.
|
||||
|
||||
---
|
||||
|
||||
## 2. Where the problem came from
|
||||
|
||||
The [preceding project](/essays/branch-based-local-capture-in-tree-balls/index.html) studied a rigid local certificate in the cops-and-robbers game. Around a fixed robber position, every outward nonbacktracking path indexed a descendant "tube," and if every tube held a cop, then every surviving round ended in capture, blockage, or pressure from at least two branches. It was a clean local statement, and it left an obvious global question hanging over it: could one cop placement, chosen once, hit every such tube for every possible robber root at the same time?
|
||||
|
||||
At first this looked like a monstrous hypergraph problem — many roots, exponentially many paths through each — and I spent far longer than I would like to admit intimidated by it. Then, as they somehow often do, the combinatorics simply collapsed. Once the root is allowed to vary, a tube remembers nothing^[I think we mathematicians should really have a term for amnesia. We have the [Markov Property](https://en.wikipedia.org/wiki/Markov_property), but can we facetciously generalize further?] about its interior. It remembers only its **terminal directed edge** and the unused residual radius
|
||||
|
||||
$$
|
||||
h=R-t.
|
||||
$$
|
||||
|
||||
For a directed edge $p\to w$, the associated set is the forward cone from $w$ of depth $h$, with the branch back through $p$ excluded. A vertex set hits every such directed cone at $w$ exactly when either $w$ is itself selected, or at least two distinct branches at $w$ contain selected vertices within distance $h$. The exponential path structure was never really there. It was an artifact of insisting on a fixed root, and the clean way to say this uses no tree at all.
|
||||
|
||||
A set $S$ is **internally two-path $(h,2)$-dominating** if every $v\notin S$ has two paths of length at most $h$ from $v$ to $S$ whose only shared vertex is $v$. Any such set is automatically ordinary distance-$h$ dominating, because either path on its own already witnesses a selected vertex within distance $h$.
|
||||
|
||||
So, a lower bound for ordinary distance-$h$ domination is, for free, a lower bound for this stronger global tube certificate. The theorem therefore says something I find satisfying: exhaustive static tube coverage keeps its logarithmic overhead even after you allow all roots to share a single placement. Collapsing the hypergraph bought no asymptotic mercy.
|
||||
|
||||
It does **not** say that cops need $\sqrt n\log n$ in the actual game. Adaptive strategies, partial coverage, motion between epochs, and entirely different certificates all remain outside the argument, exactly as they did in the previous paper. I will return to how narrow this obstruction is, deliberately, at the end. For now it is enough that the pursuit question handed the domination problem a reason to care about the scale $B_h\asymp\sqrt n$, and then got out of the way.
|
||||
|
||||
---
|
||||
|
||||
## 3. Why the coupon heuristic is not a proof
|
||||
|
||||
The direct concentration approach dies on the scale of the problem, and it dies quickly. Changing a single pairing in the configuration model can flip the radius-$h$ coverage status of on the order of $B_h$ vertices. The natural Lipschitz constant therefore grows on precisely the scale I need to resolve, which means the usual bounded-differences machinery is not merely weak here, but rather calibrated to lose.
|
||||
|
||||
The overlap is just as unforgiving. The events "the ball around $v$ is uncovered" and "the ball around $w$ is uncovered" share most of their mass. At $B_h\asymp\sqrt n$ this is exactly the birthday-collision scale for random regular neighborhoods. What made the problem exact was giving up on balls entirely and tracking **distances** instead.
|
||||
|
||||
Given a candidate set $S$, label every vertex by
|
||||
|
||||
$$
|
||||
\ell(v)=\operatorname{dist}(v,S)\in\{0,1,\ldots,h\}.
|
||||
$$
|
||||
|
||||
These labels obey two entirely local rules:
|
||||
|
||||
1. labels on adjacent vertices differ by at most one;
|
||||
2. every vertex of positive label $i$ has a neighbor of label $i-1$.
|
||||
|
||||
And the rules run in reverse: any labeling obeying them is forced to be genuine graph distance. Descending labels trace a path down to label zero of exactly the stated length, and no path leaving label zero can climb by more than one per edge. The two inequalities pin the label to the distance from both sides.
|
||||
|
||||
That observation may come across as elementary, but it is the hinge of the whole paper. It removes any need to pretend random neighborhoods are trees. The labels are honest distances on the honest graph, and it turns the first moment into an exact method-of-types calculation. The price, which I did not appreciate until much later, is a nonconcave variational problem whose optimizer I would have to pin down uniformly as the number of levels grew. I traded an intractable probabilistic estimate for a hard but finite optimization.
|
||||
|
||||
---
|
||||
|
||||
## 4. The local entropy that appears in the type count
|
||||
|
||||
At a vertex of label $i>0$, at least one of its $d$ incident half-edges has to descend to label $i-1$. Suppose each coordinate carries descending marginal $a$. The maximum-entropy distribution over nonempty subsets of the $d$ half-edges is then an exponentially tilted law, and its marginal and entropy are
|
||||
|
||||
$$
|
||||
a=
|
||||
\frac{\lambda(1+\lambda)^{d-1}}
|
||||
{(1+\lambda)^d-1},
|
||||
$$
|
||||
|
||||
$$
|
||||
s_d(a)=
|
||||
\log\big((1+\lambda)^d-1\big)
|
||||
-da\log\lambda.
|
||||
$$
|
||||
|
||||
The numerics need care near the Moore-growth boundary $a=1/d$, where $\lambda$ can be exponentially small and a naive inversion silently underflows.
|
||||
|
||||
```python
|
||||
|
||||
def _conditioned_marginal(d: int, lam: mp.mpf) -> mp.mpf:
|
||||
"""Marginal in a nonempty tilted subset of [d]."""
|
||||
log1p_lam = mp.log1p(lam)
|
||||
numerator = lam * mp.exp((d - 1) * log1p_lam)
|
||||
denominator = mp.expm1(d * log1p_lam)
|
||||
return numerator / denominator
|
||||
|
||||
|
||||
def lambda_from_marginal(d: int, a: mp.mpf) -> mp.mpf:
|
||||
"""Invert the conditioned-subset marginal in log lambda."""
|
||||
a = mp.mpf(a)
|
||||
lower = mp.mpf(1) / d
|
||||
if a < lower or a > 1:
|
||||
raise ValueError("a must lie in [1/d, 1]")
|
||||
|
||||
tol = mp.power(10, -(mp.mp.dps - 12))
|
||||
if abs(a - lower) <= tol:
|
||||
return mp.mpf(0)
|
||||
if abs(a - 1) <= tol:
|
||||
return mp.inf
|
||||
|
||||
lo = -mp.mpf(2) * mp.mp.dps * mp.log(10)
|
||||
hi = mp.mpf(2) * mp.mp.dps * mp.log(10)
|
||||
for _ in range(max(160, 3 * mp.mp.dps)):
|
||||
mid = (lo + hi) / 2
|
||||
lam = mp.exp(mid)
|
||||
if _conditioned_marginal(d, lam) < a:
|
||||
lo = mid
|
||||
else:
|
||||
hi = mid
|
||||
return mp.exp((lo + hi) / 2)
|
||||
|
||||
|
||||
def conditioned_subset_entropy(d: int, a: mp.mpf) -> mp.mpf:
|
||||
"""Maximum entropy s_d(a) with the all-zero pattern excluded."""
|
||||
a = mp.mpf(a)
|
||||
lower = mp.mpf(1) / d
|
||||
tol = mp.power(10, -(mp.mp.dps - 12))
|
||||
if abs(a - lower) <= tol:
|
||||
return mp.log(d)
|
||||
if abs(a - 1) <= tol:
|
||||
return mp.mpf(0)
|
||||
|
||||
lam = lambda_from_marginal(d, a)
|
||||
log_partition = mp.log(mp.expm1(d * mp.log1p(lam)))
|
||||
return log_partition - d * a * mp.log(lam)
|
||||
```
|
||||
|
||||
The precision-scaled endpoint is not fussiness for its own sake. An early independent audit by [Claude Fable 5](https://www.anthropic.com/claude/fable) found that a fixed lower cutoff in the $\lambda$ solver was quietly clamping the deep layers and manufacturing false KKT residuals. The whole computation lives exactly where the asymptotic coordinates separate exponentially. How you handle the endpoints *is* part of the mathematics, not just an implementation detail beneath it.[^audit]
|
||||
|
||||
[^audit]: I have come to treat "it's just a numerical detail" as a (not so) small alarm bell. For this problem in particular, it was never once true.
|
||||
|
||||
---
|
||||
|
||||
## 5. The exact compact functional
|
||||
|
||||
Let
|
||||
|
||||
$$
|
||||
x_i=q_{i-1,i},\qquad 1\le i\le h,
|
||||
$$
|
||||
|
||||
be the directed-edge mass between adjacent distance levels, and let
|
||||
|
||||
$$
|
||||
\ell_i=q_{i,i},\qquad 0\le i\le h,
|
||||
$$
|
||||
|
||||
be the same-level edge mass. The layer masses are
|
||||
|
||||
$$
|
||||
p_i=\ell_i+x_i+x_{i+1},
|
||||
$$
|
||||
|
||||
with $x_0=x_{h+1}=0$. Put
|
||||
|
||||
$$
|
||||
a_i=\frac{x_i}{p_i},
|
||||
\qquad
|
||||
y_i=p_i-x_i.
|
||||
$$
|
||||
|
||||
After optimizing out the full local neighbor-count profile, the exact annealed functional collapses to
|
||||
|
||||
$$
|
||||
\begin{aligned}
|
||||
\mathcal F_{d,h}
|
||||
={}&(d-1)\alpha\log\alpha\\
|
||||
&+\sum_{i=1}^h
|
||||
\left[
|
||||
-p_i\log p_i
|
||||
+p_i s_d(a_i)
|
||||
+d y_i\log y_i
|
||||
\right]\\
|
||||
&-\frac d2\sum_{i=0}^h\ell_i\log\ell_i,
|
||||
\end{aligned}
|
||||
$$
|
||||
|
||||
with $\alpha=p_0$. The reduction from the full profile to this $2h+1$-variable object is exact, not an approximation I am hoping is harmless. The exactness is proved in the preprint and it is what lets me trust the code below to be computing the real thing.
|
||||
|
||||
The implementation follows the displayed formula line for line.
|
||||
|
||||
```python
|
||||
|
||||
def _xlogx(x: mp.mpf) -> mp.mpf:
|
||||
return mp.mpf(0) if x == 0 else x * mp.log(x)
|
||||
|
||||
|
||||
def compact_microcanonical_value(d, h, x, ell):
|
||||
"""Evaluate the exact compact functional."""
|
||||
xx = [mp.mpf(0)] + [mp.mpf(v) for v in x] + [mp.mpf(0)]
|
||||
ll = [mp.mpf(v) for v in ell]
|
||||
p = [ll[i] + xx[i] + xx[i + 1] for i in range(h + 1)]
|
||||
|
||||
if abs(sum(p) - 1) > mp.power(10, -(mp.mp.dps // 2)):
|
||||
raise ValueError("profile is not normalized")
|
||||
|
||||
alpha = p[0]
|
||||
value = (d - 1) * _xlogx(alpha)
|
||||
|
||||
for i in range(1, h + 1):
|
||||
if p[i] == 0:
|
||||
continue
|
||||
lower = mp.mpf(1) / d
|
||||
a_i = xx[i] / p[i]
|
||||
numerical_tol = mp.power(10, -(mp.mp.dps // 3))
|
||||
if a_i < lower - numerical_tol or a_i > 1 + numerical_tol:
|
||||
raise ValueError("descent marginal is infeasible")
|
||||
a_i = min(mp.mpf(1), max(lower, a_i))
|
||||
|
||||
y_i = p[i] - xx[i]
|
||||
value += -_xlogx(p[i])
|
||||
value += p[i] * conditioned_subset_entropy(d, a_i)
|
||||
value += d * _xlogx(y_i)
|
||||
|
||||
value -= mp.mpf(d) / 2 * sum(_xlogx(v) for v in ll)
|
||||
return value, alpha
|
||||
```
|
||||
|
||||
The full type count is two-sided. For each integer type, its expected count in the pairing model is an explicit ratio of factorials; uniform Stirling estimates identify its exponential rate with $\mathcal F_{d,h}$ up to $O_d(h\log n)$. The lower side of that estimate is the one I nearly overlooked. Because the expected number of any single type cannot exceed the total number of subsets of the corresponding size, it hands over the pointwise anchor
|
||||
|
||||
$$
|
||||
\Psi_{d,h}(\alpha)\le H(\alpha).
|
||||
$$
|
||||
|
||||
An earlier working note of mine called this "the trivial counting bound" and moved on. It was not trivial, and it was not something to move on from. It turned out to be the last missing lemma in the first-moment theorem, and I had worked it out long before I understood I already had it.[^trivial]
|
||||
|
||||
[^trivial]: There is probably a general lesson here about the things one labels "trivial" in one's own notes. I have chosen not to learn it and will surely repeat the mistake.
|
||||
|
||||
---
|
||||
|
||||
## 6. A failed concavity conjecture
|
||||
|
||||
Once the profile was down to $2h+1$ variables, the functional looked numerically docile near the low density stationary branch. The obvious conjecture, and the one I wanted to be true, was that it might simply be concave throughout the slack region.
|
||||
|
||||
An adversarial Hessian search turned up profiles with genuinely positive constrained curvature. They had a recognizable shape: most layers sitting near capacity-saturated Moore growth, with a few isolated layers carrying much heavier same-level edge mass. Finite differences confirmed the positive eigenvalues of the analytic Hessian, so this was not a precision artifact, and the outcome I had wished for was, as is often the case in mathematics, completely dead.
|
||||
|
||||
The failure was, perhaps, the most useful thing that happened to the proof. It is interesting how failure is often the most useful outcome, contrary to how we perceive it. The positive curvature profiles were not stationary, as they were bumps off to the side, not competing maxima. This meant the right target was never concavity at all. It was **uniqueness of stationary points, together with boundary repulsion**. The proof then reassembled itself into something much cleaner than the one I had been trying to force:
|
||||
|
||||
- entropy singularities repel every boundary face;
|
||||
- so every maximizer is an interior KKT point;
|
||||
- the stationary equations admit an exact reverse transfer;
|
||||
- that reverse transfer is order preserving;
|
||||
- activity increases strictly along its single terminal parameter;
|
||||
- so there is exactly one stationary point at each activity;
|
||||
- and it has to be the unique global maximizer.
|
||||
|
||||
A nonconcave function is perfectly entitled to a unique global optimizer. Here it is monotone dynamics, not curvature, that supplies globality. I do not think I would ever have gone looking for the dynamics had the concavity conjecture not first embarrassed me out of the supposed easy route.
|
||||
|
||||
---
|
||||
|
||||
## 7. The exact reverse transfer
|
||||
|
||||
The stationary equations can be carried by two positive messages per distance level. Taking ratios, let
|
||||
|
||||
$$
|
||||
\rho_i=\frac{B_i}{A_i},
|
||||
\qquad
|
||||
u_i=\frac{A_{i+1}}{A_i},
|
||||
\qquad
|
||||
v_i=\rho_i+u_i.
|
||||
$$
|
||||
|
||||
At the terminal wall, $u_h=0$ and $v_h=\rho_h$. Given the next state $(\rho',v')$, the preceding state is fully explicit.
|
||||
|
||||
```python
|
||||
|
||||
def _w_e(rho: mp.mpf, b: int):
|
||||
"""w=(1-rho)^(1/b), e=1-w, evaluated stably."""
|
||||
log_w = mp.log1p(-rho) / b
|
||||
return mp.exp(log_w), -mp.expm1(log_w)
|
||||
|
||||
|
||||
def reverse_step(rho_next: mp.mpf, v_next: mp.mpf, b: int):
|
||||
"""Invert one stationary transfer step."""
|
||||
w, e = _w_e(rho_next, b)
|
||||
R = v_next * e / w
|
||||
M = mp.power(e + w / v_next, b)
|
||||
denominator = R + M
|
||||
rho = R / denominator
|
||||
v = (1 + R) / denominator
|
||||
return rho, v
|
||||
```
|
||||
|
||||
This map is coordinatewise order preserving, and that single property does most of the load-bearing work in the globality proof. (I couldn't help myself quoting Fable here -- world, take note, THIS was the result out of them all that Fable, under the role of auditor, called load-bearing.)^[This is a humorous interjection, for those of you who don't use Claude often. Claude models have a tendency to call anything that has some type of impact on the bigger picture "load-bearing," using this phrase obsessively to the point that it should probably be considered the new em dash. It is somewhat less of a tragedy; at least I and other reasonable humans greatly enjoyed usage of the dash in our personal writing. I don't think I've ever used the phrase "load bearing" in an organic sense, ever.] Start from a larger terminal value and every preceding $\rho$ coordinate comes out larger. The reconstructed activity is strictly increasing right along with it. The entire positive stationary family is therefore one-dimensional and uniquely parametrized by activity, which is exactly the uniqueness the failed concavity conjecture had been standing in for.
|
||||
|
||||
For computation, it is far better to parametrize the terminal value as
|
||||
|
||||
$$
|
||||
t=-\log(1-s),
|
||||
\qquad s=1-e^{-t}.
|
||||
$$
|
||||
|
||||
Near high density, $s$ becomes indistinguishable from one at ordinary precision while $t$ stays comfortably moderate. This is a small change with a large payoff, the kind of thing one only learns by watching a doomed solver quietly and blissfully lose all its digits at large $h$.
|
||||
|
||||
---
|
||||
|
||||
## 8. Reconstructing the stationary point
|
||||
|
||||
The central routine does four things in order:
|
||||
|
||||
1. reverse-iterate from the terminal wall;
|
||||
2. reconstruct the messages $A_i,B_i$;
|
||||
3. compute density and activity at the root;
|
||||
4. evaluate the free energy through *both* a direct partition-function route and a stable root-only route.
|
||||
|
||||
The shipped file carries the full dataclass with all the residual fields; the core calculation is below.
|
||||
|
||||
```python
|
||||
from dataclasses import dataclass
|
||||
|
||||
|
||||
@dataclass
|
||||
class StationaryPoint:
|
||||
d: int
|
||||
h: int
|
||||
terminal_t: mp.mpf
|
||||
terminal: mp.mpf
|
||||
z: mp.mpf
|
||||
alpha: mp.mpf
|
||||
phi_direct: mp.mpf
|
||||
psi_direct: mp.mpf
|
||||
phi_root: mp.mpf
|
||||
psi_root: mp.mpf
|
||||
kappa: mp.mpf
|
||||
r0: mp.mpf
|
||||
Zv: mp.mpf
|
||||
Ze: mp.mpf
|
||||
telescoping_residual: mp.mpf
|
||||
root_pressure_residual: mp.mpf
|
||||
root_micro_residual: mp.mpf
|
||||
stationarity_residual: mp.mpf
|
||||
compact_residual: mp.mpf
|
||||
rho: list[mp.mpf]
|
||||
u: list[mp.mpf]
|
||||
A: list[mp.mpf]
|
||||
Bmsg: list[mp.mpf]
|
||||
|
||||
|
||||
def _stable_power_difference(S, B, power):
|
||||
"""Compute S^power-(S-B)^power without cancellation."""
|
||||
ratio = B / S
|
||||
return mp.power(S, power) * (
|
||||
-mp.expm1(power * mp.log1p(-ratio))
|
||||
)
|
||||
```
|
||||
|
||||
```python
|
||||
|
||||
def stationary_from_terminal_t(
|
||||
d: int,
|
||||
h: int,
|
||||
terminal_t: mp.mpf,
|
||||
*,
|
||||
validate_compact: bool = True,
|
||||
) -> StationaryPoint:
|
||||
b = d - 1
|
||||
t = mp.mpf(terminal_t)
|
||||
terminal = -mp.expm1(-t)
|
||||
|
||||
rho = [mp.mpf(0)] * (h + 1)
|
||||
v = [mp.mpf(0)] * (h + 1)
|
||||
u = [mp.mpf(0)] * (h + 1)
|
||||
rho[h] = v[h] = terminal
|
||||
|
||||
for i in range(h - 1, 0, -1):
|
||||
rho[i], v[i] = reverse_step(rho[i + 1], v[i + 1], b)
|
||||
u[i] = v[i] - rho[i]
|
||||
u[h] = mp.mpf(0)
|
||||
|
||||
# Root reconstruction.
|
||||
w1, e1 = _w_e(rho[1], b)
|
||||
r0 = v[1] * e1 / w1
|
||||
kappa = mp.power(v[1] / w1, b)
|
||||
z = r0 * kappa / mp.power(1 + r0, b)
|
||||
|
||||
# Normalize A_1=1 and rebuild the messages.
|
||||
A = [mp.mpf(0)] * (h + 2)
|
||||
Bmsg = [mp.mpf(0)] * (h + 1)
|
||||
A[1] = mp.mpf(1)
|
||||
for i in range(1, h):
|
||||
A[i + 1] = u[i] * A[i]
|
||||
Bmsg[0] = r0
|
||||
for i in range(1, h + 1):
|
||||
Bmsg[i] = rho[i] * A[i]
|
||||
|
||||
S = [mp.mpf(0)] * (h + 1)
|
||||
S[0] = Bmsg[0] + A[1]
|
||||
for i in range(1, h):
|
||||
S[i] = Bmsg[i - 1] + Bmsg[i] + A[i + 1]
|
||||
S[h] = Bmsg[h - 1] + Bmsg[h]
|
||||
|
||||
vertex_terms = [z * mp.power(S[0], d)]
|
||||
for i in range(1, h + 1):
|
||||
vertex_terms.append(
|
||||
_stable_power_difference(S[i], Bmsg[i - 1], d)
|
||||
)
|
||||
|
||||
Zv = mp.fsum(vertex_terms)
|
||||
Ze = mp.fsum(value * value for value in Bmsg)
|
||||
Ze += 2 * mp.fsum(Bmsg[i] * A[i + 1] for i in range(h))
|
||||
|
||||
alpha = Bmsg[0] * (Bmsg[0] + A[1]) / Ze
|
||||
phi_direct = mp.log(Zv) - mp.mpf(d) / 2 * mp.log(Ze)
|
||||
psi_direct = phi_direct - alpha * mp.log(z)
|
||||
```
|
||||
|
||||
The exact telescoping identity is
|
||||
|
||||
$$
|
||||
Z_v=\kappa Z_e.
|
||||
$$
|
||||
|
||||
It is what lets me trade the direct partition-function route for root-only formulas that are far better conditioned than subtracting two enormous logarithms and praying.
|
||||
|
||||
```python
|
||||
phi_root = (
|
||||
mp.log(z)
|
||||
+ mp.mpf(d) / 2 * mp.log((1 + r0) / r0)
|
||||
+ mp.mpf(d - 2) / 2 * mp.log(alpha)
|
||||
)
|
||||
|
||||
psi_root = (
|
||||
(1 - alpha) * mp.log(kappa)
|
||||
- (mp.mpf(d - 2) / 2 + alpha) * mp.log(r0)
|
||||
+ mp.mpf(d - 2) / 2 * mp.log(alpha)
|
||||
+ (
|
||||
mp.mpf(d - 1) * alpha
|
||||
- mp.mpf(d - 2) / 2
|
||||
) * mp.log(1 + r0)
|
||||
)
|
||||
```
|
||||
|
||||
The routine then checks the stationary equations, reconstructs the compact profile
|
||||
|
||||
$$
|
||||
x_i=\frac{B_{i-1}A_i}{Z_e},
|
||||
\qquad
|
||||
\ell_i=\frac{B_i^2}{Z_e},
|
||||
$$
|
||||
|
||||
and confirms that the compact functional agrees with the root-only free energy.
|
||||
|
||||
```python
|
||||
stationarity = [
|
||||
abs(kappa * Bmsg[0] - z * mp.power(S[0], b))
|
||||
]
|
||||
for i in range(1, h + 1):
|
||||
stationarity.append(
|
||||
abs(kappa * A[i] - mp.power(S[i], b))
|
||||
)
|
||||
stationarity.append(
|
||||
abs(
|
||||
kappa * Bmsg[i]
|
||||
- _stable_power_difference(
|
||||
S[i], Bmsg[i - 1], b
|
||||
)
|
||||
)
|
||||
)
|
||||
|
||||
if validate_compact:
|
||||
x = [
|
||||
Bmsg[i - 1] * A[i] / Ze
|
||||
for i in range(1, h + 1)
|
||||
]
|
||||
ell = [
|
||||
Bmsg[i] * Bmsg[i] / Ze
|
||||
for i in range(h + 1)
|
||||
]
|
||||
compact_value, compact_alpha = compact_microcanonical_value(
|
||||
d, h, x, ell
|
||||
)
|
||||
compact_residual = max(
|
||||
abs(compact_value - psi_root),
|
||||
abs(compact_alpha - alpha),
|
||||
)
|
||||
else:
|
||||
compact_residual = mp.nan
|
||||
```
|
||||
|
||||
This is in many ways the computational core. It finds one exact finite-$h$ stationary orbit and audits every identity that the theory says must hold on it. The asymptotic proof does not lean on any of these numbers. What the code provides is thus a hostile, independent witness that the analytic estimates are describing the object I think they are.
|
||||
|
||||
---
|
||||
|
||||
## 9. Solving for a target density
|
||||
|
||||
The theorem is phrased in terms of
|
||||
|
||||
$$
|
||||
C=\alpha B_h.
|
||||
$$
|
||||
|
||||
Because terminal parameter, activity, and density all increase together, I can solve $\alpha B_h=C$ by bisection in the stable coordinate $t$ and never worry about which branch I am on.
|
||||
|
||||
```python
|
||||
|
||||
def solve_for_alpha_B(d, h, C, iterations=None):
|
||||
"""Solve alpha B_h = C by bisection in terminal_t."""
|
||||
B = tree_ball_volume(d, h)
|
||||
target_alpha = mp.mpf(C) / B
|
||||
if iterations is None:
|
||||
iterations = max(140, 2 * mp.mp.dps)
|
||||
|
||||
D = mp.mpf(d) / (d - 2)
|
||||
lo = mp.mpf("0.05") * C / D
|
||||
hi = mp.mpf("3.0") * C / D + 1
|
||||
|
||||
while stationary_from_terminal_t(
|
||||
d, h, lo, validate_compact=False
|
||||
).alpha > target_alpha:
|
||||
lo /= 2
|
||||
|
||||
while stationary_from_terminal_t(
|
||||
d, h, hi, validate_compact=False
|
||||
).alpha < target_alpha:
|
||||
hi *= 2
|
||||
|
||||
for _ in range(iterations):
|
||||
mid = (lo + hi) / 2
|
||||
point = stationary_from_terminal_t(
|
||||
d, h, mid, validate_compact=False
|
||||
)
|
||||
if point.alpha < target_alpha:
|
||||
lo = mid
|
||||
else:
|
||||
hi = mid
|
||||
|
||||
return stationary_from_terminal_t(
|
||||
d, h, (lo + hi) / 2, validate_compact=True
|
||||
)
|
||||
```
|
||||
|
||||
At the near-critical coordinate, the activity law is
|
||||
|
||||
$$
|
||||
-\log z
|
||||
=
|
||||
\log\frac1\alpha
|
||||
+B_h(1-\alpha)^{B_h}(1+o(1)),
|
||||
$$
|
||||
|
||||
so the quantity
|
||||
|
||||
$$
|
||||
\frac{-\log z-\log(1/\alpha)}
|
||||
{B_h(1-\alpha)^{B_h}}
|
||||
$$
|
||||
|
||||
ought to tend to one.
|
||||
|
||||
---
|
||||
|
||||
## 10. Numerical conditioning is part of the result
|
||||
|
||||
Two computational failures materially shaped how I worked, and I am keeping them in the write-up rather than sanding them out. This is the spirit of the website; much like the [living documents](https://levineuwirth.org/colophon.html#living-documents) that the Colophon describes, I will not hide the work that I supersede.
|
||||
|
||||
The first was the infuriating variant of mundane. An early comparison table mixed stationary values generated by two different solver versions, and one cubic $h=12$ entry came out wrong by about $2\times10^{-4}$. The asymptotic conclusion was untouched, as the error was far too small to matter for the theorem, but the table was no longer reproducible from the shipped code, which is its own kind of unacceptable.
|
||||
|
||||
The second was more serious. Direct evaluation of
|
||||
|
||||
$$
|
||||
\Psi=
|
||||
\log Z_v-\frac d2\log Z_e-\alpha\log z
|
||||
$$
|
||||
|
||||
turned catastrophically ill-conditioned near criticality. The expression subtracts two large logarithms to recover a very small number, and at cubic $h=52$, simply raising the working precision changed the reported ratio visibly. This is a glaring red flag. Lost digits! The stable root-only formula was right, and the direct route was quietly bleeding precision the whole time.
|
||||
|
||||
My response was the small principle that this section is really about: make the residuals **gate** the output rather than merely decorate it.
|
||||
|
||||
```python
|
||||
|
||||
def gate_point(point, route_tolerance="1e-6"):
|
||||
"""Reject a row if independent routes disagree too much."""
|
||||
tol = mp.mpf(route_tolerance)
|
||||
if point.psi_root == 0:
|
||||
raise RuntimeError("zero root-formula exponent")
|
||||
|
||||
route_disagreement = abs(
|
||||
point.root_micro_residual / point.psi_root
|
||||
)
|
||||
if route_disagreement > tol:
|
||||
raise RuntimeError(
|
||||
"microcanonical routes disagree: "
|
||||
f"relative discrepancy={route_disagreement}; "
|
||||
"increase mp.dps"
|
||||
)
|
||||
|
||||
if point.compact_residual > mp.sqrt(tol):
|
||||
raise RuntimeError(
|
||||
"compact reconstruction failed: "
|
||||
f"{point.compact_residual}"
|
||||
)
|
||||
```
|
||||
|
||||
The diagnostic row then keeps both routes, every major residual, the working precision, and a SHA-256 digest of the source file (aren't you amazed I didn't use BLAKE3?), so that any number I report can be traced back to the exact bytes that produced it.
|
||||
|
||||
```python
|
||||
|
||||
def diagnostic_row(d, h, W=None):
|
||||
B = tree_ball_volume(d, h)
|
||||
L = mp.log(B)
|
||||
if W is None:
|
||||
W = mp.log(L)
|
||||
C = near_critical_coordinate(d, h, W)
|
||||
|
||||
point = solve_for_alpha_B(d, h, C)
|
||||
gate_point(point)
|
||||
|
||||
coupon = B * mp.power(1 - point.alpha, B)
|
||||
activity_ratio = (
|
||||
-mp.log(point.z) - mp.log(1 / point.alpha)
|
||||
) / coupon
|
||||
scale = mp.exp(-C)
|
||||
|
||||
return {
|
||||
"d": d,
|
||||
"h": h,
|
||||
"dps": mp.mp.dps,
|
||||
"B_h": mp.nstr(B, 30),
|
||||
"C": mp.nstr(C, 24),
|
||||
"alpha_B_h": mp.nstr(point.alpha * B, 24),
|
||||
"terminal_t": mp.nstr(point.terminal_t, 30),
|
||||
"activity_ratio": mp.nstr(activity_ratio, 20),
|
||||
"minus_psi_over_exp_minus_C": mp.nstr(
|
||||
-point.psi_root / scale, 20
|
||||
),
|
||||
"psi_root": mp.nstr(point.psi_root, 24),
|
||||
"psi_direct": mp.nstr(point.psi_direct, 24),
|
||||
"route_relative_disagreement": mp.nstr(
|
||||
abs(point.root_micro_residual / point.psi_root), 12
|
||||
),
|
||||
"stationarity_residual": mp.nstr(
|
||||
point.stationarity_residual, 12
|
||||
),
|
||||
"telescoping_residual": mp.nstr(
|
||||
point.telescoping_residual, 12
|
||||
),
|
||||
"compact_residual": mp.nstr(
|
||||
point.compact_residual, 12
|
||||
),
|
||||
"source_sha256": hashlib.sha256(
|
||||
Path(__file__).read_bytes()
|
||||
).hexdigest(),
|
||||
}
|
||||
```
|
||||
|
||||
> If an internal cross-check is comparable in size to the quantity you are reporting, it should be a gate, not an unprinted field in a dataclass.
|
||||
|
||||
---
|
||||
|
||||
## 11. What the computation shows
|
||||
|
||||
Running the demonstration script with $W_h=\log\log B_h$ gives the following.
|
||||
|
||||
| $d$ | $h$ | $C$ | activity-law ratio | $-\Psi/e^{-C}$ |
|
||||
|---:|---:|---:|---:|---:|
|
||||
| 3 | 12 | 2.6889 | 0.4376 | 0.3870 |
|
||||
| 3 | 20 | 6.8451 | 0.5873 | 0.6484 |
|
||||
| 3 | 30 | 12.6345 | 0.9520 | 0.9418 |
|
||||
| 3 | 40 | 18.7408 | 0.9956 | 0.9757 |
|
||||
| 4 | 12 | 5.9859 | 0.6816 | 0.7285 |
|
||||
| 4 | 20 | 13.3029 | 0.9915 | 0.9706 |
|
||||
| 4 | 28 | 21.1087 | 0.9999 | 0.9800 |
|
||||
|
||||
The asymptotic theorem asks only for a conservative negative bound, so in a sense these numbers are more than it needs. But I'd argue they clearly earn their place: both ratios march toward one, and the convergence is visibly slow precisely when the coefficient $C/\log B_h$ is small. This is exactly what the proof's fixed-compact-window uniformity predicts, and exactly what a claim of uniformity all the way down to zero would *not* look like. The table is quietly consistent with the shape of the theorem, not just its sign.
|
||||
|
||||
The generator is satisfyingly short.
|
||||
|
||||
```python
|
||||
|
||||
def make_table(cases, csv_path=None):
|
||||
rows = []
|
||||
for d, h, dps in cases:
|
||||
mp.mp.dps = dps
|
||||
row = diagnostic_row(d, h)
|
||||
rows.append(row)
|
||||
print(
|
||||
f"d={d} h={h:>2} C={row['C']} "
|
||||
f"activity={row['activity_ratio']} "
|
||||
f"-Psi/e^-C={row['minus_psi_over_exp_minus_C']}"
|
||||
)
|
||||
|
||||
if csv_path:
|
||||
with open(csv_path, "w", newline="") as handle:
|
||||
writer = csv.DictWriter(
|
||||
handle,
|
||||
fieldnames=list(rows[0].keys()),
|
||||
)
|
||||
writer.writeheader()
|
||||
writer.writerows(rows)
|
||||
return rows
|
||||
```
|
||||
|
||||
The complete script also carries optional plotting for the activity-law and free-energy ratios, for those who would rather watch the convergence than read it off a table.
|
||||
|
||||
---
|
||||
|
||||
## 12. How the proof uses the orbit
|
||||
|
||||
The code finds one exact finite-$h$ stationary orbit. The proof has to understand that orbit *uniformly* as $h\to\infty$, which is a different and harder demand, and it is where the analysis actually happens.
|
||||
|
||||
There are two exact outer regimes. On the **free side**, a transverse defect $q_i$ is tiny and the main coordinate rides the free-coverage orbit. On the **stable side**, the occupied-message coordinate $\rho_i$ is tiny and the orbit approaches the empty-root manifold. I choose a matching layer where both transverse quantities are exponentially small at once, and shadow the true orbit against whichever exact outer solution is nearby.
|
||||
|
||||
The reverse map supplies one-step estimates that stay uniform even as the main coordinate itself shrinks toward nothing. That uniformity, rather than any single clever identity, is what makes the whole scheme go. It yields
|
||||
|
||||
$$
|
||||
\alpha=a_h(t)(1+o(1)),
|
||||
$$
|
||||
|
||||
with
|
||||
|
||||
$$
|
||||
a_h(t)=1-e^{-t/(d-1)^h},
|
||||
$$
|
||||
|
||||
and
|
||||
|
||||
$$
|
||||
\frac{x}{(d-1)^{h-1}}
|
||||
=(1-\alpha)^{B_h}(1+o(1)),
|
||||
$$
|
||||
|
||||
where $x=-\log u_1$ is the stable root coordinate. The edge normalizer localizes at the overlap layer, which is what converts the transported message density into the actual selected density. Root reconstruction then delivers the activity law
|
||||
|
||||
$$
|
||||
-\log z(\alpha)
|
||||
=
|
||||
\log\frac1\alpha
|
||||
+B_h(1-\alpha)^{B_h}(1+o(1)).
|
||||
$$
|
||||
|
||||
The globality theorem supplies the envelope identity
|
||||
|
||||
$$
|
||||
\Psi'_{d,h}(\alpha)=-\log z(\alpha),
|
||||
$$
|
||||
|
||||
and integrating it from a slightly larger density, anchored at the upper endpoint by
|
||||
|
||||
$$
|
||||
\Psi_{d,h}(\alpha)\le H(\alpha),
|
||||
$$
|
||||
|
||||
gives
|
||||
|
||||
$$
|
||||
\Psi_{d,h}(C/B_h)
|
||||
\le
|
||||
-\left(\frac12-o(1)\right)e^{-C}
|
||||
$$
|
||||
|
||||
uniformly across the near-critical window. The first moment then closes the random-graph statement, provided the exponential negative term outweighs the $O_d(h\log n)$ type-counting error — which is precisely the growth condition on $W_h$ that the theorem carries out front, now visibly earning its place.
|
||||
|
||||
---
|
||||
|
||||
## 13. What the theorem says about exhaustive tube coverage
|
||||
|
||||
Finally! I can pay off the pursuit problem it started with, as promised..
|
||||
|
||||
The original local certificate placed a cop in every path tube at a *known* root. Make the root unknown and force all roots to share one placement, and the path prefix collapses — as in Section 2 — to a single directed terminal edge. Hitting every resulting forward cone is equivalent, in the tree geometry, to having selected vertices in at least two branches at every unselected vertex, and the graph-general form of that condition is internally two-path $(h,2)$-domination.
|
||||
|
||||
Since $(h,2)$-domination implies ordinary distance-$h$ domination, the theorem hands the same lower bound to the exhaustive global certificate at no extra cost. At $B_h\asymp\sqrt n$, that certificate needs
|
||||
|
||||
$$
|
||||
\Omega(\sqrt n\log n)
|
||||
$$
|
||||
|
||||
selected vertices with high probability. One tempting route from the local tube theorem to a root-independent, Meyniel-scale placement is therefore closed.
|
||||
|
||||
---
|
||||
|
||||
## 14. Three remaining problems
|
||||
|
||||
### The bounded critical window
|
||||
|
||||
The theorem reaches
|
||||
|
||||
$$
|
||||
\log B_h-2\log\log B_h-W_h
|
||||
$$
|
||||
|
||||
for every $W_h\to\infty$, and stops one diverging step short of the bounded additive term. The stationary computation points hard at a specific answer: the annealed transition looks governed by the scalar balance
|
||||
|
||||
$$
|
||||
H(C/B_h)=(1-C/B_h)^{B_h},
|
||||
$$
|
||||
|
||||
whose solution satisfies
|
||||
|
||||
$$
|
||||
C=
|
||||
\log B_h-2\log\log B_h+o(1).
|
||||
$$
|
||||
|
||||
Proving the full bounded window, though, needs uniform matching sharper than the slack theorem ever required — errors controlled down to the same order as the competing entropy and coupon terms.
|
||||
|
||||
### Quenched matching
|
||||
|
||||
What I have is a lower bound. A simple random placement followed by patching the uncovered vertices gives an upper bound of order
|
||||
|
||||
$$
|
||||
\frac{n\log B_h}{B_h},
|
||||
$$
|
||||
|
||||
but it does not pin the actual domination number to the annealed transition. A matching quenched result likely wants a second moment, small-subgraph conditioning, or a genuinely problem-specific construction. I do not yet know which, so let me know if you do.
|
||||
|
||||
### The direct two-branch constant
|
||||
|
||||
Ordinary distance domination is only a relaxation of internally two-path domination, so the direct branch problem should carry its own coupon balance and, quite possibly, a different leading constant. One has to remember branch multiplicity, and cycles can make the message representations non-unique — I have left it as its own piece of work rather than forcing it into this one.
|
||||
|
||||
---
|
||||
|
||||
## 15. Why I am showing so much Python
|
||||
|
||||
The code was never a numerical appendix bolted onto a finished proof. It kept reaching back into the mathematics and rearranging it, and I can name some of the specific occasions:
|
||||
|
||||
- exact ILPs first revealed that the global path hypergraph collapsed to a domination problem;
|
||||
- transfer numerics suggested the $\log B_h-2\log\log B_h$ scale before I could prove it;
|
||||
- an independent implementation caught a stale stationary value hiding in a comparison table;
|
||||
- Hessian probes killed a plausible, load-bearing, and entirely false concavity conjecture;
|
||||
- that dead conjecture is what redirected the proof toward boundary repulsion and reverse shooting;
|
||||
- KKT checks exposed the last mechanical gap in the globality argument;
|
||||
- residual-gated near-critical runs caught catastrophic cancellation in the direct free-energy route;
|
||||
- and the stable root-only formula that fixed it became both a proof tool and the canonical way to evaluate everything.
|
||||
|
||||
The formal theorem depends on none of these numbers. The path to the theorem depended on almost nothing else — on code explicit enough to be reimplemented by some AI system who (rightfully so) did not trust me, and hostile enough to keep trying to break whatever conjecture I was currently in love with.
|
||||
|
||||
That is the role I want this page to preserve, and it is the reason it exists apart from the preprint at all. The paper records what turned out to be true. This page records how I found out which statements were even worth trying to prove — which is the part no theorem-and-proof ever shows you, and, if I am honest, the part I would most have wanted to read.
|
||||
|
|
@ -0,0 +1,90 @@
|
|||
<svg viewBox="0 0 280 280" xmlns="http://www.w3.org/2000/svg" role="img" aria-labelledby="mark-title-levcs">
|
||||
<title id="mark-title-levcs">A federated commit graph: multiple peer roots connected by signed edges, with no privileged origin.</title>
|
||||
<desc>Frontmatter mark for the essay "LeVCS: A Distributed Version Control System".</desc>
|
||||
|
||||
<g fill="none" stroke="currentColor" stroke-linecap="round" stroke-linejoin="round">
|
||||
|
||||
<!-- Outer roundel -->
|
||||
<circle cx="140" cy="140" r="128" stroke-width="0.6"/>
|
||||
|
||||
<!-- Three peer roots, equally weighted, at 90, 210, 330 degrees on r=78.
|
||||
Each is the source of its own commit chain that intersects with the others.
|
||||
Identity sigil (small filled square) inside each root marks
|
||||
"identity in the protocol". -->
|
||||
|
||||
<!-- Root A: top -->
|
||||
<circle cx="140" cy="62" r="9" stroke-width="1.2"/>
|
||||
<rect x="136.5" y="58.5" width="7" height="7" fill="currentColor" stroke="none"/>
|
||||
|
||||
<!-- Root B: lower-left -->
|
||||
<circle cx="72.4" cy="179" r="9" stroke-width="1.2"/>
|
||||
<rect x="68.9" y="175.5" width="7" height="7" fill="currentColor" stroke="none"/>
|
||||
|
||||
<!-- Root C: lower-right -->
|
||||
<circle cx="207.6" cy="179" r="9" stroke-width="1.2"/>
|
||||
<rect x="204.1" y="175.5" width="7" height="7" fill="currentColor" stroke="none"/>
|
||||
|
||||
<!-- Commits along each chain. Each root spawns a chain of 3 commits walking
|
||||
toward the center along its inward radial. Commits are small open circles. -->
|
||||
|
||||
<!-- Chain A: from top root walking down toward (140, 140). 3 commits. -->
|
||||
<g stroke-width="0.9">
|
||||
<line x1="140" y1="71" x2="140" y2="89"/>
|
||||
<line x1="140" y1="98" x2="140" y2="116"/>
|
||||
<line x1="140" y1="125" x2="140" y2="131"/>
|
||||
</g>
|
||||
<circle cx="140" cy="93.5" r="3.4" stroke-width="0.9"/>
|
||||
<circle cx="140" cy="120.5" r="3.4" stroke-width="0.9"/>
|
||||
|
||||
<!-- Chain B: from lower-left root, walking toward center along the
|
||||
radial. Direction vector = (1, -1)/sqrt(2). Step ~13.5 along the line.
|
||||
B is at (72.4, 179); center at (140, 140); distance ~78. Step inward. -->
|
||||
<g stroke-width="0.9">
|
||||
<line x1="79.8" y1="174.4" x2="95.4" y2="165.5"/>
|
||||
<line x1="103" y1="160.6" x2="118.6" y2="151.7"/>
|
||||
<line x1="126.2" y1="146.8" x2="131.8" y2="143.6"/>
|
||||
</g>
|
||||
<circle cx="99.2" cy="163.05" r="3.4" stroke-width="0.9"/>
|
||||
<circle cx="122.4" cy="149.25" r="3.4" stroke-width="0.9"/>
|
||||
|
||||
<!-- Chain C: from lower-right root, walking toward center.
|
||||
C at (207.6, 179); direction = (-1, -1)/sqrt(2). -->
|
||||
<g stroke-width="0.9">
|
||||
<line x1="200.2" y1="174.4" x2="184.6" y2="165.5"/>
|
||||
<line x1="177" y1="160.6" x2="161.4" y2="151.7"/>
|
||||
<line x1="153.8" y1="146.8" x2="148.2" y2="143.6"/>
|
||||
</g>
|
||||
<circle cx="180.8" cy="163.05" r="3.4" stroke-width="0.9"/>
|
||||
<circle cx="157.6" cy="149.25" r="3.4" stroke-width="0.9"/>
|
||||
|
||||
<!-- Confluence point: a small open square at the geometric center where the
|
||||
three chains meet. This is the merge: cascading, format-aware, but does
|
||||
not collapse history — the chains remain distinct above. -->
|
||||
<rect x="135.5" y="135.5" width="9" height="9" stroke-width="1.2"/>
|
||||
|
||||
<!-- Cross-edges: peer-to-peer references between the three chains.
|
||||
These are the federation edges — direct refs between commits on different
|
||||
chains, not routed through any hub. Drawn as dashed thin lines. -->
|
||||
<g stroke-width="0.5" stroke-dasharray="2 3" opacity="0.75">
|
||||
<!-- A's first commit ↔ C's first commit -->
|
||||
<path d="M 143 92 Q 168 110 178 161"/>
|
||||
<!-- A's first commit ↔ B's first commit -->
|
||||
<path d="M 137 92 Q 112 110 102 161"/>
|
||||
<!-- B's second ↔ C's second -->
|
||||
<path d="M 122 149 L 158 149" stroke-dasharray="2 3"/>
|
||||
</g>
|
||||
|
||||
<!-- Signature ticks: short hash marks on each chain's middle-segment,
|
||||
indicating signed history (BLAKE3). One small tick perpendicular to each
|
||||
chain segment. -->
|
||||
<g stroke-width="0.7">
|
||||
<!-- Chain A tick: between commits 1 and 2 -->
|
||||
<line x1="146" y1="107" x2="150" y2="107"/>
|
||||
<!-- Chain B tick -->
|
||||
<line x1="111.4" y1="160.0" x2="113.8" y2="156.8"/>
|
||||
<!-- Chain C tick -->
|
||||
<line x1="168.6" y1="160.0" x2="166.2" y2="156.8"/>
|
||||
</g>
|
||||
|
||||
</g>
|
||||
</svg>
|
||||
|
After Width: | Height: | Size: 4.2 KiB |
|
|
@ -0,0 +1,365 @@
|
|||
---
|
||||
title: "LeVCS: A Distributed Version Control System"
|
||||
date: 2026-05-01
|
||||
abstract: >
|
||||
LeVCS is a distributed version control system in the lineage of git, fossil, pijul, and sapling — content-addressed
|
||||
objects, signed history, three-way merge — with five things deliberately rebuilt: identity in the protocol, federation
|
||||
as a first-class concept, a cascading merge engine that dispatches per-file to format-aware and tree-sitter handlers,
|
||||
BLAKE3 hashing throughout, and releases as signed artifacts rather than mutable name pointers. v0.1.0 ships the
|
||||
protocol substrate; the workflow surface — review, issues, web UI — is the next layer up.
|
||||
tags: [systems, projects, releases]
|
||||
status: "Working model"
|
||||
confidence: 85
|
||||
importance: 3
|
||||
evidence: 5
|
||||
scope: broad
|
||||
novelty: innovative
|
||||
practicality: moderate
|
||||
---
|
||||
|
||||
Naming a version control system after yourself is a mild hubris. Linus has been candid that this is exactly what he did with git — *"I'm an egotistical bastard, and I name all my projects after myself. First 'Linux', now 'git'."* LeVCS — Levi's [VCS]{.smallcaps} — admits the same joke up front and gets on with it. The work itself is the substrate: an object model, a federation [API]{.smallcaps}, a merge engine, and a small instance server, written in Rust, sized to fit on a [VPS]{.smallcaps} and audit-able by one person in a weekend.
|
||||
|
||||
The first instance will be at [levcs.levineuwirth.org](https://levcs.levineuwirth.org); the source lives at [git.levineuwirth.org/neuwirth/levcs](https://git.levineuwirth.org/neuwirth/levcs) and is mirrored by the same git infrastructure that hosts everything else. v0.1.0 is what is described here. The workflow surface — the [PR]{.smallcaps} review object, issue tracking, web [UI]{.smallcaps}, [CI]{.smallcaps} conventions — is intentionally not part of v0.1.0 and is the next document in the series.
|
||||
|
||||
This page describes what LeVCS is, why it diverges from git on five specific axes, and how to use and operate it today.
|
||||
|
||||
---
|
||||
|
||||
## What It Is
|
||||
|
||||
LeVCS is a distributed version control system. It uses the same conceptual primitives as every other modern [DVCS]{.smallcaps}: content-addressed objects, a directed-acyclic-graph history, signed commits, three-way merge. The substrate is small — about ten Rust crates, ~194 passing tests at v0.1.0, full `cargo test` under a minute on a laptop.
|
||||
|
||||
The short version of what comes with it:
|
||||
|
||||
- **Content addressing with [BLAKE3]{.smallcaps}** — 32-byte object identifiers everywhere, tree-hashed, ~5 [GiB]{.smallcaps}/s on a laptop. No [SHA-1]{.smallcaps} transition story to live through.
|
||||
- **Signed authority chain** — repository membership is a first-class object with explicit roles (Reader / Contributor / Maintainer / Owner), versioned, signed with [Ed25519]{.smallcaps}, chained by predecessor. Push authorization is protocol-level, not server policy.
|
||||
- **Federation as the normal mode** — every repository has a global `repo_id` (the [BLAKE3]{.smallcaps} of its genesis authority), and instances mirror each other in three storage modes: *full* (everything reachable), *release* (only releases and their trees), *metadata* (authority chain and ref headers, no content).
|
||||
- **Cascading merge engine** — per-file dispatch to a handler ranked by aggressiveness: textual fallback, format-aware ([JSON]{.smallcaps} / [YAML]{.smallcaps} / [TOML]{.smallcaps} / [XML]{.smallcaps} / Markdown / prose), tree-sitter for source code (Rust, Python, [JS]{.smallcaps}/[TS]{.smallcaps}, Go, C/[C++]{.smallcaps}, Java, Ruby, Bash), and wasm-sandboxed plugins for the long tail. Each merged file produces a `FileRecord` in `.levcs/merge-record`, signed with the resulting commit.
|
||||
- **Releases as signed objects** — not mutable name pointers. Each release carries the released tree, the predecessor commit, the parent release in the chain, the authority hash at release time, the declarer's public key, and signed release notes.
|
||||
- **A small federation server** — a single binary, `levcs-instance`, fronted by Caddy or nginx, with a protocol surface of ten endpoints. Storage is a directory tree; a consistent backup is an `rsync` of `/var/lib/levcs`.
|
||||
- **A reproducible benchmark suite** — `scripts/bench.sh` with metadata capture (rustc version, kernel, [CPU]{.smallcaps}, git rev), parsed summaries, and optional flamegraphs.
|
||||
|
||||
The dependency list is short: a recent stable Rust toolchain (workspace [MSRV]{.smallcaps} is 1.75) and a [C]{.smallcaps} compiler for the tree-sitter grammars. No database server, no message broker, no external service.
|
||||
|
||||
---
|
||||
|
||||
## Why a New VCS?
|
||||
|
||||
Git is the dominant [DVCS]{.smallcaps}, and there is no good case for replacing it on the strength of taste alone. The case for replacing it rests on five specific places where its 2005 design has aged poorly enough that bolt-on solutions have stopped paying their freight:
|
||||
|
||||
- **Identity is no longer optional.** Many projects need to know not just *who claims to have authored a commit* but who is *authorized to alter the repository's history*. Signed commits are an opt-in (`gpg-sign` since 2014, `ssh-sign` since 2021), but even when signed they answer "did *some* key sign this?" — not "is the signer authorized to write to this repo right now?"^[The right question — *is this writer in the current authority — turns into a hosting-platform toggle in practice. GitHub branch-protection rules are the de-facto authority chain for most projects, and the chain doesn't travel with the repository.]
|
||||
- **Replication is more complex than push/pull.** Mirrors, archives, cold-storage replicas, and read-only forks are all common; git treats them with the same primitives as the source-of-truth remote. There is no first-class concept of *kinds* of mirror, and no protocol-level enforcement that a mirror remains consistent with what it is mirroring.
|
||||
- **Merge conflicts are still mostly resolved at the line level.** [JSON]{.smallcaps}, [YAML]{.smallcaps}, [TOML]{.smallcaps}, source code with semantic structure — the line-diff treatment is wrong for all of these and produces false conflicts on reformats every team has hit. Custom merge drivers exist (`gitattributes`) but are awkward, single-purpose, and don't compose.
|
||||
- **[SHA-1]{.smallcaps} is broken.** [SHAttered]{.smallcaps} (2017) was a practical collision. Git's [SHA-256]{.smallcaps} transition has been "in progress" for most of a decade and is unlikely to ever finish for the long tail of git infrastructure.
|
||||
- **Tags are names, not artifacts.** A git tag is a string ref that points to a commit (or, if you remember `-a`, to a tag object). Either way, releases are conventions sitting on top of name pointers — not first-class signed objects with a chain you can audit.
|
||||
|
||||
LeVCS is an attempt at a clean restart that takes the [DAG]{.smallcaps} model and content addressing as obvious wins, and rebuilds identity, federation, merging, hashing, and releases as **protocol-level concerns** rather than conventions or sidecar tools.
|
||||
|
||||
---
|
||||
|
||||
## The Shape of the System
|
||||
|
||||
LeVCS is layered. Each layer has a clean interface to the one below it:
|
||||
|
||||
```
|
||||
┌───────────────────────────────────────────────────────────┐
|
||||
│ Workflow tools (TBD: review, issues, web UI) │
|
||||
├───────────────────────────────────────────────────────────┤
|
||||
│ CLI: `levcs init / commit / push / merge / release` │
|
||||
├───────────────────────────────────────────────────────────┤
|
||||
│ Federation HTTP API (instances, mirrors, releases) │
|
||||
├───────────────────────────────────────────────────────────┤
|
||||
│ Object model: Blob / Tree / Commit / Release / Authority │
|
||||
│ Merge engine: textual → format-aware → tree-sitter │
|
||||
│ Trust root: signed authority chain (Ed25519) │
|
||||
│ Content addressing: BLAKE3 │
|
||||
└───────────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
Five object kinds, all content-addressed by their [BLAKE3]{.smallcaps} digest:
|
||||
|
||||
- **Blob** — raw file contents.
|
||||
- **Tree** — `(name, type, mode, hash)` entries; sorted, no duplicates.
|
||||
- **Commit** — tree + parents + authority + author key + message, signed.
|
||||
- **Release** — first-class artifact: tree + predecessor commit + parent release + label + notes, signed by a maintainer or owner.
|
||||
- **Authority** — the *membership document* for a repository: who has what role, signed and chained by predecessor.
|
||||
|
||||
A repository is the set of these objects plus a `refs/` map (`branches/*`, `releases/*`, `authority/{genesis,current}`) indexing into them. The `repo_id` is the [BLAKE3]{.smallcaps} of the genesis authority — globally unique by construction, no central registrar needed.
|
||||
|
||||
---
|
||||
|
||||
## What's Different from Git
|
||||
|
||||
| Axis | git | LeVCS |
|
||||
|:-----|:----|:------|
|
||||
| Hash | [SHA-1]{.smallcaps} (deprecated, transitioning) | [BLAKE3]{.smallcaps} |
|
||||
| Identity | Author string in commit | Signed authority object with explicit roles |
|
||||
| Push authorization | Server-side hook or hosting platform | Protocol-level role check |
|
||||
| Force-push rule | Server policy (off-protocol) | Protocol enforces maintainer-or-owner role |
|
||||
| Federation | [URL]{.smallcaps}-bound remotes | Global `repo_id` + replicating instances |
|
||||
| Mirror replication | `git fetch --mirror` (best-effort) | First-class with three storage modes |
|
||||
| Tags / releases | Mutable string refs (often) | Signed objects with predecessor + parent-release chain |
|
||||
| Merge granularity | Line-level (myers / patience) | Cascade: textual → format → tree-sitter → plugin |
|
||||
| Merge audit | No artifact | `.levcs/merge-record` [TOML]{.smallcaps}, signed with the commit |
|
||||
| Web [UI]{.smallcaps} / issues | Hosting platform | Out of scope for v1 |
|
||||
|
||||
The rest of this section unpacks each axis worth unpacking.
|
||||
|
||||
### Identity in the Protocol, Not on Top
|
||||
|
||||
Git stores `Author: Name <email>` and `Committer: Name <email>` strings in commits. There is nothing cryptographic about either. Even signed commits answer the wrong question — *is some key behind this signature?* — instead of the right one: *is this signer currently authorized to write to this repository?*
|
||||
|
||||
LeVCS makes membership a first-class object. An **authority body** has:
|
||||
|
||||
```
|
||||
schema_version repo_id previous_authority version created_micros
|
||||
members: [(public_key, handle, role, added_micros, added_by), ...]
|
||||
policy: [(key, value), ...]
|
||||
```
|
||||
|
||||
Roles form a strict order: `Reader < Contributor < Maintainer < Owner`. Every commit references the authority hash that was current when it was signed. Updating membership is a versioned operation: you write a new authority object, signed by an Owner, with `previous_authority` pointing at the prior one. The instance walks the chain on push and rejects any push whose author key isn't a current member.
|
||||
|
||||
The practical consequence is that *"give Bob push access"* is not a hosting-platform toggle. It is a signed authority update that travels in the repository and is auditable for the lifetime of the project.^[This is the design choice I care most about. The alternative — that authorization is a dashboard somewhere — means the repository is not actually self-describing. You can't tell, from the repository alone, who could have written this history. With a chained authority object you can.]
|
||||
|
||||
### Federation, Not "Remotes"
|
||||
|
||||
A git remote is a [URL]{.smallcaps} plus some credentials. There is no fact-of-the-matter about whether two URLs refer to the *same* repository — git checks by walking commits, but "same project" is a convention.
|
||||
|
||||
LeVCS has a **global `repo_id`** — the [BLAKE3]{.smallcaps} of the genesis authority object, so two clones of the same project have the same `repo_id` even if they live on instances on opposite continents. An instance is a federation peer: it serves `/levcs/v1/repos/<repo_id>/...` endpoints and replicates state from other instances when configured to. Mirroring is the protocol's normal mode, not a `git fetch --mirror` cron job.
|
||||
|
||||
This composes with three **storage modes**:
|
||||
|
||||
- **Full** — every reachable object. The source-of-truth instance.
|
||||
- **Release** — only release objects, their reachable trees and blobs, and the authority chain. Skips inter-release commits. For long-lived archive replicas.
|
||||
- **Metadata** — authority objects, release headers, signed refs only. No content. For "is this project still alive?" pings.
|
||||
|
||||
The instance enforces these on push. A release-mode replica refuses pushes that update branches; a metadata-mode replica refuses all pushes (it is populated entirely by mirroring). A migrating maintainer can move the source-of-truth role from one instance to another with `levcs migrate`, replaying the full history at the destination — the `repo_id` is unchanged, because the genesis authority is unchanged.
|
||||
|
||||
### The Merge Cascade
|
||||
|
||||
This is the technical centerpiece.
|
||||
|
||||
A traditional three-way merge — git, mercurial, fossil — works at the line level. It is correct for prose and acceptable for code, but it generates false conflicts on reformats (linters, prettifiers, whitespace-policy bumps), key reorderings in [JSON]{.smallcaps} / [YAML]{.smallcaps} / [TOML]{.smallcaps}, imports lists in source files that two branches both edited, and Markdown files where two contributors modified disjoint sections of the same paragraph.
|
||||
|
||||
LeVCS dispatches per-file to a **handler cascade** ranked by aggressiveness:
|
||||
|
||||
```
|
||||
rank 0 textual universal line-level fallback
|
||||
rank 1 format-aware json | yaml | toml | xml | markdown | prose
|
||||
rank 2 tree-sitter rust | python | js | ts | go | c | cpp |
|
||||
java | ruby | bash
|
||||
rank 3 plugin wasm-sandboxed, user-supplied
|
||||
```
|
||||
|
||||
A repository's `.levcs/merge.toml` maps glob patterns to handlers. Per-user `.levcs/merge.local.toml` can **demote** but never promote, so a distrusted plugin can be locally turned off without a repo edit. Each merged file produces a `FileRecord` in `.levcs/merge-record` listing the handler used and its hash; the merge-record blob is committed alongside the resolved tree, so every merge in history is auditable.
|
||||
|
||||
Two examples illustrate the practical difference:
|
||||
|
||||
**Format-aware example.** `package.json` where Alice adds a dependency at the top of `dependencies` and Bob adds one at the bottom. Git produces a conflict because the lines are adjacent. The [JSON]{.smallcaps} handler parses both sides, computes the structural diff, and merges them — both new entries appear in the output, no conflict.
|
||||
|
||||
**Tree-sitter example.** Two contributors add unrelated `use` statements to a Rust file. Line diff conflicts. The tree-sitter handler treats the `use_declaration` list as an ordered set, merges both additions, no conflict.
|
||||
|
||||
The cascade is fail-safe. A tree-sitter handler that bails on a syntax error falls through to the format-aware handler if applicable, then to textual. The textual handler always merges — it might produce conflicts, but it never fails to produce *some* output. This matters for [CI]{.smallcaps} and for automated mirror sync: there is no merge that the engine simply refuses to attempt.
|
||||
|
||||
### Hashing
|
||||
|
||||
Git uses [SHA-1]{.smallcaps}. [SHAttered]{.smallcaps} (2017) was a practical collision, and the [SHA-256]{.smallcaps} transition is incomplete in 2026. LeVCS uses [BLAKE3]{.smallcaps} from day one — faster than [SHA-256]{.smallcaps} in practice (~5 [GiB]{.smallcaps}/s on a laptop for blob serialize-plus-hash), tree-hashed, no commitment to a specific length-tag convention. Object [ID]{.smallcaps}s are 32 bytes everywhere, with no migration story to live through.
|
||||
|
||||
### Releases as Objects
|
||||
|
||||
Git tags are refs that point to commits — or to tag objects, if you remember to use `-a`. Either way, they are *names*, not artifacts. A release in LeVCS is a signed object:
|
||||
|
||||
```
|
||||
tree commit's root tree
|
||||
predecessor commit being released
|
||||
parent_release prior release in the chain (or zero)
|
||||
authority authority hash at release time
|
||||
declarer_key public key of the signing maintainer/owner
|
||||
timestamp Unix micros
|
||||
label "v1.0.0" or similar
|
||||
notes release notes (UTF-8, up to 4 GiB)
|
||||
```
|
||||
|
||||
The chain `parent_release → parent_release → ...` gives a clean release history independent of branch topology. The replica modes above can replicate just releases (and their trees and authority) for archive instances that don't need the inter-release commit history — a useful primitive for long-tail preservation.
|
||||
|
||||
---
|
||||
|
||||
## How You Use It
|
||||
|
||||
### Bootstrap
|
||||
|
||||
```sh
|
||||
levcs key generate --label primary
|
||||
levcs init --key primary
|
||||
levcs track --all
|
||||
levcs commit -m "initial import"
|
||||
```
|
||||
|
||||
After `init`, `.levcs/` exists alongside the working tree. The genesis authority names the chosen key as the sole Owner; the `repo_id` is fixed forever. After `commit`, the repository has one commit on `refs/branches/main`.
|
||||
|
||||
### Branch and Merge
|
||||
|
||||
```sh
|
||||
levcs branch feature/x
|
||||
# ... edit files ...
|
||||
levcs commit -m "wip on x"
|
||||
levcs branch main
|
||||
levcs merge feature/x
|
||||
```
|
||||
|
||||
If the merge produces conflicts, drop into the resolution [TUI]{.smallcaps}:
|
||||
|
||||
```sh
|
||||
levcs merge --resolve
|
||||
```
|
||||
|
||||
The [TUI]{.smallcaps} shows each conflicted file with the ours/base/theirs panes the handler emitted, plus the cascade decision (which handler ran, and why it fell through if it did). On accept, it writes the resolved file and a signed `.levcs/merge-record` entry.
|
||||
|
||||
### Release
|
||||
|
||||
```sh
|
||||
levcs release v1.0.0 --notes "first release"
|
||||
```
|
||||
|
||||
Writes a Release object with the current commit as `predecessor`, signs it with the active key, and adds `refs/releases/v1.0.0`. If prior releases exist, `parent_release` chains to the most recent one automatically.
|
||||
|
||||
### Federation
|
||||
|
||||
```sh
|
||||
levcs instance --set https://levcs.levineuwirth.org/levcs/v1
|
||||
levcs push refs/branches/main
|
||||
```
|
||||
|
||||
The first push to a fresh instance auto-inits the repository using the genesis authority. Subsequent pushes are role-checked. Pulls are public-read by default (the `public_read` policy bit on the genesis authority).
|
||||
|
||||
To migrate to a new home:
|
||||
|
||||
```sh
|
||||
levcs migrate https://new-host.example.com/levcs/v1 --set-active
|
||||
```
|
||||
|
||||
`migrate` re-inits and replays the full history at the destination, then points the local repository at it. The `repo_id` is unchanged — same project, new location.
|
||||
|
||||
---
|
||||
|
||||
## Operating an Instance
|
||||
|
||||
A single binary, `levcs-instance`, reads a [TOML]{.smallcaps} config and listens on [HTTP]{.smallcaps}. Production deployments terminate [TLS]{.smallcaps} at a reverse proxy; the instance binds to localhost. The full walkthrough — systemd unit, Caddy and nginx examples, firewall, laptop-side bootstrap — lives in `deploy/README.md` in the repository.
|
||||
|
||||
The protocol surface is small:
|
||||
|
||||
```
|
||||
GET /health
|
||||
GET /levcs/v1/instance/info
|
||||
GET /levcs/v1/instance/peers
|
||||
GET /levcs/v1/repos/<repo_id>/info
|
||||
GET /levcs/v1/repos/<repo_id>/refs
|
||||
GET /levcs/v1/repos/<repo_id>/objects/<hash>
|
||||
GET /levcs/v1/repos/<repo_id>/pack?have=...&want=...
|
||||
POST /levcs/v1/repos/<repo_id>/init
|
||||
POST /levcs/v1/repos/<repo_id>/push
|
||||
```
|
||||
|
||||
That is the whole [API]{.smallcaps}. No admin endpoints, no users-and-passwords table, no web [UI]{.smallcaps} to firewall. POSTs require a signed `LeVCS-Signature` header ([Ed25519]{.smallcaps}-over-canonical-request, with timestamp and nonce for replay protection); GETs are public unless the genesis authority's policy turned that off.
|
||||
|
||||
Storage is a directory tree. Per-object atomic writes via temp-then-rename, per-repository serializing mutex on push. A consistent backup is just a snapshot of `/var/lib/levcs`. The first instance — `levcs.levineuwirth.org` — is configured exactly this way, fronted by Caddy on a small [VPS]{.smallcaps}, dogfooding the federation surface against the source-of-truth Forgejo at [git.levineuwirth.org](https://git.levineuwirth.org).
|
||||
|
||||
---
|
||||
|
||||
## What LeVCS Isn't (Yet)
|
||||
|
||||
The honest list of things you would want for a full project home that LeVCS does not provide:
|
||||
|
||||
- **Code review.** No [PR]{.smallcaps} object, no review threads, no comments. The workflow spec coming next defines these.
|
||||
- **Issue tracking.** Same — protocol substrate doesn't cover it.
|
||||
- **[CI]{.smallcaps} integration.** No webhooks. [CI]{.smallcaps} systems would need to poll `/refs` on a cadence, which works but isn't turnkey.
|
||||
- **Web [UI]{.smallcaps}.** No branch browser, no diff view, no blame. These can be built atop the existing [GET]{.smallcaps} endpoints; nothing in the protocol is hostile to a [UI]{.smallcaps}, but none ship.
|
||||
- **Search.** No `git grep` equivalent on the server side. Local-only.
|
||||
- **Submodules / monorepo tooling.** No analog yet.
|
||||
|
||||
If a use case requires any of the above today, the right pattern is to run LeVCS *parallel* to an existing platform. Forgejo, GitHub, or Gitea continues to host the workflow; the LeVCS instance acts as a dogfood replica that gets the same commits via a `push-both` wrapper. When the workflow surface lands, the migration story flips. This is how `levcs.levineuwirth.org` will be operated for the foreseeable future.
|
||||
|
||||
---
|
||||
|
||||
## What Is True Today (and How We Know)
|
||||
|
||||
The repository at v0.1.0 has 194 passing tests covering:
|
||||
|
||||
- The full §2–§7 object model and protocol surface.
|
||||
- A 14-scenario merge conformance corpus, eight of which are git-false-conflict cases the cascade resolves cleanly.
|
||||
- Property tests on the pack codec and object parsers (fuzz plus structured proptest round-trip).
|
||||
- An end-to-end "dogfood" integration test that stands up three instances (source-of-truth, peer, mirror), pushes a chain of commits plus a release, replicates via mirror sync, migrates to the peer, and asserts byte-for-byte object equality across all three.
|
||||
|
||||
A baseline microbenchmark suite lives in `scripts/bench.sh` with metadata capture (rustc version, kernel, [CPU]{.smallcaps}, git rev) for run-to-run comparison. On a Ryzen 7 laptop, headline numbers:
|
||||
|
||||
- **Pack decode** of a 10 × 1 [MiB]{.smallcaps} pack: ~2.3 ms (4.3 [GiB]{.smallcaps}/s).
|
||||
- **[BLAKE3]{.smallcaps} + serialize** on 1 [MiB]{.smallcaps} blobs: ~190 µs (5.1 [GiB]{.smallcaps}/s).
|
||||
- **Textual three-way merge** of a 100 [KiB]{.smallcaps} document: ~4.6 ms (~80 [MiB]{.smallcaps}/s).
|
||||
- **Pack encode** is the throughput floor at ~380 [MiB]{.smallcaps}/s — bottlenecked by zstd level 3 on incompressible data.
|
||||
|
||||
Numbers are reproducible via `scripts/bench.sh --quick`.
|
||||
|
||||
---
|
||||
|
||||
## The Roadmap
|
||||
|
||||
The immediate priorities, in order:
|
||||
|
||||
1. **Workflow spec** — the missing layer above. [PR]{.smallcaps}/review object, discussion threads, [CI]{.smallcaps} hook conventions, web [UI]{.smallcaps} design. This is the document the rest of v1 builds toward.
|
||||
2. **Reference workflow tools** — a minimal web [UI]{.smallcaps} that reads the federation [API]{.smallcaps} and lets you browse, review, and merge. Probably a separate repository and process, not bundled into the instance binary.
|
||||
3. **[CI]{.smallcaps} conventions** — a published webhook protocol so existing [CI]{.smallcaps} systems can integrate without polling.
|
||||
4. **Plugin handler examples** — a few real wasm handlers (e.g. protobuf, [SQL]{.smallcaps} migrations) to validate the plugin protocol against real formats.
|
||||
5. **Git import** — a one-way import path so existing projects can adopt LeVCS without hand-replaying history.
|
||||
|
||||
The substrate guarantees the workflow layer can lean on:
|
||||
|
||||
- Signed objects with a verifiable authority chain.
|
||||
- Per-file merge records that travel with each commit.
|
||||
- A content-addressed object store that doesn't care what kind of content it stores.
|
||||
- Federation as a normal operating mode rather than a special case.
|
||||
|
||||
A "[PR]{.smallcaps}" is just an object kind LeVCS doesn't have yet; an "issue" is another; the storage modes already define how a [CI]{.smallcaps} system would replicate the metadata it needs without pulling source.
|
||||
|
||||
---
|
||||
|
||||
## Trying It
|
||||
|
||||
Build:
|
||||
|
||||
```sh
|
||||
git clone https://git.levineuwirth.org/neuwirth/levcs
|
||||
cd levcs
|
||||
cargo build --release
|
||||
sudo install -m 0755 \
|
||||
target/release/levcs target/release/levcs-instance \
|
||||
/usr/local/bin/
|
||||
```
|
||||
|
||||
Local single-machine tour:
|
||||
|
||||
```sh
|
||||
levcs key generate --label me
|
||||
mkdir /tmp/demo && cd /tmp/demo
|
||||
echo "hello" > a.txt
|
||||
levcs init --key me
|
||||
levcs track --all
|
||||
levcs commit -m "first"
|
||||
levcs log
|
||||
```
|
||||
|
||||
Push to the public instance once it lands at `levcs.levineuwirth.org`:
|
||||
|
||||
```sh
|
||||
levcs instance --set https://levcs.levineuwirth.org/levcs/v1
|
||||
levcs push refs/branches/main
|
||||
```
|
||||
|
||||
Read the technical report in the repository at `doc/technical-report.md`. Read the code: every crate is small and documented. `crates/levcs-core` is the object model, `crates/levcs-merge` is the cascade, `crates/levcs-instance` is the server, `crates/levcs-cli` is the user-facing tool.
|
||||
|
||||
---
|
||||
|
||||
## License and Repository
|
||||
|
||||
The code is released under the [Apache License 2.0]{.smallcaps} — see `LICENSE` in the repository for the full text. The choice is deliberate: the patent grant and the explicit contributor license are worth the slight ceremony for a substrate other people may build on. Frameworks should not take a stake in the work they compile, but they should be unambiguous about what compiling against them does and doesn't permit.
|
||||
|
||||
The repository is at [git.levineuwirth.org/neuwirth/levcs](https://git.levineuwirth.org/neuwirth/levcs). The first federation instance will be at [levcs.levineuwirth.org](https://levcs.levineuwirth.org). The next document in the series is the workflow spec; until it lands, comments and corrections on the substrate itself are welcome, addressable to your's truly!
|
||||
|
|
@ -0,0 +1,61 @@
|
|||
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 280 280" role="img" aria-labelledby="mark-title-near-critical">
|
||||
<title id="mark-title-near-critical">A radius-2 ball in the 3-regular tree with a soft circumscribing arc, above a horizontal threshold ruler marking the near-critical whisker between log B_h and log B_h minus 2 log log B_h minus W_h</title>
|
||||
<desc>A frontispiece mark for "Near-Critical First-Moment Lower Bounds for Growing-Radius Domination in Random Regular Graphs." The central figure is a radius-2 ball in the 3-regular tree — root at the bottom, three children at the first level, six grandchildren at the second level (B_2 = 10 for d=3), enclosed by a soft circumscribing arc. Above the ball, a horizontal ruler runs from left to right with two calibrated tick marks: the leftmost tick at the theorem's reachable coordinate log B_h - 2 log log B_h - W_h, and the rightmost tick at the predicted critical coordinate log B_h. Between them, a small horizontal bracket marks the diverging whisker W_h — the bounded critical window the theorem cannot close. The mark says: here is the ball, here is where the theorem stops, and here is what remains open.</desc>
|
||||
|
||||
<circle cx="140" cy="140" r="128" stroke="currentColor" stroke-width="0.6" fill="none"/>
|
||||
|
||||
<g stroke="currentColor" fill="none" stroke-linecap="round" stroke-linejoin="round">
|
||||
<line x1="140" y1="196" x2="102" y2="176" stroke-width="1.0"/>
|
||||
<line x1="140" y1="196" x2="140" y2="170" stroke-width="1.0"/>
|
||||
<line x1="140" y1="196" x2="178" y2="176" stroke-width="1.0"/>
|
||||
|
||||
<line x1="102" y1="176" x2="82" y2="150" stroke-width="0.8"/>
|
||||
<line x1="102" y1="176" x2="112" y2="146" stroke-width="0.8"/>
|
||||
<line x1="140" y1="170" x2="126" y2="140" stroke-width="0.8"/>
|
||||
<line x1="140" y1="170" x2="154" y2="140" stroke-width="0.8"/>
|
||||
<line x1="178" y1="176" x2="168" y2="146" stroke-width="0.8"/>
|
||||
<line x1="178" y1="176" x2="198" y2="150" stroke-width="0.8"/>
|
||||
</g>
|
||||
|
||||
<g fill="currentColor" stroke="none">
|
||||
<circle cx="140" cy="196" r="2.4"/>
|
||||
|
||||
<circle cx="102" cy="176" r="1.7"/>
|
||||
<circle cx="140" cy="170" r="1.7"/>
|
||||
<circle cx="178" cy="176" r="1.7"/>
|
||||
|
||||
<circle cx="82" cy="150" r="1.3"/>
|
||||
<circle cx="112" cy="146" r="1.3"/>
|
||||
<circle cx="126" cy="140" r="1.3"/>
|
||||
<circle cx="154" cy="140" r="1.3"/>
|
||||
<circle cx="168" cy="146" r="1.3"/>
|
||||
<circle cx="198" cy="150" r="1.3"/>
|
||||
</g>
|
||||
|
||||
<path
|
||||
d="M 70 152 Q 70 128 90 118 Q 120 108 140 108 Q 160 108 190 118 Q 210 128 210 152"
|
||||
stroke="currentColor" stroke-width="0.5" fill="none" stroke-linecap="round"
|
||||
stroke-dasharray="3 2" opacity="0.55"/>
|
||||
|
||||
<line x1="52" y1="72" x2="228" y2="72" stroke="currentColor" stroke-width="0.9" stroke-linecap="round"/>
|
||||
|
||||
<g stroke="currentColor" fill="none" stroke-linecap="round" stroke-width="0.9">
|
||||
<line x1="102" y1="66" x2="102" y2="78"/>
|
||||
<line x1="212" y1="66" x2="212" y2="78"/>
|
||||
</g>
|
||||
|
||||
<circle cx="102" cy="72" r="2.0" fill="currentColor" stroke="none"/>
|
||||
<circle cx="212" cy="72" r="2.0" stroke="currentColor" stroke-width="0.9" fill="none"/>
|
||||
|
||||
<g stroke="currentColor" fill="none" stroke-linecap="round" stroke-width="0.7">
|
||||
<line x1="102" y1="52" x2="212" y2="52"/>
|
||||
<line x1="102" y1="49" x2="102" y2="55"/>
|
||||
<line x1="212" y1="49" x2="212" y2="55"/>
|
||||
</g>
|
||||
|
||||
<g stroke="currentColor" fill="none" stroke-linecap="round" stroke-width="0.5" opacity="0.7">
|
||||
<line x1="140" y1="49" x2="140" y2="55"/>
|
||||
<line x1="176" y1="49" x2="176" y2="55"/>
|
||||
</g>
|
||||
|
||||
</svg>
|
||||
|
After Width: | Height: | Size: 3.5 KiB |
File diff suppressed because it is too large
Load Diff
|
|
@ -0,0 +1,18 @@
|
|||
---
|
||||
title: "Networking Stack from Scratch"
|
||||
date: 2026-04-21
|
||||
abstract: >
|
||||
TCP/IP, RIP, UDP, and DNS implementations in Go, supporting file transmission of up to 1 GB across networks of up to 8 virtual machines. Extended with a fully RFC-compliant SSH implementation (2,000+ additional lines) supporting sustained sessions of arbitrary length.
|
||||
tags:
|
||||
- tech
|
||||
authors:
|
||||
- "Levi Neuwirth | /me.html"
|
||||
status: "Draft"
|
||||
confidence: 85
|
||||
importance: 1
|
||||
scope: personal
|
||||
novelty: conventional
|
||||
practicality: moderate
|
||||
---
|
||||
|
||||
A fuller write-up follows. In the meantime, see the [projects index](/cv/projects/).
|
||||
|
|
@ -0,0 +1,89 @@
|
|||
<svg viewBox="0 0 280 280" xmlns="http://www.w3.org/2000/svg" role="img" aria-labelledby="mark-title-networking-stack">
|
||||
<title id="mark-title-networking-stack">A four-layer protocol stack with addressing arrows passing vertically through, headers nested as concentric brackets at each layer.</title>
|
||||
<desc>Frontmatter mark for the essay "Networking Stack from Scratch".</desc>
|
||||
|
||||
<g fill="none" stroke="currentColor" stroke-linecap="round" stroke-linejoin="round">
|
||||
|
||||
<!-- Outer roundel -->
|
||||
<circle cx="140" cy="140" r="128" stroke-width="0.6"/>
|
||||
|
||||
<!-- Four protocol layers as horizontal bands, stacked.
|
||||
Width 130, centered at x=140. Vertical band spacing: 30 px.
|
||||
Layer order (top to bottom): Application, Transport, Internet, Link.
|
||||
Visual encoding: stroke-style varies per layer to distinguish
|
||||
without color and without text. -->
|
||||
|
||||
<!-- Layer 1: Application (e.g. SSH, DNS) — solid rule -->
|
||||
<line x1="75" y1="80" x2="205" y2="80" stroke-width="1.2"/>
|
||||
<line x1="75" y1="98" x2="205" y2="98" stroke-width="0.4"/>
|
||||
|
||||
<!-- Layer 2: Transport (TCP, UDP) — dashed rule -->
|
||||
<line x1="75" y1="118" x2="205" y2="118" stroke-width="1.2" stroke-dasharray="6 3"/>
|
||||
<line x1="75" y1="136" x2="205" y2="136" stroke-width="0.4"/>
|
||||
|
||||
<!-- Layer 3: Internet (IP, RIP) — densely-dashed rule -->
|
||||
<line x1="75" y1="156" x2="205" y2="156" stroke-width="1.2" stroke-dasharray="2 2"/>
|
||||
<line x1="75" y1="174" x2="205" y2="174" stroke-width="0.4"/>
|
||||
|
||||
<!-- Layer 4: Link — double rule -->
|
||||
<line x1="75" y1="194" x2="205" y2="194" stroke-width="1.2"/>
|
||||
<line x1="75" y1="198" x2="205" y2="198" stroke-width="0.6"/>
|
||||
<line x1="75" y1="212" x2="205" y2="212" stroke-width="0.4"/>
|
||||
|
||||
<!-- Headers nesting: at each layer, a small bracket on the left margin
|
||||
of the band indicates encapsulation. They get progressively wider
|
||||
as we descend (each layer wraps the one above). -->
|
||||
<g stroke-width="0.7">
|
||||
<!-- Layer 1 bracket -->
|
||||
<path d="M 88 84 L 84 84 L 84 94 L 88 94"/>
|
||||
<!-- Layer 2 bracket -->
|
||||
<path d="M 86 122 L 80 122 L 80 132 L 86 132"/>
|
||||
<!-- Layer 3 bracket -->
|
||||
<path d="M 84 160 L 76 160 L 76 170 L 84 170"/>
|
||||
<!-- Layer 4 bracket -->
|
||||
<path d="M 82 198 L 72 198 L 72 208 L 82 208"/>
|
||||
</g>
|
||||
|
||||
<!-- Address arrows: a downward flow on the left, an upward flow on the right.
|
||||
These are the data path — packet descending the stack on send,
|
||||
ascending on receive. Each is interrupted at every layer with a
|
||||
small horizontal tick (per-layer header attachment / strip). -->
|
||||
|
||||
<!-- Down arrow (left side): outbound -->
|
||||
<g stroke-width="1.0">
|
||||
<line x1="58" y1="62" x2="58" y2="226"/>
|
||||
<!-- Arrowhead at bottom -->
|
||||
<line x1="58" y1="226" x2="54" y2="220"/>
|
||||
<line x1="58" y1="226" x2="62" y2="220"/>
|
||||
<!-- Per-layer ticks crossing the arrow shaft -->
|
||||
<line x1="54" y1="89" x2="62" y2="89"/>
|
||||
<line x1="54" y1="127" x2="62" y2="127"/>
|
||||
<line x1="54" y1="165" x2="62" y2="165"/>
|
||||
<line x1="54" y1="203" x2="62" y2="203"/>
|
||||
</g>
|
||||
|
||||
<!-- Up arrow (right side): inbound -->
|
||||
<g stroke-width="1.0">
|
||||
<line x1="222" y1="226" x2="222" y2="62"/>
|
||||
<!-- Arrowhead at top -->
|
||||
<line x1="222" y1="62" x2="218" y2="68"/>
|
||||
<line x1="222" y1="62" x2="226" y2="68"/>
|
||||
<!-- Per-layer ticks -->
|
||||
<line x1="218" y1="89" x2="226" y2="89"/>
|
||||
<line x1="218" y1="127" x2="226" y2="127"/>
|
||||
<line x1="218" y1="165" x2="226" y2="165"/>
|
||||
<line x1="218" y1="203" x2="226" y2="203"/>
|
||||
</g>
|
||||
|
||||
<!-- Below the stack, two filled small circles connected by a horizontal line
|
||||
indicating the wire / virtual machines. Two endpoints, one segment. -->
|
||||
<g>
|
||||
<line x1="100" y1="240" x2="180" y2="240" stroke-width="0.7"/>
|
||||
<circle cx="100" cy="240" r="2.5" fill="currentColor" stroke="none"/>
|
||||
<circle cx="180" cy="240" r="2.5" fill="currentColor" stroke="none"/>
|
||||
<!-- A tiny squiggle between them indicating signal in transit -->
|
||||
<path d="M 130 240 q 5 -3 10 0 t 10 0" stroke-width="0.5"/>
|
||||
</g>
|
||||
|
||||
</g>
|
||||
</svg>
|
||||
|
After Width: | Height: | Size: 4.1 KiB |
|
|
@ -0,0 +1,23 @@
|
|||
---
|
||||
title: "NeuroPose"
|
||||
date: 2026-04-21
|
||||
abstract: >
|
||||
3D pose-estimation and kinematic-analysis system for neurological-recovery research, developed in Liqi Shu's laboratory at the Brown University Department of Neurology. Python/TensorFlow inference, MATLAB-based statistical post-processing, Rust backend with HTML/JS frontends. Four externally-funded sub-projects since 2023; clinical-implications manuscript in preparation.
|
||||
tags:
|
||||
- research
|
||||
- research/machine-learning
|
||||
authors:
|
||||
- "Levi Neuwirth | /me.html"
|
||||
- "Liqi Shu"
|
||||
affiliation:
|
||||
- "Department of Neurology, Warren Alpert Medical School, Brown University"
|
||||
status: "Draft"
|
||||
confidence: 75
|
||||
importance: 4
|
||||
evidence: 4
|
||||
scope: broad
|
||||
novelty: innovative
|
||||
practicality: high
|
||||
---
|
||||
|
||||
A fuller write-up follows with the clinical-implications manuscript. In the meantime, see the [projects index](/cv/projects/).
|
||||
|
|
@ -0,0 +1,106 @@
|
|||
<svg viewBox="0 0 280 280" xmlns="http://www.w3.org/2000/svg" role="img" aria-labelledby="mark-title-neuropose">
|
||||
<title id="mark-title-neuropose">An articulated kinematic figure with a small dendritic node at the head — pose tracked through the body, signal originating from the brain.</title>
|
||||
<desc>Frontmatter mark for the essay "NeuroPose".</desc>
|
||||
|
||||
<g fill="none" stroke="currentColor" stroke-linecap="round" stroke-linejoin="round">
|
||||
|
||||
<!-- Outer roundel -->
|
||||
<circle cx="140" cy="140" r="128" stroke-width="0.6"/>
|
||||
|
||||
<!-- Kinematic skeleton: head, spine, two arms, two legs.
|
||||
Drawn in a slight asymmetric stance to suggest motion-tracking
|
||||
rather than a static anatomical diagram.
|
||||
|
||||
Joint coordinates (the markers tracked):
|
||||
head (140, 78)
|
||||
neck (140, 102)
|
||||
shoulder L (122, 110)
|
||||
shoulder R (158, 110)
|
||||
elbow L (108, 134)
|
||||
elbow R (172, 132)
|
||||
wrist L (96, 158)
|
||||
wrist R (180, 156)
|
||||
hip L (130, 158)
|
||||
hip R (150, 158)
|
||||
knee L (124, 192)
|
||||
knee R (158, 190)
|
||||
ankle L (118, 222)
|
||||
ankle R (164, 224)
|
||||
-->
|
||||
|
||||
<!-- Spine (neck → mid-hip) -->
|
||||
<line x1="140" y1="102" x2="140" y2="158" stroke-width="1.2"/>
|
||||
|
||||
<!-- Shoulders crossbar -->
|
||||
<line x1="122" y1="110" x2="158" y2="110" stroke-width="1.0"/>
|
||||
|
||||
<!-- Hips crossbar -->
|
||||
<line x1="130" y1="158" x2="150" y2="158" stroke-width="1.0"/>
|
||||
|
||||
<!-- Left arm -->
|
||||
<line x1="122" y1="110" x2="108" y2="134" stroke-width="1.0"/>
|
||||
<line x1="108" y1="134" x2="96" y2="158" stroke-width="1.0"/>
|
||||
|
||||
<!-- Right arm: slightly forward stance for asymmetry -->
|
||||
<line x1="158" y1="110" x2="172" y2="132" stroke-width="1.0"/>
|
||||
<line x1="172" y1="132" x2="180" y2="156" stroke-width="1.0"/>
|
||||
|
||||
<!-- Left leg -->
|
||||
<line x1="130" y1="158" x2="124" y2="192" stroke-width="1.0"/>
|
||||
<line x1="124" y1="192" x2="118" y2="222" stroke-width="1.0"/>
|
||||
|
||||
<!-- Right leg: slight stride -->
|
||||
<line x1="150" y1="158" x2="158" y2="190" stroke-width="1.0"/>
|
||||
<line x1="158" y1="190" x2="164" y2="224" stroke-width="1.0"/>
|
||||
|
||||
<!-- Joint markers as small filled circles — these are the tracked points -->
|
||||
<g fill="currentColor" stroke="none">
|
||||
<circle cx="140" cy="102" r="1.6"/>
|
||||
<circle cx="122" cy="110" r="1.6"/>
|
||||
<circle cx="158" cy="110" r="1.6"/>
|
||||
<circle cx="108" cy="134" r="1.6"/>
|
||||
<circle cx="172" cy="132" r="1.6"/>
|
||||
<circle cx="96" cy="158" r="1.6"/>
|
||||
<circle cx="180" cy="156" r="1.6"/>
|
||||
<circle cx="130" cy="158" r="1.6"/>
|
||||
<circle cx="150" cy="158" r="1.6"/>
|
||||
<circle cx="124" cy="192" r="1.6"/>
|
||||
<circle cx="158" cy="190" r="1.6"/>
|
||||
<circle cx="118" cy="222" r="1.6"/>
|
||||
<circle cx="164" cy="224" r="1.6"/>
|
||||
</g>
|
||||
|
||||
<!-- Head as the neural node: an open ring with a small dendritic spray
|
||||
radiating upward, encoding "neuro" — origin of the motor signal.
|
||||
Larger and more detailed than the joint markers to distinguish. -->
|
||||
<circle cx="140" cy="78" r="9" stroke-width="1.0"/>
|
||||
<!-- Dendrites: short branching strokes from the top half of the head -->
|
||||
<g stroke-width="0.6">
|
||||
<line x1="135" y1="71" x2="129" y2="62"/>
|
||||
<line x1="129" y1="62" x2="124" y2="58"/>
|
||||
<line x1="129" y1="62" x2="131" y2="55"/>
|
||||
|
||||
<line x1="140" y1="69" x2="140" y2="56"/>
|
||||
<line x1="140" y1="56" x2="136" y2="50"/>
|
||||
<line x1="140" y1="56" x2="144" y2="50"/>
|
||||
|
||||
<line x1="145" y1="71" x2="151" y2="62"/>
|
||||
<line x1="151" y1="62" x2="156" y2="58"/>
|
||||
<line x1="151" y1="62" x2="149" y2="55"/>
|
||||
</g>
|
||||
<!-- A central dot at the cortex — the source -->
|
||||
<circle cx="140" cy="78" r="1.8" fill="currentColor" stroke="none"/>
|
||||
|
||||
<!-- Trajectory traces: thin dashed arcs at three of the limb endpoints,
|
||||
indicating tracked motion across frames. Sparse — three is enough. -->
|
||||
<g stroke-width="0.4" stroke-dasharray="1.5 2.5" opacity="0.7">
|
||||
<!-- Right wrist arc -->
|
||||
<path d="M 188 150 Q 192 156 180 156"/>
|
||||
<!-- Left ankle arc -->
|
||||
<path d="M 110 222 Q 114 226 118 222"/>
|
||||
<!-- Right ankle arc -->
|
||||
<path d="M 168 232 Q 165 226 164 224"/>
|
||||
</g>
|
||||
|
||||
</g>
|
||||
</svg>
|
||||
|
After Width: | Height: | Size: 4.3 KiB |
|
|
@ -0,0 +1,51 @@
|
|||
---
|
||||
title: "The Philosophical Legacy of Dostoevsky's Implicit Rejection of Logic and Science in Part I of <em>Notes from Underground</em>"
|
||||
date: 2025-10-24
|
||||
abstract: >
|
||||
*Notes from Underground* is widely admired as a cornerstone of literature, culture, and philosophy. This paper develops the argument that the primary philosophical undercurrent is a rejection of logic and science as end-alls in modern life, traced through comparison with the more explicitly articulated works of Dostoevsky's contemporaries and successors: Nietzsche, Heidegger, Shestov, Ellul, Sartre, Camus, Husserl, and Arendt.
|
||||
tags:
|
||||
- nonfiction
|
||||
- nonfiction/philosophy
|
||||
authors:
|
||||
- "Levi Neuwirth | /me.html"
|
||||
history:
|
||||
- date: "2026-04-17"
|
||||
note: "expanded section on Shestov's divergence from Nietzsche"
|
||||
- date: "2025-12-03"
|
||||
---
|
||||
|
||||
*Notes from Underground* is widely admired as a cornerstone of literature, culture, and philosophy.[@Frank] Dostoevsky masterfully combines the narrative of the now-iconic Underground Man with implicit philosophical undercurrents. The result is a piece which is simultaneously a masterwork of fiction and a deeply influential philosophical treatise. Intriguingly, Dostoevsky never explicitly defines what the philosophical positions that he is advocating for or against are in his work, instead exclusively expressing them implicitly through storytelling. This paper will develop the argument that the primary philosophical undercurrent in *Notes from Underground* is a rejection of logic and science as end-alls in modern life through comparison of Dostoevsky's implicit revelation to the more explicitly articulated philosophical works of his contemporaries.
|
||||
|
||||
*Notes from Underground* opens with a contemplation on suffering, while also hinting at the notion of futility. In sections III and IV of part 1, the motif of the stone wall is introduced, one which will recur throughout the remainder of the part. The stone wall is emblematic of a barrier; it is the "only" object which "perhaps will stop" one from their goals.[@Notes] But the stone wall introduces an immediate double entendre. Dostoevsky initially refers to it as a "calming influence." The calming influence to which he refers is the supposed illusion of stability that the stone wall provides. The stability follows from scientific laws that are uniformly true --- Underground Man provides the example of the equation $2+2=4$, elaborating: "...I shall never be able to break through such a stone wall... but I shall not reconcile myself to it... As though such a stone wall were really the same thing as peace of mind."[@Notes]
|
||||
|
||||
The Underground Man has thus, in the opening four sections of the work, established a contradiction. The apparent resultant stability that logic and science provide through laws of arithmetic and reason is one that, in the eyes of the Underground Man, incurs an unacceptable price in the reduction of creativity, of freedom of expression. Indeed, the primary grievance that the Underground Man indicates is the forced acceptance of advocates of such laws and, "consequently, all [their] results." This forced acceptance disregards, even outright dismisses his opinion on the laws.^[This is arguably a mischaracterization of scientific ideals, one repeated by many scientific credential-lacking philosophers. We withhold development of such an argument here; it would need to be a distinct paper.] There is apparently nothing which the stone wall is *actually* useful for.
|
||||
|
||||
Dostoevsky, through his introduction and subsequent discussion of the stone wall in relation to scientific laws, has commenced his implicit philosophical treatise. Throughout the remainder of the work, as will be discussed further, the Underground Man's actions and anecdotes serve as illustrative evidence that the stability provided by the stone wall of science as a framework of thinking (and one which greatly captivates the Underground Man) is nothing more than illusion, for the Underground Man's actions are indicative of anything but stability, morality, etc. He further demonstrates, through the explicit discontent and boredom that Underground Man expresses in section V, that the stone wall of reason is insufficient even as mere entertainment.
|
||||
|
||||
Nietzsche is a contemporary of Dostoevsky who was influenced not only by this philosophical argument, but additionally by the character of the Underground Man himself. In his *On the Genealogy of Morals*, he develops an explicit comparison of science to the "ascetic ideal," proclaiming that "science rests on the same basis as does the ascetic ideal: a certain impoverishment of life is the presupposition of the latter as of the former."[@Nietzsche_Gene] One might describe the Underground Man's situation as something of a cursed asceticism. He is certainly a hermit to the same extent that a religious ascetic would be. Yet he is not an ascetic due to religious ideals, nor is he one due to lack of social compulsion --- would such an ascetic not only be compelled to write about himself to such an extent, but describe it as the "greatest possible pleasure?" He is almost narcissistic in his self-indulgence through writing, a quality not advocated for by any major religion of which he could be an ascetic. Therefore, what Nietzsche really means is that science promotes a lifestyle that is akin to the sacrifice and abnegation that asceticism entails, while providing no real stability --- the stone wall *is* an illusion, as Dostoevsky claims --- and, thus, rejects science as an end-all. It seems that Dostoevsky's example deeply resonated with Nietzsche. He describes the lifestyle of sacrifice and abnegation as "periods of exhaustion, often of sunset, of decay --- the effervescing strength, the confidence in life, the confidence in the future are no more."[@Nietzsche_Gene] Over ten years before this writing, he had already begun to form his opinion on science in *The Birth of Tragedy,* writing "What I then laid hands on, something terrible and dangerous, a problem with horns... it was the problem of science."[@Nietzsche_Tragedy]
|
||||
|
||||
Martin Heidegger is another of Dostoevsky's contemporaries who outlines, perhaps even formalizes^[Though Heidegger, demonstrated by his argument in "Letter on Humanism," would *hate* reduction to mere "formalization" of his predecessor, and is probably rolling in his grave at the thought.] a similar, aggressive perspective. In "Letter on Humanism," Heidegger argues that "such names as 'logic,' 'ethics,' and 'physics' begin to flourish only when original thinking comes to an end."[@Heidegger_Letter] Heidegger goes further within "Building Dwelling Thinking," indicating therein that "...building is closer to the essence of spaces and the essential origins of 'space' than any geometry and mathematics."[@Heidegger_Building]^[It may be debated the extent to which Heidegger intends for "building" in this work to come across in a metaphorical sense. He does remark, 8 pages before the chosen quote, that "We limit ourselves to building in the sense of...", which implies that further, implicit meanings are present (an argument with which I strongly concur). He never explicitly mentions any such metaphorical intention.] Heidegger's insistence that the consideration of science and logic as end-alls is a reductive position appears to have been influenced by Dostoevsky in *Notes from Underground*. Heidegger, once again in "Building Dwelling Thinking," presents an ingenious discourse on the necessity of some higher ulterior motivation than mere fulfillment of rational inquiry without exterior purpose, asserting that "The essence of building is letting dwell"[@Heidegger_Building]^[My proposed interpretation of "dwelling" in this more metaphorical sense.] --- or, to paraphrase, an interior motivation and fascination with one's pursuits is a prerequisite for the derivation of any sense of fulfillment from said pursuits. This is in stark contrast to the image of asceticism developed by Dostoevsky and Nietzsche, wherein science is a pursuit which reduces the extent of one's livelihood, thereby diminishing the qualities Heidegger presents as necessities.
|
||||
|
||||
In section 5 the Underground Man expands his digression on science, therein beginning the expansion in scope of Dostoevsky's philosophical argument. Yet, before doing this, Dostoevsky feels it necessary to explicitly characterize the "laws of nature," doing so through the Underground Man's description of them as "a disgusting business" due to the "infinite worry and trouble" they have caused him.[@Notes] Here, Nietzsche and Dostoevsky begin to diverge. Nietzsche concedes that "...as for these celebrated victories of science; there is no doubt that they are victories..."[@Nietzsche_Gene] acknowledging that there is some merit to this "disgusting business." But Nietzsche is hardly the only philosopher to have felt the influence of *Notes from Underground*, and other philosophers have continued to agree more strongly with the Underground Man's classification to our current day and age.^[This is, once again, arguably another broad and disappointing mischaracterization of science. We withhold arguments here and refer you to Richard Feynman's 1988 book *What Do You Care What Other People Think?*]
|
||||
|
||||
One such example of a philosopher in deep agreement with Dostoevsky is Lev Shestov. Shestov holds, contrary to Nietzsche, that the victories of science are better classified as stagnations, writing "In the 'ultimate questions of life' we are not a bit nearer the truth than our ancestors were... reason is a laggard, without much foresight..."[@Shestov] Shestov, in his work, develops an argument much closer in sentiment to Underground Man, and this is no coincidence; Dostoevsky is mentioned by name countless times in his work. Yet, for all of Shestov's agreement with Dostoevsky, in comparing *All Things Are Possible* and *Notes from Underground* an interesting contradiction arises. Shestov develops his claim that we have made no progress in answering the questions of life in a way that suggests he believes that ancient people had few answers and we have not added any. Dostoevsky, on the other hand, would almost certainly argue that his ancestors, say those who were around at the same time as Christ, had *all* of the answers, and over the next nearly 2,000 years no further answers were contributed because none were necessary; the questions were already satisfactorily answered. In section 7 of part I, amidst a lengthy development of his implicit philosophical treatise, Dostoevsky interjects with an intriguing reference to religion. The Underground Man asks "who was it who first said... the only reason man behaves dishonorably is because he does not know his own interests..."[@Notes] The discussion evolves into a wide-ranging one, touching upon enlightenment, innocence, and human nature in the subsequent lines. It is no accident that, to begin the section of part one directly at the center of a sequence of sections (5--9) entirely focused on development of his philosophical position against science and reason, he refers to religion in the abstract, never opting for the imagery of any one specific religion, but instead using religion to center and ground his argument. Dostoevsky's subtle yet crucial grounding of his argument in ancient ideas demonstrates an agreement with Shestov, if only one that is viewed from a different perspective.
|
||||
|
||||
The Underground Man goes on, still in section 7, the center of his philosophical argument, to presciently discuss the dangers of unrestricted science and reason. In one of his most brilliant quotes, the Underground Man argues that "man is so obsessed by systems and abstract deductions that he is ready to distort the truth deliberately...", going further to emphasize the necessity of reliance on the senses.[@Notes] Interestingly, the Underground Man uses the example of war, illustrating through the imagery of bloodshed the pitfalls of reason's application and interaction with human nature. We may once again turn to Heidegger as an example of a philosopher who expanded upon this position.^[With the obvious irony that, of course, Heidegger had at least *some* (and likely *plenty* of) affiliation with the Nazi party.] In *The Question Concerning Technology*, Heidegger constructs an interpretation of technology as a mechanicism of "revealing," and, after much development, presents the incredible claim that "The destining of revealing is in itself not just any danger, but *the* danger."[@Heidegger_Building] Underground Man, throughout section 5, gradually transitions his discussion from the individual (which was the focus of the immediately preceding sections 3 and 4) to the collective, discussing the consideration of warfare and violence by civilization at large as an "abomination" only after his previous meditation on the individualistic impact of reason; Heidegger seems more interested in society from the start, neglecting to explicitly speak to the impact of technological progress through scientific innovation on the individual.
|
||||
|
||||
Ellul was a philosopher who was, more similarly to Dostoevsky, equally concerned with the individual and the collective. Dostoevsky begins to discuss the collective impact through his brilliant "conversation" between the Underground Man and the reader, actively addressing and involving the latter in the second person. The reader ("you") postulates that science will have "completely re-educated human nature and directed it along the road of normal behavior," before the Underground Man deconstructs any notion of normality in such "normal" behavior.[@Notes] We may compare this to Ellul, who, not too unlike Heidegger, describes how "Science brings to the light of day everything man had believed sacred... Technique takes possession of it and enslaves it."[@Ellul] The Underground Man himself will develop a stunningly similar argument in which he concludes that "there will be no more independent actions or adventures in the world." Where Dostoevsky and Ellul differ is that Dostoevsky, once again implicitly through the Underground Man, argues that it is the laws of nature themselves (or, science in the abstract) that opens the possibility of such slavery to determinism, whereas Ellul (and, to a lesser extent, Heidegger) argues that it is technology (or, science in tangible application) that does so. Both come to strikingly similar conclusions, regardless of the means by which they reached them.
|
||||
|
||||
The Underground Man subsequently presents a depiction of resistance, creating a character who advocates for sending "'all these logarithms to the devil so that we can again live according to our foolish will'."[@Notes] He goes on to argue that such a character "would certainly find followers." Interestingly, while Ellul himself never made such explicit predictions nor advocated for an anti-Technique revolution, it is well known and documented that Ted Kaczynski was profoundly influenced by Ellul's work, particularly *The Technological Society*, and these influences were part of his motivation to commit atrocities in the name of eco-terrorist ideology. Indeed, in Kaczynski's own *Industrial Society and Its Future*^[More commonly referred to as the "Unabomber Manifesto."], the phrase "technological society" occurs 10 times.[@Kacz] Whether the revolution called for in this document has found any followers, as a more extreme case of "sending all these logarithms to the devil," is not clear.
|
||||
|
||||
The Underground Man continues into section 8 with a discussion of the absolute necessity of free will, describing in the process the shortcomings of reason. He argues that "Reason is only reason... whereas volition is a manifestation of the whole of life."[@Notes] This argument, once again, is not much unlike Heidegger's argument in *Building Dwelling Thinking*. The strength of Dostoevsky's conviction on behalf of volition and free will also deeply influenced Jean-Paul Sartre. Sartre develops an argument that he is "condemned to be free," elaborating that "no limits to my freedom can be found except freedom itself... that we are not free to cease being free."[@Sartre] The Underground Man is also an advocate of such unrestricted freedom, describing this absolute freedom as "the right to desire for himself even what is very stupid and not to be bound by an obligation to desire only what is sensible." Sartre goes on to describe how "man being condemned to be free carries the weight of the whole world on his shoulders; he is responsible for the world."[@Sartre] This explicitly solidifies a recurring implicit theme across Dostoevsky's work: that all is interconnected, and that everyone is to blame for everything. In *Notes from Underground*, this is continually reiterated through the depiction of the Underground Man's *underground* condition in incredible detail. The Underground Man has a seemingly contradictory self-awareness of his state, an awareness that in some ways he himself has created the conditions and the environment that proliferated such a state, and yet, at the same time, the meditations of the laws of nature and the interactions depicted in part II are used to implicitly build an argument of interconnection. The influence of this construction on Sartre is, once again, not one that I merely postulate; Sartre mentions Dostoevsky by name twice in *Being and Nothingness*.
|
||||
|
||||
If Sartre was an advocate of unrestricted freedom as an absolute necessity in the same way that Dostoevsky was (implicitly through the Underground Man, of course), then Camus was much like our earlier example of Shestov, coming to near identical conclusions from a different perspective. Camus, much like the Underground Man, found suffering to be absolutely essential to happiness, writing in perhaps the most famous words of 20th century philosophy "The struggle itself toward the heights is enough to fill a man's heart. One must imagine Sisyphus happy."[@Camus_Myth] Nearly 80 years earlier, the Underground Man had demonstrated a similar, if less strong, conclusion through his illustration of the "'pleasure even in toothache.'" In fact, while the majority of sections 3 and 4 of part I are devoted to this meditation on suffering, the Underground Man elaborates on it again in section 9, stating "Does reason not make mistakes...? Is it not possible that man loves something besides prosperity? Perhaps he is just as fond of suffering?" Where Camus and Dostoevsky differ is in their view of the source of suffering. Where Dostoevsky relates the example of the toothache as an inevitable, almost mundane instance of suffering, one which the Utopian future that science promises cannot ever truly aspire to negate, Camus develops suffering as the result of existential inquiry --- and he argues that such inquiry is vital. Camus never takes a stance so explicitly against science and reason as, say, Nietzsche, rather elaborating, once again in *The Myth of Sisyphus*, that "I realize that if through science I can seize phenomena and enumerate them, I cannot, for all that, apprehend the world."[@Camus_Myth] Thus, Camus' stance on science is somewhere between Nietzsche's and Dostoevsky's. He acknowledges that science alone is insufficient for answering questions of stature similar to the "ultimate" ones raised by Shestov. This shortcoming of science, Camus argues, is a source of suffering, as it proliferates the existential dread that such questions impose. Yet Camus advocates for suffering elegantly in *Return to Tipasa*, beautifully proclaiming "In the depths of winter, I finally learned that within me there lay an invincible summer."[@Camus_Myth]^[This prose is even more beautiful in French, so we refer you to the original *Retour à Tipasa* from *L'Été.*] Dostoevsky, rather than appreciating (or, at least, feeling truly neutral) science as an emanation of suffering, argues that suffering is the ideological inverse of science. The Underground Man, again in section 9, discusses the Crystal Palace --- an unmistakable emblem of science and reason. In doing so, he refers to how "suffering is not permitted... In the Crystal Palace it is unthinkable: suffering is doubt, it is negation."[@Notes] Thus the Underground Man and, implicitly, Dostoevsky, take issue with what they perceive as the end goal of science: the cessation of suffering, the construction of a Utopian world, and the inevitable reduction, perhaps even cessation, of freedom as a consequence.^[Once again, I would argue that this is a mischaracterization --- perhaps to some this is an end goal of *technology*, but not of *science*. I refer you, in lieu of an extended argument, to chapter IV of Carl Sagan's 1994 *Pale Blue Dot.*]
|
||||
|
||||
The Underground Man goes on, in section 10 now, to argue (in an unusually explicit way) that the primary conflict that arises between this notion of freedom and free will and science and reason is the objectivity of science. The Underground Man acknowledges and disregards this objectivity when he states "What do I care whether it is against the laws of nature? What does it matter so long as it exists in my desires?"[@Notes] He elaborates on his rejection of the rigidity and seriousness in objectivity of science and reason a few lines later: "I rejected the Crystal Palace myself for the sole reason that one would not be allowed to stick one's tongue out at it." These examples reinforce the previous development of the argument that Dostoevsky perceives freedom as inevitable, much like Sartre. The Underground Man goes further to characterize why science and reason in particular are the target of his scrutiny by saying "Perhaps what I resented was that among all our buildings there has never been one at which one could not stick out one's tongue."
|
||||
|
||||
This scrutiny of the harshly perceived immutability of science and reason likely influenced the philosopher Edmund Husserl. Husserl, originally trained as a mathematician,[@Cooper] was in some ways a stark contrast to Dostoevsky. He believed in the necessity of the philosopher "withdrawing into himself and attempting, within himself, to overthrow and build anew all the sciences that, up until then, he has been accepting."[@Husserl_Meditations] The belief in such a necessity implies a belief in the validity and worthiness of science that is contrary to Dostoevsky's implicit arguments. Yet this perspective on science, the perspective that one must develop their own scientific views for themselves, is emblematic of the broader skepticism that Dostoevsky advocates through the Underground Man. For Husserl does not advocate blind acceptance of science, but rather the replication and verification of any ends which are to be considered objectively true, like "laws of nature." Of course, Husserl does not pay any attention to the fact that building anew "all the sciences" is likely an impossible task, and one that is directly in opposition to the collaborative nature of science as an enterprise.^[As a mathematician more so than a scientist in training myself, I would argue this is very much the thinking and position of a mathematician and *not* a scientist. There is a difference, if subtle!] Yet, although Husserl does not appear to explicitly indicate this necessity as one that is necessary only as an intellectual exercise or means of philosophical development otherwise, he does express in his later work a broader skepticism and disdain for the objectivity of science and reason, much like the Underground Man. Where the Underground Man expresses his disdain for science's perceived immutability compared to other institutions of society, Husserl expresses his disdain in a more explicit and straightforward way, borrowing from Kant (with acknowledgment) when he states that "the objective sciences (no matter how much they... may consider themselves... to be in possession of the only true method) are not seriously sciences at all... not cognitions of what exists in ultimate truth."[@Husserl_Crisis] Yet, like Nietzsche, Husserl concedes the "virtue of their obvious theoretical and practical accomplishments."[@Husserl_Crisis] Husserl's notion of skepticism and rejection of objectivity, regardless of his perceived virtues of the *objective* sciences, creates an interesting juxtaposition of Dostoevsky's ideas with other ideas which Dostoevsky would've disapproved of. The Underground Man's rejection is perhaps more emotionally charged when, at the end of a paragraph of rhetorical questions, he asks "Can this be the sole purpose? I don't believe it."[@Notes] Although the Underground Man's (and thus, Dostoevsky's) ideas are distinctly differentiable from Husserl's, in another intriguing, deliberate contradiction that Dostoevsky introduces to his text through another dialogue between reader and Underground Man, he acknowledges that the Underground Man "longs for life, yet [he tries] to solve the problems of life by a logical tangle!"[@Notes] This (as he describes it, "insolent") logical tangle[@Notes] is not much unlike Husserl's advocated individualized building of the sciences anew from scratch.
|
||||
|
||||
Finally, at the close of part I, Dostoevsky provides one further reason to be critical of science and reason, particularly in their perceived role as the progenitors of technology and automation. Similarly to his view on suffering, the Underground Man is, like Heidegger, an advocate of work, justifying his composition of part II with "Writing down things is, in fact, a sort of work. People say work makes man better and more honest. Well, here's a chance for me..."[@Notes] It should come as no surprise, then, that Hannah Arendt, a philosopher heavily influenced by (and personally involved with) Heidegger, is another party who shares similar views. Arendt captures her vision of the failed Utopia that technological progress catalyzed by science would provide with strong language, stating "What we are confronted with is the prospect of a society of laborers without labor, that is, without the only activity left to them. Surely, nothing could be worse."[@Arendt]
|
||||
|
||||
Thus, at the end of the first part of *Notes from Underground* which we have confined our analysis to, all of the implicit philosophical undercurrents have been developed and connected full circle. We have now returned to the futility and boredom that both Dostoevsky and Arendt argue (the former implicitly and the latter explicitly) would arise from uninhibited forward progress in the realms of science and reason. Where in the later sections of part I the Underground Man returns to being primarily concerned with himself on the individual level, his self-centered contemplation of the Crystal Palace and philosophical discussions make an implicit argument concerning the collective. Arendt, likely influenced by this argument, whether directly or transitively through Heidegger, saves the strongest of words in her work for the perceived discrepancy, one she shares with Dostoevsky, between what science and reason promise and what they *actually* provide.
|
||||
|
||||
*Notes from Underground* is a work that has retained its emotional and philosophical power, and, as a consequence, one that has stood the test of time. It is a work that is timelessly prescient, drawing unbelievably accurate conclusions about the discrepancies between the perceived end-goals of the end-alls of science and reason, end-goals that seemingly come from a place of daydreaming and fantasy, and the reality of the century following its composition: two world wars, intense geopolitical climate in the years where the wars were not active, immense economic turmoil and suffering, etc. If science *was* truly promising Utopia during Dostoevsky's time, it certainly failed to deliver in the subsequent century, and Dostoevsky presciently observed the inevitable shortcomings before they had even had a chance to occur. Regardless of the degree to which one agrees or disagrees with Dostoevsky's implicit conclusions and the more explicit conclusions of his contemporaries, one must acknowledge the incredible stature of his work, and its lasting legacy that continues to endure to the present day.
|
||||
|
|
@ -0,0 +1,33 @@
|
|||
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 280 280" role="img" aria-labelledby="mark-title-ozymandias">
|
||||
<title id="mark-title-ozymandias">A half-buried fluted column on a desert horizon, with a low sun-disk behind</title>
|
||||
<desc>A frontispiece mark for "Ozymandias: A Static Site Framework".</desc>
|
||||
|
||||
<circle cx="140" cy="140" r="128" stroke="currentColor" stroke-width="0.6" fill="none"/>
|
||||
|
||||
<circle cx="140" cy="155" r="34" stroke="currentColor" stroke-width="1.0" fill="none"/>
|
||||
|
||||
<line x1="32" y1="180" x2="248" y2="180" stroke="currentColor" stroke-width="1.2" stroke-linecap="round"/>
|
||||
|
||||
<g stroke="currentColor" fill="none" stroke-linecap="round" stroke-linejoin="round">
|
||||
<path d="M 110 100 L 168 96 L 172 110 L 106 114 Z" stroke-width="1.4"/>
|
||||
<line x1="108" y1="107" x2="170" y2="103" stroke-width="1.4"/>
|
||||
|
||||
<line x1="118" y1="114" x2="120" y2="180" stroke-width="1.4"/>
|
||||
<line x1="160" y1="110" x2="162" y2="180" stroke-width="1.4"/>
|
||||
|
||||
<line x1="128" y1="116" x2="130" y2="180" stroke-width="0.6"/>
|
||||
<line x1="138" y1="115" x2="140" y2="180" stroke-width="0.6"/>
|
||||
<line x1="148" y1="114" x2="150" y2="180" stroke-width="0.6"/>
|
||||
</g>
|
||||
|
||||
<path d="M 32 180 Q 70 196 110 188 T 180 192 T 248 184" stroke="currentColor" stroke-width="0.8" fill="none" stroke-linecap="round"/>
|
||||
<path d="M 50 180 Q 80 210 140 205 T 230 198" stroke="currentColor" stroke-width="0.5" fill="none" stroke-linecap="round" opacity="0.7"/>
|
||||
|
||||
<g fill="currentColor" stroke="none">
|
||||
<circle cx="80" cy="220" r="0.8"/>
|
||||
<circle cx="200" cy="225" r="0.8"/>
|
||||
<circle cx="155" cy="235" r="0.8"/>
|
||||
<circle cx="100" cy="240" r="0.8"/>
|
||||
<circle cx="180" cy="245" r="0.8"/>
|
||||
</g>
|
||||
</svg>
|
||||
|
After Width: | Height: | Size: 1.7 KiB |
|
|
@ -0,0 +1,285 @@
|
|||
---
|
||||
title: "Ozymandias: A Static Site Framework"
|
||||
date: 2026-04-12
|
||||
abstract: >
|
||||
Ozymandias is the static site framework underlying this website, now extracted and released under the MIT license. It
|
||||
is a full-featured Hakyll and Pandoc setup for long-form writing: sidenotes, epistemic profiles, backlinks,
|
||||
wikilinks, a swipeable score reader, a semantic search pipeline, and more — configurable from a single YAML file
|
||||
and deployable with a single make command.
|
||||
tags: [meta, projects, releases]
|
||||
status: "Working model"
|
||||
confidence: 80
|
||||
importance: 2
|
||||
evidence: 3
|
||||
scope: average
|
||||
novelty: moderate
|
||||
practicality: high
|
||||
---
|
||||
|
||||
<figure class="poem-excerpt">
|
||||
<blockquote>
|
||||
<p>My name is Ozymandias, King of Kings;<br>
|
||||
Look on my Works, ye Mighty, and despair!<br>
|
||||
Nothing beside remains. Round the decay<br>
|
||||
Of that colossal Wreck, boundless and bare<br>
|
||||
The lone and level sands stretch far away.</p>
|
||||
</blockquote>
|
||||
<figcaption><a href="/poetry/ozymandias.html">Ozymandias</a> — Percy Bysshe Shelley</figcaption>
|
||||
</figure>
|
||||
|
||||
The name is a joke. Every framework is a monument that its author believes will outlast the work produced in it. The name is also a warning: the writing you put in a framework might actually outlast the framework itself, which is why the framework should be small, coherent, and legible — not a cathedral built to impress.
|
||||
|
||||
The core of this website has been extracted and released as [Ozymandias](https://git.levineuwirth.org/neuwirth/ozymandias), a static site framework under the MIT license. It is the full pipeline: the Haskell build system, the Pandoc filter stack, all templates, all stylesheets, all client-side JavaScript — minus my personal content. If you want a website that works like this one and want to understand exactly how it works, Ozymandias is where to start.
|
||||
|
||||
This page describes what Ozymandias is, how it diverged from this site during extraction, and how to use it.
|
||||
|
||||
---
|
||||
|
||||
## What It Is
|
||||
|
||||
Ozymandias is a static site generator for long-form writing. It is built on two mature Haskell tools: [Hakyll](https://jaspervdj.be/hakyll/) for build orchestration and [Pandoc](https://pandoc.org/) for document processing. The framework handles routing, templating, and pagination through Hakyll, and applies a custom sequence of Pandoc [AST]{.smallcaps} transforms during compilation. The output is a directory of plain [HTML]{.smallcaps} files that can be served by any web server.
|
||||
|
||||
The short version of what comes with it:
|
||||
|
||||
- **Sidenotes** — Pandoc footnotes render as margin notes on wide screens; on narrow screens they collapse to a numbered footnotes section at the bottom.
|
||||
- **Epistemic profiles** — essays can declare confidence, evidence quality, importance, scope, novelty, and practicality; these appear as a structured footer before the reader commits to the full text.
|
||||
- **Backlinks** — every page accumulates a list of other pages that link to it, with surrounding paragraph context, via a two-pass compilation strategy.
|
||||
- **Wikilinks** — `[[Page Title]]` and `[[Page Title|display text]]` syntax resolved at build time.
|
||||
- **Citations** — Pandoc citeproc with Chicago Notes; footnote-style in-text markers with a bibliography section and a separate "further reading" block.
|
||||
- **Score reader** — a swipeable [SVG]{.smallcaps} viewer for music compositions with dark-mode-compatible notation.
|
||||
- **Typography** — dropcaps, automatic smallcaps detection for abbreviations, Latin abbreviation tooltips, old-style figures via the `onum` OpenType feature.
|
||||
- **Mathematics** — [KaTeX]{.smallcaps} rendering at build time; no math rendering in the browser.
|
||||
- **Full-text search** — Pagefind, client-side, no external service.
|
||||
- **Semantic search** — an optional embedding pipeline using `sentence-transformers` and [FAISS]{.smallcaps} for a "similar pages" section and semantic query matching.
|
||||
- **Hierarchical tags** — `research/mathematics` expands to both `research` and `research/mathematics`; tag pages are generated and paginated automatically.
|
||||
- **Library portal** — a configurable taxonomy page that groups all content by tag hierarchy.
|
||||
- **Dark mode, reading mode, settings** — client-side, persisted in `localStorage`.
|
||||
- **GPG signing** — optional per-page detached signatures, with pubkey linked from the footer.
|
||||
- **Atom feeds** — site-wide and per-section (music gets its own feed by default).
|
||||
|
||||
The prerequisite list is short: [GHC]{.smallcaps} 9.6+, cabal-install, and Pagefind. Image conversion and the embedding pipeline are both optional and add their own dependencies (`cwebp` and Python with `uv`, respectively).
|
||||
|
||||
---
|
||||
|
||||
## How It Diverged From This Site
|
||||
|
||||
When I extracted Ozymandias, the primary engineering work was disentangling site-specific configuration from the framework machinery. In levineuwirth.org, several values — the site [URL]{.smallcaps}, the author name, the navigation structure, the feed title — were compiled directly into the Haskell source. That is fine for a personal site and irritating for a reusable framework. The extraction introduced a `Config.hs` module and a `site.yaml` file that together hold all identity and navigation configuration. The rest of the build system reads from these at startup and never hardcodes a domain or author name.
|
||||
|
||||
The result is that you can fork the repository, edit one file, and have a working site with a completely different identity. The Haskell source does not need to be touched unless you want to extend or modify the framework itself.
|
||||
|
||||
Beyond configuration, the content in `content/` was replaced with a small set of demo pages that exercise the filter pipeline without constituting a personal corpus. The `data/bibliography.bib` file was emptied and replaced with a placeholder. Everything in `static/` — the fonts, stylesheets, scripts, and link icons — shipped intact. No features were removed during the extraction. Ozymandias has the full pipeline.
|
||||
|
||||
### What Remains Shared
|
||||
|
||||
The two repositories share the same filter modules, the same templates (minus identity strings), and the same static assets. Changes to the filter pipeline in one are intended to be ported to the other. The practical result is that this site is an Ozymandias instance — it runs on the same engine, only with the configuration file pointing at `levineuwirth.org` rather than `example.com`. This page is compiled by the same code that compiles an Ozymandias site built from the framework.
|
||||
|
||||
### What Diverges Intentionally
|
||||
|
||||
Several features of this site are too specific to my personal corpus to include in the framework defaults. The similarity embedding index — which requires running a neural model over all page content — is present in Ozymandias as an optional pipeline but ships with an empty index. The music catalog, the commonplace book, and the statistics page are included in the framework because they are useful to authors in general, but they contain no data by default. The semantic search [ONNX]{.smallcaps} model weights are downloaded by a separate `make download-model` target rather than committed to the repository.
|
||||
|
||||
---
|
||||
|
||||
## The Filter Pipeline
|
||||
|
||||
The filters are the heart of the framework. Pandoc compiles Markdown to an abstract syntax tree, and the filters walk and transform that tree before Pandoc serializes it to [HTML]{.smallcaps}. They are applied in a fixed sequence; the order matters.
|
||||
|
||||
**Source-level preprocessors** run before Pandoc sees the file. They transform raw Markdown strings:
|
||||
|
||||
- **Wikilinks** — converts `[[Page Name]]` and `[[Page Name|display text]]` to standard Markdown links using slugification: lowercase, spaces to hyphens, punctuation stripped. The destination path follows the same routing rules as the content item it targets.
|
||||
- **EmbedPdf** — converts `{{pdf:/path/to/file.pdf}}` syntax (optionally with a page anchor) to an iframe pointed at the vendored PDF.js viewer, preserving the original path in a `data-pdf-src` attribute for the popup thumbnail system.
|
||||
- **Transclusion** — converts `{{essay-slug}}` or `{{essay-slug#section}}` to placeholder divs that the client-side `transclude.js` script resolves at page load. This allows shared content to be authored once and embedded anywhere without duplicating the source.
|
||||
|
||||
**[AST]{.smallcaps}-level filters** run after parsing. They are pure functions over the Pandoc [AST]{.smallcaps}:
|
||||
|
||||
- **Images** — wraps each image in a `<picture>` element with a [WebP]{.smallcaps} source if a `.webp` companion file exists alongside the original. Adds `loading="lazy"` to images below the fold and marks them for the lightbox system.
|
||||
- **Sidenotes** — transforms Pandoc's footnote syntax (`[^1]: note text`) into inline `<span class="sidenote">` elements with alphabetic labels (a, b, c, … z, aa, ab, …). A `<section class="footnotes">` fallback is preserved at document end for narrow screens where margin placement is impractical.
|
||||
- **Typography** — matches exact Pandoc `Str` tokens against a table of Latin abbreviations and wraps them in `<abbr title="…">` elements. The table covers *e.g.*, *i.e.*, *cf.*, *viz.*, *NB*, *et al.*, and the rest of the common scholarly shorthand.
|
||||
- **Links** — classifies external links (any `http`/`https` [URL]{.smallcaps} not on the site's own domain) and adds `class="link-external"`, `target="_blank"`, `rel="noopener noreferrer"`, and a `data-link-icon` attribute that the [CSS]{.smallcaps} uses to render a per-domain icon. A separate pass rewrites root-relative [PDF]{.smallcaps} links to the viewer [URL]{.smallcaps}. Domain classification is by exact hostname match, not substring, so lookalike domains are correctly identified as external.
|
||||
- **Smallcaps** — detects runs of three or more uppercase letters and wraps them in `<abbr class="smallcaps">`. Trailing punctuation is stripped before matching so `HTML,` and `API.` are caught correctly. Short all-caps tokens (`OK`, `I`) and mixed-case tokens (`JavaScript`) are not converted.
|
||||
- **Dropcaps** — the filter itself is an identity transform; the real work is done by the [CSS]{.smallcaps} `.dropcap` class applied via fenced div syntax (`::: dropcap`). The filter's presence in the pipeline documents the intent.
|
||||
- **Math** — another near-identity transform; inline and display math is passed through as-is for [KaTeX]{.smallcaps} to process at render time.
|
||||
- **Code** — prepends `language-` to code block class names so Prism.js can pick up the language for syntax highlighting without each author needing to write `language-haskell` instead of just `haskell`.
|
||||
- **Score** (music-specific) — reads [SVG]{.smallcaps} score fragment files from disk and inlines them into the document, replacing `#000000` and `black` fills and strokes with `currentColor` so notation renders correctly in both light and dark mode.
|
||||
- **Viz** (visualization-specific) — executes Python scripts referenced in fenced code blocks and captures stdout. A Matplotlib script produces an [SVG]{.smallcaps} that is inlined directly; a Vega-Lite script produces a [JSON]{.smallcaps} spec that is embedded for Vega-Embed to render client-side.
|
||||
|
||||
The IO-performing filters (Score, Viz, Images) run before the pure ones. This ordering ensures that downstream filters see a stable [AST]{.smallcaps} without pending file reads.
|
||||
|
||||
---
|
||||
|
||||
## Epistemic Profiles
|
||||
|
||||
The epistemic profile is a structured block that appears in the footer of any essay or post whose frontmatter includes a `status` field. It is the most distinctive feature of the framework philosophically, and the one most worth understanding before deploying it.
|
||||
|
||||
The fields:
|
||||
|
||||
- **Status** — a controlled vocabulary: *Draft*, *Working model*, *Durable*, *Refined*, *Superseded*, *Deprecated*. The distinction between *Working model* and *Durable* matters: the former is a position I currently hold but would not stake much on; the latter is something I expect to hold up under scrutiny.
|
||||
- **Confidence** — an integer from 0 to 100 representing credence in the central thesis. When a `confidence-history` list is present in the frontmatter, the framework derives a trend arrow (↑ ↓ →) from the last two entries automatically.
|
||||
- **Importance** — a 1–5 dot scale for how much the work matters.
|
||||
- **Evidence** — a 1–5 dot scale for how well-evidenced the claims are. An essay with high importance and low evidence is a speculative position and should be read accordingly.
|
||||
- **Trust score** — derived automatically as (confidence × 0.6) + (rescaled evidence × 0.4). It is a narrow answer to "how much should you trust the central claim?" and deliberately does not incorporate importance, scope, novelty, or practicality, which are separate axes intentionally not blended into a composite.
|
||||
- **Scope, Novelty, Practicality** — orientation fields, not ratings. *Scope* ranges from *personal* to *civilizational*; *novelty* from *conventional* to *innovative*; *practicality* from *abstract* to *exceptional*. They appear in the footer alongside the numeric fields.
|
||||
- **Stability** — auto-computed from `git log --follow` at every build. The heuristic: a very new or barely-touched document is *volatile*; an actively-revised document is *revising*; older documents with more commits settle into *fairly stable*, *stable*, or *established*. This requires no manual maintenance.
|
||||
|
||||
The version history block, just above the epistemic footer, uses a three-tier fallback: authored `history:` notes in the frontmatter, then the raw git log, then the `date:` field as a creation record.
|
||||
|
||||
The point is not precision — a 72% confidence rating is not false exactness. It is an attempt to make explicit what most writing leaves implicit: where the author actually stands, and whether that position is stable or still shifting.
|
||||
|
||||
---
|
||||
|
||||
## Backlinks
|
||||
|
||||
Backlinks require a two-pass architecture, because a page cannot know which pages will link to it until all pages have been compiled.
|
||||
|
||||
Pass one compiles every content item in a special "links" version that extracts all internal links together with the surrounding paragraph [HTML]{.smallcaps}. Pass two inverts this map — grouping sources by their targets — and produces `data/backlinks.json`. The final compilation pass loads this file as a dependency and injects the backlinks section into each page's template context.
|
||||
|
||||
The practical consequence for authors is that internal links automatically generate backlink sections with source titles and context snippets, without any manual cross-referencing. The `[[Wikilinks]]` syntax makes it natural to link between pages; the backlinks system makes those connections visible to readers moving in either direction.
|
||||
|
||||
---
|
||||
|
||||
## Semantic Search and Similar Links
|
||||
|
||||
Both features are optional and require Python with `uv`:
|
||||
|
||||
```sh
|
||||
uv sync # install dependencies from pyproject.toml
|
||||
make download-model # fetch ONNX weights for client-side search
|
||||
```
|
||||
|
||||
**Full-text search** uses Pagefind, which indexes the compiled [HTML]{.smallcaps} and produces a static search index that runs entirely in the browser. No external service is involved.
|
||||
|
||||
**Semantic search** runs a `sentence-transformers` model (`all-MiniLM-L6-v2`, 384 dimensions) over extracted page text, builds a [FAISS]{.smallcaps} similarity index, and stores page-level neighbors in `data/similar-links.json`. At render time, this file is loaded as a Hakyll dependency and the top similar pages are injected into each essay's template context as a "Related" section. The same model can be run client-side in the browser via [ONNX]{.smallcaps} Runtime Web for semantic query matching — the weights are served from the same origin, which means no external [API]{.smallcaps} calls.^[This is the design decision I care most about. Bolting semantic search onto a static site usually means sending queries to a third-party service. Serving the model weights from the same origin means the feature works without any network request beyond what is needed to load the page.]
|
||||
|
||||
---
|
||||
|
||||
## Content Types
|
||||
|
||||
Ozymandias supports six content types, each with its own template and routing convention:
|
||||
|
||||
| Type | Path | Route | Template |
|
||||
|:-----|:-----|:------|:---------|
|
||||
| Essay | `content/essays/*.md` | `/essays/{slug}.html` | `essay.html` |
|
||||
| Blog post | `content/blog/*.md` | `/blog/YYYY-MM-DD-{slug}.html` | `blog-post.html` |
|
||||
| Poetry | `content/poetry/*.md` | `/poetry/{slug}.html` | `reading.html` |
|
||||
| Fiction | `content/fiction/*.md` | `/fiction/{slug}.html` | `reading.html` |
|
||||
| Composition | `content/music/{slug}/index.md` | `/music/{slug}/index.html` | `composition.html` |
|
||||
| Page | `content/*.md` | `/{slug}.html` | `page.html` |
|
||||
|
||||
Essays and blog posts support the full feature set: [TOC]{.smallcaps}, epistemic profiles, backlinks, similar links, citations, version history. Poetry and fiction use a `reading` [CSS]{.smallcaps} class that adjusts line spacing and disables indentation, making stanza structure visible. Music compositions get a separate score-reader view at `/music/{slug}/score/` — a minimal interface with swipe navigation through [SVG]{.smallcaps} score pages.
|
||||
|
||||
Several pages are generated automatically without source files: `/essays/index.html`, `/blog/index.html` (paginated, 20 per page), `/new.html` (all content sorted by creation date), `/library.html` (portal taxonomy), tag index pages at `/{tag}/index.html`, author pages at `/authors/{slug}/index.html`, and `/feed.xml`.
|
||||
|
||||
Drafts live in `content/drafts/essays/` and are only visible when the `SITE_ENV=dev` environment variable is set. Production builds exclude them entirely — they do not appear in feeds, tag pages, backlinks, or the library.
|
||||
|
||||
---
|
||||
|
||||
## Configuration
|
||||
|
||||
All site identity and navigation lives in `site.yaml`. The full schema:
|
||||
|
||||
```yaml
|
||||
site-name: "My Site"
|
||||
site-url: "https://example.com"
|
||||
site-description: "A personal site built with Ozymandias"
|
||||
site-language: "en"
|
||||
|
||||
author-name: "Your Name"
|
||||
author-email: "you@example.com"
|
||||
|
||||
feed-title: "My Site"
|
||||
feed-description: "Essays, notes, and creative work"
|
||||
|
||||
license: "CC BY-SA 4.0"
|
||||
source-url: "" # optional link to git repository
|
||||
|
||||
gpg-fingerprint: "" # leave empty to omit sig links
|
||||
gpg-pubkey-url: "/gpg/pubkey.asc"
|
||||
|
||||
nav:
|
||||
- { href: "/", label: "Home" }
|
||||
- { href: "/library.html", label: "Library" }
|
||||
- { href: "/new.html", label: "New" }
|
||||
- { href: "/search.html", label: "Search" }
|
||||
|
||||
portals:
|
||||
- { slug: "writing", name: "Writing" }
|
||||
- { slug: "code", name: "Code" }
|
||||
- { slug: "notes", name: "Notes" }
|
||||
```
|
||||
|
||||
Portals are the library taxonomy. Each portal collects all content whose tags include the portal's slug or any tag with that slug as a prefix. Content tagged `writing/essays` and `writing/fiction` both appear under the `writing` portal.
|
||||
|
||||
---
|
||||
|
||||
## Getting Started
|
||||
|
||||
```sh
|
||||
git clone https://git.levineuwirth.org/neuwirth/ozymandias my-site
|
||||
cd my-site
|
||||
$EDITOR site.yaml # set site-name, site-url, author-name, author-email
|
||||
make dev # build with drafts visible; serve on :8000
|
||||
```
|
||||
|
||||
`make dev` builds with `SITE_ENV=dev` (so drafts are included) and starts a local server. `make build` produces the production output in `_site/`. `make watch` adds incremental rebuilds on file changes.
|
||||
|
||||
For deployment, the included `make deploy` target runs `make clean && make build`, optionally signs each page with [GPG]{.smallcaps}, rsyncs `_site/` to a [VPS]{.smallcaps} configured via `.env`, and pushes to the git remote. Set `VPS_USER`, `VPS_HOST`, and `VPS_PATH` in `.env` to configure the destination.^[The `make deploy` target always begins with `make clean` to avoid stale build artifacts. Incremental Hakyll rebuilds are safe for development but can produce subtly incorrect output — particularly for pages whose template context depends on the full backlink graph — if the dependency graph is not fully consistent. The clean ensures the graph is always recomputed from scratch for production.]
|
||||
|
||||
---
|
||||
|
||||
## Writing Content
|
||||
|
||||
An essay with the full feature set looks like this:
|
||||
|
||||
```yaml
|
||||
---
|
||||
title: "On the Virtues of Careful Writing"
|
||||
date: 2026-04-12
|
||||
abstract: >
|
||||
A brief description that appears on index pages and in the epistemic header.
|
||||
tags: [writing, research/rhetoric]
|
||||
authors: ["Your Name", "Collaborator | https://example.com"]
|
||||
affiliation: "Institution | https://institution.edu"
|
||||
|
||||
status: "Working model"
|
||||
confidence: 65
|
||||
importance: 4
|
||||
evidence: 3
|
||||
scope: average
|
||||
novelty: moderate
|
||||
practicality: high
|
||||
confidence-history: [50, 65]
|
||||
|
||||
history:
|
||||
- date: "2026-04-12"
|
||||
note: Initial draft
|
||||
|
||||
bibliography: data/bibliography.bib
|
||||
further-reading: [key1, key2]
|
||||
---
|
||||
|
||||
::: dropcap
|
||||
Opening paragraph here. Sidenotes use the standard Pandoc footnote syntax.^[Like this.]
|
||||
:::
|
||||
|
||||
## First Section
|
||||
|
||||
Wikilinks to other pages: [[About This Site]]. External links work normally.
|
||||
Citations use Pandoc's citeproc syntax: [@author2024].
|
||||
```
|
||||
|
||||
The `authors` field defaults to the `author-name` in `site.yaml` when absent. The `affiliation` field takes a `Name | URL` format. The `history:` block overrides git-derived version history when the git log alone would not convey what changed.
|
||||
|
||||
---
|
||||
|
||||
## License
|
||||
|
||||
The framework code — everything in `build/`, `templates/`, `static/`, `tools/`, and the configuration files — is [MIT]{.smallcaps} licensed. The demo content under `content/` is public domain. Your content is yours; add whatever license you choose.
|
||||
|
||||
The [MIT]{.smallcaps} license was chosen deliberately: it imposes no obligations, carries no viral clauses, and makes no claims on the writing produced with it. Frameworks should not take a stake in the work they compile.
|
||||
|
||||
---
|
||||
|
||||
## The Relationship Between Ozymandias and This Site
|
||||
|
||||
This site is Ozymandias with my configuration and my content. Changes flow in both directions, with the understanding that the framework is the more conservative of the two repositories: features that turn out to be site-specific stay in levineuwirth.org; features that generalize get ported to Ozymandias. The filter pipeline and the template system are intended to stay in sync.
|
||||
|
||||
The divergence is, in a sense, the point. A personal website is a *position*, as I elaborate upon in the [[Colophon]]. Ozymandias is the mechanism; the position is what you put in it.
|
||||
|
|
@ -0,0 +1,32 @@
|
|||
---
|
||||
title: "Speculative Reluctance"
|
||||
date: 2026-04-15
|
||||
abstract: >
|
||||
AI labs are likely deliberately reluctant to scale because they are aware that any imminient shift to locally run models as the norm would render their compute redundant. We take Anthropic as a principal case study to validate this hypothesis.
|
||||
tags:
|
||||
- ai
|
||||
- tech
|
||||
- speculative
|
||||
- open
|
||||
status: "Draft"
|
||||
confidence: 55
|
||||
importance: 3
|
||||
evidence: 1
|
||||
scope: broad
|
||||
novelty: moderate
|
||||
practicality: high
|
||||
---
|
||||
|
||||
Running a lab that develops frontier LLMs is somewhat like playing a game that, by all measurable metrics external, you are bound to lose. The amount of compute required to train a frontier LLM is unbelievably expensive. The expense of inference is even more astronomical. OpenAI claims at the time of this writing to have somewhere between 900 Million and 1 Billion active users, all of whom require some amount of inference cost, and some small subset of whom consume an enormous amount of compute - to use their words, this is ["commercial scale."](https://openai.com/index/accelerating-the-next-phase-ai/). This isn't to mention the immense amount of competition - there are many major players in the United States alone contributing models that push the boundaries. OpenAI may have been the first, but Anthropic, Google, Meta, xAI, and, yes, even Amazon and Bytedance are following right along.
|
||||
|
||||
Then there's the news that the stock market doesn't want to hear. Ask yourself: who is deliberately left off the above list? If you're thinking of models like GLM, Qwen, MiniMax, and the notorious Deepseek, then we're on the same page. These models are rapidly approaching the capabilities of the frontier models that remain behind intrusive "competitive moats"^[This phrasing is adopted from Jared James Grogan's 2026 paper ["The End of the Foundation Model Era](https://arxiv.org/abs/2604.06217)] that do little more than violate the rights of their users. The advantages that such models provide are immense, and labs of the first list cannot ignore the likelihood of their precedence increasing in the weeks and months to come. In fact, I hypothesize that we are already seeing the reaction of frontier labs to these increasing capabilities, through the lense of juxtaposition: the jargon has remained constant, as if to negate any possibility of an "AI Bubble" bursting, but the quiet actions of the companies that aren't notoriously announced and decreed have shifted.
|
||||
|
||||
## The Dilemma
|
||||
### Inference is the Name of the Game
|
||||
Very few users of an LLM have ever attempted to train an LLM. Even those users who are technical powerhouses - and there are many of these^[Per OpenAI's account, Codex [has reached](https://openai.com/index/accelerating-the-next-phase-ai/) 2,000,000 active weekly users, and while I could not find any specific numbers that Anthropic has released regarding Claude Code's weekly user count, I presume it is higher than that of Codex.] -
|
||||
likely are not intricately familiar with the inner workings of transformers. Even those who, perhaps from coursework, perhaps from curiosity, perhaps from [a chat](https://claude.ai/share/5282e1b8-24ce-4cf8-983e-55df95f5fbdc) with an LLM of choice have enough technical prowess to in theory write code that could facilitate the training of a naive transformer are unlikely to be able to train any model of substance, due to computational constraints. Consider, for instance, that [over 200,000 GPUs](https://x.ai/news/grok-3) were used to train Grok 3, which is a model from early 2025; the [aspirations of xAI](https://www.spacex.com/updates#xai-joins-spacex) in particular with regards to expansion of compute (into outer space) have, more recently, been the source of much controversy. To be absolutely precise, the inherent computational cost of training a model does not provide companies that do train models any safeguards nor guarantees that users cannot find more open alternatives.
|
||||
|
||||
Inference is the primary concern for multiple reasons. Inference is what creates the opportunity for an AI lab to generate revenue. Training a model, in principle, enables the capability for inference to be provided as a service to paying users, but there is no inherent revenue that is generated as a direct consequence of the training pipeline. Inference is also the primary logistical and computational concern. We have neglected in our previous discussion of training that training clusters may be provisioned; powerful GPUs are available to rent by the hour, and though doing this at the scale of training a frontier LLM is economically out of reach for the general population, for venture-capital backed startups, cash is abundantly available as a resource to burn. Inference, on the other hand, is not provisional; to provide inference at a scale that enables revenue, GPUs must be available to serve the requests of paying customers at all times. This is often not the case, as we will soon explore^[A detailed analysis of how even minute per-request inference costs scale to unfathomable overall costs is provided in CMU's ["Agents of Change."](https://www.cmu.edu/cmist/tech-and-policy/agents-of-change/index.html)].
|
||||
|
||||
## A deficit of compute
|
||||
We are already seeing the extensive effects of the fact that inference cannot truly be provisioned at scale. Inference can be provisioned at smaller scales - indeed, as a student at Brown University, I make extensive use of our own [self-hosted interface](https://docs.ccv.brown.edu/ai-tools/services/librechat), which provides access to various frontier LLMs.
|
||||
Some files were not shown because too many files have changed in this diff Show More
Loading…
Reference in New Issue