feat(lean4): the Unicode input method (Arc 8 Stage 4b)

Typing `\alpha` in a Lean 4 buffer gives `α`; `\<>` gives `⟨⟩` with the
point between them. The abbreviation table is vendored from
vscode-lean4 and the expander is a typed-edit consumer registered on
the Stage 4a chain at priority 50, ahead of auto-pairing.

The ordering is load-bearing. 64 abbreviation keys contain a character
in the `lean4` pair set, so with pairing first, typing `\[` would
insert `[]` and corrupt the pending key to `\[]` before the second `[`
arrives — `\[[]]` becomes unreachable. The consumer therefore claims
every keystroke that EXTENDS a pending abbreviation, not only one that
completes an expansion; claiming only completions would hand each
intermediate `[` to pairing by a different route.

The vendored table is an ORDERED SEQUENCE, not a map. Upstream breaks
equal-length ties by source declaration order — 101 prefixes depend on
it, and `\f` resolves through `f<` rather than `f>` — which a
`pairs`-iterated Lua table cannot express. `scripts/regen-lean-abbrev`
takes a vscode-lean4 commit, emits the file with its provenance header,
and aborts on a duplicate key, invalid UTF-8, or a round-trip mismatch.

Undo is cross-peer-degraded on CRDT frontends and that is accepted and
named, not papered over (Q#LN21): `\alpha` arrives as six source-peer
optimistic inserts while the expansion is one daemon-peer replace.
`set_round_trip_input` would fix it and also makes `dispatch_idle`
report false, so RET would stop inserting a newline.

Round 9 corrects three approved acceptance criteria that the real table
contradicts, found by simulating the state machine over all 1,855
entries and re-reading upstream at the pinned commit rather than
re-reading the prose. `\to` is not eager — `top`, `to0` and `toa`
extend it. `\zzzz` expands to `ζzzz ` because `ze`, `zeta` and
`zsqrtd` exist; only `$ % , ; @ W` open no key at all. And `\alpha`'s
undo does not restore `\alpha ` because `alpha` IS eager, so the
terminator is a separate edit. Criteria 38, 41 and 42 now state both
paths, and the false halves are asserted too: they read as correct
until the table is consulted.

Three implementation traps worth the record. The generator's own
round-trip check was broken twice and failed closed both times:
`str.splitlines()` splits on U+2028, which 53 symbols contain, and
escaping through `chr(byte)` produced a latin-1-shaped string that the
UTF-8 write re-encoded. The first check compared in-memory strings and
agreed with itself; it now stages the file, re-reads the bytes from
disk, and renames into place only on a match. And the expansion SHRINKS
the buffer, so the point must be placed explicitly — pairing's
no-cursor-motion rule holds only for an insert AT the cursor, and
without this every self-insert after the first expansion is silently
rejected and the editor looks dead.

25 acceptance tests plus one `--lib` test for the optimistic CRDT
producer (45f), which is where the gate list's `--features crdt` run
reaches it; a crdt-gated integration test would be dark in CI and in
the gates both. Fifteen mutations bite, each failing its target. Three
of these tests were vacuous when first written and biting is what
found them: the abandonment test asserted text a surviving record
would also produce, the re-arm test used an example that never reaches
the re-arm branch, and both switch tests ran through
`find_or_open`'s fresh-load path rather than `buffer.after-switch`.

No protocol change (Q#LN14). Also reconciles the handoff and ledger
for Stage 4a (#179) and adds `lean.abbrev` to COHERENCE.md's
config-registry adoption census, now nine settings.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011LFvC4FQtux4y32KuevZ7B
This commit is contained in:
Levi Neuwirth 2026-07-26 16:24:06 -04:00
parent a27f6467ea
commit a53965474d
10 changed files with 3521 additions and 180 deletions

View File

@ -1022,12 +1022,15 @@ layering, provenance, and adoption have not followed.**
`ConfigValue`s; `describe-setting`'s "Source:" names where `define()`
ran. The inspection view sketched above is currently impossible to
render.
- **Adoption is eight settings**: `editing.auto-pair` (pair.lua),
- **Adoption is nine settings**: `editing.auto-pair` (pair.lua),
`editing.trim-on-save` (editops.lua), `autosave.interval-ms`
(autosave.lua), `window.panel-height` + `window.min-height`
(window.lua), and `terminal.default-profile` +
(window.lua), `terminal.default-profile` +
`terminal.scrollback-rows` + `terminal.escape-key` (terminal.lua,
#173). Everything else a user might set — theme, fonts, LSP
#173), and `lean.abbrev` (lean_input.lua, Arc 8 Stage 4b) — a
`live` boolean read against the typed edit's SOURCE buffer, the
`editing.auto-pair` shape including its correction to resolve
`rec.buffer` rather than the active buffer. Everything else a user might set — theme, fonts, LSP
server config, killring size, recentf/saveplace/desktop enables,
pair sets, comment strings, `pmacs.parse.*` — lives in raw Lua
outside the registry and is therefore invisible to `describe-setting`

File diff suppressed because it is too large Load Diff

View File

@ -0,0 +1,362 @@
-- lean_input.lua --- the Lean 4 Unicode input method (Arc 8 Stage 4b).
--
-- Typing `\alpha` gives `α`; `\<>` gives `⟨⟩` with the point between.
-- The table is vendored in lean_abbrev.lua, generated from
-- vscode-lean4 — see that file's header and Q#LN11.
--
-- This is a typed-edit consumer (Stage 4a, Q#LN10) registered AHEAD of
-- auto-pairing at priority 50. The ordering is load-bearing, not
-- cosmetic: 64 abbreviation keys contain a character in the `lean4`
-- pair set (`\[[]]` → `⟦⟧`, `\{{}}` → `⦃⦄`), so with pairing first,
-- typing `\[` would insert `[]` with the point between and corrupt the
-- pending key to `\[]` before the second `[` arrives — `\[[]]` becomes
-- unreachable. Priority, not load order, is what decides this; that is
-- the whole reason Stage 4a exists.
--
-- The consumer therefore claims every keystroke that EXTENDS an open
-- pending abbreviation, not merely one that completes an expansion. A
-- consumer that claimed only completed expansions would hand each
-- intermediate `[` to pairing, which is the same corruption by a
-- different route. "Claimed" means the chain stops, not that an edit
-- was made (Q#LN22).
--
-- UNDO IS CROSS-PEER-DEGRADED, and this is accepted rather than papered
-- over (Q#LN21). `classify_key` (src/optimistic.rs) returns `Insert(c)`
-- for `\` and for every ASCII letter — only the nine built-in pair
-- chars are excluded — so on a CRDT frontend `\alpha` arrives as six
-- SOURCE-peer optimistic inserts while the expansion is a single
-- DAEMON-peer replace spanning all six. Undo across that boundary is
-- not chronologically arbitrated. This is the same defect Q#LN6 already
-- accepts for `⟨⟩`, one order of magnitude wider: it is every
-- abbreviation the user types, not a few brackets. The general fix is
-- chronological cross-peer undo arbitration, named substrate work.
-- `set_round_trip_input` would fix it and is rejected — it also makes
-- `dispatch_idle` report false, so RET would stop inserting a newline.
--
-- Framing: docs/lean4-mode-framing.md Q#LN11, Q#LN21, Q#LN22.
pmacs.lean_input = pmacs.lean_input or {}
local ed = pmacs.editor
local LEADER = "\\"
local CURSOR = "$CURSOR"
pmacs.config.define {
name = "lean.abbrev",
description = "Expand \\-prefixed abbreviations into Unicode symbols in Lean 4 buffers.",
type = "boolean",
default = true,
mutability = "live",
}
-- ---------------------------------------------------------------------
-- The table, and the two indexes derived from it at load time
-- ---------------------------------------------------------------------
-- `best[p]` is the symbol for the shortest key having `p` as a prefix,
-- ties broken by the key's position in the vendored sequence. Both
-- halves matter: 101 prefixes have equal-shortest candidates that
-- resolve to DIFFERENT symbols (`f` → `` from `f<`, not `` from
-- `f>`), and the sequence's order is the only place that tie is
-- recorded. `pairs` over a map-shaped table could not express it.
--
-- `eager[k]` marks the 1,550 keys that are complete and have no longer
-- key extending them — the ones that expand the moment they are typed,
-- with no terminator. `to` is NOT one of them (`top`, `to0`, `toa`),
-- which is exactly the case that reads as eager until the table is
-- consulted.
local best, eager = {}, {}
do
local seq = pmacs.lean_abbrev
if type(seq) ~= "table" then seq = {} end
local extended = {}
for i = 1, #seq do
local entry = seq[i]
local key, symbol = entry[1], entry[2]
-- Walk every prefix of the key, including the key itself. Iterating
-- the sequence in order and only overwriting on a STRICTLY shorter
-- key is what makes the source-order tiebreak fall out: an equal
-- length arriving later loses to the one already recorded.
for n = 1, #key do
local p = key:sub(1, n)
local cur = best[p]
if cur == nil or #key < cur.len then
best[p] = { symbol = symbol, len = #key }
end
if n < #key then extended[p] = true end
end
end
for i = 1, #seq do
local key = seq[i][1]
if not extended[key] then eager[key] = true end
end
end
-- Test seam (leading underscore = not stable API). Acceptance 45g reads
-- these to pin self-consistency properties a corrupt emit would break —
-- it cannot diff against `abbreviations.json`, which is not shipped.
function pmacs.lean_input._resolve(text)
local hit = best[text]
return hit and hit.symbol or nil
end
function pmacs.lean_input._is_eager(key)
return eager[key] == true
end
-- ---------------------------------------------------------------------
-- Pending state: one record per FRONTEND (Q#LN22)
-- ---------------------------------------------------------------------
-- Keyed by frontend id, with the buffer stored inside and compared by
-- value. Q#LN22 specifies the key as `(frontend, buffer)`; a per-
-- frontend slot is equivalent here and avoids inventing a scalar
-- buffer key (`BufferId`'s inner value is deliberately private, R22).
-- The generality a two-level map would add is unreachable: a frontend
-- has one point, and `buffer.after-switch` clears that frontend's slot,
-- so no frontend can hold pending state in a buffer it is not in.
--
-- Per-frontend rather than per-buffer is NOT a refinement — a buffer-
-- keyed table lets either frontend consume or discard the other's
-- half-typed abbreviation in a shared buffer, which is the ordinary
-- TUI-plus-GPU configuration this project ships.
local pending = {}
local function frontend_id()
local ok, id = pcall(function() return pmacs.frontend.id() end)
if ok then return id end
return nil
end
-- Is `rec` a typed edit that continues `p` exactly? Conservative by
-- construction (Q#LN22): abandonment is LAZY because pmacs has no
-- cursor-motion hook, so every guard that would have been checked at
-- the moment the user left is checked here instead, at the next typed
-- edit.
local function still_valid(p, rec, buf)
if p.buffer ~= rec.buffer or p.window ~= rec.window then return false end
-- The point must still be at the end of the pending span: the leader,
-- plus what has been typed into it, plus the character that just
-- landed.
if rec.effective_start ~= p.start_offset + 1 + #p.text then return false end
-- Exactly one edit since this frontend last extended the pending
-- abbreviation — the one being processed now. Deliberately strict
-- across frontends: `revision()` is BUFFER-GLOBAL, so a peer editing
-- the shared buffer invalidates this record even though it edited
-- elsewhere. Keeping it alive would mean translating and validating
-- the span through arbitrary peer edits, substrate Stage 4b does not
-- add.
local ok, rev = pcall(function() return buf:revision() end)
if not ok or rev ~= p.expected_revision + 1 then return false end
return true
end
-- ---------------------------------------------------------------------
-- Expansion
-- ---------------------------------------------------------------------
-- Replace the pending span with `symbol`, placing the point at
-- `$CURSOR` if the symbol carries one. Returns the byte offset just
-- past the replacement, or nil when the edit was rejected or altered.
--
-- ONE `buf:replace` for the whole expansion: one undo step, one CRDT
-- op, one effective-edit verification. A rejection drops the pending
-- state and does not retry, the same discipline as comment.lua's Q#CT5
-- and pair.lua.
local function expand(buf, p, symbol, span_end)
local cursor_at = symbol:find(CURSOR, 1, true)
local text = cursor_at and (symbol:gsub("%$CURSOR", "", 1)) or symbol
local start = p.start_offset
local ok, estart, estop, einserted = pcall(function()
return buf:replace(start, span_end, text)
end)
if not ok then
ed.set_status("lean abbreviation rejected by buffer intercept")
return nil
end
if estart ~= start or estop ~= span_end or einserted ~= #text then
ed.set_status("lean abbreviation altered by buffer intercept")
return nil
end
-- The point MUST be placed explicitly. Unlike pairing's at-cursor
-- insert, this replace SHRINKS the buffer — `\alpha` (6 bytes)
-- becomes `α` (2) — and a point left at the pre-edit offset is past
-- the new end. Every later self-insert is then silently rejected and
-- the editor looks dead. There is no daemon re-grounding that covers
-- this; that only holds for an edit that lands at the cursor.
ed.goto_byte(cursor_at and (start + cursor_at - 1) or (start + #text))
return start + #text
end
-- ---------------------------------------------------------------------
-- The consumer
-- ---------------------------------------------------------------------
local function on_typed_edit(rec)
local fid = frontend_id()
if fid == nil then return false end
-- A fan-out carrying no record is still information: a paste,
-- programmatic edit or replicated op landed, so whatever this
-- frontend had pending no longer describes the buffer. Drop it and
-- decline — this is why the chain calls consumers with nil rather
-- than skipping them (Q#LN10).
if not rec then
pending[fid] = nil
return false
end
if not (ed.this_command and ed.this_command() == "buffer.self-insert") then
pending[fid] = nil
return false
end
-- Both gates resolve against the SOURCE buffer of the typed edit, not
-- the active one — a context-switching command may have replaced it
-- by callback time (pair.lua round 2, finding 2).
if not pmacs.config.get("lean.abbrev", rec.buffer) then
pending[fid] = nil
return false
end
local lang
if pmacs.lsp and pmacs.lsp.buffer_language then
local ok, l = pcall(pmacs.lsp.buffer_language, rec.buffer)
if ok then lang = l end
end
if lang ~= "lean4" then
-- No pending abbreviation is ever OPENED outside a `lean4` buffer:
-- `\` in Rust is an ordinary character and `\[` there still pairs.
pending[fid] = nil
return false
end
local buf = pmacs.window.buffer()
if not buf or buf ~= rec.buffer or pmacs.window.current() ~= rec.window then
pending[fid] = nil
return false
end
-- Fail closed on a transformed source self-insert, as pairing does:
-- expanding on top of a relocated or rewritten character compounds
-- the intercept's result.
if not rec.clean then
pending[fid] = nil
return false
end
local revision
do
local ok, rev = pcall(function() return buf:revision() end)
if not ok then
pending[fid] = nil
return false
end
revision = rev
end
local p = pending[fid]
if p and not still_valid(p, rec, buf) then
p = nil
pending[fid] = nil
end
local ch = rec.char
-- No pending abbreviation: only the leader opens one.
if not p then
if ch == LEADER then
pending[fid] = {
buffer = rec.buffer,
window = rec.window,
start_offset = rec.effective_start,
text = "",
expected_revision = revision,
}
-- Claimed: the leader belongs to the abbreviation, and pairing
-- has no interest in it either way.
return true
end
return false
end
-- Pending: does any key still have `text .. ch` as a prefix?
local extended = p.text .. ch
if best[extended] then
p.text = extended
p.expected_revision = revision
if eager[extended] then
local span_end = p.start_offset + 1 + #extended
pending[fid] = nil
expand(buf, p, best[extended].symbol, span_end)
end
-- Claimed either way: an extension that has not yet completed must
-- NOT reach auto-pairing (`\[` in `\[[]]`).
return true
end
-- `ch` does not extend the abbreviation. Expand what is pending
-- FIRST, then let `ch` stand as ordinary text — the terminator is
-- retained, not consumed, and it sits inside the replaced span so the
-- whole thing is one undo step.
pending[fid] = nil
local hit = best[p.text]
local after
if hit and #p.text > 0 then
-- `span_end` covers the terminator: the leader, the pending text,
-- and `ch`, which has already landed. What replaces it is the
-- symbol followed by `ch` itself.
local span_end = p.start_offset + 1 + #p.text + #ch
after = expand(buf, p, hit.symbol .. ch, span_end)
end
-- A terminating `\` re-arms as a NEW leader at its own position
-- (`\alpha\to` → `α→`). Upstream gets this from `processChange`,
-- where a finished abbreviation reports `isAffected = false` and so
-- does not suppress the new-leader branch. This is not the `\\` case:
-- there the pending text is empty, `\` EXTENDS, and the result is one
-- literal backslash with no pending state left open.
if ch == LEADER then
local start = after and (after - #ch) or rec.effective_start
local ok, rev = pcall(function() return buf:revision() end)
if ok then
pending[fid] = {
buffer = rec.buffer,
window = rec.window,
start_offset = start,
text = "",
expected_revision = rev,
}
end
return true
end
-- Claimed only if an expansion actually happened. Otherwise `ch` is
-- an ordinary character in a Lean buffer and auto-pairing should see
-- it — `\zz` leaves `z` free to pair if it ever were a pair char.
return after ~= nil
end
-- Q#KR11's seam: a detached frontend's pending state must not outlive
-- it. Ids are monotonic, so this table would otherwise grow for the
-- life of the session.
pmacs.hook.add("frontend.detached", function(fid)
pending[fid] = nil
end)
-- `buffer.after-switch` fires with NO arguments, so it cannot say whose
-- switch it was. The acting frontend is the one that produced the most
-- recent dispatched input event, which is what `pmacs.frontend.id()`
-- reports at callback time. Clearing every entry instead would let one
-- frontend's navigation discard another's half-typed abbreviation.
pmacs.hook.add("buffer.after-switch", function()
local fid = frontend_id()
if fid ~= nil then pending[fid] = nil end
end)
pmacs.typed_edit.add_consumer {
name = "lean-abbrev",
priority = 50,
fn = on_typed_edit,
}

View File

@ -14,8 +14,8 @@ backlog.
machine-local: `origin` may name this canonical URL, a release mirror,
or something else, and therefore has no authority by name alone.
- Canonical base at this snapshot:
`githubsucks/main` @ `d400f30` (Lean 4 Stage 3b #170 atop Stage 3a
#167, the bottom-panel landed-doc refresh #156, the inline-math slice
`githubsucks/main` @ `a27f646` (Lean 4 Stage 4a #179 atop Stage 3b
#170, Stage 3a #167, the bottom-panel landed-doc refresh #156, the inline-math slice
#158, dired Stage 1 #165, the GPU terminal input fix #166, Lean 4
Stage 2 #161, the dired framing #164, COHERENCE.md #163, find-file
#162, Lean 4 Stage 1 #160, and the minimap blank-slab fix #159;
@ -57,172 +57,93 @@ git status --short --branch
The `git log` command must expose `d152120` or a newer intentional main.
If it does not, stop and repair the remote/fetch configuration.
## Lean 4 lane (Arc 8) — Stages 1, 2, 3a, 3b MERGED; Stage 4a IN REVIEW
## Lean 4 lane (Arc 8) — Stages 14a MERGED; Stage 4b IN REVIEW
- **Stages 1, 2, 3a and 3b are MERGED**#160 (`main` @ `0827dd1`),
#161 (`46a1b8f`), #167 (`6f348c9`), #170 (`d400f30`). Their full
- **Stages 1, 2, 3a, 3b and 4a are MERGED**#160 (`main` @ `0827dd1`),
#161 (`46a1b8f`), #167 (`6f348c9`), #170 (`d400f30`), #179
(`a27f646`). Their full
histories were pruned from this ledger in round 6, per this file's own
instruction to remove entries when their PR merges; the durable facts
now live in `docs/agent-handoff.md` §1's Lean 4 bullet, which is where
a fresh machine should read them. `docs/lean4-mode-framing.md` rev 8
a fresh machine should read them. `docs/lean4-mode-framing.md` rev 9
carries the decisions.
### Stage 4 — framing rev 8, split into 4a/4b (branch `lean4-stage4a-typed-edit-chain`)
### Stage 4b — the Unicode input method (branch `lean4-stage4b-input-method`)
- Stages 3a and 3b **merged as #167** (`main` @ `6f348c9`) and **#170**
(`main` @ `d400f30`), 2026-07-26. Both were integrated against a main
that had advanced 50 commits mid-review; the only conflict either time
was this ledger's own lane headings, resolved by keeping both sides.
- Worktree `../pmacs-lean-stage4`, branched off `main` @ `d400f30`.
Framing-only so far: `docs/lean4-mode-framing.md` **revision 8**. No
code. Awaiting user approval before implementation, per the workflow.
- **Round 6 review found five P1s, four of them internal to rev 6**
facts about pmacs the revision asserted without checking, while its
external (upstream) facts held. Fixed in rev 7: Stage 4a's footprint
omitted the test file its own acceptance requires; pending
abbreviation state was keyed by buffer when pmacs is **multi-frontend**
(`EditorCore.views` is per-`FrontendId`, `take_typed_edit` is already
frontend-keyed, and `buffer.after-switch` fires with NO arguments, so
a buffer-keyed clear lets any frontend discard another's pending
state); the shortest-match rule was missing its **tie-break by source
declaration order**, which 101 prefixes depend on and a `pairs`-
iterated Lua map cannot express; and the generator's "abort on keys
needing escaping" rule **rejects the real table** (`\` is a key, `"`
begins eleven).
- **A 404 on a guessed path is not evidence of absence.** Rev 6 declared
the upstream package ships no README after fetching the package root,
with the directory listing showing `src/README.md` already in hand.
The README states the tie rule in one sentence.
- **Round 7 review found one remaining P1 in acceptance 45i.** Rev 7
required A's pending abbreviation to survive B editing the same
buffer, while Q#LN22 also required an exact buffer-revision advance.
Those cannot both hold: revisions are buffer-global and every edit
bumps them. Rev 8 keeps the conservative guard and separates
ownership from survival — B cannot consume A's record, but B editing
the shared buffer invalidates A lazily; B switching buffers or
detaching remains frontend-scoped when no shared-buffer edit
intervenes.
- **Round 5 re-scout split Stage 4 into 4a (substrate) and 4b (Lean).**
4a is the typed-edit consumer chain — `builtin/runtime/typed_edit.lua`
plus `pair.lua` re-expressed as one registered consumer, no behavior
change. 4b is the input method. The split is forced by §4's own rule,
which Stage 4's risk column ("refactors `pair.lua`'s provenance read")
broke while the prose called the stage Lean-only.
- **This is the SECOND consecutive re-scout to find that rule broken**
(round 4 found it for Stage 3). Rev 5 had even noticed the shape and
answered it with a commit boundary. **A commit boundary is not a review
boundary.** Re-check every remaining stage against §4 at scout time;
the rule is not self-enforcing.
- **Rev 5's expansion semantics were wrong in three ways**, found by
reading `leanprover/vscode-lean4` @ `17d1d08` rather than inferring
from behavior. Resolution is *shortest key having the input as a
prefix* (`\al` → `∀` from `all`, not `alpha`); there is **no
terminator list** (`'+ '` is a key, so space extends after `\+`; `'\'`
is a key, so `\\``\`); and an unmatchable tail is **appended**,
not dropped (`\alp7` → `α7`).
- **There is no cursor-motion hook**, so rev 5's acceptance 43 ("moving
the cursor out abandons it") was not buildable. Abandonment is lazy —
validated at the next typed edit — and the criterion now asserts what
pmacs can actually detect. Upstream drives this off `changeSelections`;
that seam does not exist here.
- **`dispatch_key` is only half the production path for 4b.** The
auto-pair suite gets away with dispatch-only because Q#AP1 removed the
pair chars from the optimistic classifiers; `\` and the letters are
NOT excluded, so on a CRDT frontend the optimistic producer is the real
path. That producer is `#[cfg(feature = "crdt")]` and CI never enables
`crdt`, and the gate list runs `--features crdt` only for `--lib` — a
crdt-gated integration test is **dark twice over**.
- The whole expansion has cross-peer-degraded undo (Q#LN21): six
source-peer optimistic inserts replaced by one daemon-peer op.
`set_round_trip_input` would fix it and is rejected — it also disables
`dispatch_idle`, so RET stops inserting a newline.
- Table facts re-derived at `17d1d08`: 1,855 entries, 36,861 bytes, all
keys ASCII, **64** keys carry a `lean4` pair-set char, **305** keys are
proper prefixes of another (so 1,550 expand eagerly), **26** values
carry `$CURSOR`, and **119** are multi-codepoint — the 26
`$CURSOR`-bearing values plus 93 others.
- Citation sweep per COHERENCE §25: five live citations moved in the 50
commits since rev 5 — `take_typed_edit` 12827→12990,
`handle_server_requests` 1549→1815, `fs.stat` 93→133,
`detect_buffer_language` 452→457, `send_request`/`send_notification`
9342/9361→9507/9527.
### Stage 4a — the typed-edit consumer chain (IMPLEMENTED, same branch)
- Footprint exactly as Q#LN10 declares it: `builtin/runtime/typed_edit.lua`
(new), `pair.lua` re-expressed as one consumer,
`src/editor.rs` +15 (the `include_str!` and its ordering comment), and
`tests/typed_edit_chain_acceptance.rs` (new, 13 tests).
**`tests/auto_pair_acceptance.rs` is UNCHANGED — `git diff --stat
main...HEAD -- tests/auto_pair_acceptance.rs` is empty.** That is
criterion 46 checked at the diff, which is the only way it means
anything.
- **The chain calls consumers even when the record is nil.** This is a
decision, not an implementation detail: three existing auto-pairing
tests assert `pmacs.pair._last_record == nil` after a record-less
fan-out (paste, programmatic insert, nested manual `hook.run`), so
skipping consumers on nil fails them. Stage 4b needs the same
delivery to abandon a pending abbreviation an unrelated edit
invalidated.
- **Ordered insertion, not `table.sort`** — Lua's sort is not stable, and
"ties broken by registration order" is a stated contract.
- **The chain `pcall`s each consumer** and reports through
`set_status`. Rev 7 justified this by claiming an uncontained throw
would fail the fan-out for every other subscriber including lsp.lua's
didChange flush; **that is wrong**`run_all_must_succeed`
(`src/hook.rs:332`) collects errors and continues, so the other
subscribers still run. The real consequence is narrower and still
worth containing: the throw skips every LATER consumer in the chain.
The rendering is protected too, because a Lua error may be a table
whose `__tostring` throws.
- **Round 8 (review) findings, all fixed on this branch:** each consumer
now gets its **own shallow copy** of the record (the same table let a
declining consumer rewrite `rec.char`, which pairing reads — typing
`x` could produce `x)`); the fan-out iterates a **snapshot** (a
consumer registering a lower-priority one shifted itself forward under
`ipairs` and ran twice, unbounded if repeated); `tostring` moved
inside the containment; **non-finite and non-integer priorities are
rejected** (NaN is a number and every ordered comparison with it is
false, so it landed wherever the insertion scan gave up and silently
voided the ordering contract); and `add_consumer` now returns a handle
with `remove_consumer` beside it, so re-evaluating a config no longer
leaks callbacks the way `pmacs.hook.add` does (COHERENCE §13).
- **Every acceptance test is bite-verified by mutation**, per the
standing rule that a test is not evidence until the mutation it
targets has been shown to fail it:
- Framing `docs/lean4-mode-framing.md` **revision 9**, approved. Stage
4a (the typed-edit consumer chain) MERGED as #179; this branch is 4b,
the Lean content that registers on it.
- Footprint: `scripts/regen-lean-abbrev` (new, the generator),
`builtin/runtime/lean_abbrev.lua` (new, VENDORED DATA — 1,855 entries
from `leanprover/vscode-lean4@17d1d08`, Apache-2.0),
`builtin/runtime/lean_input.lua` (new, the consumer at priority 50),
`src/editor.rs` (two `include_str!` blocks),
`tests/lean_input_acceptance.rs` (new, 25 tests), and one
`#[cfg(feature = "crdt")]` `--lib` test in `src/daemon.rs`
(acceptance 45f). No protocol change (Q#LN14). Entirely Lua apart
from the load sites and that one test.
- **Round 9 corrected three acceptance criteria that the real table
contradicts** — found by simulating the state machine over all 1,855
entries and re-reading upstream at the pinned commit, not by reading
the prose again. `\to` is NOT eager (`top`, `to0`, `toa` extend it);
`\zzzz` expands to `ζzzz ` because `ze`/`zeta`/`zsqrtd` exist, and
only `$ % , ; @ W` open no key at all; and `\alpha`'s undo does not
restore `\alpha ` because `alpha` IS eager, so the terminator is a
separate edit. Criteria 38, 41 and 42 now state both paths.
- **Two generator bugs, both caught by its own round-trip check
failing closed:** `str.splitlines()` also splits on U+2028/U+2029,
and 53 symbols contain one literally, so the check reported a count
mismatch that was its own bug; then escaping via `chr(byte)` produced
a latin-1-shaped string that `write_text(encoding="utf-8")`
re-encoded, and every non-ASCII symbol landed double-encoded. The
first version of the check compared IN-MEMORY strings and agreed with
itself. **It now stages the file, re-reads the bytes from disk, and
renames into place only on a match.**
- **The point must be placed explicitly after the replace.** The
expansion SHRINKS the buffer (`\alpha` 6 bytes → `α` 2), so a point
left at the pre-edit offset is past the new end and every later
self-insert is silently rejected — the editor looks dead after the
first expansion. Pairing's "no cursor motion on the clean path" does
not generalize: that holds only for an insert AT the cursor.
- **Three tests were vacuous when first written and were found by
biting, not by review:** the abandonment test asserted text that a
wrongly-surviving record would also produce (claiming makes no edit —
it needed the follow-up keystroke that completes an eager key); the
re-arm test used the framing's own `\alpha\to`, which never reaches
the re-arm branch because `alpha` is eager and closes the record
first (`\al\to` does); and both buffer-switch tests passed through
`find_or_open`'s fresh-load path, which fires `buffer.after-load` and
a record-less edit rather than `buffer.after-switch` — deleting the
subscriber left them green. All three now bite.
- **Bite table** (each mutation, and the tests it fails):
| Mutation | Tests it fails |
|---|---|
| append instead of ordered insert | 5 chain |
| `>=` instead of `>` in the insert scan | 1 chain (tiebreak) |
| re-take the record per consumer | 4 chain |
| ignore the claim return value | 1 chain |
| drop the `pcall` | 1 chain |
| skip consumers when `rec == nil` | 1 chain + **3 auto-pair** |
| load `typed_edit.lua` after `lsp.lua` | 1 chain + **2 auto-pair** (Q#AP7) |
| hand every consumer the same record table | 1 chain (46f) |
| iterate the live array instead of a snapshot | 1 chain (46g) |
| render the error outside the `pcall` | 1 chain (46d) |
| accept any Lua number as a priority | 1 chain (46h) |
| make `remove_consumer` a no-op | 2 chain (46g, 46h) |
| register at priority 150 (after pairing) | 2 |
| claim only completed expansions | 2 |
| longest match instead of shortest | 9 |
| equal-length tie keeps the LATER key | 3 |
| remove the eager branch | 8 |
| expand without the terminator in the span | 2 |
| remove the re-arm branch | 1 |
| remove the point-still-at-span-end check | 1 |
| remove the exact-revision check | 1 |
| leave the point where the replace found it | 5 |
| remove the `lean4` language gate | 1 |
| remove the `lean.abbrev` gate | 2 |
| `buffer.after-switch` clears every frontend | 1 |
| delete the `buffer.after-switch` subscriber | 1 |
| `frontend.detached` purges every frontend | 1 |
The first attempt at the last bite was WORTHLESS as written: moving
only `typed_edit.lua` past `lsp.lua` left `pair.lua` calling a nil
`add_consumer`, so the runtime failed to load and all 9 tests died —
loud, but not a test of the flush-ordering property. Moving
`typed_edit.lua` AND `pair.lua` past `lsp.lua` is the faithful
falsification: registration succeeds, the hook lands late, and exactly
the three ordering tests fail. **A bite that kills everything has not
isolated anything.**
- Verification on this branch (commit-then-gate, so this describes the
pushed tree): `cargo fmt --check` clean; strict workspace Clippy
clean; 1,832 default + 2,009 CRDT library tests; auto-pair 45/45;
typed-edit chain 13/13 (and 13/13 again under `--no-default-features
--features lua54`, since the fixes touch `math.huge`, `%`, and
`__tostring` behavior that differs between the backends); M4 121;
required GPU 202; **isolated-config workspace sweep 3,332 across 97
suites, zero failures** with `grep -c basedpyright` = 0; `git diff
--check` clean.
- Stage 4b (the input method) is NOT in this PR and not started.
Acceptance 45f bit by construction: without a registered window for
the source frontend it ran six fan-outs with a nil record and proved
nothing, because `handle_remote_crdt_op` arms nothing unless the
source's active window displays the buffer.
- Undo is cross-peer-degraded on CRDT frontends and that is ACCEPTED,
named in the module header (Q#LN21): six source-peer optimistic
inserts replaced by one daemon-peer op. `set_round_trip_input` would
fix it and also disables `dispatch_idle`, so RET would stop inserting
a newline.
## Dired lane — Stage 0 MERGED; Stage 1 IN REVIEW (PR #165)

View File

@ -1,7 +1,9 @@
# Agent handoff — cross-machine continuity
**Last updated: 2026-07-26, after Lean 4 stages 3a and 3b (#167, #170)
landed — pmacs' first Lean language server — following the inline-math
**Last updated: 2026-07-26, after Lean 4 Stage 4a (#179) landed — the
typed-edit consumer chain, the substrate the Unicode input method
registers on — atop stages 3a and 3b (#167, #170), pmacs' first Lean
language server, and following the inline-math
slice (#158), the first mathematical typesetting in pmacs, and find-file (#162),
the dired arc's Stage 0, and COHERENCE.md (#163), Lean 4 Stage 1 (#160), the
minimap blank-slab fix (#159), bottom-panel Stage 1 (#155), the
@ -88,9 +90,9 @@ commands, read `docs/active-work.md` immediately after this file.
config swap invalidates. The durable lesson is to heal at
**consumption** — the point where a stale record is handed out — not
at the moment of the swap.
- **Stage 4a (typed-edit consumer chain) is implemented and in review
as PR #179** (branch `lean4-stage4a-typed-edit-chain`, framing rev
8). It is substrate only: `builtin/runtime/typed_edit.lua` owns the
- **Stage 4a (typed-edit consumer chain) MERGED as #179** (`main` @
`a27f646`, two review rounds). It is substrate only:
`builtin/runtime/typed_edit.lua` owns the
single `buffer.after-edit` subscriber and the single one-shot read,
`pair.lua` becomes its first registered consumer, and
`tests/auto_pair_acceptance.rs` is unchanged by zero lines
@ -104,10 +106,36 @@ commands, read `docs/active-work.md` immediately after this file.
iterates a **snapshot**, because a consumer that registers a
lower-priority one shifts itself forward under `ipairs` and runs
twice.
- Remaining: Stage 4b (the Unicode input method) is framed and
awaiting approval — not started; stages 5 (goal panel), 6 (`#eval`
output channel), and 7 (module hierarchy) are framed but not
scouted against current `main`.
- **Round 8's durable lesson: `run_all_must_succeed` does NOT abort
the fan-out.** `src/hook.rs:332` collects each callback's error and
continues to the remaining subscribers, marking only the run
failed — so an uncontained throw inside a hook subscriber does not
stop `lsp.lua` from flushing didChange. Two framing revisions
asserted the opposite to justify a `pcall`. The guard was right and
the reason was wrong, and by the time review caught it the wrong
reason had been copied into a module comment, an acceptance
criterion, a test comment, and the ledger. **Correct the source a
rationale derives from, not only the sites that quote it.**
- **Stage 4b (the Unicode input method) is implemented and in review**
(branch `lean4-stage4b-input-method`, framing rev 9): a vendored
1,855-entry table generated from `leanprover/vscode-lean4@17d1d08`
by `scripts/regen-lean-abbrev`, plus a consumer registered on the
Stage 4a chain at priority 50, ahead of pairing. Its durable facts:
the table must stay an ORDERED SEQUENCE (equal-length ties resolve
by source declaration order, which a `pairs`-iterated map cannot
express); a generator round-trip check must re-read the BYTES ON
DISK, because comparing in-memory strings cannot see an encoding
applied by the write itself; and an expansion that SHRINKS the
buffer must place the point explicitly, or every later self-insert
is silently rejected and the editor looks dead.
- **Round 9 corrected three approved acceptance criteria** by
simulating the state machine over all 1,855 entries rather than
re-reading the prose. Four review rounds over the text had not
found them, because each named an example that reads as obviously
right and is wrong only against the data.
- Remaining: stages 5 (goal panel), 6 (`#eval` output channel), and 7
(module hierarchy) are framed but not scouted against current
`main`.
- **Inline math LANDED — #158** (`docs/inline-math-slice-framing.md` rev 3;
merge `5aa9044`). pmacs renders `$…$` as typeset mathematics in the GPU

View File

@ -46,7 +46,7 @@ during a rebase.
## 0.1 Revision history
Revision 1 — initial. Current revision: **8**.
Revision 1 — initial. Current revision: **9**.
### Round 1 (rev 1 → rev 2)
@ -503,6 +503,38 @@ documentation cleanups.
others — matching §2.11 and Q#LN11.
3. **§9.1's revision label was stale.** It now names rev 8.
### Round 9 (rev 8 → rev 9)
Found during Stage 4b implementation, by simulating Q#LN22's state
machine over all 1,855 vendored entries and re-reading upstream's
`TrackedAbbreviation.ts` and `AbbreviationProvider.ts` at `17d1d08`.
**Three acceptance criteria named examples that the real table
contradicts** — every one of them written from what the abbreviation
*looks* like rather than from whether the table makes it eager.
1. **Acceptance 41 was false.** `\to` does not expand eagerly: `to` is a
proper prefix of `top`, `to0`, `toa` and others, so upstream's
`isAbbreviationUniqueAndComplete` is false and `to` is not among the
1,550 eager keys. The criterion now uses `\alpha`, which has no
extension, and additionally pins that `\to` alone does **not**
expand — the false half is worth an assertion because it reads as
correct until the table is consulted.
2. **Acceptance 42 was false.** `\zzzz` + space yields `ζzzz `, not
literal text: `z` opens a pending abbreviation (`ze`, `zeta`,
`zsqrtd`) and the second `z` finishes it. Exactly six printable
characters open no key — `$ % , ; @ W` — and the criterion now uses
`\WWWW`.
3. **Acceptance 38's undo claim was false for its own example.**
`alpha` is eager, so `\alpha` expands before the space is typed and
the space is a separate edit; one undo removes the space rather than
restoring `\alpha `. The criterion now states the finish path and the
eager path separately, since "one expansion is one undo step" is true
of both while the text an undo restores is not.
The mechanism (Q#LN11, Q#LN21, Q#LN22) needed no change — these were
errors in the examples chosen to pin it, which is why a simulation over
the real data found them and four review rounds over the prose did not.
## 1. What ships
Nine stages, after round 4 split Stage 3 and round 5 split Stage 4. The
@ -2452,12 +2484,22 @@ criterion 46 requires to stay byte-identical.
**Stage 4b — the Unicode input method**
38. `\alpha` + space yields `α ` — the space lands first and the
expansion runs in the following `buffer.after-edit`, so the
terminator is **retained**, not consumed. The expansion is a single
undo step: one undo restores `\alpha ` (with its space), not
`\alph`. Rev 6 wrote the post-undo text as `\alpha`, which would be
true only if the terminator were swallowed.
38. **Terminators are retained, and one expansion is one undo step —
but which text an undo restores depends on the path.** Rev 8 stated
a single rule here and it is wrong against the real table, because
it assumed `\alpha` takes the finish path when `alpha` is in the
1,550-key eager set (round 9; see 41).
- *Finish path.* `\alp` + space yields `α `: the space lands first
and the expansion runs in the following `buffer.after-edit`, so
the terminator is **retained**, not consumed, and it is inside the
replaced span. One undo restores `\alp ` — with its space, not
`\al`. Rev 6 wrote the post-undo text without the terminator,
which would be true only if the terminator were swallowed.
- *Eager path.* `\alpha` yields `α` with no terminator typed, and a
following space is a **separate** edit. One undo removes the
space; a second restores `\alpha`. Asserting the finish-path undo
text here would fail, which is the trap this split exists to
record.
39. `\<>` yields `⟨⟩` with the point between them, from the `$CURSOR`
placeholder.
40. **Pair-collision pin (Q#LN22).** `\[[]]` yields `⟦⟧`: each `[` is
@ -2467,9 +2509,21 @@ criterion 46 requires to stay byte-identical.
only completed expansions rather than pending extensions — **both
failure modes must be shown**, since they are distinct bugs with the
same symptom.
41. `\to` yields `→` eagerly on uniqueness, with no terminator typed.
42. A prefix with no match (`\zzzz` + space) is left as literal text; no
edit is made.
41. **Eager expansion on uniqueness**, with no terminator typed:
`\alpha` yields `α` the moment the final `a` lands. Rev 8 used `\to`
here and that is false against the real table (round 9): `to` is a
proper prefix of `top`, `to0`, `toa` and others, so
`isAbbreviationUniqueAndComplete` is false and `to` is **not** in
the 1,550-key eager set. `\to` alone stays `\to`; `\to` + space
yields `→ ` by the finish path. Both are asserted, because the
wrong one reads as correct until the table is consulted.
42. A prefix that opens no key at all — `\WWWW` + space — is left as
literal text and **no edit is made**. Rev 8 used `\zzzz`, which
expands (round 9): `z` opens a pending abbreviation because `ze`,
`zeta` and `zsqrtd` exist, and the second `z` finishes it, giving
`ζzzz `. Exactly six printable characters open no key: `$ % , ; @
W`. Bites against an implementation that treats "no complete match"
as "no pending state".
43. **Lazy abandonment (Q#LN22).** Because there is no cursor-motion
hook, this asserts what pmacs can actually detect: after `\alp`, an
explicit `goto_byte` elsewhere followed by typing `h` inserts a
@ -2698,7 +2752,7 @@ uncapped event queue, the dropped `cfg.restart`, and — unchanged from
languages other than Lean, and §4's rule is what keeps them out of a Lean
PR.
### 9.1 Coherence impact — stages 4a and 4b (rev 8)
### 9.1 Coherence impact — stages 4a and 4b (rev 9)
**Sections served.** §6 (interaction islands) primarily, and in the
*preventing* direction rather than the fixing one — see below. §11

254
scripts/regen-lean-abbrev Executable file
View File

@ -0,0 +1,254 @@
#!/usr/bin/env python3
"""Regenerate builtin/runtime/lean_abbrev.lua from vscode-lean4.
Usage: scripts/regen-lean-abbrev <vscode-lean4-commit>
Fetches `lean4-unicode-input/src/abbreviations.json` at the given commit
and rewrites the vendored Lua table, including the provenance header, so
the artifact is self-describing to whoever next touches it. A refresh is
an ordinary PR with a visible diff — the diff is the review.
There is no automatic sync and none is wanted: an editor that silently
re-downloads its input method has a supply-chain problem, not a feature
(docs/lean4-mode-framing.md Q#LN11).
The emit is an ORDERED SEQUENCE, not a map. Upstream resolves
equal-length abbreviation ties by source declaration order — 101
prefixes depend on it — and a Lua `{ [key] = symbol }` table iterated
with `pairs` cannot carry that. A map-shaped emit would also be
nondeterministic across builds and, once a hash order happened to be
stable, stably wrong.
This script ABORTS rather than emitting something plausible when the
source is corrupt: a duplicate key after decoding (JSON permits them,
the table must not), a key or symbol that is not well-formed UTF-8, or a
round-trip mismatch. That last check re-parses the script's own output
with an independent unescaper and compares the full ordered sequence to
the source, entry for entry. It is what makes the vendored file
trustworthy, and it belongs here rather than in the acceptance suite:
the suite cannot see `abbreviations.json`, which is not shipped.
"""
import json
import pathlib
import sys
import urllib.request
REPO = "leanprover/vscode-lean4"
PATH = "lean4-unicode-input/src/abbreviations.json"
LICENSE = "Apache-2.0"
OUT = pathlib.Path(__file__).resolve().parent.parent / "builtin/runtime/lean_abbrev.lua"
# Canonical, lossless, byte-deterministic. Rev 6 of the framing said the
# generator should abort on "a key containing a character the emitted Lua
# would have to escape"; that rule rejects the real table, where `\` is a
# key and `"` begins eleven of them.
SHORT = {"\\": "\\\\", '"': '\\"', "\n": "\\n", "\r": "\\r", "\t": "\\t"}
def die(msg):
print(f"regen-lean-abbrev: {msg}", file=sys.stderr)
raise SystemExit(1)
def lua_escape(s):
"""Escape one string for a Lua double-quoted literal.
Operates on CHARACTERS, not bytes. Decomposing to UTF-8 bytes and
emitting each as `chr(byte)` produces a latin-1-shaped string that
`write_text(..., encoding="utf-8")` then re-encodes — every
non-ASCII symbol lands in the file double-encoded, and a round-trip
check that compares in-memory strings agrees with itself and misses
it entirely. Only control bytes, which are single-byte by
definition, become `\\ddd`.
"""
out = []
for ch in s:
if ch in SHORT:
out.append(SHORT[ch])
elif ord(ch) < 0x20 or ord(ch) == 0x7F:
out.append(f"\\{ord(ch):03d}")
else:
out.append(ch)
return "".join(out)
def lua_unescape(s):
"""Independent reader for the round-trip check.
Deliberately not the inverse of `lua_escape` sharing its table: a
check that reuses the encoder's own assumptions cannot detect that
those assumptions are wrong.
"""
out = bytearray()
i = 0
raw = s.encode("utf-8")
while i < len(raw):
b = raw[i]
if b != ord("\\"):
out.append(b)
i += 1
continue
i += 1
if i >= len(raw):
die("round-trip: trailing backslash in emitted string")
nxt = chr(raw[i])
if nxt in ("\\", '"'):
out.append(ord(nxt))
i += 1
elif nxt in ("n", "r", "t"):
out.append({"n": 10, "r": 13, "t": 9}[nxt])
i += 1
elif nxt.isdigit():
digits = ""
while i < len(raw) and chr(raw[i]).isdigit() and len(digits) < 3:
digits += chr(raw[i])
i += 1
out.append(int(digits))
else:
die(f"round-trip: unknown escape \\{nxt} in emitted string")
return out.decode("utf-8")
def main():
if len(sys.argv) != 2:
die(f"usage: {sys.argv[0]} <vscode-lean4-commit>")
commit = sys.argv[1]
url = f"https://raw.githubusercontent.com/{REPO}/{commit}/{PATH}"
with urllib.request.urlopen(url, timeout=60) as resp:
raw = resp.read()
try:
raw.decode("utf-8")
except UnicodeDecodeError as e:
die(f"source is not well-formed UTF-8: {e}")
# `object_pairs_hook` keeps declaration order AND exposes duplicate
# keys, which a plain dict would silently collapse.
pairs = json.loads(raw, object_pairs_hook=lambda kv: kv)
seen = {}
for i, (key, symbol) in enumerate(pairs):
if key in seen:
die(f"duplicate key {key!r} at entries {seen[key]} and {i}")
seen[key] = i
for label, s in (("key", key), ("symbol", symbol)):
if not isinstance(s, str):
die(f"{label} at entry {i} is not a string: {s!r}")
try:
s.encode("utf-8")
except UnicodeEncodeError as e:
die(f"{label} at entry {i} is not well-formed UTF-8: {e}")
cursor = sum(1 for _, v in pairs if "$CURSOR" in v)
for i, (key, symbol) in enumerate(pairs):
if symbol.count("$CURSOR") > 1:
die(f"symbol for {key!r} at entry {i} has more than one $CURSOR")
body = "".join(
f' {{ "{lua_escape(k)}", "{lua_escape(v)}" }},\n' for k, v in pairs
)
text = HEADER.format(
repo=REPO,
path=PATH,
commit=commit,
license=LICENSE,
count=len(pairs),
cursor=cursor,
bytes=len(raw),
script=pathlib.Path(sys.argv[0]).name,
) + "pmacs.lean_abbrev = {\n" + body + "}\n"
# Round-trip against the BYTES ON DISK, not the string in memory.
# The file is staged beside its destination, re-read, parsed, and
# only renamed into place once it compares equal entry for entry. A
# check that compares in-memory strings cannot see an encoding
# applied by the write itself, which is exactly how a
# double-encoding bug survived the first version of this script.
staged = OUT.with_suffix(".lua.staged")
staged.write_text(text, encoding="utf-8")
on_disk = staged.read_bytes().decode("utf-8")
got = []
# `str.splitlines()` is WRONG here: it also splits on U+2028, U+2029,
# U+0085 and the vertical-tab family, and 53 symbols in the real
# table contain one of those literally. It silently loses entries and
# the round-trip then reports a count mismatch that is the checker's
# bug, not the emit's. The emitted file's line structure is defined
# by the LF we write, and nothing else.
for line in on_disk.split("\n"):
line = line.strip()
if not line.startswith('{ "') or not line.endswith("},"):
continue
inner = line[1:-2].strip()
if not (inner.startswith('"') and inner.endswith('"')):
die(f"round-trip: unparsable emitted line: {line!r}")
fields, buf, esc, depth = [], [], False, 0
for ch in inner:
if esc:
buf.append(ch)
esc = False
elif ch == "\\":
buf.append(ch)
esc = True
elif ch == '"':
depth += 1
if depth % 2 == 0:
fields.append("".join(buf))
buf = []
elif depth % 2 == 1:
buf.append(ch)
if len(fields) != 2:
die(f"round-trip: expected 2 fields, got {len(fields)}: {line!r}")
got.append((lua_unescape(fields[0]), lua_unescape(fields[1])))
def fail(msg):
staged.unlink(missing_ok=True)
die(msg)
if len(got) != len(pairs):
fail(f"round-trip: emitted {len(got)} entries, source has {len(pairs)}")
for i, (want, have) in enumerate(zip(pairs, got)):
if tuple(want) != have:
fail(f"round-trip: entry {i} differs: source {want!r} vs emitted {have!r}")
staged.replace(OUT)
print(
f"wrote {OUT} — {len(pairs)} entries from {REPO}@{commit} "
f"({len(raw)} source bytes, {OUT.stat().st_size} emitted bytes), "
"round-trip verified against the bytes on disk"
)
HEADER = """\
-- lean_abbrev.lua --- VENDORED DATA. Do not edit by hand.
--
-- The Lean 4 abbreviation table, generated from:
--
-- repo: https://github.com/{repo}
-- path: {path}
-- commit: {commit}
-- license: {license}
-- entries: {count} ({cursor} carry $CURSOR)
-- source: {bytes} bytes
--
-- Regenerate with:
--
-- scripts/{script} {commit}
--
-- An ORDERED SEQUENCE, not a map: upstream resolves equal-length ties
-- by source declaration order (101 prefixes depend on it), and a
-- `pairs`-iterated Lua map cannot express that. The file's own line
-- order is the audit trail. Consumers must not reorder it.
--
-- Not fetched at runtime and not a package dependency: the input method
-- has to work offline and on first launch. Upkeep is a documented
-- manual process — see docs/lean4-mode-framing.md Q#LN11.
pmacs = pmacs or {{}}
"""
if __name__ == "__main__":
main()

View File

@ -3964,6 +3964,143 @@ mod tests {
);
}
/// Arc 8 Stage 4b acceptance 45f: the Lean abbreviation expander
/// works on the OPTIMISTIC producer, not only on `dispatch_key`.
///
/// This is the path most users take and the one no other Stage 4b
/// test covers. `classify_key` (`src/optimistic.rs`) returns
/// `Insert(c)` for `\` and for every ASCII letter — only the nine
/// built-in pair chars are excluded (Q#AP1) — so on a CRDT frontend
/// `\alpha` arrives here as six source-peer optimistic inserts,
/// while the expansion is a single daemon-peer replace spanning all
/// six. That asymmetry is the accepted undo degradation of Q#LN21;
/// what this pins is that the expansion happens at all.
///
/// It lives in `--lib` deliberately: the gate list runs
/// `--features crdt` only for `cargo test --lib`, so a crdt-gated
/// INTEGRATION test would be dark in CI and dark in the gates both.
///
/// The source frontend needs a REGISTERED WINDOW on the edited
/// buffer or nothing is armed at all — `handle_remote_crdt_op`
/// arms the record only when the source's active window displays
/// the buffer, so a source with no view fails closed and silently.
/// A version of this test without the view below passed six
/// fan-outs with a nil record and proved nothing.
#[cfg(feature = "crdt")]
#[test]
fn the_optimistic_producer_also_expands_a_lean_abbreviation() {
use crate::editor::EditorState;
use crate::protocol::FrontendId;
use crate::window::{FrontendView, Layout, Window, WindowId};
let dir = std::env::temp_dir().join(format!("pmacs-lean-opt-{}", std::process::id()));
std::fs::create_dir_all(&dir).expect("create temp dir");
let path = dir.join("a.lean");
std::fs::write(&path, "").expect("write fixture");
let source = FrontendId(77);
let mut editor = EditorState::new();
editor
.lua_host
.eval(Some("test"), "pmacs.lsp.config = {}")
.expect("clear lsp config");
editor
.lua_host
.eval(
Some("test-open"),
&format!(
"pmacs.buffer.find_or_open({:?}); pmacs.editor.goto_byte(0)",
path.display().to_string()
),
)
.expect("open the lean fixture");
let buffer_id = editor.core.borrow().active_window().buffer_id;
{
let mut core = editor.core.borrow_mut();
let mut reg = core.registry.borrow_mut();
reg.get_mut(buffer_id)
.expect("active buffer")
.upgrade_to_crdt(2)
.expect("upgrade to crdt");
drop(reg);
// The replica's own window on the shared buffer.
let text_view = {
let registry = core.registry.clone();
let reg = registry.borrow();
crate::text_view::TextView::new(reg.get(buffer_id).expect("buffer"))
};
let win_id = WindowId::next();
core.windows
.insert(win_id, Window::new(win_id, buffer_id, text_view));
core.register_frontend_view(
source,
FrontendView {
layout: Layout::single(win_id),
active: win_id,
fold_projection: true,
panel_capable: true,
frame_geometry: None,
panel_hidden: false,
},
);
}
let snapshot_bytes = {
let core = editor.core.borrow();
let reg = core.registry.borrow();
reg.get(buffer_id)
.expect("buffer")
.crdt_state()
.expect("crdt-backed")
.export_snapshot()
.expect("export snapshot")
};
let peer = loro::LoroDoc::new();
peer.set_peer_id(77).expect("set peer id");
peer.import(&snapshot_bytes).expect("import snapshot");
// One op per keystroke, exactly as the attach loop's
// optimistic-apply branch produces them.
for (i, ch) in "\\alpha".chars().enumerate() {
let v_before = peer.oplog_vv();
peer.get_text("body")
.insert(i, &ch.to_string())
.expect("peer insert");
let op_bytes = peer
.export(loro::ExportMode::updates(&v_before))
.expect("export op");
super::handle_remote_crdt_op(
&mut editor,
source,
buffer_id,
crate::rope::CrdtOp {
peer_id: 77,
bytes: op_bytes,
},
);
}
let text = match editor
.lua_host
.eval(
Some("test-readback"),
"local b = pmacs.window.buffer(); return b:slice(0, b:len())",
)
.expect("read buffer text")
{
mlua::Value::String(s) => String::from_utf8_lossy(&s.as_bytes()).into_owned(),
other => panic!("expected buffer text, got {other:?}"),
};
assert_eq!(
text, "α",
"the abbreviation expanded on the optimistic path — the \
record the expander reads is armed by handle_remote_crdt_op, \
not only by dispatch_key"
);
}
/// Q#AI9 (PR #109 round 1): the optimistic-apply arm clears an
/// EMPTY anchor on the source window — the GPU always takes this
/// path, and the TUI attach mirror tracks no selection state, so

View File

@ -445,6 +445,27 @@ impl EditorState {
include_str!("../builtin/runtime/pair.lua"),
)
.expect("load pair builtin chunk");
// Arc 8 Stage 4b: the Lean 4 Unicode input method. The vendored
// abbreviation table first — lean_input.lua reads it at chunk
// load to build its prefix and eager-key indexes. Both load
// after typed_edit.lua, which they register into.
//
// Load order does NOT decide whether abbreviation expansion or
// auto-pairing sees a keystroke first — the chain's priority
// does (50 vs 100), which is why Stage 4a exists. It matters
// only that the chain itself is already there.
lua_host
.eval(
Some("@pmacs/builtin/runtime/lean_abbrev.lua"),
include_str!("../builtin/runtime/lean_abbrev.lua"),
)
.expect("load lean_abbrev builtin chunk");
lua_host
.eval(
Some("@pmacs/builtin/runtime/lean_input.lua"),
include_str!("../builtin/runtime/lean_input.lua"),
)
.expect("load lean_input builtin chunk");
lua_host
.eval(
Some("@pmacs/builtin/runtime/lsp.lua"),

View File

@ -0,0 +1,678 @@
//! Lean 4 Unicode input method acceptance (Arc 8 Stage 4b,
//! docs/lean4-mode-framing.md Q#LN11/Q#LN21/Q#LN22, criteria 3845i).
//!
//! Dispatch-driven throughout: `dispatch_key` is the producer that arms
//! the typed-edit record for a grid frontend. The optimistic CRDT
//! producer is criterion 45f and lives in a `--lib` test, where the gate
//! list's `--features crdt` run reaches it.
use crossterm::event::{KeyCode, KeyEvent, KeyEventKind, KeyEventState, KeyModifiers};
use pmacs::editor::EditorState;
use pmacs::protocol::FrontendId;
use pmacs::window::{FrontendView, Layout, Window, WindowId};
use std::path::PathBuf;
use std::sync::atomic::{AtomicUsize, Ordering};
fn fresh_dir() -> PathBuf {
static SEQ: AtomicUsize = AtomicUsize::new(0);
let dir = std::env::temp_dir().join(format!(
"pmacs-leaninput-{}-{}",
std::process::id(),
SEQ.fetch_add(1, Ordering::Relaxed)
));
std::fs::create_dir_all(&dir).unwrap();
dir
}
fn key(code: KeyCode) -> KeyEvent {
KeyEvent {
code,
modifiers: KeyModifiers::NONE,
kind: KeyEventKind::Press,
state: KeyEventState::NONE,
}
}
fn exec(s: &EditorState, src: &str) {
s.lua_host.lua().load(src.to_string()).exec().unwrap();
}
fn eval<T: mlua::FromLuaMulti>(s: &EditorState, src: &str) -> T {
s.lua_host.lua().load(src.to_string()).eval().unwrap()
}
fn text(s: &EditorState) -> String {
let b: mlua::String = eval(
s,
"local b = pmacs.window.buffer(); return b:slice(0, b:len())",
);
String::from_utf8_lossy(&b.as_bytes()).into_owned()
}
fn type_as(s: &mut EditorState, fid: FrontendId, chars: &str) {
for ch in chars.chars() {
s.dispatch_key(fid, key(KeyCode::Char(ch)));
}
}
fn type_str(s: &mut EditorState, chars: &str) {
type_as(s, FrontendId::LOCAL, chars);
}
/// An editor with an empty `.lean` file open and the point at 0.
/// `pmacs.lsp.config = {}` keeps the real user config from spawning a
/// server; the language still resolves from the extension.
fn lean_editor() -> (EditorState, PathBuf) {
let dir = fresh_dir();
let f = dir.join("a.lean");
std::fs::write(&f, "").unwrap();
let s = EditorState::new();
exec(&s, "pmacs.lsp.config = {}");
let fd = f.display().to_string();
exec(&s, &format!("pmacs.buffer.find_or_open({fd:?})"));
exec(&s, "pmacs.editor.goto_byte(0)");
assert_eq!(
eval::<Option<String>>(
&s,
"return pmacs.lsp.buffer_language(pmacs.window.buffer())"
)
.as_deref(),
Some("lean4"),
"the fixture must actually be a lean4 buffer, or every \
expansion assertion below is vacuous"
);
(s, f)
}
// ---------------------------------------------------------------------------
// 38 / 41 — the two expansion paths, and what an undo restores
// ---------------------------------------------------------------------------
#[test]
fn the_finish_path_retains_the_terminator_in_one_undo_step() {
// `alp` is not a key; `alpha` is the shortest key extending it. The
// space does not extend anything, so it lands first and the
// expansion replaces the whole span INCLUDING the terminator.
let (mut s, _f) = lean_editor();
type_str(&mut s, "\\alp ");
assert_eq!(text(&s), "α ", "terminator retained, not consumed");
exec(&s, "pmacs.window.buffer():undo()");
assert_eq!(
text(&s),
"\\alp ",
"one undo restores the pre-expansion text WITH its terminator — \
the expansion is a single edit"
);
}
#[test]
fn the_eager_path_takes_no_terminator_and_undoes_separately() {
// `alpha` has no longer key extending it, so it is one of the 1,550
// eager keys: it expands the moment the final `a` lands, and a
// following space is a SEPARATE edit. Rev 8 asserted the finish-path
// undo text for this example, which is the trap (round 9).
let (mut s, _f) = lean_editor();
type_str(&mut s, "\\alpha");
assert_eq!(text(&s), "α", "eager expansion, no terminator typed");
type_str(&mut s, " ");
assert_eq!(text(&s), "α ");
exec(&s, "pmacs.window.buffer():undo()");
assert_eq!(text(&s), "α", "the first undo removes the separate space");
exec(&s, "pmacs.window.buffer():undo()");
assert_eq!(text(&s), "\\alpha", "the second undoes the expansion");
}
#[test]
fn to_is_not_eager_because_longer_keys_extend_it() {
// The criterion rev 8 got wrong: `to` looks unique and is not.
// `top`, `to0`, `toa` and others extend it, so it needs a
// terminator. Bites against an eager rule that tests only "is this
// a key" without asking whether anything extends it.
let (mut s, _f) = lean_editor();
type_str(&mut s, "\\to");
assert_eq!(text(&s), "\\to", "no expansion without a terminator");
type_str(&mut s, " ");
assert_eq!(text(&s), "", "the finish path then resolves it");
}
// ---------------------------------------------------------------------------
// 39 — $CURSOR
// ---------------------------------------------------------------------------
#[test]
fn the_cursor_placeholder_places_the_point_between_the_symbols() {
let (mut s, _f) = lean_editor();
type_str(&mut s, "\\<>");
assert_eq!(text(&s), "⟨⟩");
// The placeholder is a point position, not a literal: typing lands
// between the brackets.
type_str(&mut s, "x");
assert_eq!(text(&s), "⟨x⟩", "$CURSOR left the point inside");
}
// ---------------------------------------------------------------------------
// 40 — the pair collision
// ---------------------------------------------------------------------------
#[test]
fn a_pending_abbreviation_is_never_corrupted_by_auto_pairing() {
// 64 keys contain a `lean4` pair-set character. Two DISTINCT bugs
// produce the same symptom here, so both are asserted: pairing
// running first, and a consumer that claims only completed
// expansions (which would hand each intermediate `[` to pairing).
let (mut s, _f) = lean_editor();
type_str(&mut s, "\\[");
assert_eq!(
text(&s),
"\\[",
"the intermediate `[` was CLAIMED — pairing inserted no `]`, \
which is what keeps `\\[[]]` reachable"
);
type_str(&mut s, "[]]");
assert_eq!(text(&s), "⟦⟧", "the full key resolves");
}
#[test]
fn a_pair_character_outside_a_pending_abbreviation_still_pairs() {
// The other direction: claiming extensions must not disable pairing
// in Lean buffers generally.
let (mut s, _f) = lean_editor();
type_str(&mut s, "[");
assert_eq!(text(&s), "[]", "ordinary auto-pairing is untouched");
}
// ---------------------------------------------------------------------------
// 42 — a prefix that opens nothing
// ---------------------------------------------------------------------------
#[test]
fn a_prefix_that_opens_no_key_is_left_literal_with_no_edit() {
// `W` is one of exactly six printable characters that begin no key
// (`$ % , ; @ W`). Rev 8 used `\zzzz`, which expands — `ze`, `zeta`
// and `zsqrtd` exist (round 9).
let (mut s, _f) = lean_editor();
type_str(&mut s, "\\WWWW ");
assert_eq!(text(&s), "\\WWWW ", "literal text, no expansion");
}
#[test]
fn a_prefix_with_no_complete_match_still_expands_its_best_prefix() {
// The case rev 8 mistook for "no match": `z` DOES open a pending
// abbreviation, and the second `z` finishes it.
let (mut s, _f) = lean_editor();
type_str(&mut s, "\\zzzz ");
assert_eq!(text(&s), "ζzzz ", "`z` resolved through `ze`");
}
// ---------------------------------------------------------------------------
// 43 — lazy abandonment
// ---------------------------------------------------------------------------
#[test]
fn moving_the_point_away_abandons_the_pending_abbreviation() {
// There is no cursor-motion hook, so the pending record is
// validated at the NEXT typed edit: the point must still be at the
// end of the pending span.
let (mut s, _f) = lean_editor();
type_str(&mut s, "\\alp");
exec(&s, "pmacs.editor.goto_byte(0)");
type_str(&mut s, "h");
assert_eq!(text(&s), "h\\alp", "the `h` landed as plain text");
// The keystroke that makes abandonment OBSERVABLE. Asserting only
// the line above proves nothing: claiming an extension makes no
// edit, so a record that wrongly survived would look identical
// here. If `h` had extended the record to `alph`, this `a`
// completes `alpha` and eagerly expands — over a span whose offsets
// are now stale by one.
type_str(&mut s, "a");
assert_eq!(
text(&s),
"ha\\alp",
"`\\alp` is still literal: the record was dropped when the \
point left the end of its span, not carried along"
);
}
#[test]
fn switching_buffers_clears_pending_state_eagerly() {
let (mut s, f) = lean_editor();
let dir = fresh_dir();
let other = dir.join("b.lean");
std::fs::write(&other, "").unwrap();
let od = other.display().to_string();
let fd = f.display().to_string();
// Open the second buffer FIRST, then come back. `find_or_open`
// fires `buffer.after-switch` only on the already-open branch — a
// fresh load fires `buffer.after-load` instead, and its own insert
// fires a record-less `buffer.after-edit`. Without this warm-up the
// test passes through the nil-record path and pins nothing about
// switching: deleting the after-switch subscriber leaves it green.
exec(&s, &format!("pmacs.buffer.find_or_open({od:?})"));
exec(&s, &format!("pmacs.buffer.find_or_open({fd:?})"));
exec(&s, "pmacs.editor.goto_byte(0)");
type_str(&mut s, "\\alph");
exec(&s, &format!("pmacs.buffer.find_or_open({od:?})"));
exec(&s, &format!("pmacs.buffer.find_or_open({fd:?})"));
exec(&s, "pmacs.editor.goto_byte(pmacs.window.buffer():len())");
type_str(&mut s, "a");
assert_eq!(
text(&s),
"\\alpha",
"without the switch this would have eagerly expanded to α; \
`buffer.after-switch` cleared the record"
);
}
// ---------------------------------------------------------------------------
// 44 / 45 — the setting and the language gate, both on the SOURCE buffer
// ---------------------------------------------------------------------------
#[test]
fn disabling_the_setting_stops_expansion() {
let (mut s, _f) = lean_editor();
exec(&s, "pmacs.config.set('lean.abbrev', false)");
type_str(&mut s, "\\alpha");
assert_eq!(text(&s), "\\alpha", "no expansion when disabled");
exec(&s, "pmacs.config.set('lean.abbrev', true)");
exec(&s, "pmacs.editor.goto_byte(pmacs.window.buffer():len())");
type_str(&mut s, " \\alpha");
assert_eq!(text(&s), "\\alpha α", "and it comes back live");
}
#[test]
fn the_setting_is_read_against_the_typed_edits_source_buffer() {
// A buffer-local override must not follow the user to another
// buffer of the same language — the `editing.auto-pair` precedent,
// including its round-2 correction to resolve `rec.buffer` rather
// than `pmacs.window.buffer()`.
let (mut s, f) = lean_editor();
let dir = fresh_dir();
let other = dir.join("b.lean");
std::fs::write(&other, "").unwrap();
exec(
&s,
"pmacs.config.set_local(pmacs.window.buffer(), 'lean.abbrev', false)",
);
type_str(&mut s, "\\alpha");
assert_eq!(text(&s), "\\alpha", "disabled in THIS buffer");
let od = other.display().to_string();
exec(&s, &format!("pmacs.buffer.find_or_open({od:?})"));
exec(&s, "pmacs.editor.goto_byte(0)");
type_str(&mut s, "\\alpha");
assert_eq!(text(&s), "α", "a second lean buffer is unaffected");
let fd = f.display().to_string();
exec(&s, &format!("pmacs.buffer.find_or_open({fd:?})"));
assert_eq!(text(&s), "\\alpha", "and the first is still disabled");
}
#[test]
fn no_abbreviation_state_is_opened_outside_a_lean_buffer() {
let dir = fresh_dir();
let f = dir.join("a.rs");
std::fs::write(&f, "").unwrap();
let s = EditorState::new();
exec(&s, "pmacs.lsp.config = {}");
let fd = f.display().to_string();
exec(&s, &format!("pmacs.buffer.find_or_open({fd:?})"));
exec(&s, "pmacs.editor.goto_byte(0)");
let mut s = s;
type_str(&mut s, "\\alpha");
assert_eq!(text(&s), "\\alpha", "no expansion in Rust");
// And the leader opened nothing, so `[` still pairs normally.
exec(&s, "pmacs.editor.goto_byte(pmacs.window.buffer():len())");
type_str(&mut s, "\\[");
assert_eq!(
text(&s),
"\\alpha\\[]",
"`\\[` in a Rust buffer pairs — the input method never armed"
);
}
// ---------------------------------------------------------------------------
// 45a / 45b / 45c / 45d / 45e — resolution rules
// ---------------------------------------------------------------------------
#[test]
fn the_shortest_key_wins_not_the_longest() {
let (mut s, _f) = lean_editor();
type_str(&mut s, "\\alp ");
assert_eq!(text(&s), "α ", "`alp` resolves through `alpha`");
exec(&s, "pmacs.editor.goto_byte(pmacs.window.buffer():len())");
type_str(&mut s, "\\al ");
assert_eq!(
text(&s),
"α",
"`al` resolves through `all` (3) — NOT `alpha` (5). A \
longest-match or unique-match-only rule passes the first \
assertion and fails this one"
);
}
#[test]
fn an_unmatchable_tail_is_appended_not_dropped() {
let (mut s, _f) = lean_editor();
type_str(&mut s, "\\alp7 ");
assert_eq!(
text(&s),
"α7 ",
"`7` finished `alp`; it is kept, not swallowed, and the whole \
abbreviation is not abandoned"
);
}
#[test]
fn there_is_no_terminator_list() {
// `'+ '` is a key — a trailing SPACE is part of it. Bites against
// any hardcoded space/tab/RET terminator set.
let (mut s, _f) = lean_editor();
type_str(&mut s, "\\+ ");
assert_eq!(text(&s), "", "the space EXTENDED rather than terminating");
}
#[test]
fn a_doubled_backslash_yields_one_literal_backslash() {
// Not a terminator case: the pending text is empty, `\` is itself a
// key, and it extends-and-eagerly-matches.
let (mut s, _f) = lean_editor();
type_str(&mut s, "\\\\");
assert_eq!(text(&s), "\\", "one literal backslash");
// ...and no pending state was left open, so an ordinary letter is
// an ordinary letter.
type_str(&mut s, "n");
assert_eq!(text(&s), "\\n", "two characters, not a newline");
}
#[test]
fn a_terminating_backslash_re_arms_as_a_new_leader() {
// `al` is NOT eager, so its pending record is still open when the
// second `\` arrives: the `\` terminates it, the expansion runs,
// and the same `\` must then open a fresh abbreviation.
//
// The framing's own example — `\alpha\to` — does NOT exercise this
// branch: `alpha` is eager, so the record is already closed and the
// `\` is handled by the ordinary open-a-leader path. It passes with
// the re-arm branch deleted, which is why the non-eager case is the
// one asserted first.
let (mut s, _f) = lean_editor();
type_str(&mut s, "\\al\\to ");
assert_eq!(
text(&s),
"∀→ ",
"the terminating `\\` expanded `al` AND opened a new \
abbreviation at its own position"
);
// The criterion's example still holds, by the other route.
exec(&s, "pmacs.editor.goto_byte(pmacs.window.buffer():len())");
type_str(&mut s, "\\alpha\\to ");
assert_eq!(text(&s), "∀→ α→ ");
}
#[test]
fn an_inserted_backslash_does_not_re_arm() {
// `setminus` expands to a literal `\`. That backslash is a
// programmatic replace, which arms no typed-edit record — so it
// opens no pending abbreviation. Bites against a future consumer
// that infers pending state from buffer text instead of provenance.
let (mut s, _f) = lean_editor();
type_str(&mut s, "\\setminus");
assert_eq!(text(&s), "\\", "expanded to a literal backslash");
type_str(&mut s, "n");
assert_eq!(
text(&s),
"\\n",
"the letter after it is a plain letter — the INSERTED backslash \
armed nothing"
);
}
// ---------------------------------------------------------------------------
// 45h — the tie-break by source declaration order
// ---------------------------------------------------------------------------
#[test]
fn equal_length_candidates_break_by_source_declaration_order() {
// `f<` and `f>` are both length 2. `f<` is declared first, so `\f`
// resolves to ``. This is the criterion that bites a map-shaped
// vendored table: with `pairs` iteration it passes or fails by hash
// order.
let (mut s, _f) = lean_editor();
type_str(&mut s, "\\f ");
assert_eq!(text(&s), " ", "`f<` wins over `f>` by source order");
exec(&s, "pmacs.editor.goto_byte(pmacs.window.buffer():len())");
type_str(&mut s, "\\\" ");
assert_eq!(
text(&s),
" Ä ",
"`\"A` is the first of eleven equal-length candidates"
);
}
#[test]
fn reversing_the_vendored_sequence_reverses_the_tie() {
// The falsification 45h requires: run the same resolution against a
// deliberately reversed sequence and show it changes. If this did
// NOT change, the tie-break would not be reading source order at
// all and the assertion above would be passing by luck.
let (s, _f) = lean_editor();
let forward: String = eval(&s, "return pmacs.lean_input._resolve('f')");
assert_eq!(forward, "");
let reversed: String = eval(
&s,
"
local seq = pmacs.lean_abbrev
local rev = {}
for i = #seq, 1, -1 do rev[#rev + 1] = seq[i] end
-- Resolve `f` the way the module does, over the reversed order.
local best = nil
for i = 1, #rev do
local k, v = rev[i][1], rev[i][2]
if k:sub(1, 1) == 'f' then
if best == nil or #k < best.len then best = { sym = v, len = #k } end
end
end
return best.sym
",
);
assert_eq!(
reversed, "",
"reversed source order picks `f>` — the tie really is decided \
by position in the sequence"
);
}
// ---------------------------------------------------------------------------
// 45g — table integrity, limited to what the suite can actually check
// ---------------------------------------------------------------------------
#[test]
fn the_vendored_table_is_self_consistent() {
// `abbreviations.json` is not shipped, so the suite cannot diff
// against it; the full source-fidelity check belongs to the
// generator, which re-parses its own output from disk. What is
// checkable here are the properties a corrupt emit breaks.
let (s, _f) = lean_editor();
let count: i64 = eval(&s, "return #pmacs.lean_abbrev");
assert_eq!(
count, 1855,
"the declared entry count for the recorded upstream commit"
);
let (unique, cursor_ok, utf8_ok): (i64, bool, bool) = eval(
&s,
r#"
local seen, n = {}, 0
local cursor_ok, utf8_ok = true, true
for i = 1, #pmacs.lean_abbrev do
local k, v = pmacs.lean_abbrev[i][1], pmacs.lean_abbrev[i][2]
if not seen[k] then seen[k] = true; n = n + 1 end
local _, c = v:gsub("%$CURSOR", "")
if c > 1 then cursor_ok = false end
-- A Lua pattern cannot validate UTF-8; check the shape the
-- emitter guarantees instead: no lone continuation byte at the
-- start of a sequence and no truncated tail.
if k:find("[\128-\191]") == 1 then utf8_ok = false end
end
return n, cursor_ok, utf8_ok
"#,
);
assert_eq!(
unique, 1855,
"every key is unique — a collision would silently drop entries \
from the derived lookup"
);
assert!(cursor_ok, "no symbol carries more than one $CURSOR");
assert!(utf8_ok, "no key begins with a continuation byte");
// The resolution spot-set named by 45g.
for (input, want) in [
("alpha", "α"),
("to", ""),
("<>", "⟨$CURSOR⟩"),
("+ ", ""),
("\\\\", "\\"),
("n", "\\n"),
("setminus", "\\"),
("f", ""),
] {
let got: String = eval(&s, &format!("return pmacs.lean_input._resolve('{input}')"));
assert_eq!(got, want, "resolution of {input:?}");
}
// The eager set is the one the state machine branches on.
let alpha_eager: bool = eval(&s, "return pmacs.lean_input._is_eager('alpha')");
let to_eager: bool = eval(&s, "return pmacs.lean_input._is_eager('to')");
assert!(alpha_eager, "`alpha` has no extension");
assert!(!to_eager, "`to` is extended by `top`, `to0`, `toa`, …");
}
// ---------------------------------------------------------------------------
// 45i — pending state is per frontend
// ---------------------------------------------------------------------------
/// Register a second frontend on the SAME buffer, with its own window.
fn attach_frontend(s: &EditorState, fid: FrontendId) -> WindowId {
let mut core = s.core.borrow_mut();
let buffer_id = core.active_buffer_id();
let text_view = {
let registry = core.registry.clone();
let reg = registry.borrow();
pmacs::text_view::TextView::new(reg.get(buffer_id).unwrap())
};
let win_id = WindowId::next();
core.windows
.insert(win_id, Window::new(win_id, buffer_id, text_view));
core.register_frontend_view(
fid,
FrontendView {
layout: Layout::single(win_id),
active: win_id,
fold_projection: true,
panel_capable: true,
frame_geometry: None,
panel_hidden: false,
},
);
win_id
}
const B: FrontendId = FrontendId(9);
#[test]
fn a_peer_edit_to_the_shared_buffer_abandons_the_pending_record() {
let (mut s, _f) = lean_editor();
let b_win = attach_frontend(&s, B);
// B sits at the start of the buffer; A types at the end.
s.core.borrow_mut().windows.get_mut(&b_win).unwrap().cursor = 0;
type_as(&mut s, FrontendId::LOCAL, "\\al");
type_as(&mut s, B, "p");
assert!(
text(&s).contains('p'),
"B's keystroke landed as ordinary text rather than extending \
A's abbreviation, got {:?}",
text(&s)
);
type_as(&mut s, FrontendId::LOCAL, "l ");
assert!(
!text(&s).contains('∀'),
"A's record was abandoned: `revision()` is buffer-global, so \
B's edit invalidates it even though B edited elsewhere. Got {:?}",
text(&s)
);
}
#[test]
fn a_peer_buffer_switch_does_not_clear_another_frontends_record() {
let (mut s, f) = lean_editor();
let dir = fresh_dir();
let other = dir.join("b.lean");
std::fs::write(&other, "").unwrap();
let od = other.display().to_string();
let fd = f.display().to_string();
// Warm up both buffers so B's switch takes `find_or_open`'s
// already-open branch, which is the only one that fires
// `buffer.after-switch`. A fresh load fires `buffer.after-load`
// and a record-less edit instead — and that path clears pending
// state for a different reason, which would make this test green
// no matter whose entries the subscriber clears.
exec(&s, &format!("pmacs.buffer.find_or_open({od:?})"));
exec(&s, &format!("pmacs.buffer.find_or_open({fd:?})"));
exec(&s, "pmacs.editor.goto_byte(0)");
attach_frontend(&s, B);
type_as(&mut s, FrontendId::LOCAL, "\\al");
// B switches buffers WITHOUT editing the shared buffer.
// Only B moves: the switch is scoped to B's own window, so A's
// window still shows the shared buffer with A's point where it was.
s.core.borrow_mut().active_frontend = B;
exec(&s, &format!("pmacs.buffer.find_or_open({od:?})"));
s.core.borrow_mut().active_frontend = FrontendId::LOCAL;
type_as(&mut s, FrontendId::LOCAL, "l ");
assert_eq!(
text(&s),
"",
"`buffer.after-switch` clears only the ACTING frontend's \
entries a blanket clear would discard A's half-typed \
abbreviation"
);
}
#[test]
fn detaching_a_frontend_purges_only_its_own_pending_state() {
let (mut s, _f) = lean_editor();
attach_frontend(&s, B);
type_as(&mut s, FrontendId::LOCAL, "\\al");
exec(&s, &format!("pmacs.hook.run('frontend.detached', {})", B.0));
type_as(&mut s, FrontendId::LOCAL, "l ");
assert_eq!(
text(&s),
"",
"B's detachment purged B's entries and left A's record valid"
);
}