docs: consolidate in-buffer search framing (substring + regex)

Folds incremental-search-framing.md (#70) and regex-search-framing.md
into a single in-buffer-search-framing.md that records the design as
shipped, and removes the two superseded docs. The incremental doc in
particular described the C-f + minibuffer-hosted plan that never
shipped (C-s/C-r + a dedicated core SearchSession did), which was
actively misleading.

The consolidated doc adds an "As-built divergences" section capturing
the four places implementation departed from the framing passes:
C-f → C-s/C-r (veto resolved), minibuffer-hosted → frontend-agnostic
core mode (the GUI has no minibuffer), regex deferred → shipped, and
single-line substring → multi-line regex wash.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
Levi Neuwirth 2026-06-27 15:03:31 -04:00
parent 6e47fb4725
commit 6b7d3fb95d
3 changed files with 133 additions and 261 deletions

View File

@ -0,0 +1,133 @@
# In-buffer search — consolidated framing + as-built
Consolidates the two framing passes for in-buffer search:
**incremental substring isearch** (PR #70) and **regex search** (this
arc). Supersedes the separate `incremental-search-framing.md` and
`regex-search-framing.md`. Where the implementation diverged from a
framing stance, the "As-built" notes record what actually shipped and
why.
User-decided up front:
- **Incremental isearch** — highlight live as you type, the same key
steps to the next match, `RET` accepts, `Esc`/`C-g` restores origin.
- **Smart-case** matching (case-insensitive unless the query has an
uppercase letter), for both substring and regex.
- **Regex** with **both** dedicated entry keys (`C-M-s` / `C-M-r`)
**and** a mid-search toggle (`M-r`), matching **multi-line**.
## Architecture (as-built)
- **`search::SearchStore`** — per-buffer `HashMap<BufferId,
SearchState>` (query + sorted `Vec<ByteRange>` matches + active
index), shared `Arc<Mutex>`, mirroring `diag::DiagnosticStore`. The
active index is navigation state on the store (two windows on one
buffer share the active highlight — the diagnostics tradeoff). Edits
mark the entry stale (M11.8) so matches at pre-edit byte positions
never paint until a re-search.
- **`EditorCore::SearchSession { query, origin, forward, regex,
invalid }`** — the live input state. `search_begin/input_char/
backspace/step/finish/toggle_regex/recompute` drive it. `recompute`
runs the matcher over an `O(1)` rope snapshot, writes the store,
refocuses from the origin cursor, moves the cursor to the active
match.
- **Shared dispatch.** Keys are intercepted in `EditorState::
dispatch_key` → `dispatch_search_key` (`SearchKey::from_chord`).
This is the *same* path the daemon runs for round-tripped GUI
keystrokes, so isearch behaves identically in both frontends; only
the prompt *surface* differs.
## Matching
- **`find_all(haystack, query)`** — smart-case ASCII substring,
non-overlapping. Case-insensitive unless `query` has an uppercase
char.
- **`find_all_regex(haystack, pattern) -> Option<Vec<ByteRange>>`** —
`regex::bytes::Regex` over the whole buffer. `Some` for a valid
pattern (possibly empty), `None` when it won't compile, so the caller
distinguishes *invalid* (show `[invalid]`) from *zero matches*.
Smart-case via a `(?i)` prefix unless the pattern carries an
uppercase letter. Multi-line is free — the regex runs over the whole
byte slice, so an explicit `\n` (or `(?s).`) spans lines while `.`
stays line-bound. Zero-width matches (`a*`, `^`, `$`) are filtered.
The `regex` crate's linear-time engine makes a pathological pattern
slow at worst, never catastrophic. An uppercase letter inside an
escape/class (`\D`, `[A-Z]`) trips case-sensitivity — accepted, the
same coarse rule as the literal path.
## Input & bindings
- `C-s` / `C-r` — start a literal isearch forward / backward; once
running, the same keys step next / previous (intercepted in Rust, no
binding). `RET` accepts (keeps cursor + highlights until the next
edit); `C-g` / `Esc` cancels (restores the origin cursor, clears the
store); `BS` shortens the query.
- `C-M-s` / `C-M-r` — start a **regex** isearch (`search.forward-regex`
/ `search.backward-regex``ed.search_start(forward, regex)`).
- `M-r` — toggle literal ↔ regex mid-search (a `SearchKey` decoded in
`dispatch_search_key`, so it works the same in both frontends).
- Both keys were free in the default map (save is `C-x C-s`, redo is
`C-x r`), so isearch landed without disturbing the CUA / Emacs
editing keys.
## Frontend surfaces
- **TUI** — a `SearchView` overlay attached to the active window on
`search_begin` (deduped; self-suppresses with no matches / when
stale) washes matches; a bottom-row prompt reads `[Regex] I-search:
<query> (n/m)` (or `[no match]` / `[invalid]`). The terminal cursor
stays in the buffer at the active match. Multi-line matches wash each
spanned row (mirrors `paint_local_selection`'s per-row clip, newline
excluded).
- **GUI (pmacs-gpu)** — matches wash via `SearchMatch` /
`SearchMatchActive` decorations through `push_glyph_extent_rects`,
which already fans a byte range across visual lines, so **multi-line
needed no GUI rendering change**. The query reaches the band via the
`SearchPrompt` wire message. Key routing reuses the M11.6
`DispatchIdle` gate: `daemon_intercepts_keys` (a live `SearchPrompt`
or `!dispatch_idle`) round-trips every key into the daemon's search
while it runs, and `is_search_entry_chord` (`C-s`/`C-r`/`C-M-s`/
`C-M-r`) forwards the entry chords that are otherwise withheld.
Escape cancels an active search instead of quitting the window.
## Wire (`InstanceMessage::SearchPrompt`)
`{ buffer_id, query: Option<String>, active: Option<u32>, total: u32,
regex: bool, invalid: bool }`. Emitted by the semantic producer
(cached-compare suppressed like `StatusFacts`); `query: None` clears
the band. Protocol **v9** added the message (query/active/total);
**v10** added `regex` / `invalid` (an encoding change to the variant),
so the daemon's per-session filter gates it at `>= 10` — a v9 peer is
sent no `SearchPrompt` (decorations still highlight) rather than
mis-decoding the wider shape. `SUPPORTED = [6, 7, 8, 9, 10]`.
## As-built divergences from the framing passes
1. **Entry binding: `C-f` → `C-s` / `C-r`.** The incremental framing
penciled `C-f` (CUA "Find", rebinding `cursor.right`) "for veto."
`C-s` / `C-r` shipped instead: both were unbound, Emacs-faithful,
and need no `cursor.right` rebind. User-validated.
2. **Input model: minibuffer-hosted → dedicated core search mode.**
The framing (Q#SR5) proposed hosting the query in the minibuffer
with a new `on_changed` hook. Shipped as a frontend-agnostic
`SearchSession` on `EditorCore` driven by `dispatch_search_key`,
because **pmacs-gpu has no minibuffer** — a shared core mode was the
only way to make search work identically in both frontends.
3. **Regex: deferred → shipped.** Q#SR2 deferred regex ("literal-text
is the 95% case"); this arc added it as `find_all_regex` + a mode
flag, keeping the literal path the default.
4. **Multi-line: substring single-line → regex multi-line.** Substring
matches never span lines; regex can. The TUI `SearchView` (which
assumed single-line) gained per-row washing; the GUI was already
multi-line-capable.
## Bets that held (validation gate)
- Stale-after-edit linger — closed by `apply_active_edit` marking the
store stale.
- Invalid-regex incremental states — `foo(` shows `[invalid]`, never
panics, recovers on completion.
- Multi-line TUI wash — per-row clip at line boundaries, no phantom
trailing cell.
- GUI key routing — `dispatch_idle` flips during search; the optimistic
path round-trips instead of editing the buffer.

View File

@ -1,123 +0,0 @@
# Incremental in-buffer search — framing pass
Date: 2026-06-15. The last deferred GUI item: `SearchMatch` /
`SearchMatchActive` decorations exist on the wire (message.rs) and are
gated to `None` in both frontends "waiting on a search feature." This
builds that feature. Decided up front (user): **incremental isearch**
(highlight live as you type, same key steps to next, Enter accepts,
Esc/C-g restores origin) with **smart-case substring** matching
(case-insensitive unless the query has an uppercase letter).
## Survey facts (anchors)
- Greenfield in-buffer search; only `project.search` (cross-file grep)
exists. No `Buffer::find` / rope search.
- `DiagnosticStore` (diag.rs) is a near-exact template: keyed
`Arc<Mutex>` store, sorted entries, `next_after`/`previous_before`,
stale tracking, Lua nav bindings, overlay `_attach_view`.
- Producer `scoped_decorations` (semantic_render.rs) + TUI
`DiagnosticView::render` are the emit/paint templates (viewport clip,
line-start cache, stale-skip).
- GPU `decoration_kind_to_bg_color` already draws bg decorations
through the quad pipeline; SearchMatch/Active just need their arms.
- The minibuffer (minibuffer.rs) is keystroke-driven and pseudo-modal
(dispatch_minibuffer_key intercepts all keys while active) but has
**no live-preview/on_changed hook** — the one missing piece for
incremental highlight.
## Q#SR1 — store shape & ownership
**Stance: a per-buffer `SearchStore` mirroring `DiagnosticStore`.**
`by_buffer: HashMap<BufferId, SearchState>` where `SearchState` holds
the resolved query, the sorted `Vec<ByteRange>` matches, and the
active index. Shared `Arc<Mutex>`. The active index lives on the store
(navigation state), not per-window — v1 accepts that two windows on
the same buffer share the active highlight (note it; selection is the
per-window concept, search mirrors diagnostics). Edits mark the
buffer's entry stale (M11.8 model) so matches at pre-edit byte
positions aren't painted until re-search.
## Q#SR2 — search primitive
**Stance: smart-case substring over a rope snapshot, regex deferred.**
`find_all(haystack, query) -> Vec<ByteRange>`: case-insensitive unless
`query` contains an uppercase char (then exact). Built on
`snapshot_rope().slice` bytes (the diagnostics path's cheap snapshot).
Recomputed on query change; invalidated on edit. No `regex` crate in
v1 (literal-text is the 95% case; regex is a later toggle). Overlapping
matches: advance past each match's start+1 (standard non-overlapping).
## Q#SR3 — decoration emission
**Stance: mirror the diagnostics producer.** In `scoped_decorations`,
read the search store for the viewport buffer, emit `SearchMatch` for
every visible match and `SearchMatchActive` for the active one
(emitted last / higher z so it wins the overlap). Reuse the line cache
+ `clip_to_viewport`. Stale-skip exactly like diagnostics. TUI gets a
`SearchView` overlay (mirrors `DiagnosticView`) painting bg, attached
via `pmacs.search._attach_view`.
## Q#SR4 — colors
**Stance: a single search palette.** SearchMatch = translucent yellow
wash; SearchMatchActive = stronger amber/orange. GPU: the two
`decoration_kind_to_bg_color` arms. TUI: reverse-ish colored bg in the
`SearchView`. Distinct from selection (blue) and diagnostics
(severity).
## Q#SR5 — input & modality (incremental)
**Stance: host the query in the minibuffer + a small `on_changed`
hook + targeted next/prev interception.** Entry opens a minibuffer
search session (prompt `I-search: `); `on_changed` (new optional
session callback, fired after each content mutation in
dispatch_minibuffer_key) recomputes matches → updates the store →
re-decorate. While that session is active, the entry chord again =
`search.next`, its shift/`C-r` variant = `search.prev` (control keys,
not self-insert, so safe to intercept in the search branch of
dispatch_minibuffer_key). `Enter` accepts (close, leave cursor at the
active match); `Esc`/`C-g` cancels (close, restore the origin cursor
saved at entry, clear the store). Reusing the minibuffer's input
editing + prompt avoids reimplementing a modal query line.
## Q#SR6 — navigation
**Stance: `search.next`/`search.prev` mirror `diag.next/prev`.**
Advance the active index with wrap, move the active window's cursor to
the active match start, scroll it into view. The active index drives
which match is `SearchMatchActive`. Usable both during the live
session and afterward (matches persist until cleared / next search).
## Binding (proposal, flagged for veto)
`C-f` → search (CUA "Find"), rebound from `cursor.right`. Consistent
with the editor's CUA direction (arrows move; Ctrl+F finds); the
Emacs-holdover `C-f = forward-char` is the inconsistent one. Easy to
change — call out in validation.
## Predicted findings (categorical bets)
1. **Stale-after-edit linger** (the squiggle lesson again): matches at
pre-edit byte positions paint over shifted text until re-search —
the store's stale gate + re-search-on-change must be right, or
highlights drift during typing.
2. **Minibuffer `on_changed` × completion**: the hook interacts with
the existing per-keystroke candidate recompute; the search session
must opt out of completion cleanly (a session "kind" seam).
3. **Per-buffer active match across windows** surfaces as navigating
in one window moving the active highlight in another — accepted for
v1, but worth eyeballing.
4. **Empty / all-match queries**: empty query → no matches (not all);
a 1-char common letter → many matches → viewport-clipped emission
must stay cheap (line cache + only-visible).
## Session plan
Three green commits:
1. Core `SearchStore` + `find_all` smart-case primitive + unit tests.
2. Producer emission + GPU bg colors + TUI `SearchView` + attach.
3. Incremental UX: minibuffer `on_changed`, search session,
next/prev interception + commands, cancel-restores-origin, binding.
Manual validation gate as usual (type to highlight live, step matches,
edit mid-search, Esc restores).

View File

@ -1,138 +0,0 @@
# Regex in-buffer search — framing pass
Date: 2026-06-27. Extends the merged incremental isearch (PR #70):
`C-s`/`C-r` drive a smart-case **substring** search over a per-buffer
`SearchStore`, highlighted live in both frontends. This adds **regex**
matching alongside the literal path. Decided up front (user):
- **Both** a dedicated regex entry (`C-M-s` / `C-M-r`) **and** a
mid-search toggle (`M-r`, Emacs `isearch-toggle-regexp`).
- **Multi-line** matches (a pattern may span newlines).
## Survey facts (anchors — what shipped in #70)
- `find_all(haystack, query) -> Vec<ByteRange>` (search.rs): smart-case
ASCII substring, non-overlapping. The one matcher today.
- `EditorCore::SearchSession { query, origin, forward }` +
`search_begin/input_char/backspace/step/finish/recompute`. Recompute
runs `find_all` over an `O(1)` rope snapshot, writes the store,
refocuses from origin, moves the cursor.
- Input is intercepted in the shared `dispatch_key`
`dispatch_search_key` (`SearchKey::from_chord`), so TUI-local and
daemon (GUI round-trip) keystrokes drive the same core.
- TUI `SearchView` overlay washes matches; it assumes **single-line**
matches ("the query carries no newline") — the one place multi-line
breaks.
- GPU washes via `SearchMatch`/`SearchMatchActive` decorations through
`push_glyph_extent_rects`, which **already fans a byte range across
visual lines** (multi-line selections use the same path) — so the
GPU needs **no** rendering change for multi-line.
- `InstanceMessage::SearchPrompt { buffer_id, query, active, total }`
(protocol v9) carries the query + readout to the GUI status band.
- `regex` 1.12.3 is already in `Cargo.lock` (transitive); promote to a
direct dependency.
## Q#RX1 — the matcher
**Stance: a sibling `find_all_regex(haystack, pattern) -> Option<Vec<ByteRange>>`
on `regex::bytes::Regex` over the whole buffer.** `Some(matches)` for a
valid pattern (possibly empty), `None` for an invalid one (so the
caller can show `[invalid]` rather than `[no match]`). `find_iter`
gives leftmost non-overlapping byte ranges directly; **zero-width
matches are filtered** (start == end — `a*`, `^`, anchors — wash
nothing and would spam). Multi-line falls out for free: the regex runs
over the whole byte slice, so `foo\n\s*bar` matches across the newline.
`.` keeps its default (no `\n`); the user opts into dotall with `(?s)`.
## Q#RX2 — smart-case for regex
**Stance: the same uppercase heuristic, via an `(?i)` prefix.**
Case-insensitive unless the *pattern string* contains an uppercase
ASCII letter, implemented by compiling `(?i){pattern}` vs `{pattern}`.
Accepted imprecision: an uppercase letter inside an escape/class (`\D`,
`[A-Z]`) trips case-sensitivity — same spirit as the literal path's
"any uppercase ⇒ exact", and a regex author writing `[A-Z]` plausibly
wants case to matter anyway. Documented, not solved, in v1.
## Q#RX3 — session mode & toggle
**Stance: a `regex: bool` on `SearchSession`; `recompute` dispatches.**
`search_begin(forward, regex)` records the mode; `recompute` calls
`find_all_regex` (regex) or `find_all` (literal). `M-r`
`SearchKey::ToggleRegex``search_toggle_regex()` flips the flag and
re-runs recompute on the same query (re-anchored from origin). The
session tracks an `invalid` bool (last recompute's pattern failed to
compile) so the prompt can distinguish invalid from zero-match.
## Q#RX4 — multi-line TUI wash
**Stance: `SearchView` iterates the rows each match spans, mirroring
`paint_local_selection`.** Per match `[start, end)`: for each display
row from `line(start)` to `line(end)`, wash that row's clipped slice
(`paint_start = max(start, line_start)`, `paint_end = min(end,
line_end)`), mapping to display cols with the existing
`byte_range_to_display_cols`. Single-line matches (every literal match,
and most regex matches) hit exactly one row — no behavior change there.
The GPU is untouched (Q#RX-anchor: `push_glyph_extent_rects` already
multi-lines).
## Q#RX5 — entry keys & both frontends
**Stance: `C-M-s`/`C-M-r` start a regex search; `M-r` toggles within
any search.** Daemon keymap binds `C-M-s``search.forward-regex`,
`C-M-r``search.backward-regex` (Lua → `ed.search_start(forward,
regex=true)`); `M-r` is handled *inside* `dispatch_search_key` (a
`SearchKey`, not a global binding — it is only meaningful mid-search).
GUI: `is_search_entry_chord` also forwards `C-M-s`/`C-M-r` (Ctrl+Alt);
`M-r` already round-trips via the intercept path once a search is
running, so it needs no GUI change. The TUI/GUI prompt reads
**`Regex I-search:`** in regex mode and **`[invalid]`** when the
pattern won't compile.
## Q#RX6 — the wire (GUI)
**Stance: extend `SearchPrompt` with `regex: bool` + `invalid: bool`,
protocol v10.** `SearchPrompt` is new in v9 (this is the first
encoding change to it), so the additive-but-encoding-changing rule
bumps `PROTOCOL_VERSION` 9 → 10 and grows `SUPPORTED` to
`[6,7,8,9,10]`, daemon-gated per session exactly like v9. The producer
fills both from the active `SearchSession`; the GUI renders the regex
label + invalid indicator from them.
## Predicted findings (categorical bets)
1. **Multi-line TUI wash correctness** (headline): row-by-row clipping
at line boundaries — off-by-one on the trailing newline / a match
ending exactly at a line end / an empty middle line must wash
cleanly, not bleed a column or skip a row.
2. **Invalid-regex incremental states**: typing `foo(` passes through
invalid mid-keystroke constantly; recompute must degrade to "no
matches + `[invalid]`", never panic or surface a raw regex error,
and recover the instant the pattern compiles again.
3. **Smart-case heuristic × escapes/classes**: `\D`, `[A-Z]` flip
case-sensitivity under the simple "any uppercase" rule — acceptable,
noted.
4. **Zero-width / pathological matches**: `a*`, `^`, `$` produce empty
or per-position matches; filter empties, and lean on `find_iter`'s
non-overlapping advance so a catastrophic pattern can't loop.
5. **Regex compile per keystroke**: a fresh compile each character is
fine for interactive small patterns; no cache in v1 (note it).
## Session plan
Four green commits (mirrors the #70 arc: core → TUI → GUI):
1. This framing doc.
2. `find_all_regex` (multi-line, smart-case, `Option` for invalid) +
`regex` direct dep + unit tests.
3. TUI: `SearchSession.regex` + `invalid` + `search_begin(fwd, regex)`
+ `search_toggle_regex` + `SearchKey::ToggleRegex` (`M-r`) +
`search.forward-regex`/`backward-regex` commands + `C-M-s`/`C-M-r`
bindings + `Regex I-search:` / `[invalid]` prompt + multi-line
`SearchView` wash + tests.
4. GUI: protocol v10 (`SearchPrompt` + `regex`/`invalid`), producer
fill, GUI prompt label + indicator, `C-M-s`/`C-M-r` entry chords.
Manual validation gate as usual (regex highlight live, multi-line
pattern washes across rows, `M-r` toggles, invalid pattern shows
`[invalid]`, both frontends).