165 lines
8.1 KiB
Markdown
165 lines
8.1 KiB
Markdown
# pmacs-gpu mouse input — framing pass
|
||
|
||
Date: 2026-06-10. Follows the Phase B framing
|
||
(`pmacs-gpu-phase-b-framing.md`), which deferred the mouse wire
|
||
decision as Q#B5. The optimistic-typing arc is merged (PR #60,
|
||
main @ `7fdae76`); this is the next capability session.
|
||
|
||
## What exists today (fact-checked)
|
||
|
||
- **Wire**: `FrontendEvent::Mouse(MouseEvent)` carries a `CellCoord`
|
||
(`pmacs-protocol/src/message.rs:210-219`) — a *cell-grid*
|
||
coordinate. `PROTOCOL_VERSION = 4` (`message.rs:970`); the
|
||
DispatchIdle variant set the precedent for gating new variants on
|
||
`negotiated_protocol_version` (`daemon.rs:998`).
|
||
- **Daemon semantics**: `EditorState::dispatch_mouse` owns the full
|
||
gesture state machine — Down activates the window, positions the
|
||
cursor, begins a selection; Drag grows the region; Up collapses an
|
||
empty region; a second Down in the same cell within 500ms is a
|
||
double-click and calls `select_word_at_cursor`; ScrollUp/Down move
|
||
the viewport (`src/editor.rs`, CUA arc). For a *semantic* session,
|
||
`apply_semantic_input_event` already routes
|
||
`FrontendEvent::Mouse → mouse_to_crossterm → dispatch_mouse`
|
||
(`daemon.rs:1983-1986`) — against the session's placeholder 24×80
|
||
cell geometry, which pmacs-gpu does not have.
|
||
- **GPU layout**: the shaped buffer holds the *visible slice* of the
|
||
source (`current_text[vstart..vend]`) with **inline adornment text
|
||
injected** by `projected_rich_chunks` (`pmacs-gpu/src/main.rs:3202`)
|
||
— so shaped-buffer byte offsets ≠ source bytes whenever inlay hints
|
||
are visible. Text is drawn at `TEXT_LEFT/TEXT_TOP` pixel offsets.
|
||
- **Hit testing**: `cosmic_text::Buffer::hit(x, y) -> Option<Cursor>`
|
||
exists (cosmic-text 0.18.2, `buffer.rs:923`) and returns a cursor in
|
||
the shaped buffer's (line, byte-index-within-line) space — the same
|
||
line-relative space QB3 taught us to rebase via the slice line
|
||
table.
|
||
- **Scroll wheel**: the GPU owns viewport + scroll state (S1:
|
||
`scroll_top` / slice reshape / viewport re-declaration), but
|
||
**no winit `MouseWheel` handler exists today** (fact-check: the
|
||
window_event catch-all swallows it; scrolling is keyboard-driven).
|
||
M-2 adds the handler — it composes from existing pieces
|
||
(`scroll_top` adjust → reshape → `viewport_send_if_changed`) and
|
||
needs no wire.
|
||
|
||
## Q#M1 — the wire shape (resolves Q#B5)
|
||
|
||
How does a pixel frontend tell the instance where a click landed?
|
||
|
||
- **(α) Reuse `FrontendEvent::Mouse` with synthesized cell coords.**
|
||
The GPU would approximate `(row, col)` from pixels. REJECTED: the
|
||
daemon's cell math runs against window geometry the GPU doesn't
|
||
share (placeholder 24×80), inline adornments shift visual columns
|
||
with no cell-space representation, and the design doc's contract is
|
||
explicit — *the instance never learns a pixel, and there is no
|
||
hit-test round trip*. α is broken-by-design, not merely lossy.
|
||
- **(β1) Local hit-test → byte-position pointer events.** The GPU
|
||
resolves pixels to a **source byte offset** locally (it owns the
|
||
layout) and ships gesture-level events; the daemon replays its
|
||
existing mouse state machine in byte space. Selection semantics
|
||
(CUA, double-click word select, future triple-click) stay
|
||
instance-side, exactly like the Key round-trip philosophy.
|
||
- **(β2) Frontend-computed selection states.** The GPU computes
|
||
cursor+selection itself and ships final states. REJECTED: forks the
|
||
gesture semantics into a second implementation the TUI doesn't
|
||
share, and fights the daemon-authoritative selection model the CUA
|
||
arc just consolidated.
|
||
|
||
**Stance: β1.** New wire variant:
|
||
|
||
```rust
|
||
FrontendEvent::Pointer {
|
||
frontend_id: FrontendId,
|
||
buffer_id: BufferId,
|
||
/// Source byte offset the frontend hit-tested locally.
|
||
byte: u64,
|
||
kind: PointerKind, // Down | Drag | Up | DoubleDown
|
||
mods: Modifiers,
|
||
}
|
||
```
|
||
|
||
`DoubleDown` is detected frontend-side (the frontend knows pixel
|
||
proximity and its own double-click interval; the daemon's cell-based
|
||
double-click detection cannot see pixels). The daemon maps:
|
||
Down → activate semantic window + set cursor + `begin_selection`;
|
||
Drag → cursor move (region grows); Up → collapse-if-empty;
|
||
DoubleDown → `select_word_at_cursor`. This reuses the same core
|
||
primitives `dispatch_mouse` calls today — no new selection logic.
|
||
|
||
Protocol: bump `PROTOCOL_VERSION` to 5; the GPU sends `Pointer` only
|
||
when `hello.protocol_version >= 5`; the daemon ignores the variant
|
||
for sessions that lack `semantic_render`.
|
||
|
||
## Q#M2 — projected→source byte mapping
|
||
|
||
`Buffer::hit` answers in *shaped* coordinates: line index + byte
|
||
offset within the shaped line, where shaped text = source slice with
|
||
adornment text spliced in. Two-step mapping:
|
||
|
||
1. Shaped (line, index) → shaped-buffer byte offset via the shaped
|
||
line table (the QB3 rebase, but over the *projected* text).
|
||
2. Projected byte → **source byte** via a run map built during
|
||
`reshape`. **This is new work, not a current by-product**
|
||
(fact-check: `RichChunk` today carries only `text` + `color`, no
|
||
source range — `main.rs:2269`): `projected_rich_chunks` walks
|
||
(source-run | adornment-run) chunks in order, so M-2 extends it to
|
||
also emit
|
||
`Vec<ProjectedRun { projected_start, len, source_start: Option<u64> }>`.
|
||
A hit inside a source run maps linearly; a hit inside an adornment
|
||
run snaps to the adornment's source anchor.
|
||
|
||
**Stance:** build the run map in `reshape` (it is O(chunks), walked
|
||
there anyway) rather than re-deriving adornment offsets at click
|
||
time. Clicks land between keystrokes, so the map being rebuilt per
|
||
reshape costs nothing new.
|
||
|
||
## Q#M3 — gesture scope for the first session
|
||
|
||
**In:** left-click cursor placement, left-drag selection,
|
||
double-click word select, wheel scroll (local-only, no wire).
|
||
**Deferred:** triple-click line select (needs a core
|
||
`select_line_at_cursor`), middle-click paste (needs clipboard
|
||
session), right-click menu (needs GUI chrome), drag auto-scroll at
|
||
window edges (needs a repeat timer; record as a known gap),
|
||
minimap click-to-jump (local-only; natural follow-up).
|
||
|
||
## Q#M4 — interaction with optimistic state
|
||
|
||
A pointer event is a round-trip input: it must (a) mark
|
||
`cursor_fresh = false` until the daemon's `CursorByte` answers, and
|
||
(b) defer behind the optimistic-cursor floor exactly like round-trip
|
||
keys do (`defer_round_trip_key_if_needed` shape) — a click landing
|
||
between unconfirmed keystrokes would otherwise race the cursor
|
||
confirmation. Deferred pointer events should keep only the **latest**
|
||
Down/Drag (coalescing, like the TUI's frame-boundary mouse
|
||
coalescing) rather than queueing every drag sample.
|
||
|
||
## Predicted findings (categorical bets, scored at session close)
|
||
|
||
1. **Coordinate-space bug** (the QB3 family): some offset in the
|
||
pixel→shaped→projected→source chain is wrong on first
|
||
implementation — most likely the adornment run snap or the
|
||
`TEXT_LEFT/TEXT_TOP` subtraction. Surfaces only in manual
|
||
validation with inlay hints visible.
|
||
2. **Window-activation gap** (the B1 family): the daemon-side Pointer
|
||
handler forgets some part of what `activate_and_position` does for
|
||
grid windows (window focus, `goal_col` reset), so a click works
|
||
but a subsequent arrow key moves from the wrong anchor.
|
||
3. **Selection-wash latency**: the wash arrives a round trip after
|
||
the drag (Decorations cadence), reading as rubber-band lag. If it
|
||
surfaces, the fix is frontend-local provisional selection — out of
|
||
scope, record it.
|
||
4. **Coalescing**: drag streams at pixel rate (hundreds of
|
||
events/sec) flood the socket or the dispatcher. Mitigation
|
||
in-scope: send Drag only when the hit byte changes.
|
||
|
||
## Session plan
|
||
|
||
- **M-1 (wire + daemon)**: `PointerKind`/`Pointer` variant, version
|
||
bump, daemon handler reusing core primitives, unit tests through
|
||
`handle_dispatcher_event`. Compile-green alone.
|
||
- **M-2 (GPU)**: winit mouse events → hit test → run map (new, from
|
||
`projected_rich_chunks`) → Pointer sends; drag coalescing on byte
|
||
change; NEW local wheel-scroll handler; optimistic-floor deferral.
|
||
Manual validation gate: click placement with inlay hints on the
|
||
same line, drag selection, double-click, wheel — in both pmacs-gpu
|
||
and a TUI peer simultaneously.
|