Merge pull request #61 from levineuwirth/session-m-mouse-pointer

pmacs-gpu mouse input: byte-position pointer events (protocol v5)

docs/pmacs-gpu-mouse-framing.md

@@ -0,0 +1,164 @@
+# pmacs-gpu mouse input — framing pass
+
+Date: 2026-06-10. Follows the Phase B framing
+(`pmacs-gpu-phase-b-framing.md`), which deferred the mouse wire
+decision as Q#B5. The optimistic-typing arc is merged (PR #60,
+main @ `7fdae76`); this is the next capability session.
+
+## What exists today (fact-checked)
+
+- **Wire**: `FrontendEvent::Mouse(MouseEvent)` carries a `CellCoord`
+  (`pmacs-protocol/src/message.rs:210-219`) — a *cell-grid*
+  coordinate. `PROTOCOL_VERSION = 4` (`message.rs:970`); the
+  DispatchIdle variant set the precedent for gating new variants on
+  `negotiated_protocol_version` (`daemon.rs:998`).
+- **Daemon semantics**: `EditorState::dispatch_mouse` owns the full
+  gesture state machine — Down activates the window, positions the
+  cursor, begins a selection; Drag grows the region; Up collapses an
+  empty region; a second Down in the same cell within 500ms is a
+  double-click and calls `select_word_at_cursor`; ScrollUp/Down move
+  the viewport (`src/editor.rs`, CUA arc). For a *semantic* session,
+  `apply_semantic_input_event` already routes
+  `FrontendEvent::Mouse → mouse_to_crossterm → dispatch_mouse`
+  (`daemon.rs:1983-1986`) — against the session's placeholder 24×80
+  cell geometry, which pmacs-gpu does not have.
+- **GPU layout**: the shaped buffer holds the *visible slice* of the
+  source (`current_text[vstart..vend]`) with **inline adornment text
+  injected** by `projected_rich_chunks` (`pmacs-gpu/src/main.rs:3202`)
+  — so shaped-buffer byte offsets ≠ source bytes whenever inlay hints
+  are visible. Text is drawn at `TEXT_LEFT/TEXT_TOP` pixel offsets.
+- **Hit testing**: `cosmic_text::Buffer::hit(x, y) -> Option<Cursor>`
+  exists (cosmic-text 0.18.2, `buffer.rs:923`) and returns a cursor in
+  the shaped buffer's (line, byte-index-within-line) space — the same
+  line-relative space QB3 taught us to rebase via the slice line
+  table.
+- **Scroll wheel**: the GPU owns viewport + scroll state (S1:
+  `scroll_top` / slice reshape / viewport re-declaration), but
+  **no winit `MouseWheel` handler exists today** (fact-check: the
+  window_event catch-all swallows it; scrolling is keyboard-driven).
+  M-2 adds the handler — it composes from existing pieces
+  (`scroll_top` adjust → reshape → `viewport_send_if_changed`) and
+  needs no wire.
+
+## Q#M1 — the wire shape (resolves Q#B5)
+
+How does a pixel frontend tell the instance where a click landed?
+
+- **(α) Reuse `FrontendEvent::Mouse` with synthesized cell coords.**
+  The GPU would approximate `(row, col)` from pixels. REJECTED: the
+  daemon's cell math runs against window geometry the GPU doesn't
+  share (placeholder 24×80), inline adornments shift visual columns
+  with no cell-space representation, and the design doc's contract is
+  explicit — *the instance never learns a pixel, and there is no
+  hit-test round trip*. α is broken-by-design, not merely lossy.
+- **(β1) Local hit-test → byte-position pointer events.** The GPU
+  resolves pixels to a **source byte offset** locally (it owns the
+  layout) and ships gesture-level events; the daemon replays its
+  existing mouse state machine in byte space. Selection semantics
+  (CUA, double-click word select, future triple-click) stay
+  instance-side, exactly like the Key round-trip philosophy.
+- **(β2) Frontend-computed selection states.** The GPU computes
+  cursor+selection itself and ships final states. REJECTED: forks the
+  gesture semantics into a second implementation the TUI doesn't
+  share, and fights the daemon-authoritative selection model the CUA
+  arc just consolidated.
+
+**Stance: β1.** New wire variant:
+
+```rust
+FrontendEvent::Pointer {
+    frontend_id: FrontendId,
+    buffer_id: BufferId,
+    /// Source byte offset the frontend hit-tested locally.
+    byte: u64,
+    kind: PointerKind, // Down | Drag | Up | DoubleDown
+    mods: Modifiers,
+}
+```
+
+`DoubleDown` is detected frontend-side (the frontend knows pixel
+proximity and its own double-click interval; the daemon's cell-based
+double-click detection cannot see pixels). The daemon maps:
+Down → activate semantic window + set cursor + `begin_selection`;
+Drag → cursor move (region grows); Up → collapse-if-empty;
+DoubleDown → `select_word_at_cursor`. This reuses the same core
+primitives `dispatch_mouse` calls today — no new selection logic.
+
+Protocol: bump `PROTOCOL_VERSION` to 5; the GPU sends `Pointer` only
+when `hello.protocol_version >= 5`; the daemon ignores the variant
+for sessions that lack `semantic_render`.
+
+## Q#M2 — projected→source byte mapping
+
+`Buffer::hit` answers in *shaped* coordinates: line index + byte
+offset within the shaped line, where shaped text = source slice with
+adornment text spliced in. Two-step mapping:
+
+1. Shaped (line, index) → shaped-buffer byte offset via the shaped
+   line table (the QB3 rebase, but over the *projected* text).
+2. Projected byte → **source byte** via a run map built during
+   `reshape`. **This is new work, not a current by-product**
+   (fact-check: `RichChunk` today carries only `text` + `color`, no
+   source range — `main.rs:2269`): `projected_rich_chunks` walks
+   (source-run | adornment-run) chunks in order, so M-2 extends it to
+   also emit
+   `Vec<ProjectedRun { projected_start, len, source_start: Option<u64> }>`.
+   A hit inside a source run maps linearly; a hit inside an adornment
+   run snaps to the adornment's source anchor.
+
+**Stance:** build the run map in `reshape` (it is O(chunks), walked
+there anyway) rather than re-deriving adornment offsets at click
+time. Clicks land between keystrokes, so the map being rebuilt per
+reshape costs nothing new.
+
+## Q#M3 — gesture scope for the first session
+
+**In:** left-click cursor placement, left-drag selection,
+double-click word select, wheel scroll (local-only, no wire).
+**Deferred:** triple-click line select (needs a core
+`select_line_at_cursor`), middle-click paste (needs clipboard
+session), right-click menu (needs GUI chrome), drag auto-scroll at
+window edges (needs a repeat timer; record as a known gap),
+minimap click-to-jump (local-only; natural follow-up).
+
+## Q#M4 — interaction with optimistic state
+
+A pointer event is a round-trip input: it must (a) mark
+`cursor_fresh = false` until the daemon's `CursorByte` answers, and
+(b) defer behind the optimistic-cursor floor exactly like round-trip
+keys do (`defer_round_trip_key_if_needed` shape) — a click landing
+between unconfirmed keystrokes would otherwise race the cursor
+confirmation. Deferred pointer events should keep only the **latest**
+Down/Drag (coalescing, like the TUI's frame-boundary mouse
+coalescing) rather than queueing every drag sample.
+
+## Predicted findings (categorical bets, scored at session close)
+
+1. **Coordinate-space bug** (the QB3 family): some offset in the
+   pixel→shaped→projected→source chain is wrong on first
+   implementation — most likely the adornment run snap or the
+   `TEXT_LEFT/TEXT_TOP` subtraction. Surfaces only in manual
+   validation with inlay hints visible.
+2. **Window-activation gap** (the B1 family): the daemon-side Pointer
+   handler forgets some part of what `activate_and_position` does for
+   grid windows (window focus, `goal_col` reset), so a click works
+   but a subsequent arrow key moves from the wrong anchor.
+3. **Selection-wash latency**: the wash arrives a round trip after
+   the drag (Decorations cadence), reading as rubber-band lag. If it
+   surfaces, the fix is frontend-local provisional selection — out of
+   scope, record it.
+4. **Coalescing**: drag streams at pixel rate (hundreds of
+   events/sec) flood the socket or the dispatcher. Mitigation
+   in-scope: send Drag only when the hit byte changes.
+
+## Session plan
+
+- **M-1 (wire + daemon)**: `PointerKind`/`Pointer` variant, version
+  bump, daemon handler reusing core primitives, unit tests through
+  `handle_dispatcher_event`. Compile-green alone.
+- **M-2 (GPU)**: winit mouse events → hit test → run map (new, from
+  `projected_rich_chunks`) → Pointer sends; drag coalescing on byte
+  change; NEW local wheel-scroll handler; optimistic-floor deferral.
+  Manual validation gate: click placement with inlay hints on the
+  same line, drag selection, double-click, wheel — in both pmacs-gpu
+  and a TUI peer simultaneously.

pmacs-gpu/src/attach.rs

@@ -23,7 +23,7 @@ use std::thread;
 
 use pmacs_protocol::{
     AttachRequest, BufferId, ByteRange, CrdtOp, FrontendCapabilities, FrontendEvent, FrontendId,
-    Hello, InstanceMessage, Key, KeyEvent, Modifiers, PROTOCOL_VERSION,
+    Hello, InstanceMessage, Key, KeyEvent, Modifiers, PROTOCOL_VERSION, PointerKind,
     SUPPORTED_PROTOCOL_VERSIONS, TransportError, is_supported_protocol_version, read_message,
     write_message,
 };
@@ -202,6 +202,7 @@ pub fn connect(
     Ok(AttachClient {
         writer_tx,
         frontend_id: hello.assigned_frontend_id,
+        server_protocol_version: hello.protocol_version,
     })
 }
 
@@ -213,6 +214,10 @@ pub struct AttachClient {
     /// `FrontendEvent` carries this so the daemon can route input back
     /// to the per-session `SemanticRenderState`.
     frontend_id: FrontendId,
+    /// The daemon's `Hello.protocol_version`. Wire variants newer
+    /// than the daemon (e.g. `Pointer`, v5) must be gated on this —
+    /// an older daemon hard-errors decoding an unknown variant.
+    server_protocol_version: u32,
 }
 
 impl AttachClient {
@@ -255,6 +260,30 @@ impl AttachClient {
         }))
     }
 
+    /// Send a `FrontendEvent::Pointer` (session M-2): a locally
+    /// hit-tested gesture in source bytes. Callers gate on
+    /// [`Self::server_protocol_version`] `>= 5`.
+    pub fn send_pointer(
+        &self,
+        buffer_id: BufferId,
+        byte: u64,
+        kind: PointerKind,
+        mods: Modifiers,
+    ) -> Result<(), TransportError> {
+        self.send_event(FrontendEvent::Pointer {
+            frontend_id: self.frontend_id,
+            buffer_id,
+            byte,
+            kind,
+            mods,
+        })
+    }
+
+    /// The daemon's negotiated wire version from `Hello`.
+    pub fn server_protocol_version(&self) -> u32 {
+        self.server_protocol_version
+    }
+
     /// Send a locally-authored CRDT operation to the daemon. The GPU
     /// uses this for idle plain-text insertion after applying the same
     /// op to its local Loro replica, avoiding a Key round trip on the

pmacs-gpu/src/main.rs

@@ -36,7 +36,7 @@ use loro::{ContainerTrait, ExportMode};
 use pmacs_protocol::{
     AdornmentContent, AdornmentPlacement, BufferId, ByteRange, CrdtOp, Decoration, DecorationKind,
     DecorationSegment, FrontendId, InlineAdornment, InstanceMessage, Key as ProtocolKey, Modifiers,
-    SelectionSnapshot, StyleSegment, StyleSpan,
+    PointerKind, SelectionSnapshot, StyleSegment, StyleSpan,
     cell::{Color as CellColor, Style as CellStyle},
 };
 use wgpu::MultisampleState;
@@ -391,6 +391,24 @@ struct State {
     /// ops can mis-prune by one frame; the next generation-keyed
     /// full resync self-corrects.
     unconfirmed_edits: Vec<(u64, TextProjectionEdit)>,
+    /// Q#M2 — projected→source hit map for the currently shaped
+    /// slice. Rebuilt by every `reshape` from the same chunks that
+    /// feed glyphon; source offsets are slice-relative (pair with
+    /// `view_range.0`).
+    current_hit_runs: Vec<ProjectedRun>,
+    /// Line-start byte offsets of the *projected* text (cosmic-text
+    /// reports hits as line index + byte-within-line).
+    projected_line_starts: Vec<u64>,
+    /// Last reported pointer position, in window pixels.
+    pointer_pos: Option<(f64, f64)>,
+    /// Primary button is held after a Down inside the text area.
+    pointer_drag_active: bool,
+    /// Hit byte of the last Pointer event sent — Drag coalescing:
+    /// pixel-rate motion only ships when the hit byte changes.
+    last_pointer_sent_byte: Option<u64>,
+    /// `(when, byte)` of the last primary Down, for frontend-side
+    /// double-click detection (same-hit within the interval).
+    last_pointer_down: Option<(std::time::Instant, u64)>,
 }
 
 /// pmacs-gpu's own cursor position, mirrored from `CursorByte`.
@@ -422,6 +440,23 @@ struct FileStyleSummaryState {
     lines: Vec<CellStyle>,
 }
 
+impl App {
+    /// Ship a Pointer event if the daemon speaks protocol v5+ — the
+    /// Q#M1 frontend-side gate (an older instance cannot decode the
+    /// variant and would drop the connection).
+    fn send_pointer(&self, buffer_id: BufferId, byte: u64, kind: PointerKind, mods: Modifiers) {
+        let Some(client) = self.attach_client.as_ref() else {
+            return;
+        };
+        if client.server_protocol_version() < 5 {
+            return;
+        }
+        if let Err(e) = client.send_pointer(buffer_id, byte, kind, mods) {
+            eprintln!("pmacs-gpu: send_pointer failed: {e}");
+        }
+    }
+}
+
 impl ApplicationHandler<AppEvent> for App {
     fn resumed(&mut self, event_loop: &ActiveEventLoop) {
         if self.state.is_some() {
@@ -456,6 +491,7 @@ impl ApplicationHandler<AppEvent> for App {
         }
     }
 
+    #[allow(clippy::too_many_lines)] // linear per-event dispatch; splitting hides the input flow.
     fn window_event(&mut self, event_loop: &ActiveEventLoop, _id: WindowId, event: WindowEvent) {
         match event {
             WindowEvent::CloseRequested => event_loop.exit(),
@@ -536,6 +572,100 @@ impl ApplicationHandler<AppEvent> for App {
                     eprintln!("pmacs-gpu: resize send_viewport failed: {e}");
                 }
             }
+            // Session M-2 — pointer input (docs/pmacs-gpu-mouse-framing.md).
+            WindowEvent::CursorMoved { position, .. } => {
+                let Some(state) = self.state.as_mut() else {
+                    return;
+                };
+                state.pointer_pos = Some((position.x, position.y));
+                if !state.pointer_drag_active {
+                    return;
+                }
+                // Drag coalescing (predicted finding #4): pixel-rate
+                // motion only ships when the hit byte changes.
+                let Some(byte) = state.hit_test_source_byte(position.x, position.y) else {
+                    return;
+                };
+                if state.last_pointer_sent_byte == Some(byte) {
+                    return;
+                }
+                state.last_pointer_sent_byte = Some(byte);
+                state.note_pointer_round_trip();
+                let buffer_id = state.current_buffer_id;
+                let mods = translate_mods(self.modifiers);
+                if let Some(buffer_id) = buffer_id {
+                    self.send_pointer(buffer_id, byte, PointerKind::Drag, mods);
+                }
+            }
+            WindowEvent::MouseInput {
+                state: button_state,
+                button: winit::event::MouseButton::Left,
+                ..
+            } => {
+                let Some(state) = self.state.as_mut() else {
+                    return;
+                };
+                let Some((x, y)) = state.pointer_pos else {
+                    return;
+                };
+                let mods = translate_mods(self.modifiers);
+                match button_state {
+                    ElementState::Pressed => {
+                        let Some(byte) = state.hit_test_source_byte(x, y) else {
+                            return;
+                        };
+                        let kind = state.classify_pointer_down(byte);
+                        state.pointer_drag_active = true;
+                        state.last_pointer_sent_byte = Some(byte);
+                        state.note_pointer_round_trip();
+                        if let Some(buffer_id) = state.current_buffer_id {
+                            if debug_input() {
+                                eprintln!("pmacs-gpu pointer: {kind:?} byte={byte}");
+                            }
+                            self.send_pointer(buffer_id, byte, kind, mods);
+                        }
+                    }
+                    ElementState::Released => {
+                        if !state.pointer_drag_active {
+                            return;
+                        }
+                        state.pointer_drag_active = false;
+                        let byte = state
+                            .hit_test_source_byte(x, y)
+                            .or(state.last_pointer_sent_byte);
+                        let buffer_id = state.current_buffer_id;
+                        if let (Some(byte), Some(buffer_id)) = (byte, buffer_id) {
+                            self.send_pointer(buffer_id, byte, PointerKind::Up, mods);
+                        }
+                    }
+                }
+            }
+            WindowEvent::MouseWheel { delta, .. } => {
+                let Some(state) = self.state.as_mut() else {
+                    return;
+                };
+                // Wheel scroll is local-only: the GPU owns the
+                // viewport. Positive winit y = scroll up = smaller
+                // scroll_top.
+                let lines = match delta {
+                    winit::event::MouseScrollDelta::LineDelta(_, y) => {
+                        (-y * WHEEL_LINES_PER_TICK).round() as i64
+                    }
+                    winit::event::MouseScrollDelta::PixelDelta(p) => {
+                        (-(p.y as f32) / CODE_LINE_HEIGHT).round() as i64
+                    }
+                };
+                if lines == 0 {
+                    return;
+                }
+                let vp = state.scroll_by_lines(lines);
+                if let Some(vp) = vp
+                    && let Some(client) = self.attach_client.as_ref()
+                    && let Err(e) = client.send_viewport(vp.buffer_id, vp.visible, vp.generation)
+                {
+                    eprintln!("pmacs-gpu: wheel send_viewport failed: {e}");
+                }
+            }
             WindowEvent::RedrawRequested => {
                 if let Some(state) = self.state.as_mut() {
                     state.render();
@@ -622,6 +752,14 @@ struct CrdtOpSend {
 /// tiny against a human noticing wedged keys.
 const FLOOR_CONFIRM_TIMEOUT: std::time::Duration = std::time::Duration::from_millis(500);
 
+/// Frontend-side double-click interval (Q#M1: the daemon cannot see
+/// pixels, so the frontend decides what a double-click is). Matches
+/// the TUI's `DOUBLE_CLICK_MAX_DELAY`.
+const DOUBLE_CLICK_WINDOW: std::time::Duration = std::time::Duration::from_millis(500);
+
+/// Wheel lines scrolled per `MouseScrollDelta::LineDelta` unit.
+const WHEEL_LINES_PER_TICK: f32 = 3.0;
+
 /// Byte range an optimistic Backspace/Delete removes at `cursor`, or
 /// `None` when it can't be predicted locally: buffer edge (the
 /// daemon's behavior is a no-op there anyway), a modifier variant
@@ -862,6 +1000,12 @@ impl State {
             deferred_round_trip_keys: Vec::new(),
             optimistic_floor_set_at: None,
             unconfirmed_edits: Vec::new(),
+            current_hit_runs: Vec::new(),
+            projected_line_starts: vec![0],
+            pointer_pos: None,
+            pointer_drag_active: false,
+            last_pointer_sent_byte: None,
+            last_pointer_down: None,
         }
     }
 
@@ -1587,6 +1731,67 @@ impl State {
         self.scroll_top != old
     }
 
+    /// Resolve a window-pixel position to an **absolute source byte**
+    /// (Q#M2): pixel → cosmic-text hit (shaped line + byte within
+    /// line) → projected byte → run map → slice byte → + `vstart`.
+    /// `None` when no buffer is attached or the position is outside
+    /// anything hit-testable.
+    fn hit_test_source_byte(&self, x: f64, y: f64) -> Option<u64> {
+        self.current_buffer_id?;
+        let rel_x = x as f32 - TEXT_LEFT;
+        let rel_y = y as f32 - TEXT_TOP;
+        let cursor = self.buffer.hit(rel_x, rel_y)?;
+        let line_start = *self.projected_line_starts.get(cursor.line)?;
+        let projected = line_start + cursor.index as u64;
+        let slice_byte = projected_to_source(&self.current_hit_runs, projected)?;
+        let (vstart, vend) = self.view_range;
+        Some((vstart + slice_byte).min(vend))
+    }
+
+    /// Wheel scroll (local-only — the GPU owns the viewport; no wire
+    /// event exists or is needed). Positive `delta` scrolls down.
+    fn scroll_by_lines(&mut self, delta: i64) -> Option<ViewportSend> {
+        let max_top = self.current_line_starts.len().saturating_sub(1);
+        let new_top = self
+            .scroll_top
+            .saturating_add_signed(delta as isize)
+            .min(max_top);
+        if new_top == self.scroll_top {
+            return None;
+        }
+        self.scroll_top = new_top;
+        self.reshape();
+        self.current_buffer_id
+            .and_then(|bid| self.viewport_send_if_changed(bid))
+    }
+
+    /// Bookkeeping for an outgoing Pointer event: it supersedes any
+    /// unconfirmed optimistic-cursor prediction (the daemon's answer
+    /// will be the click position, not the typing prediction), and
+    /// the cursor is not authoritative again until that `CursorByte`
+    /// lands.
+    fn note_pointer_round_trip(&mut self) {
+        self.cursor_fresh = false;
+        self.optimistic_cursor_floor = None;
+        self.optimistic_floor_set_at = None;
+    }
+
+    /// Frontend-side double-click detection: a second Down at the
+    /// same hit byte within the interval upgrades to `DoubleDown`.
+    fn classify_pointer_down(&mut self, byte: u64) -> PointerKind {
+        let now = std::time::Instant::now();
+        let is_double = self.last_pointer_down.take().is_some_and(|(at, prev)| {
+            prev == byte && now.duration_since(at) <= DOUBLE_CLICK_WINDOW
+        });
+        if is_double {
+            // A third click starts over (triple-click is deferred).
+            PointerKind::DoubleDown
+        } else {
+            self.last_pointer_down = Some((now, byte));
+            PointerKind::Down
+        }
+    }
+
     fn apply_file_style_summary(
         &mut self,
         buffer_id: BufferId,
@@ -1834,8 +2039,13 @@ impl State {
             .collect();
 
         let default_attrs = Attrs::new().family(Family::Name("JetBrains Mono"));
-        let chunks: Vec<(String, Attrs<'static>)> =
+        let rich = projected_rich_chunks(slice, &spans, &decorations, &adornments);
-            projected_rich_chunks(slice, &spans, &decorations, &adornments)
+        // Q#M2 — the pointer hit map is derived from the SAME chunks
+        // the shaped buffer is built from, so the two cannot disagree.
+        let (hit_runs, projected_line_starts) = build_hit_runs(&rich);
+        self.current_hit_runs = hit_runs;
+        self.projected_line_starts = projected_line_starts;
+        let chunks: Vec<(String, Attrs<'static>)> = rich
             .into_iter()
             .map(|chunk| {
                 let mut attrs = default_attrs.clone();
@@ -2269,6 +2479,74 @@ struct MinimapLineShape {
 struct RichChunk {
     text: String,
     color: Option<glyphon::Color>,
+    /// Where this chunk's text came from — the seam the pointer
+    /// hit-test walks back through (Q#M2).
+    source: ChunkSource,
+}
+
+/// Origin of one [`RichChunk`] in the shaped (projected) text.
+/// Offsets are slice-relative (the same space `projected_rich_chunks`
+/// works in); the hit test rebases with the slice's `vstart`.
+#[derive(Clone, Copy, Debug, PartialEq, Eq)]
+enum ChunkSource {
+    /// Verbatim source text starting at this slice byte offset.
+    Source { start: u64 },
+    /// Injected adornment text (inlay hint) anchored at this slice
+    /// byte offset. Hits inside it snap to the anchor.
+    Adornment { anchor: u64 },
+}
+
+/// One run of the projected→source hit map (Q#M2), built by
+/// [`build_hit_runs`] from the same chunks `reshape` feeds glyphon —
+/// so the map and the shaped buffer can never disagree.
+#[derive(Clone, Copy, Debug, PartialEq, Eq)]
+struct ProjectedRun {
+    /// Byte offset of this run in the shaped (projected) text.
+    projected_start: u64,
+    /// Run length in projected bytes.
+    len: u64,
+    source: ChunkSource,
+}
+
+/// Build the projected→source run map plus the projected text's line
+/// start table (cosmic-text reports hits as line + byte-within-line).
+fn build_hit_runs(chunks: &[RichChunk]) -> (Vec<ProjectedRun>, Vec<u64>) {
+    let mut runs = Vec::with_capacity(chunks.len());
+    let mut line_starts = vec![0u64];
+    let mut projected = 0u64;
+    for chunk in chunks {
+        let len = chunk.text.len() as u64;
+        runs.push(ProjectedRun {
+            projected_start: projected,
+            len,
+            source: chunk.source,
+        });
+        for (i, b) in chunk.text.bytes().enumerate() {
+            if b == b'\n' {
+                line_starts.push(projected + i as u64 + 1);
+            }
+        }
+        projected += len;
+    }
+    (runs, line_starts)
+}
+
+/// Map a projected byte offset back to a slice-relative source byte
+/// (Q#M2). Hits inside an adornment run snap to its anchor; offsets
+/// past the last run clamp to its end.
+fn projected_to_source(runs: &[ProjectedRun], projected: u64) -> Option<u64> {
+    if runs.is_empty() {
+        return None;
+    }
+    let idx = runs
+        .partition_point(|r| r.projected_start <= projected)
+        .saturating_sub(1);
+    let run = runs[idx];
+    let within = projected.saturating_sub(run.projected_start).min(run.len);
+    match run.source {
+        ChunkSource::Source { start } => Some(start + within),
+        ChunkSource::Adornment { anchor } => Some(anchor),
+    }
 }
 
 fn minimap_left(surface_width: u32) -> Option<f32> {
@@ -2591,10 +2869,7 @@ fn debug_input() -> bool {
 /// has no representation for (the daemon ignores `Key::Unknown`, so
 /// there's no value in forwarding them). `translate_key` covers the
 /// full editing set; session B1 gates the send on [`is_motion_key`].
-fn translate_key(
+fn translate_mods(mods: winit::keyboard::ModifiersState) -> Modifiers {
-    logical: &Key,
-    mods: winit::keyboard::ModifiersState,
-) -> Option<(ProtocolKey, Modifiers)> {
     let mut bits = 0u8;
     if mods.shift_key() {
         bits |= Modifiers::SHIFT.bits();
@@ -2608,7 +2883,14 @@ fn translate_key(
     if mods.super_key() {
         bits |= Modifiers::META.bits();
     }
-    let pmods = Modifiers::from_bits_truncate(bits);
+    Modifiers::from_bits_truncate(bits)
+}
+
+fn translate_key(
+    logical: &Key,
+    mods: winit::keyboard::ModifiersState,
+) -> Option<(ProtocolKey, Modifiers)> {
+    let pmods = translate_mods(mods);
 
     let pkey = match logical {
         Key::Named(named) => match named {
@@ -3248,6 +3530,7 @@ fn projected_rich_chunks(
             chunks.push(RichChunk {
                 text: text[a as usize..b as usize].to_owned(),
                 color: source_color_at(a, spans, decorations),
+                source: ChunkSource::Source { start: a },
             });
         }
     }
@@ -3261,6 +3544,7 @@ fn projected_rich_chunks(
         chunks.push(RichChunk {
             text: String::new(),
             color: None,
+            source: ChunkSource::Source { start: 0 },
         });
     }
     chunks
@@ -3292,6 +3576,7 @@ fn push_adornments_at(
             chunks.push(RichChunk {
                 text: text.clone(),
                 color: Some(adornment_text_color(style.fg)),
+                source: ChunkSource::Adornment { anchor },
             });
         }
         *next += 1;
@@ -3854,6 +4139,57 @@ mod tests {
         );
     }
 
+    #[test]
+    fn hit_runs_map_projected_bytes_back_to_source() {
+        // Source slice "ab\ncd" with an inlay hint ": i32 " anchored
+        // at byte 2 (end of "ab"): projected text = "ab: i32 \ncd".
+        let chunks = vec![
+            RichChunk {
+                text: "ab".into(),
+                color: None,
+                source: ChunkSource::Source { start: 0 },
+            },
+            RichChunk {
+                text: ": i32 ".into(),
+                color: None,
+                source: ChunkSource::Adornment { anchor: 2 },
+            },
+            RichChunk {
+                text: "\ncd".into(),
+                color: None,
+                source: ChunkSource::Source { start: 2 },
+            },
+        ];
+        let (runs, line_starts) = build_hit_runs(&chunks);
+
+        assert_eq!(
+            line_starts,
+            vec![0, 9],
+            "projected line table counts the newline at projected byte 8"
+        );
+
+        // Hits inside source runs map linearly.
+        assert_eq!(projected_to_source(&runs, 0), Some(0));
+        assert_eq!(projected_to_source(&runs, 1), Some(1));
+        assert_eq!(
+            projected_to_source(&runs, 9),
+            Some(3),
+            "projected 'c' (byte 9) maps to source byte 3"
+        );
+        // Hits inside the adornment snap to its anchor.
+        for projected in 2..8 {
+            assert_eq!(
+                projected_to_source(&runs, projected),
+                Some(2),
+                "adornment hit at projected {projected} snaps to the anchor"
+            );
+        }
+        // Past-the-end hits clamp into the last run.
+        assert_eq!(projected_to_source(&runs, 999), Some(5));
+        // Empty map: nothing to hit.
+        assert_eq!(projected_to_source(&[], 0), None);
+    }
+
     #[test]
     fn optimistic_delete_range_covers_single_codepoints_only() {
         let none = Modifiers::NONE;

pmacs-protocol/src/lib.rs

@@ -50,7 +50,7 @@ pub use message::{
     DecorationKind, DecorationSegment, FrontendCapabilities, FrontendEvent, GoodbyeReason, Hello,
     InlineAdornment, InstanceCapabilities, InstanceIdentity, InstanceMessage, InstanceSignal, Key,
     KeyEvent, Modifiers, MouseButton, MouseEvent, MouseKind, NegotiatedCapabilities,
-    PROTOCOL_VERSION, ResourceBody, SUPPORTED_PROTOCOL_VERSIONS, SelectionSnapshot, StyleSegment,
+    PROTOCOL_VERSION, PointerKind, ResourceBody, SUPPORTED_PROTOCOL_VERSIONS, SelectionSnapshot,
-    StyleSpan, is_supported_protocol_version, negotiate_capabilities,
+    StyleSegment, StyleSpan, is_supported_protocol_version, negotiate_capabilities,
 };
 pub use transport::{MAX_FRAME_BYTES, TransportError, read_message, write_message};

pmacs-protocol/src/message.rs

@@ -307,6 +307,51 @@ pub enum FrontendEvent {
         /// CRDT generation the frontend computed `visible` against.
         generation: u64,
     },
+    /// Q#M1 (protocol v5): a pointer gesture a *semantic* frontend
+    /// hit-tested locally to a **source byte offset**. The pixel→byte
+    /// resolution happens entirely frontend-side (the frontend owns
+    /// layout: fonts, inline adornments, scroll) — consistent with
+    /// `Viewport`'s no-pixels contract; the instance replays its
+    /// existing mouse gesture semantics in byte space, so selection
+    /// behavior stays single-sourced with the grid path.
+    ///
+    /// Cell-grid frontends keep using [`FrontendEvent::Mouse`]; the
+    /// two variants are per-session-kind, never mixed. Only sent when
+    /// the instance's `Hello.protocol_version >= 5` (an older
+    /// instance cannot decode the variant).
+    Pointer {
+        /// Which frontend produced the gesture (untrusted; the
+        /// instance routes by the authenticated session, matching
+        /// the `CrdtOp` / `Viewport` source-trust rule).
+        frontend_id: FrontendId,
+        /// Buffer the frontend was displaying when it hit-tested.
+        buffer_id: crate::BufferId,
+        /// Source byte offset of the hit (frontend-local hit test;
+        /// adornment runs already snapped to their anchors).
+        byte: u64,
+        /// Which gesture step this is.
+        kind: PointerKind,
+        /// Modifiers held during the gesture. Carried for future
+        /// Shift-click extension; the v5 instance ignores them.
+        mods: Modifiers,
+    },
+}
+
+/// Gesture step for [`FrontendEvent::Pointer`]. Double-click
+/// detection is frontend-side (`DoubleDown` instead of a second
+/// `Down`): only the frontend knows pixel proximity and its own
+/// double-click interval.
+#[derive(serde::Serialize, serde::Deserialize, Debug, Clone, Copy, PartialEq, Eq)]
+pub enum PointerKind {
+    /// Primary button pressed at `byte`.
+    Down,
+    /// Pointer moved to `byte` with the primary button held.
+    Drag,
+    /// Primary button released at `byte`.
+    Up,
+    /// Second press at the same hit within the frontend's
+    /// double-click window — selects the word at `byte`.
+    DoubleDown,
 }
 
 impl FrontendEvent {
@@ -322,7 +367,8 @@ impl FrontendEvent {
             | Self::FocusLost(frontend_id)
             | Self::Detach(frontend_id)
             | Self::CrdtOp { frontend_id, .. }
-            | Self::Viewport { frontend_id, .. } => *frontend_id,
+            | Self::Viewport { frontend_id, .. }
+            | Self::Pointer { frontend_id, .. } => *frontend_id,
         }
     }
 }
@@ -967,7 +1013,14 @@ pub enum ResourceBody {
 /// checks the session's negotiated version and skips the variant for
 /// older peers. An old peer would hard-error on decode of an unknown
 /// postcard variant; gating prevents that.
-pub const PROTOCOL_VERSION: u32 = 4;
+///
+/// Q#M1 (mouse framing): bumped from 4 to 5 for
+/// [`FrontendEvent::Pointer`]. The gate runs in the *frontend* this
+/// time (the new variant travels frontend→instance): a semantic
+/// frontend sends `Pointer` only when the instance's
+/// `Hello.protocol_version >= 5`, because an older instance would
+/// hard-error decoding the unknown variant.
+pub const PROTOCOL_VERSION: u32 = 5;
 
 /// T M10.5: the set of protocol versions a v1.0 binary accepts on
 /// the wire. v0.1 binaries only accepted `[1]`; v1.0 binaries accept
@@ -987,7 +1040,11 @@ pub const PROTOCOL_VERSION: u32 = 4;
 /// T M11.6: extended to `[1, 2, 3, 4]`. v4 sessions receive
 /// `InstanceMessage::DispatchIdle`; older sessions are filtered out
 /// of that emission.
-pub const SUPPORTED_PROTOCOL_VERSIONS: &[u32] = &[1, 2, 3, 4];
+///
+/// Mouse framing Q#M1: extended to `[1, 2, 3, 4, 5]`. v5 peers may
+/// send `FrontendEvent::Pointer`; the frontend-side gate (see
+/// [`PROTOCOL_VERSION`]) keeps the variant off wires negotiated `< 5`.
+pub const SUPPORTED_PROTOCOL_VERSIONS: &[u32] = &[1, 2, 3, 4, 5];
 
 /// T M10.5: predicate for the handshake check. Returns `true` if
 /// `peer_version` is in [`SUPPORTED_PROTOCOL_VERSIONS`].

pmacs-protocol/src/transport.rs

@@ -186,6 +186,19 @@ mod tests {
         round_trip(&h);
     }
 
+    #[test]
+    fn pointer_event_round_trips_through_transport() {
+        // Q#M1 (protocol v5): the byte-position pointer gesture.
+        let ev = FrontendEvent::Pointer {
+            frontend_id: FrontendId(7),
+            buffer_id: crate::BufferId::next(),
+            byte: 4096,
+            kind: crate::PointerKind::DoubleDown,
+            mods: Modifiers::SHIFT,
+        };
+        round_trip(&ev);
+    }
+
     #[test]
     fn attach_request_round_trips_through_transport() {
         let req = AttachRequest {

src/daemon.rs

@@ -1359,6 +1359,25 @@ fn handle_dispatcher_event(
                         }
                     }
                 }
+                FrontendEvent::Pointer {
+                    buffer_id,
+                    byte,
+                    kind,
+                    mods,
+                    ..
+                } => {
+                    // Mouse framing Q#M1 — a semantic frontend's
+                    // locally hit-tested gesture, in source bytes.
+                    // Routed by the authenticated `source` (the
+                    // client-supplied frontend_id is untrusted — the
+                    // CrdtOp / Viewport source-trust rule). The window
+                    // aligns to the buffer the frontend says it was
+                    // displaying: a click can race a buffer switch.
+                    if semantic_states.contains_key(&source) {
+                        align_semantic_window_to_buffer(editor, source, buffer_id);
+                        editor.dispatch_pointer(source, buffer_id, byte, kind, mods);
+                    }
+                }
                 _ => {
                     let term_size = *term_sizes
                         .get(&source)
@@ -2052,6 +2071,17 @@ fn apply_event(
                  handle_dispatcher_event. Dropping op."
             );
         }
+        FrontendEvent::Pointer { .. } => {
+            // Mouse framing Q#M1 — only semantic sessions emit
+            // Pointer, and `handle_dispatcher_event` routes those via
+            // `apply_semantic_input_event` (with the authenticated
+            // source). A grid session sending one is a protocol
+            // violation; drop it like the CrdtOp arm above.
+            eprintln!(
+                "pmacs daemon: FrontendEvent::Pointer from a grid session; dropping \
+                 (semantic sessions route via apply_semantic_input_event)"
+            );
+        }
     }
 }
 

src/editor.rs

@@ -779,6 +779,81 @@ impl EditorState {
             && prev.at.elapsed() <= DOUBLE_CLICK_MAX_DELAY
     }
 
+    /// Mouse framing Q#M1 — apply a semantic frontend's locally
+    /// hit-tested pointer gesture (`FrontendEvent::Pointer`) to its
+    /// window. The byte-space twin of [`Self::dispatch_mouse`]: same
+    /// gesture semantics, but the position arrives as a source byte
+    /// offset the frontend resolved against its own layout (fonts,
+    /// inline adornments, scroll), so no cell geometry is consulted.
+    ///
+    ///   * `Down` places the cursor and anchors a selection there
+    ///     (a following drag grows it).
+    ///   * `Drag` moves the cursor; the anchor stays.
+    ///   * `Up` collapses an empty selection (a click without drag).
+    ///   * `DoubleDown` selects the word at the hit (frontend-side
+    ///     double-click detection — only it knows pixel proximity).
+    ///
+    /// The hit byte is clamped into the buffer and snapped back to a
+    /// UTF-8 boundary: the frontend's hit may race an in-flight edit.
+    /// `mods` is carried for future Shift-click extension and ignored
+    /// today, matching `dispatch_mouse`.
+    pub fn dispatch_pointer(
+        &mut self,
+        frontend_id: FrontendId,
+        buffer_id: crate::buffer::BufferId,
+        byte: u64,
+        kind: crate::protocol::PointerKind,
+        _mods: crate::protocol::Modifiers,
+    ) {
+        use crate::protocol::PointerKind;
+        let mut core = self.core.borrow_mut();
+        core.active_frontend = frontend_id;
+        let Some(win_id) = core.views.get(&frontend_id).map(|v| v.active) else {
+            return;
+        };
+        // The dispatcher aligns the session's window to the declared
+        // buffer before calling here; re-check defensively (a click
+        // can race a buffer switch).
+        if core.windows.get(&win_id).map(|w| w.buffer_id) != Some(buffer_id) {
+            return;
+        }
+        core.set_active_window_id(win_id);
+        let byte = {
+            let registry = core.registry.clone();
+            let reg = registry.borrow();
+            let Ok(buf) = reg.get(buffer_id) else {
+                return;
+            };
+            snap_to_char_boundary(buf, byte)
+        };
+        match kind {
+            PointerKind::Down => {
+                let aw = core.active_window_mut();
+                aw.cursor = byte;
+                aw.goal_col = None;
+                core.begin_selection(byte);
+            }
+            PointerKind::Drag => {
+                let aw = core.active_window_mut();
+                aw.cursor = byte;
+                aw.goal_col = None;
+            }
+            PointerKind::Up => {
+                if let Some(sel) = core.active_window().selection
+                    && sel.anchor == core.cursor()
+                {
+                    core.clear_selection();
+                }
+            }
+            PointerKind::DoubleDown => {
+                let aw = core.active_window_mut();
+                aw.cursor = byte;
+                aw.goal_col = None;
+                core.select_word_at_cursor();
+            }
+        }
+    }
+
     /// Make `win_id` the active window and place its cursor at the
     /// buffer position corresponding to `(local_row, local_col)`,
     /// where the coordinates are relative to the window's viewport
@@ -1245,6 +1320,24 @@ fn paint_status_line(
 
 /// Rows available for buffer text inside `rect`, after subtracting
 /// the per-window mode line (one row).
+/// Clamp `pos` into `buf` and walk back to the nearest UTF-8
+/// codepoint boundary. Pointer hits arrive from a frontend whose text
+/// may be a few unconfirmed edits ahead of or behind the instance, so
+/// a raw byte offset can land mid-codepoint; a snapped position is
+/// always safe to assign to a window cursor.
+fn snap_to_char_boundary(buf: &crate::buffer::Buffer, pos: u64) -> u64 {
+    let len = buf.len();
+    let mut pos = pos.min(len);
+    while pos > 0 && pos < len {
+        match buf.snapshot_rope().byte_at(pos) {
+            // UTF-8 continuation byte (0b10xx_xxxx) ⇒ mid-codepoint.
+            Some(b) if b & 0b1100_0000 == 0b1000_0000 => pos -= 1,
+            _ => break,
+        }
+    }
+    pos
+}
+
 fn inner_rows(rect: &crate::window::Rect) -> u32 {
     rect.size.rows.saturating_sub(1)
 }
@@ -4249,6 +4342,55 @@ mod tests {
         assert!(s.core.borrow().active_region().is_none());
     }
 
+    /// Mouse framing Q#M1 — `dispatch_pointer` replays the mouse
+    /// gesture semantics in byte space for semantic frontends.
+    #[test]
+    fn dispatch_pointer_replays_mouse_semantics_in_byte_space() {
+        use crate::protocol::{Modifiers as WireMods, PointerKind};
+        // Bytes: h=0 é=1,2 ' '=3 l=4 l=5 o=6 ' '=7 w=8 ö=9,10 r=11
+        // l=12 d=13 \n=14; len=15.
+        let mut s = fresh_with("hé llo wörld\n".as_bytes());
+        let bid = s.core.borrow().active_buffer_id();
+        let none = WireMods::NONE;
+
+        // Down places the cursor — a mid-codepoint hit (inside 'é')
+        // snaps back to the boundary — and anchors a selection.
+        s.dispatch_pointer(FrontendId::LOCAL, bid, 2, PointerKind::Down, none);
+        assert_eq!(
+            s.core.borrow().cursor(),
+            1,
+            "mid-codepoint hit snaps to the char boundary"
+        );
+
+        // Drag grows the region from the anchor; Up keeps it.
+        s.dispatch_pointer(FrontendId::LOCAL, bid, 6, PointerKind::Drag, none);
+        assert_eq!(s.core.borrow().cursor(), 6);
+        assert_eq!(s.core.borrow().active_region(), Some((1, 6)));
+        s.dispatch_pointer(FrontendId::LOCAL, bid, 6, PointerKind::Up, none);
+        assert_eq!(s.core.borrow().active_region(), Some((1, 6)));
+
+        // A plain click (Down + Up, no drag) leaves no region.
+        s.dispatch_pointer(FrontendId::LOCAL, bid, 4, PointerKind::Down, none);
+        s.dispatch_pointer(FrontendId::LOCAL, bid, 4, PointerKind::Up, none);
+        assert_eq!(s.core.borrow().cursor(), 4);
+        assert!(s.core.borrow().active_region().is_none());
+
+        // DoubleDown selects the word at the hit ("wörld").
+        s.dispatch_pointer(FrontendId::LOCAL, bid, 8, PointerKind::DoubleDown, none);
+        assert_eq!(s.core.borrow().active_region(), Some((8, 14)));
+        assert_eq!(s.core.borrow().cursor(), 14);
+
+        // Past-EOF hits clamp to the buffer length.
+        s.dispatch_pointer(FrontendId::LOCAL, bid, 999, PointerKind::Down, none);
+        assert_eq!(s.core.borrow().cursor(), 15);
+
+        // A pointer for a buffer the window isn't displaying is
+        // dropped (click racing a buffer switch).
+        let other = crate::buffer::BufferId::next();
+        s.dispatch_pointer(FrontendId::LOCAL, other, 0, PointerKind::Down, none);
+        assert_eq!(s.core.borrow().cursor(), 15, "mismatched buffer ignored");
+    }
+
     /// Acceptance bullet 3: mouse events are coalesced at frame
     /// boundaries — many drag events between renders all apply, and
     /// the cursor ends up at the last position.

src/editor_core.rs

@@ -1406,11 +1406,17 @@ fn forward_word(buf: &Buffer, mut pos: Position) -> Position {
 }
 
 fn word_range_at(buf: &Buffer, pos: Position) -> Option<(Position, Position)> {
-    let (ch, _) = char_at(buf, pos)?;
+    let (ch, ch_len) = char_at(buf, pos)?;
     if !is_word_char(ch) {
         return None;
     }
-    let start = backward_word(buf, pos);
+    // Walk back from just *past* the char under the cursor, not from
+    // `pos` itself: `backward_word(pos)` at a word's FIRST character
+    // sees the non-word char before it, skips it, and crosses into
+    // the previous word — double-clicking the 'w' of "llo world"
+    // would select "llo world". From `pos + ch_len` the char behind
+    // is this word's own first char, so the walk stops at its start.
+    let start = backward_word(buf, pos.saturating_add(ch_len));
     let end = forward_word(buf, pos);
     (start < end).then_some((start, end))
 }

src/protocol.rs

@@ -1683,29 +1683,33 @@ mod tests {
     // --- M5.5a handshake & postcard round-trips ---
 
     #[test]
-    fn protocol_version_is_four_for_dispatch_idle() {
+    fn protocol_version_is_five_for_pointer_events() {
         // Pin the value: T M10.5 bumped 1→2 (v1.0 wire: CrdtOp /
         // PresenceUpdate). T M11.1 bumped 2→3 (v1.1 wire: the
         // SemanticFrame family + FrontendEvent::Viewport). T M11.6
         // bumped 3→4 (DispatchIdle for the optimistic-apply gate).
-        // The current binary serves v1..=v4 sessions — the slice-
+        // The mouse framing Q#M1 bumped 4→5 (FrontendEvent::Pointer,
+        // byte-position gestures from semantic frontends). The
+        // current binary serves v1..=v5 sessions — the slice-
         // membership handshake makes the relaxation symmetric.
-        assert_eq!(PROTOCOL_VERSION, 4);
+        assert_eq!(PROTOCOL_VERSION, 5);
     }
 
     #[test]
-    fn supported_protocol_versions_includes_one_through_four() {
+    fn supported_protocol_versions_includes_one_through_five() {
         // T M10.5: v1.0 binaries accept v1+v2. T M11.1: v1.1 binaries
-        // accept v1+v2+v3. T M11.6: v4 binaries accept v1+v2+v3+v4.
+        // accept v1+v2+v3. T M11.6: v4 binaries accept v1..=v4. Mouse
-        // The check is slice membership, not strict equality, so
+        // framing Q#M1: v5 binaries accept v1..=v5. The check is
-        // older binaries keep connecting to current binaries
+        // slice membership, not strict equality, so older binaries
-        // unchanged. v5+ is rejected until the next bump.
+        // keep connecting to current binaries unchanged. v6+ is
+        // rejected until the next bump.
         assert!(is_supported_protocol_version(1));
         assert!(is_supported_protocol_version(2));
         assert!(is_supported_protocol_version(3));
         assert!(is_supported_protocol_version(4));
+        assert!(is_supported_protocol_version(5));
         assert!(!is_supported_protocol_version(0));
-        assert!(!is_supported_protocol_version(5));
+        assert!(!is_supported_protocol_version(6));
         assert!(!is_supported_protocol_version(u32::MAX));
     }
 
@@ -2074,10 +2078,11 @@ mod tests {
     fn m10_5_handshake_matrix_versions_outside_range_rejected() {
         // v1 daemon's strict-equality behavior is documented at the
         // v0.1 code level (different binary); the current daemon's
-        // range check accepts v1/v2/v3/v4 (T M11.1 added v3; T M11.6
+        // range check accepts v1..=v5 (T M11.1 added v3; T M11.6
-        // added v4) and rejects v5+ until the next protocol bump.
+        // added v4; the mouse framing Q#M1 added v5) and rejects v6+
+        // until the next protocol bump.
         assert!(!is_supported_protocol_version(0));
-        assert!(!is_supported_protocol_version(5));
+        assert!(!is_supported_protocol_version(6));
         assert!(!is_supported_protocol_version(u32::MAX));
     }