diff --git a/docs/pmacs-gpu-mouse-framing.md b/docs/pmacs-gpu-mouse-framing.md new file mode 100644 index 0000000..ae27052 --- /dev/null +++ b/docs/pmacs-gpu-mouse-framing.md @@ -0,0 +1,164 @@ +# pmacs-gpu mouse input — framing pass + +Date: 2026-06-10. Follows the Phase B framing +(`pmacs-gpu-phase-b-framing.md`), which deferred the mouse wire +decision as Q#B5. The optimistic-typing arc is merged (PR #60, +main @ `7fdae76`); this is the next capability session. + +## What exists today (fact-checked) + +- **Wire**: `FrontendEvent::Mouse(MouseEvent)` carries a `CellCoord` + (`pmacs-protocol/src/message.rs:210-219`) — a *cell-grid* + coordinate. `PROTOCOL_VERSION = 4` (`message.rs:970`); the + DispatchIdle variant set the precedent for gating new variants on + `negotiated_protocol_version` (`daemon.rs:998`). +- **Daemon semantics**: `EditorState::dispatch_mouse` owns the full + gesture state machine — Down activates the window, positions the + cursor, begins a selection; Drag grows the region; Up collapses an + empty region; a second Down in the same cell within 500ms is a + double-click and calls `select_word_at_cursor`; ScrollUp/Down move + the viewport (`src/editor.rs`, CUA arc). For a *semantic* session, + `apply_semantic_input_event` already routes + `FrontendEvent::Mouse → mouse_to_crossterm → dispatch_mouse` + (`daemon.rs:1983-1986`) — against the session's placeholder 24×80 + cell geometry, which pmacs-gpu does not have. +- **GPU layout**: the shaped buffer holds the *visible slice* of the + source (`current_text[vstart..vend]`) with **inline adornment text + injected** by `projected_rich_chunks` (`pmacs-gpu/src/main.rs:3202`) + — so shaped-buffer byte offsets ≠ source bytes whenever inlay hints + are visible. Text is drawn at `TEXT_LEFT/TEXT_TOP` pixel offsets. +- **Hit testing**: `cosmic_text::Buffer::hit(x, y) -> Option` + exists (cosmic-text 0.18.2, `buffer.rs:923`) and returns a cursor in + the shaped buffer's (line, byte-index-within-line) space — the same + line-relative space QB3 taught us to rebase via the slice line + table. +- **Scroll wheel**: the GPU owns viewport + scroll state (S1: + `scroll_top` / slice reshape / viewport re-declaration), but + **no winit `MouseWheel` handler exists today** (fact-check: the + window_event catch-all swallows it; scrolling is keyboard-driven). + M-2 adds the handler — it composes from existing pieces + (`scroll_top` adjust → reshape → `viewport_send_if_changed`) and + needs no wire. + +## Q#M1 — the wire shape (resolves Q#B5) + +How does a pixel frontend tell the instance where a click landed? + +- **(α) Reuse `FrontendEvent::Mouse` with synthesized cell coords.** + The GPU would approximate `(row, col)` from pixels. REJECTED: the + daemon's cell math runs against window geometry the GPU doesn't + share (placeholder 24×80), inline adornments shift visual columns + with no cell-space representation, and the design doc's contract is + explicit — *the instance never learns a pixel, and there is no + hit-test round trip*. α is broken-by-design, not merely lossy. +- **(β1) Local hit-test → byte-position pointer events.** The GPU + resolves pixels to a **source byte offset** locally (it owns the + layout) and ships gesture-level events; the daemon replays its + existing mouse state machine in byte space. Selection semantics + (CUA, double-click word select, future triple-click) stay + instance-side, exactly like the Key round-trip philosophy. +- **(β2) Frontend-computed selection states.** The GPU computes + cursor+selection itself and ships final states. REJECTED: forks the + gesture semantics into a second implementation the TUI doesn't + share, and fights the daemon-authoritative selection model the CUA + arc just consolidated. + +**Stance: β1.** New wire variant: + +```rust +FrontendEvent::Pointer { + frontend_id: FrontendId, + buffer_id: BufferId, + /// Source byte offset the frontend hit-tested locally. + byte: u64, + kind: PointerKind, // Down | Drag | Up | DoubleDown + mods: Modifiers, +} +``` + +`DoubleDown` is detected frontend-side (the frontend knows pixel +proximity and its own double-click interval; the daemon's cell-based +double-click detection cannot see pixels). The daemon maps: +Down → activate semantic window + set cursor + `begin_selection`; +Drag → cursor move (region grows); Up → collapse-if-empty; +DoubleDown → `select_word_at_cursor`. This reuses the same core +primitives `dispatch_mouse` calls today — no new selection logic. + +Protocol: bump `PROTOCOL_VERSION` to 5; the GPU sends `Pointer` only +when `hello.protocol_version >= 5`; the daemon ignores the variant +for sessions that lack `semantic_render`. + +## Q#M2 — projected→source byte mapping + +`Buffer::hit` answers in *shaped* coordinates: line index + byte +offset within the shaped line, where shaped text = source slice with +adornment text spliced in. Two-step mapping: + +1. Shaped (line, index) → shaped-buffer byte offset via the shaped + line table (the QB3 rebase, but over the *projected* text). +2. Projected byte → **source byte** via a run map built during + `reshape`. **This is new work, not a current by-product** + (fact-check: `RichChunk` today carries only `text` + `color`, no + source range — `main.rs:2269`): `projected_rich_chunks` walks + (source-run | adornment-run) chunks in order, so M-2 extends it to + also emit + `Vec }>`. + A hit inside a source run maps linearly; a hit inside an adornment + run snaps to the adornment's source anchor. + +**Stance:** build the run map in `reshape` (it is O(chunks), walked +there anyway) rather than re-deriving adornment offsets at click +time. Clicks land between keystrokes, so the map being rebuilt per +reshape costs nothing new. + +## Q#M3 — gesture scope for the first session + +**In:** left-click cursor placement, left-drag selection, +double-click word select, wheel scroll (local-only, no wire). +**Deferred:** triple-click line select (needs a core +`select_line_at_cursor`), middle-click paste (needs clipboard +session), right-click menu (needs GUI chrome), drag auto-scroll at +window edges (needs a repeat timer; record as a known gap), +minimap click-to-jump (local-only; natural follow-up). + +## Q#M4 — interaction with optimistic state + +A pointer event is a round-trip input: it must (a) mark +`cursor_fresh = false` until the daemon's `CursorByte` answers, and +(b) defer behind the optimistic-cursor floor exactly like round-trip +keys do (`defer_round_trip_key_if_needed` shape) — a click landing +between unconfirmed keystrokes would otherwise race the cursor +confirmation. Deferred pointer events should keep only the **latest** +Down/Drag (coalescing, like the TUI's frame-boundary mouse +coalescing) rather than queueing every drag sample. + +## Predicted findings (categorical bets, scored at session close) + +1. **Coordinate-space bug** (the QB3 family): some offset in the + pixel→shaped→projected→source chain is wrong on first + implementation — most likely the adornment run snap or the + `TEXT_LEFT/TEXT_TOP` subtraction. Surfaces only in manual + validation with inlay hints visible. +2. **Window-activation gap** (the B1 family): the daemon-side Pointer + handler forgets some part of what `activate_and_position` does for + grid windows (window focus, `goal_col` reset), so a click works + but a subsequent arrow key moves from the wrong anchor. +3. **Selection-wash latency**: the wash arrives a round trip after + the drag (Decorations cadence), reading as rubber-band lag. If it + surfaces, the fix is frontend-local provisional selection — out of + scope, record it. +4. **Coalescing**: drag streams at pixel rate (hundreds of + events/sec) flood the socket or the dispatcher. Mitigation + in-scope: send Drag only when the hit byte changes. + +## Session plan + +- **M-1 (wire + daemon)**: `PointerKind`/`Pointer` variant, version + bump, daemon handler reusing core primitives, unit tests through + `handle_dispatcher_event`. Compile-green alone. +- **M-2 (GPU)**: winit mouse events → hit test → run map (new, from + `projected_rich_chunks`) → Pointer sends; drag coalescing on byte + change; NEW local wheel-scroll handler; optimistic-floor deferral. + Manual validation gate: click placement with inlay hints on the + same line, drag selection, double-click, wheel — in both pmacs-gpu + and a TUI peer simultaneously. diff --git a/pmacs-gpu/src/attach.rs b/pmacs-gpu/src/attach.rs index 0e4cf84..97b1262 100644 --- a/pmacs-gpu/src/attach.rs +++ b/pmacs-gpu/src/attach.rs @@ -23,7 +23,7 @@ use std::thread; use pmacs_protocol::{ AttachRequest, BufferId, ByteRange, CrdtOp, FrontendCapabilities, FrontendEvent, FrontendId, - Hello, InstanceMessage, Key, KeyEvent, Modifiers, PROTOCOL_VERSION, + Hello, InstanceMessage, Key, KeyEvent, Modifiers, PROTOCOL_VERSION, PointerKind, SUPPORTED_PROTOCOL_VERSIONS, TransportError, is_supported_protocol_version, read_message, write_message, }; @@ -202,6 +202,7 @@ pub fn connect( Ok(AttachClient { writer_tx, frontend_id: hello.assigned_frontend_id, + server_protocol_version: hello.protocol_version, }) } @@ -213,6 +214,10 @@ pub struct AttachClient { /// `FrontendEvent` carries this so the daemon can route input back /// to the per-session `SemanticRenderState`. frontend_id: FrontendId, + /// The daemon's `Hello.protocol_version`. Wire variants newer + /// than the daemon (e.g. `Pointer`, v5) must be gated on this — + /// an older daemon hard-errors decoding an unknown variant. + server_protocol_version: u32, } impl AttachClient { @@ -255,6 +260,30 @@ impl AttachClient { })) } + /// Send a `FrontendEvent::Pointer` (session M-2): a locally + /// hit-tested gesture in source bytes. Callers gate on + /// [`Self::server_protocol_version`] `>= 5`. + pub fn send_pointer( + &self, + buffer_id: BufferId, + byte: u64, + kind: PointerKind, + mods: Modifiers, + ) -> Result<(), TransportError> { + self.send_event(FrontendEvent::Pointer { + frontend_id: self.frontend_id, + buffer_id, + byte, + kind, + mods, + }) + } + + /// The daemon's negotiated wire version from `Hello`. + pub fn server_protocol_version(&self) -> u32 { + self.server_protocol_version + } + /// Send a locally-authored CRDT operation to the daemon. The GPU /// uses this for idle plain-text insertion after applying the same /// op to its local Loro replica, avoiding a Key round trip on the diff --git a/pmacs-gpu/src/main.rs b/pmacs-gpu/src/main.rs index 923a542..33173c2 100644 --- a/pmacs-gpu/src/main.rs +++ b/pmacs-gpu/src/main.rs @@ -36,7 +36,7 @@ use loro::{ContainerTrait, ExportMode}; use pmacs_protocol::{ AdornmentContent, AdornmentPlacement, BufferId, ByteRange, CrdtOp, Decoration, DecorationKind, DecorationSegment, FrontendId, InlineAdornment, InstanceMessage, Key as ProtocolKey, Modifiers, - SelectionSnapshot, StyleSegment, StyleSpan, + PointerKind, SelectionSnapshot, StyleSegment, StyleSpan, cell::{Color as CellColor, Style as CellStyle}, }; use wgpu::MultisampleState; @@ -391,6 +391,24 @@ struct State { /// ops can mis-prune by one frame; the next generation-keyed /// full resync self-corrects. unconfirmed_edits: Vec<(u64, TextProjectionEdit)>, + /// Q#M2 — projected→source hit map for the currently shaped + /// slice. Rebuilt by every `reshape` from the same chunks that + /// feed glyphon; source offsets are slice-relative (pair with + /// `view_range.0`). + current_hit_runs: Vec, + /// Line-start byte offsets of the *projected* text (cosmic-text + /// reports hits as line index + byte-within-line). + projected_line_starts: Vec, + /// Last reported pointer position, in window pixels. + pointer_pos: Option<(f64, f64)>, + /// Primary button is held after a Down inside the text area. + pointer_drag_active: bool, + /// Hit byte of the last Pointer event sent — Drag coalescing: + /// pixel-rate motion only ships when the hit byte changes. + last_pointer_sent_byte: Option, + /// `(when, byte)` of the last primary Down, for frontend-side + /// double-click detection (same-hit within the interval). + last_pointer_down: Option<(std::time::Instant, u64)>, } /// pmacs-gpu's own cursor position, mirrored from `CursorByte`. @@ -422,6 +440,23 @@ struct FileStyleSummaryState { lines: Vec, } +impl App { + /// Ship a Pointer event if the daemon speaks protocol v5+ — the + /// Q#M1 frontend-side gate (an older instance cannot decode the + /// variant and would drop the connection). + fn send_pointer(&self, buffer_id: BufferId, byte: u64, kind: PointerKind, mods: Modifiers) { + let Some(client) = self.attach_client.as_ref() else { + return; + }; + if client.server_protocol_version() < 5 { + return; + } + if let Err(e) = client.send_pointer(buffer_id, byte, kind, mods) { + eprintln!("pmacs-gpu: send_pointer failed: {e}"); + } + } +} + impl ApplicationHandler for App { fn resumed(&mut self, event_loop: &ActiveEventLoop) { if self.state.is_some() { @@ -456,6 +491,7 @@ impl ApplicationHandler for App { } } + #[allow(clippy::too_many_lines)] // linear per-event dispatch; splitting hides the input flow. fn window_event(&mut self, event_loop: &ActiveEventLoop, _id: WindowId, event: WindowEvent) { match event { WindowEvent::CloseRequested => event_loop.exit(), @@ -536,6 +572,100 @@ impl ApplicationHandler for App { eprintln!("pmacs-gpu: resize send_viewport failed: {e}"); } } + // Session M-2 — pointer input (docs/pmacs-gpu-mouse-framing.md). + WindowEvent::CursorMoved { position, .. } => { + let Some(state) = self.state.as_mut() else { + return; + }; + state.pointer_pos = Some((position.x, position.y)); + if !state.pointer_drag_active { + return; + } + // Drag coalescing (predicted finding #4): pixel-rate + // motion only ships when the hit byte changes. + let Some(byte) = state.hit_test_source_byte(position.x, position.y) else { + return; + }; + if state.last_pointer_sent_byte == Some(byte) { + return; + } + state.last_pointer_sent_byte = Some(byte); + state.note_pointer_round_trip(); + let buffer_id = state.current_buffer_id; + let mods = translate_mods(self.modifiers); + if let Some(buffer_id) = buffer_id { + self.send_pointer(buffer_id, byte, PointerKind::Drag, mods); + } + } + WindowEvent::MouseInput { + state: button_state, + button: winit::event::MouseButton::Left, + .. + } => { + let Some(state) = self.state.as_mut() else { + return; + }; + let Some((x, y)) = state.pointer_pos else { + return; + }; + let mods = translate_mods(self.modifiers); + match button_state { + ElementState::Pressed => { + let Some(byte) = state.hit_test_source_byte(x, y) else { + return; + }; + let kind = state.classify_pointer_down(byte); + state.pointer_drag_active = true; + state.last_pointer_sent_byte = Some(byte); + state.note_pointer_round_trip(); + if let Some(buffer_id) = state.current_buffer_id { + if debug_input() { + eprintln!("pmacs-gpu pointer: {kind:?} byte={byte}"); + } + self.send_pointer(buffer_id, byte, kind, mods); + } + } + ElementState::Released => { + if !state.pointer_drag_active { + return; + } + state.pointer_drag_active = false; + let byte = state + .hit_test_source_byte(x, y) + .or(state.last_pointer_sent_byte); + let buffer_id = state.current_buffer_id; + if let (Some(byte), Some(buffer_id)) = (byte, buffer_id) { + self.send_pointer(buffer_id, byte, PointerKind::Up, mods); + } + } + } + } + WindowEvent::MouseWheel { delta, .. } => { + let Some(state) = self.state.as_mut() else { + return; + }; + // Wheel scroll is local-only: the GPU owns the + // viewport. Positive winit y = scroll up = smaller + // scroll_top. + let lines = match delta { + winit::event::MouseScrollDelta::LineDelta(_, y) => { + (-y * WHEEL_LINES_PER_TICK).round() as i64 + } + winit::event::MouseScrollDelta::PixelDelta(p) => { + (-(p.y as f32) / CODE_LINE_HEIGHT).round() as i64 + } + }; + if lines == 0 { + return; + } + let vp = state.scroll_by_lines(lines); + if let Some(vp) = vp + && let Some(client) = self.attach_client.as_ref() + && let Err(e) = client.send_viewport(vp.buffer_id, vp.visible, vp.generation) + { + eprintln!("pmacs-gpu: wheel send_viewport failed: {e}"); + } + } WindowEvent::RedrawRequested => { if let Some(state) = self.state.as_mut() { state.render(); @@ -622,6 +752,14 @@ struct CrdtOpSend { /// tiny against a human noticing wedged keys. const FLOOR_CONFIRM_TIMEOUT: std::time::Duration = std::time::Duration::from_millis(500); +/// Frontend-side double-click interval (Q#M1: the daemon cannot see +/// pixels, so the frontend decides what a double-click is). Matches +/// the TUI's `DOUBLE_CLICK_MAX_DELAY`. +const DOUBLE_CLICK_WINDOW: std::time::Duration = std::time::Duration::from_millis(500); + +/// Wheel lines scrolled per `MouseScrollDelta::LineDelta` unit. +const WHEEL_LINES_PER_TICK: f32 = 3.0; + /// Byte range an optimistic Backspace/Delete removes at `cursor`, or /// `None` when it can't be predicted locally: buffer edge (the /// daemon's behavior is a no-op there anyway), a modifier variant @@ -862,6 +1000,12 @@ impl State { deferred_round_trip_keys: Vec::new(), optimistic_floor_set_at: None, unconfirmed_edits: Vec::new(), + current_hit_runs: Vec::new(), + projected_line_starts: vec![0], + pointer_pos: None, + pointer_drag_active: false, + last_pointer_sent_byte: None, + last_pointer_down: None, } } @@ -1587,6 +1731,67 @@ impl State { self.scroll_top != old } + /// Resolve a window-pixel position to an **absolute source byte** + /// (Q#M2): pixel → cosmic-text hit (shaped line + byte within + /// line) → projected byte → run map → slice byte → + `vstart`. + /// `None` when no buffer is attached or the position is outside + /// anything hit-testable. + fn hit_test_source_byte(&self, x: f64, y: f64) -> Option { + self.current_buffer_id?; + let rel_x = x as f32 - TEXT_LEFT; + let rel_y = y as f32 - TEXT_TOP; + let cursor = self.buffer.hit(rel_x, rel_y)?; + let line_start = *self.projected_line_starts.get(cursor.line)?; + let projected = line_start + cursor.index as u64; + let slice_byte = projected_to_source(&self.current_hit_runs, projected)?; + let (vstart, vend) = self.view_range; + Some((vstart + slice_byte).min(vend)) + } + + /// Wheel scroll (local-only — the GPU owns the viewport; no wire + /// event exists or is needed). Positive `delta` scrolls down. + fn scroll_by_lines(&mut self, delta: i64) -> Option { + let max_top = self.current_line_starts.len().saturating_sub(1); + let new_top = self + .scroll_top + .saturating_add_signed(delta as isize) + .min(max_top); + if new_top == self.scroll_top { + return None; + } + self.scroll_top = new_top; + self.reshape(); + self.current_buffer_id + .and_then(|bid| self.viewport_send_if_changed(bid)) + } + + /// Bookkeeping for an outgoing Pointer event: it supersedes any + /// unconfirmed optimistic-cursor prediction (the daemon's answer + /// will be the click position, not the typing prediction), and + /// the cursor is not authoritative again until that `CursorByte` + /// lands. + fn note_pointer_round_trip(&mut self) { + self.cursor_fresh = false; + self.optimistic_cursor_floor = None; + self.optimistic_floor_set_at = None; + } + + /// Frontend-side double-click detection: a second Down at the + /// same hit byte within the interval upgrades to `DoubleDown`. + fn classify_pointer_down(&mut self, byte: u64) -> PointerKind { + let now = std::time::Instant::now(); + let is_double = self.last_pointer_down.take().is_some_and(|(at, prev)| { + prev == byte && now.duration_since(at) <= DOUBLE_CLICK_WINDOW + }); + if is_double { + // A third click starts over (triple-click is deferred). + PointerKind::DoubleDown + } else { + self.last_pointer_down = Some((now, byte)); + PointerKind::Down + } + } + fn apply_file_style_summary( &mut self, buffer_id: BufferId, @@ -1834,17 +2039,22 @@ impl State { .collect(); let default_attrs = Attrs::new().family(Family::Name("JetBrains Mono")); - let chunks: Vec<(String, Attrs<'static>)> = - projected_rich_chunks(slice, &spans, &decorations, &adornments) - .into_iter() - .map(|chunk| { - let mut attrs = default_attrs.clone(); - if let Some(c) = chunk.color { - attrs = attrs.color(c); - } - (chunk.text, attrs) - }) - .collect(); + let rich = projected_rich_chunks(slice, &spans, &decorations, &adornments); + // Q#M2 — the pointer hit map is derived from the SAME chunks + // the shaped buffer is built from, so the two cannot disagree. + let (hit_runs, projected_line_starts) = build_hit_runs(&rich); + self.current_hit_runs = hit_runs; + self.projected_line_starts = projected_line_starts; + let chunks: Vec<(String, Attrs<'static>)> = rich + .into_iter() + .map(|chunk| { + let mut attrs = default_attrs.clone(); + if let Some(c) = chunk.color { + attrs = attrs.color(c); + } + (chunk.text, attrs) + }) + .collect(); self.buffer.set_rich_text( &mut self.font_system, chunks.iter().map(|(s, a)| (s.as_str(), a.clone())), @@ -2269,6 +2479,74 @@ struct MinimapLineShape { struct RichChunk { text: String, color: Option, + /// Where this chunk's text came from — the seam the pointer + /// hit-test walks back through (Q#M2). + source: ChunkSource, +} + +/// Origin of one [`RichChunk`] in the shaped (projected) text. +/// Offsets are slice-relative (the same space `projected_rich_chunks` +/// works in); the hit test rebases with the slice's `vstart`. +#[derive(Clone, Copy, Debug, PartialEq, Eq)] +enum ChunkSource { + /// Verbatim source text starting at this slice byte offset. + Source { start: u64 }, + /// Injected adornment text (inlay hint) anchored at this slice + /// byte offset. Hits inside it snap to the anchor. + Adornment { anchor: u64 }, +} + +/// One run of the projected→source hit map (Q#M2), built by +/// [`build_hit_runs`] from the same chunks `reshape` feeds glyphon — +/// so the map and the shaped buffer can never disagree. +#[derive(Clone, Copy, Debug, PartialEq, Eq)] +struct ProjectedRun { + /// Byte offset of this run in the shaped (projected) text. + projected_start: u64, + /// Run length in projected bytes. + len: u64, + source: ChunkSource, +} + +/// Build the projected→source run map plus the projected text's line +/// start table (cosmic-text reports hits as line + byte-within-line). +fn build_hit_runs(chunks: &[RichChunk]) -> (Vec, Vec) { + let mut runs = Vec::with_capacity(chunks.len()); + let mut line_starts = vec![0u64]; + let mut projected = 0u64; + for chunk in chunks { + let len = chunk.text.len() as u64; + runs.push(ProjectedRun { + projected_start: projected, + len, + source: chunk.source, + }); + for (i, b) in chunk.text.bytes().enumerate() { + if b == b'\n' { + line_starts.push(projected + i as u64 + 1); + } + } + projected += len; + } + (runs, line_starts) +} + +/// Map a projected byte offset back to a slice-relative source byte +/// (Q#M2). Hits inside an adornment run snap to its anchor; offsets +/// past the last run clamp to its end. +fn projected_to_source(runs: &[ProjectedRun], projected: u64) -> Option { + if runs.is_empty() { + return None; + } + let idx = runs + .partition_point(|r| r.projected_start <= projected) + .saturating_sub(1); + let run = runs[idx]; + let within = projected.saturating_sub(run.projected_start).min(run.len); + match run.source { + ChunkSource::Source { start } => Some(start + within), + ChunkSource::Adornment { anchor } => Some(anchor), + } } fn minimap_left(surface_width: u32) -> Option { @@ -2591,10 +2869,7 @@ fn debug_input() -> bool { /// has no representation for (the daemon ignores `Key::Unknown`, so /// there's no value in forwarding them). `translate_key` covers the /// full editing set; session B1 gates the send on [`is_motion_key`]. -fn translate_key( - logical: &Key, - mods: winit::keyboard::ModifiersState, -) -> Option<(ProtocolKey, Modifiers)> { +fn translate_mods(mods: winit::keyboard::ModifiersState) -> Modifiers { let mut bits = 0u8; if mods.shift_key() { bits |= Modifiers::SHIFT.bits(); @@ -2608,7 +2883,14 @@ fn translate_key( if mods.super_key() { bits |= Modifiers::META.bits(); } - let pmods = Modifiers::from_bits_truncate(bits); + Modifiers::from_bits_truncate(bits) +} + +fn translate_key( + logical: &Key, + mods: winit::keyboard::ModifiersState, +) -> Option<(ProtocolKey, Modifiers)> { + let pmods = translate_mods(mods); let pkey = match logical { Key::Named(named) => match named { @@ -3248,6 +3530,7 @@ fn projected_rich_chunks( chunks.push(RichChunk { text: text[a as usize..b as usize].to_owned(), color: source_color_at(a, spans, decorations), + source: ChunkSource::Source { start: a }, }); } } @@ -3261,6 +3544,7 @@ fn projected_rich_chunks( chunks.push(RichChunk { text: String::new(), color: None, + source: ChunkSource::Source { start: 0 }, }); } chunks @@ -3292,6 +3576,7 @@ fn push_adornments_at( chunks.push(RichChunk { text: text.clone(), color: Some(adornment_text_color(style.fg)), + source: ChunkSource::Adornment { anchor }, }); } *next += 1; @@ -3854,6 +4139,57 @@ mod tests { ); } + #[test] + fn hit_runs_map_projected_bytes_back_to_source() { + // Source slice "ab\ncd" with an inlay hint ": i32 " anchored + // at byte 2 (end of "ab"): projected text = "ab: i32 \ncd". + let chunks = vec![ + RichChunk { + text: "ab".into(), + color: None, + source: ChunkSource::Source { start: 0 }, + }, + RichChunk { + text: ": i32 ".into(), + color: None, + source: ChunkSource::Adornment { anchor: 2 }, + }, + RichChunk { + text: "\ncd".into(), + color: None, + source: ChunkSource::Source { start: 2 }, + }, + ]; + let (runs, line_starts) = build_hit_runs(&chunks); + + assert_eq!( + line_starts, + vec![0, 9], + "projected line table counts the newline at projected byte 8" + ); + + // Hits inside source runs map linearly. + assert_eq!(projected_to_source(&runs, 0), Some(0)); + assert_eq!(projected_to_source(&runs, 1), Some(1)); + assert_eq!( + projected_to_source(&runs, 9), + Some(3), + "projected 'c' (byte 9) maps to source byte 3" + ); + // Hits inside the adornment snap to its anchor. + for projected in 2..8 { + assert_eq!( + projected_to_source(&runs, projected), + Some(2), + "adornment hit at projected {projected} snaps to the anchor" + ); + } + // Past-the-end hits clamp into the last run. + assert_eq!(projected_to_source(&runs, 999), Some(5)); + // Empty map: nothing to hit. + assert_eq!(projected_to_source(&[], 0), None); + } + #[test] fn optimistic_delete_range_covers_single_codepoints_only() { let none = Modifiers::NONE; diff --git a/pmacs-protocol/src/lib.rs b/pmacs-protocol/src/lib.rs index ddae434..feec39d 100644 --- a/pmacs-protocol/src/lib.rs +++ b/pmacs-protocol/src/lib.rs @@ -50,7 +50,7 @@ pub use message::{ DecorationKind, DecorationSegment, FrontendCapabilities, FrontendEvent, GoodbyeReason, Hello, InlineAdornment, InstanceCapabilities, InstanceIdentity, InstanceMessage, InstanceSignal, Key, KeyEvent, Modifiers, MouseButton, MouseEvent, MouseKind, NegotiatedCapabilities, - PROTOCOL_VERSION, ResourceBody, SUPPORTED_PROTOCOL_VERSIONS, SelectionSnapshot, StyleSegment, - StyleSpan, is_supported_protocol_version, negotiate_capabilities, + PROTOCOL_VERSION, PointerKind, ResourceBody, SUPPORTED_PROTOCOL_VERSIONS, SelectionSnapshot, + StyleSegment, StyleSpan, is_supported_protocol_version, negotiate_capabilities, }; pub use transport::{MAX_FRAME_BYTES, TransportError, read_message, write_message}; diff --git a/pmacs-protocol/src/message.rs b/pmacs-protocol/src/message.rs index 5b478c3..df07cb5 100644 --- a/pmacs-protocol/src/message.rs +++ b/pmacs-protocol/src/message.rs @@ -307,6 +307,51 @@ pub enum FrontendEvent { /// CRDT generation the frontend computed `visible` against. generation: u64, }, + /// Q#M1 (protocol v5): a pointer gesture a *semantic* frontend + /// hit-tested locally to a **source byte offset**. The pixel→byte + /// resolution happens entirely frontend-side (the frontend owns + /// layout: fonts, inline adornments, scroll) — consistent with + /// `Viewport`'s no-pixels contract; the instance replays its + /// existing mouse gesture semantics in byte space, so selection + /// behavior stays single-sourced with the grid path. + /// + /// Cell-grid frontends keep using [`FrontendEvent::Mouse`]; the + /// two variants are per-session-kind, never mixed. Only sent when + /// the instance's `Hello.protocol_version >= 5` (an older + /// instance cannot decode the variant). + Pointer { + /// Which frontend produced the gesture (untrusted; the + /// instance routes by the authenticated session, matching + /// the `CrdtOp` / `Viewport` source-trust rule). + frontend_id: FrontendId, + /// Buffer the frontend was displaying when it hit-tested. + buffer_id: crate::BufferId, + /// Source byte offset of the hit (frontend-local hit test; + /// adornment runs already snapped to their anchors). + byte: u64, + /// Which gesture step this is. + kind: PointerKind, + /// Modifiers held during the gesture. Carried for future + /// Shift-click extension; the v5 instance ignores them. + mods: Modifiers, + }, +} + +/// Gesture step for [`FrontendEvent::Pointer`]. Double-click +/// detection is frontend-side (`DoubleDown` instead of a second +/// `Down`): only the frontend knows pixel proximity and its own +/// double-click interval. +#[derive(serde::Serialize, serde::Deserialize, Debug, Clone, Copy, PartialEq, Eq)] +pub enum PointerKind { + /// Primary button pressed at `byte`. + Down, + /// Pointer moved to `byte` with the primary button held. + Drag, + /// Primary button released at `byte`. + Up, + /// Second press at the same hit within the frontend's + /// double-click window — selects the word at `byte`. + DoubleDown, } impl FrontendEvent { @@ -322,7 +367,8 @@ impl FrontendEvent { | Self::FocusLost(frontend_id) | Self::Detach(frontend_id) | Self::CrdtOp { frontend_id, .. } - | Self::Viewport { frontend_id, .. } => *frontend_id, + | Self::Viewport { frontend_id, .. } + | Self::Pointer { frontend_id, .. } => *frontend_id, } } } @@ -967,7 +1013,14 @@ pub enum ResourceBody { /// checks the session's negotiated version and skips the variant for /// older peers. An old peer would hard-error on decode of an unknown /// postcard variant; gating prevents that. -pub const PROTOCOL_VERSION: u32 = 4; +/// +/// Q#M1 (mouse framing): bumped from 4 to 5 for +/// [`FrontendEvent::Pointer`]. The gate runs in the *frontend* this +/// time (the new variant travels frontend→instance): a semantic +/// frontend sends `Pointer` only when the instance's +/// `Hello.protocol_version >= 5`, because an older instance would +/// hard-error decoding the unknown variant. +pub const PROTOCOL_VERSION: u32 = 5; /// T M10.5: the set of protocol versions a v1.0 binary accepts on /// the wire. v0.1 binaries only accepted `[1]`; v1.0 binaries accept @@ -987,7 +1040,11 @@ pub const PROTOCOL_VERSION: u32 = 4; /// T M11.6: extended to `[1, 2, 3, 4]`. v4 sessions receive /// `InstanceMessage::DispatchIdle`; older sessions are filtered out /// of that emission. -pub const SUPPORTED_PROTOCOL_VERSIONS: &[u32] = &[1, 2, 3, 4]; +/// +/// Mouse framing Q#M1: extended to `[1, 2, 3, 4, 5]`. v5 peers may +/// send `FrontendEvent::Pointer`; the frontend-side gate (see +/// [`PROTOCOL_VERSION`]) keeps the variant off wires negotiated `< 5`. +pub const SUPPORTED_PROTOCOL_VERSIONS: &[u32] = &[1, 2, 3, 4, 5]; /// T M10.5: predicate for the handshake check. Returns `true` if /// `peer_version` is in [`SUPPORTED_PROTOCOL_VERSIONS`]. diff --git a/pmacs-protocol/src/transport.rs b/pmacs-protocol/src/transport.rs index 82e06db..47bb4df 100644 --- a/pmacs-protocol/src/transport.rs +++ b/pmacs-protocol/src/transport.rs @@ -186,6 +186,19 @@ mod tests { round_trip(&h); } + #[test] + fn pointer_event_round_trips_through_transport() { + // Q#M1 (protocol v5): the byte-position pointer gesture. + let ev = FrontendEvent::Pointer { + frontend_id: FrontendId(7), + buffer_id: crate::BufferId::next(), + byte: 4096, + kind: crate::PointerKind::DoubleDown, + mods: Modifiers::SHIFT, + }; + round_trip(&ev); + } + #[test] fn attach_request_round_trips_through_transport() { let req = AttachRequest { diff --git a/src/daemon.rs b/src/daemon.rs index a1a0a93..86531a5 100644 --- a/src/daemon.rs +++ b/src/daemon.rs @@ -1359,6 +1359,25 @@ fn handle_dispatcher_event( } } } + FrontendEvent::Pointer { + buffer_id, + byte, + kind, + mods, + .. + } => { + // Mouse framing Q#M1 — a semantic frontend's + // locally hit-tested gesture, in source bytes. + // Routed by the authenticated `source` (the + // client-supplied frontend_id is untrusted — the + // CrdtOp / Viewport source-trust rule). The window + // aligns to the buffer the frontend says it was + // displaying: a click can race a buffer switch. + if semantic_states.contains_key(&source) { + align_semantic_window_to_buffer(editor, source, buffer_id); + editor.dispatch_pointer(source, buffer_id, byte, kind, mods); + } + } _ => { let term_size = *term_sizes .get(&source) @@ -2052,6 +2071,17 @@ fn apply_event( handle_dispatcher_event. Dropping op." ); } + FrontendEvent::Pointer { .. } => { + // Mouse framing Q#M1 — only semantic sessions emit + // Pointer, and `handle_dispatcher_event` routes those via + // `apply_semantic_input_event` (with the authenticated + // source). A grid session sending one is a protocol + // violation; drop it like the CrdtOp arm above. + eprintln!( + "pmacs daemon: FrontendEvent::Pointer from a grid session; dropping \ + (semantic sessions route via apply_semantic_input_event)" + ); + } } } diff --git a/src/editor.rs b/src/editor.rs index d5999bf..861af26 100644 --- a/src/editor.rs +++ b/src/editor.rs @@ -779,6 +779,81 @@ impl EditorState { && prev.at.elapsed() <= DOUBLE_CLICK_MAX_DELAY } + /// Mouse framing Q#M1 — apply a semantic frontend's locally + /// hit-tested pointer gesture (`FrontendEvent::Pointer`) to its + /// window. The byte-space twin of [`Self::dispatch_mouse`]: same + /// gesture semantics, but the position arrives as a source byte + /// offset the frontend resolved against its own layout (fonts, + /// inline adornments, scroll), so no cell geometry is consulted. + /// + /// * `Down` places the cursor and anchors a selection there + /// (a following drag grows it). + /// * `Drag` moves the cursor; the anchor stays. + /// * `Up` collapses an empty selection (a click without drag). + /// * `DoubleDown` selects the word at the hit (frontend-side + /// double-click detection — only it knows pixel proximity). + /// + /// The hit byte is clamped into the buffer and snapped back to a + /// UTF-8 boundary: the frontend's hit may race an in-flight edit. + /// `mods` is carried for future Shift-click extension and ignored + /// today, matching `dispatch_mouse`. + pub fn dispatch_pointer( + &mut self, + frontend_id: FrontendId, + buffer_id: crate::buffer::BufferId, + byte: u64, + kind: crate::protocol::PointerKind, + _mods: crate::protocol::Modifiers, + ) { + use crate::protocol::PointerKind; + let mut core = self.core.borrow_mut(); + core.active_frontend = frontend_id; + let Some(win_id) = core.views.get(&frontend_id).map(|v| v.active) else { + return; + }; + // The dispatcher aligns the session's window to the declared + // buffer before calling here; re-check defensively (a click + // can race a buffer switch). + if core.windows.get(&win_id).map(|w| w.buffer_id) != Some(buffer_id) { + return; + } + core.set_active_window_id(win_id); + let byte = { + let registry = core.registry.clone(); + let reg = registry.borrow(); + let Ok(buf) = reg.get(buffer_id) else { + return; + }; + snap_to_char_boundary(buf, byte) + }; + match kind { + PointerKind::Down => { + let aw = core.active_window_mut(); + aw.cursor = byte; + aw.goal_col = None; + core.begin_selection(byte); + } + PointerKind::Drag => { + let aw = core.active_window_mut(); + aw.cursor = byte; + aw.goal_col = None; + } + PointerKind::Up => { + if let Some(sel) = core.active_window().selection + && sel.anchor == core.cursor() + { + core.clear_selection(); + } + } + PointerKind::DoubleDown => { + let aw = core.active_window_mut(); + aw.cursor = byte; + aw.goal_col = None; + core.select_word_at_cursor(); + } + } + } + /// Make `win_id` the active window and place its cursor at the /// buffer position corresponding to `(local_row, local_col)`, /// where the coordinates are relative to the window's viewport @@ -1245,6 +1320,24 @@ fn paint_status_line( /// Rows available for buffer text inside `rect`, after subtracting /// the per-window mode line (one row). +/// Clamp `pos` into `buf` and walk back to the nearest UTF-8 +/// codepoint boundary. Pointer hits arrive from a frontend whose text +/// may be a few unconfirmed edits ahead of or behind the instance, so +/// a raw byte offset can land mid-codepoint; a snapped position is +/// always safe to assign to a window cursor. +fn snap_to_char_boundary(buf: &crate::buffer::Buffer, pos: u64) -> u64 { + let len = buf.len(); + let mut pos = pos.min(len); + while pos > 0 && pos < len { + match buf.snapshot_rope().byte_at(pos) { + // UTF-8 continuation byte (0b10xx_xxxx) ⇒ mid-codepoint. + Some(b) if b & 0b1100_0000 == 0b1000_0000 => pos -= 1, + _ => break, + } + } + pos +} + fn inner_rows(rect: &crate::window::Rect) -> u32 { rect.size.rows.saturating_sub(1) } @@ -4249,6 +4342,55 @@ mod tests { assert!(s.core.borrow().active_region().is_none()); } + /// Mouse framing Q#M1 — `dispatch_pointer` replays the mouse + /// gesture semantics in byte space for semantic frontends. + #[test] + fn dispatch_pointer_replays_mouse_semantics_in_byte_space() { + use crate::protocol::{Modifiers as WireMods, PointerKind}; + // Bytes: h=0 é=1,2 ' '=3 l=4 l=5 o=6 ' '=7 w=8 ö=9,10 r=11 + // l=12 d=13 \n=14; len=15. + let mut s = fresh_with("hé llo wörld\n".as_bytes()); + let bid = s.core.borrow().active_buffer_id(); + let none = WireMods::NONE; + + // Down places the cursor — a mid-codepoint hit (inside 'é') + // snaps back to the boundary — and anchors a selection. + s.dispatch_pointer(FrontendId::LOCAL, bid, 2, PointerKind::Down, none); + assert_eq!( + s.core.borrow().cursor(), + 1, + "mid-codepoint hit snaps to the char boundary" + ); + + // Drag grows the region from the anchor; Up keeps it. + s.dispatch_pointer(FrontendId::LOCAL, bid, 6, PointerKind::Drag, none); + assert_eq!(s.core.borrow().cursor(), 6); + assert_eq!(s.core.borrow().active_region(), Some((1, 6))); + s.dispatch_pointer(FrontendId::LOCAL, bid, 6, PointerKind::Up, none); + assert_eq!(s.core.borrow().active_region(), Some((1, 6))); + + // A plain click (Down + Up, no drag) leaves no region. + s.dispatch_pointer(FrontendId::LOCAL, bid, 4, PointerKind::Down, none); + s.dispatch_pointer(FrontendId::LOCAL, bid, 4, PointerKind::Up, none); + assert_eq!(s.core.borrow().cursor(), 4); + assert!(s.core.borrow().active_region().is_none()); + + // DoubleDown selects the word at the hit ("wörld"). + s.dispatch_pointer(FrontendId::LOCAL, bid, 8, PointerKind::DoubleDown, none); + assert_eq!(s.core.borrow().active_region(), Some((8, 14))); + assert_eq!(s.core.borrow().cursor(), 14); + + // Past-EOF hits clamp to the buffer length. + s.dispatch_pointer(FrontendId::LOCAL, bid, 999, PointerKind::Down, none); + assert_eq!(s.core.borrow().cursor(), 15); + + // A pointer for a buffer the window isn't displaying is + // dropped (click racing a buffer switch). + let other = crate::buffer::BufferId::next(); + s.dispatch_pointer(FrontendId::LOCAL, other, 0, PointerKind::Down, none); + assert_eq!(s.core.borrow().cursor(), 15, "mismatched buffer ignored"); + } + /// Acceptance bullet 3: mouse events are coalesced at frame /// boundaries — many drag events between renders all apply, and /// the cursor ends up at the last position. diff --git a/src/editor_core.rs b/src/editor_core.rs index 24798ca..c73e05a 100644 --- a/src/editor_core.rs +++ b/src/editor_core.rs @@ -1406,11 +1406,17 @@ fn forward_word(buf: &Buffer, mut pos: Position) -> Position { } fn word_range_at(buf: &Buffer, pos: Position) -> Option<(Position, Position)> { - let (ch, _) = char_at(buf, pos)?; + let (ch, ch_len) = char_at(buf, pos)?; if !is_word_char(ch) { return None; } - let start = backward_word(buf, pos); + // Walk back from just *past* the char under the cursor, not from + // `pos` itself: `backward_word(pos)` at a word's FIRST character + // sees the non-word char before it, skips it, and crosses into + // the previous word — double-clicking the 'w' of "llo world" + // would select "llo world". From `pos + ch_len` the char behind + // is this word's own first char, so the walk stops at its start. + let start = backward_word(buf, pos.saturating_add(ch_len)); let end = forward_word(buf, pos); (start < end).then_some((start, end)) } diff --git a/src/protocol.rs b/src/protocol.rs index 6ed3bcd..3dbff37 100644 --- a/src/protocol.rs +++ b/src/protocol.rs @@ -1683,29 +1683,33 @@ mod tests { // --- M5.5a handshake & postcard round-trips --- #[test] - fn protocol_version_is_four_for_dispatch_idle() { + fn protocol_version_is_five_for_pointer_events() { // Pin the value: T M10.5 bumped 1→2 (v1.0 wire: CrdtOp / // PresenceUpdate). T M11.1 bumped 2→3 (v1.1 wire: the // SemanticFrame family + FrontendEvent::Viewport). T M11.6 // bumped 3→4 (DispatchIdle for the optimistic-apply gate). - // The current binary serves v1..=v4 sessions — the slice- + // The mouse framing Q#M1 bumped 4→5 (FrontendEvent::Pointer, + // byte-position gestures from semantic frontends). The + // current binary serves v1..=v5 sessions — the slice- // membership handshake makes the relaxation symmetric. - assert_eq!(PROTOCOL_VERSION, 4); + assert_eq!(PROTOCOL_VERSION, 5); } #[test] - fn supported_protocol_versions_includes_one_through_four() { + fn supported_protocol_versions_includes_one_through_five() { // T M10.5: v1.0 binaries accept v1+v2. T M11.1: v1.1 binaries - // accept v1+v2+v3. T M11.6: v4 binaries accept v1+v2+v3+v4. - // The check is slice membership, not strict equality, so - // older binaries keep connecting to current binaries - // unchanged. v5+ is rejected until the next bump. + // accept v1+v2+v3. T M11.6: v4 binaries accept v1..=v4. Mouse + // framing Q#M1: v5 binaries accept v1..=v5. The check is + // slice membership, not strict equality, so older binaries + // keep connecting to current binaries unchanged. v6+ is + // rejected until the next bump. assert!(is_supported_protocol_version(1)); assert!(is_supported_protocol_version(2)); assert!(is_supported_protocol_version(3)); assert!(is_supported_protocol_version(4)); + assert!(is_supported_protocol_version(5)); assert!(!is_supported_protocol_version(0)); - assert!(!is_supported_protocol_version(5)); + assert!(!is_supported_protocol_version(6)); assert!(!is_supported_protocol_version(u32::MAX)); } @@ -2074,10 +2078,11 @@ mod tests { fn m10_5_handshake_matrix_versions_outside_range_rejected() { // v1 daemon's strict-equality behavior is documented at the // v0.1 code level (different binary); the current daemon's - // range check accepts v1/v2/v3/v4 (T M11.1 added v3; T M11.6 - // added v4) and rejects v5+ until the next protocol bump. + // range check accepts v1..=v5 (T M11.1 added v3; T M11.6 + // added v4; the mouse framing Q#M1 added v5) and rejects v6+ + // until the next protocol bump. assert!(!is_supported_protocol_version(0)); - assert!(!is_supported_protocol_version(5)); + assert!(!is_supported_protocol_version(6)); assert!(!is_supported_protocol_version(u32::MAX)); }