Merge pull request #61 from levineuwirth/session-m-mouse-pointer

pmacs-gpu mouse input: byte-position pointer events (protocol v5)
This commit is contained in:
Levi Neuwirth 2026-06-10 14:19:30 -04:00 committed by GitHub
commit 09852d10e4
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
10 changed files with 819 additions and 37 deletions

View File

@ -0,0 +1,164 @@
# pmacs-gpu mouse input — framing pass
Date: 2026-06-10. Follows the Phase B framing
(`pmacs-gpu-phase-b-framing.md`), which deferred the mouse wire
decision as Q#B5. The optimistic-typing arc is merged (PR #60,
main @ `7fdae76`); this is the next capability session.
## What exists today (fact-checked)
- **Wire**: `FrontendEvent::Mouse(MouseEvent)` carries a `CellCoord`
(`pmacs-protocol/src/message.rs:210-219`) — a *cell-grid*
coordinate. `PROTOCOL_VERSION = 4` (`message.rs:970`); the
DispatchIdle variant set the precedent for gating new variants on
`negotiated_protocol_version` (`daemon.rs:998`).
- **Daemon semantics**: `EditorState::dispatch_mouse` owns the full
gesture state machine — Down activates the window, positions the
cursor, begins a selection; Drag grows the region; Up collapses an
empty region; a second Down in the same cell within 500ms is a
double-click and calls `select_word_at_cursor`; ScrollUp/Down move
the viewport (`src/editor.rs`, CUA arc). For a *semantic* session,
`apply_semantic_input_event` already routes
`FrontendEvent::Mouse → mouse_to_crossterm → dispatch_mouse`
(`daemon.rs:1983-1986`) — against the session's placeholder 24×80
cell geometry, which pmacs-gpu does not have.
- **GPU layout**: the shaped buffer holds the *visible slice* of the
source (`current_text[vstart..vend]`) with **inline adornment text
injected** by `projected_rich_chunks` (`pmacs-gpu/src/main.rs:3202`)
— so shaped-buffer byte offsets ≠ source bytes whenever inlay hints
are visible. Text is drawn at `TEXT_LEFT/TEXT_TOP` pixel offsets.
- **Hit testing**: `cosmic_text::Buffer::hit(x, y) -> Option<Cursor>`
exists (cosmic-text 0.18.2, `buffer.rs:923`) and returns a cursor in
the shaped buffer's (line, byte-index-within-line) space — the same
line-relative space QB3 taught us to rebase via the slice line
table.
- **Scroll wheel**: the GPU owns viewport + scroll state (S1:
`scroll_top` / slice reshape / viewport re-declaration), but
**no winit `MouseWheel` handler exists today** (fact-check: the
window_event catch-all swallows it; scrolling is keyboard-driven).
M-2 adds the handler — it composes from existing pieces
(`scroll_top` adjust → reshape → `viewport_send_if_changed`) and
needs no wire.
## Q#M1 — the wire shape (resolves Q#B5)
How does a pixel frontend tell the instance where a click landed?
- **(α) Reuse `FrontendEvent::Mouse` with synthesized cell coords.**
The GPU would approximate `(row, col)` from pixels. REJECTED: the
daemon's cell math runs against window geometry the GPU doesn't
share (placeholder 24×80), inline adornments shift visual columns
with no cell-space representation, and the design doc's contract is
explicit — *the instance never learns a pixel, and there is no
hit-test round trip*. α is broken-by-design, not merely lossy.
- **(β1) Local hit-test → byte-position pointer events.** The GPU
resolves pixels to a **source byte offset** locally (it owns the
layout) and ships gesture-level events; the daemon replays its
existing mouse state machine in byte space. Selection semantics
(CUA, double-click word select, future triple-click) stay
instance-side, exactly like the Key round-trip philosophy.
- **(β2) Frontend-computed selection states.** The GPU computes
cursor+selection itself and ships final states. REJECTED: forks the
gesture semantics into a second implementation the TUI doesn't
share, and fights the daemon-authoritative selection model the CUA
arc just consolidated.
**Stance: β1.** New wire variant:
```rust
FrontendEvent::Pointer {
frontend_id: FrontendId,
buffer_id: BufferId,
/// Source byte offset the frontend hit-tested locally.
byte: u64,
kind: PointerKind, // Down | Drag | Up | DoubleDown
mods: Modifiers,
}
```
`DoubleDown` is detected frontend-side (the frontend knows pixel
proximity and its own double-click interval; the daemon's cell-based
double-click detection cannot see pixels). The daemon maps:
Down → activate semantic window + set cursor + `begin_selection`;
Drag → cursor move (region grows); Up → collapse-if-empty;
DoubleDown → `select_word_at_cursor`. This reuses the same core
primitives `dispatch_mouse` calls today — no new selection logic.
Protocol: bump `PROTOCOL_VERSION` to 5; the GPU sends `Pointer` only
when `hello.protocol_version >= 5`; the daemon ignores the variant
for sessions that lack `semantic_render`.
## Q#M2 — projected→source byte mapping
`Buffer::hit` answers in *shaped* coordinates: line index + byte
offset within the shaped line, where shaped text = source slice with
adornment text spliced in. Two-step mapping:
1. Shaped (line, index) → shaped-buffer byte offset via the shaped
line table (the QB3 rebase, but over the *projected* text).
2. Projected byte → **source byte** via a run map built during
`reshape`. **This is new work, not a current by-product**
(fact-check: `RichChunk` today carries only `text` + `color`, no
source range — `main.rs:2269`): `projected_rich_chunks` walks
(source-run | adornment-run) chunks in order, so M-2 extends it to
also emit
`Vec<ProjectedRun { projected_start, len, source_start: Option<u64> }>`.
A hit inside a source run maps linearly; a hit inside an adornment
run snaps to the adornment's source anchor.
**Stance:** build the run map in `reshape` (it is O(chunks), walked
there anyway) rather than re-deriving adornment offsets at click
time. Clicks land between keystrokes, so the map being rebuilt per
reshape costs nothing new.
## Q#M3 — gesture scope for the first session
**In:** left-click cursor placement, left-drag selection,
double-click word select, wheel scroll (local-only, no wire).
**Deferred:** triple-click line select (needs a core
`select_line_at_cursor`), middle-click paste (needs clipboard
session), right-click menu (needs GUI chrome), drag auto-scroll at
window edges (needs a repeat timer; record as a known gap),
minimap click-to-jump (local-only; natural follow-up).
## Q#M4 — interaction with optimistic state
A pointer event is a round-trip input: it must (a) mark
`cursor_fresh = false` until the daemon's `CursorByte` answers, and
(b) defer behind the optimistic-cursor floor exactly like round-trip
keys do (`defer_round_trip_key_if_needed` shape) — a click landing
between unconfirmed keystrokes would otherwise race the cursor
confirmation. Deferred pointer events should keep only the **latest**
Down/Drag (coalescing, like the TUI's frame-boundary mouse
coalescing) rather than queueing every drag sample.
## Predicted findings (categorical bets, scored at session close)
1. **Coordinate-space bug** (the QB3 family): some offset in the
pixel→shaped→projected→source chain is wrong on first
implementation — most likely the adornment run snap or the
`TEXT_LEFT/TEXT_TOP` subtraction. Surfaces only in manual
validation with inlay hints visible.
2. **Window-activation gap** (the B1 family): the daemon-side Pointer
handler forgets some part of what `activate_and_position` does for
grid windows (window focus, `goal_col` reset), so a click works
but a subsequent arrow key moves from the wrong anchor.
3. **Selection-wash latency**: the wash arrives a round trip after
the drag (Decorations cadence), reading as rubber-band lag. If it
surfaces, the fix is frontend-local provisional selection — out of
scope, record it.
4. **Coalescing**: drag streams at pixel rate (hundreds of
events/sec) flood the socket or the dispatcher. Mitigation
in-scope: send Drag only when the hit byte changes.
## Session plan
- **M-1 (wire + daemon)**: `PointerKind`/`Pointer` variant, version
bump, daemon handler reusing core primitives, unit tests through
`handle_dispatcher_event`. Compile-green alone.
- **M-2 (GPU)**: winit mouse events → hit test → run map (new, from
`projected_rich_chunks`) → Pointer sends; drag coalescing on byte
change; NEW local wheel-scroll handler; optimistic-floor deferral.
Manual validation gate: click placement with inlay hints on the
same line, drag selection, double-click, wheel — in both pmacs-gpu
and a TUI peer simultaneously.

View File

@ -23,7 +23,7 @@ use std::thread;
use pmacs_protocol::{
AttachRequest, BufferId, ByteRange, CrdtOp, FrontendCapabilities, FrontendEvent, FrontendId,
Hello, InstanceMessage, Key, KeyEvent, Modifiers, PROTOCOL_VERSION,
Hello, InstanceMessage, Key, KeyEvent, Modifiers, PROTOCOL_VERSION, PointerKind,
SUPPORTED_PROTOCOL_VERSIONS, TransportError, is_supported_protocol_version, read_message,
write_message,
};
@ -202,6 +202,7 @@ pub fn connect(
Ok(AttachClient {
writer_tx,
frontend_id: hello.assigned_frontend_id,
server_protocol_version: hello.protocol_version,
})
}
@ -213,6 +214,10 @@ pub struct AttachClient {
/// `FrontendEvent` carries this so the daemon can route input back
/// to the per-session `SemanticRenderState`.
frontend_id: FrontendId,
/// The daemon's `Hello.protocol_version`. Wire variants newer
/// than the daemon (e.g. `Pointer`, v5) must be gated on this —
/// an older daemon hard-errors decoding an unknown variant.
server_protocol_version: u32,
}
impl AttachClient {
@ -255,6 +260,30 @@ impl AttachClient {
}))
}
/// Send a `FrontendEvent::Pointer` (session M-2): a locally
/// hit-tested gesture in source bytes. Callers gate on
/// [`Self::server_protocol_version`] `>= 5`.
pub fn send_pointer(
&self,
buffer_id: BufferId,
byte: u64,
kind: PointerKind,
mods: Modifiers,
) -> Result<(), TransportError> {
self.send_event(FrontendEvent::Pointer {
frontend_id: self.frontend_id,
buffer_id,
byte,
kind,
mods,
})
}
/// The daemon's negotiated wire version from `Hello`.
pub fn server_protocol_version(&self) -> u32 {
self.server_protocol_version
}
/// Send a locally-authored CRDT operation to the daemon. The GPU
/// uses this for idle plain-text insertion after applying the same
/// op to its local Loro replica, avoiding a Key round trip on the

View File

@ -36,7 +36,7 @@ use loro::{ContainerTrait, ExportMode};
use pmacs_protocol::{
AdornmentContent, AdornmentPlacement, BufferId, ByteRange, CrdtOp, Decoration, DecorationKind,
DecorationSegment, FrontendId, InlineAdornment, InstanceMessage, Key as ProtocolKey, Modifiers,
SelectionSnapshot, StyleSegment, StyleSpan,
PointerKind, SelectionSnapshot, StyleSegment, StyleSpan,
cell::{Color as CellColor, Style as CellStyle},
};
use wgpu::MultisampleState;
@ -391,6 +391,24 @@ struct State {
/// ops can mis-prune by one frame; the next generation-keyed
/// full resync self-corrects.
unconfirmed_edits: Vec<(u64, TextProjectionEdit)>,
/// Q#M2 — projected→source hit map for the currently shaped
/// slice. Rebuilt by every `reshape` from the same chunks that
/// feed glyphon; source offsets are slice-relative (pair with
/// `view_range.0`).
current_hit_runs: Vec<ProjectedRun>,
/// Line-start byte offsets of the *projected* text (cosmic-text
/// reports hits as line index + byte-within-line).
projected_line_starts: Vec<u64>,
/// Last reported pointer position, in window pixels.
pointer_pos: Option<(f64, f64)>,
/// Primary button is held after a Down inside the text area.
pointer_drag_active: bool,
/// Hit byte of the last Pointer event sent — Drag coalescing:
/// pixel-rate motion only ships when the hit byte changes.
last_pointer_sent_byte: Option<u64>,
/// `(when, byte)` of the last primary Down, for frontend-side
/// double-click detection (same-hit within the interval).
last_pointer_down: Option<(std::time::Instant, u64)>,
}
/// pmacs-gpu's own cursor position, mirrored from `CursorByte`.
@ -422,6 +440,23 @@ struct FileStyleSummaryState {
lines: Vec<CellStyle>,
}
impl App {
/// Ship a Pointer event if the daemon speaks protocol v5+ — the
/// Q#M1 frontend-side gate (an older instance cannot decode the
/// variant and would drop the connection).
fn send_pointer(&self, buffer_id: BufferId, byte: u64, kind: PointerKind, mods: Modifiers) {
let Some(client) = self.attach_client.as_ref() else {
return;
};
if client.server_protocol_version() < 5 {
return;
}
if let Err(e) = client.send_pointer(buffer_id, byte, kind, mods) {
eprintln!("pmacs-gpu: send_pointer failed: {e}");
}
}
}
impl ApplicationHandler<AppEvent> for App {
fn resumed(&mut self, event_loop: &ActiveEventLoop) {
if self.state.is_some() {
@ -456,6 +491,7 @@ impl ApplicationHandler<AppEvent> for App {
}
}
#[allow(clippy::too_many_lines)] // linear per-event dispatch; splitting hides the input flow.
fn window_event(&mut self, event_loop: &ActiveEventLoop, _id: WindowId, event: WindowEvent) {
match event {
WindowEvent::CloseRequested => event_loop.exit(),
@ -536,6 +572,100 @@ impl ApplicationHandler<AppEvent> for App {
eprintln!("pmacs-gpu: resize send_viewport failed: {e}");
}
}
// Session M-2 — pointer input (docs/pmacs-gpu-mouse-framing.md).
WindowEvent::CursorMoved { position, .. } => {
let Some(state) = self.state.as_mut() else {
return;
};
state.pointer_pos = Some((position.x, position.y));
if !state.pointer_drag_active {
return;
}
// Drag coalescing (predicted finding #4): pixel-rate
// motion only ships when the hit byte changes.
let Some(byte) = state.hit_test_source_byte(position.x, position.y) else {
return;
};
if state.last_pointer_sent_byte == Some(byte) {
return;
}
state.last_pointer_sent_byte = Some(byte);
state.note_pointer_round_trip();
let buffer_id = state.current_buffer_id;
let mods = translate_mods(self.modifiers);
if let Some(buffer_id) = buffer_id {
self.send_pointer(buffer_id, byte, PointerKind::Drag, mods);
}
}
WindowEvent::MouseInput {
state: button_state,
button: winit::event::MouseButton::Left,
..
} => {
let Some(state) = self.state.as_mut() else {
return;
};
let Some((x, y)) = state.pointer_pos else {
return;
};
let mods = translate_mods(self.modifiers);
match button_state {
ElementState::Pressed => {
let Some(byte) = state.hit_test_source_byte(x, y) else {
return;
};
let kind = state.classify_pointer_down(byte);
state.pointer_drag_active = true;
state.last_pointer_sent_byte = Some(byte);
state.note_pointer_round_trip();
if let Some(buffer_id) = state.current_buffer_id {
if debug_input() {
eprintln!("pmacs-gpu pointer: {kind:?} byte={byte}");
}
self.send_pointer(buffer_id, byte, kind, mods);
}
}
ElementState::Released => {
if !state.pointer_drag_active {
return;
}
state.pointer_drag_active = false;
let byte = state
.hit_test_source_byte(x, y)
.or(state.last_pointer_sent_byte);
let buffer_id = state.current_buffer_id;
if let (Some(byte), Some(buffer_id)) = (byte, buffer_id) {
self.send_pointer(buffer_id, byte, PointerKind::Up, mods);
}
}
}
}
WindowEvent::MouseWheel { delta, .. } => {
let Some(state) = self.state.as_mut() else {
return;
};
// Wheel scroll is local-only: the GPU owns the
// viewport. Positive winit y = scroll up = smaller
// scroll_top.
let lines = match delta {
winit::event::MouseScrollDelta::LineDelta(_, y) => {
(-y * WHEEL_LINES_PER_TICK).round() as i64
}
winit::event::MouseScrollDelta::PixelDelta(p) => {
(-(p.y as f32) / CODE_LINE_HEIGHT).round() as i64
}
};
if lines == 0 {
return;
}
let vp = state.scroll_by_lines(lines);
if let Some(vp) = vp
&& let Some(client) = self.attach_client.as_ref()
&& let Err(e) = client.send_viewport(vp.buffer_id, vp.visible, vp.generation)
{
eprintln!("pmacs-gpu: wheel send_viewport failed: {e}");
}
}
WindowEvent::RedrawRequested => {
if let Some(state) = self.state.as_mut() {
state.render();
@ -622,6 +752,14 @@ struct CrdtOpSend {
/// tiny against a human noticing wedged keys.
const FLOOR_CONFIRM_TIMEOUT: std::time::Duration = std::time::Duration::from_millis(500);
/// Frontend-side double-click interval (Q#M1: the daemon cannot see
/// pixels, so the frontend decides what a double-click is). Matches
/// the TUI's `DOUBLE_CLICK_MAX_DELAY`.
const DOUBLE_CLICK_WINDOW: std::time::Duration = std::time::Duration::from_millis(500);
/// Wheel lines scrolled per `MouseScrollDelta::LineDelta` unit.
const WHEEL_LINES_PER_TICK: f32 = 3.0;
/// Byte range an optimistic Backspace/Delete removes at `cursor`, or
/// `None` when it can't be predicted locally: buffer edge (the
/// daemon's behavior is a no-op there anyway), a modifier variant
@ -862,6 +1000,12 @@ impl State {
deferred_round_trip_keys: Vec::new(),
optimistic_floor_set_at: None,
unconfirmed_edits: Vec::new(),
current_hit_runs: Vec::new(),
projected_line_starts: vec![0],
pointer_pos: None,
pointer_drag_active: false,
last_pointer_sent_byte: None,
last_pointer_down: None,
}
}
@ -1587,6 +1731,67 @@ impl State {
self.scroll_top != old
}
/// Resolve a window-pixel position to an **absolute source byte**
/// (Q#M2): pixel → cosmic-text hit (shaped line + byte within
/// line) → projected byte → run map → slice byte → + `vstart`.
/// `None` when no buffer is attached or the position is outside
/// anything hit-testable.
fn hit_test_source_byte(&self, x: f64, y: f64) -> Option<u64> {
self.current_buffer_id?;
let rel_x = x as f32 - TEXT_LEFT;
let rel_y = y as f32 - TEXT_TOP;
let cursor = self.buffer.hit(rel_x, rel_y)?;
let line_start = *self.projected_line_starts.get(cursor.line)?;
let projected = line_start + cursor.index as u64;
let slice_byte = projected_to_source(&self.current_hit_runs, projected)?;
let (vstart, vend) = self.view_range;
Some((vstart + slice_byte).min(vend))
}
/// Wheel scroll (local-only — the GPU owns the viewport; no wire
/// event exists or is needed). Positive `delta` scrolls down.
fn scroll_by_lines(&mut self, delta: i64) -> Option<ViewportSend> {
let max_top = self.current_line_starts.len().saturating_sub(1);
let new_top = self
.scroll_top
.saturating_add_signed(delta as isize)
.min(max_top);
if new_top == self.scroll_top {
return None;
}
self.scroll_top = new_top;
self.reshape();
self.current_buffer_id
.and_then(|bid| self.viewport_send_if_changed(bid))
}
/// Bookkeeping for an outgoing Pointer event: it supersedes any
/// unconfirmed optimistic-cursor prediction (the daemon's answer
/// will be the click position, not the typing prediction), and
/// the cursor is not authoritative again until that `CursorByte`
/// lands.
fn note_pointer_round_trip(&mut self) {
self.cursor_fresh = false;
self.optimistic_cursor_floor = None;
self.optimistic_floor_set_at = None;
}
/// Frontend-side double-click detection: a second Down at the
/// same hit byte within the interval upgrades to `DoubleDown`.
fn classify_pointer_down(&mut self, byte: u64) -> PointerKind {
let now = std::time::Instant::now();
let is_double = self.last_pointer_down.take().is_some_and(|(at, prev)| {
prev == byte && now.duration_since(at) <= DOUBLE_CLICK_WINDOW
});
if is_double {
// A third click starts over (triple-click is deferred).
PointerKind::DoubleDown
} else {
self.last_pointer_down = Some((now, byte));
PointerKind::Down
}
}
fn apply_file_style_summary(
&mut self,
buffer_id: BufferId,
@ -1834,17 +2039,22 @@ impl State {
.collect();
let default_attrs = Attrs::new().family(Family::Name("JetBrains Mono"));
let chunks: Vec<(String, Attrs<'static>)> =
projected_rich_chunks(slice, &spans, &decorations, &adornments)
.into_iter()
.map(|chunk| {
let mut attrs = default_attrs.clone();
if let Some(c) = chunk.color {
attrs = attrs.color(c);
}
(chunk.text, attrs)
})
.collect();
let rich = projected_rich_chunks(slice, &spans, &decorations, &adornments);
// Q#M2 — the pointer hit map is derived from the SAME chunks
// the shaped buffer is built from, so the two cannot disagree.
let (hit_runs, projected_line_starts) = build_hit_runs(&rich);
self.current_hit_runs = hit_runs;
self.projected_line_starts = projected_line_starts;
let chunks: Vec<(String, Attrs<'static>)> = rich
.into_iter()
.map(|chunk| {
let mut attrs = default_attrs.clone();
if let Some(c) = chunk.color {
attrs = attrs.color(c);
}
(chunk.text, attrs)
})
.collect();
self.buffer.set_rich_text(
&mut self.font_system,
chunks.iter().map(|(s, a)| (s.as_str(), a.clone())),
@ -2269,6 +2479,74 @@ struct MinimapLineShape {
struct RichChunk {
text: String,
color: Option<glyphon::Color>,
/// Where this chunk's text came from — the seam the pointer
/// hit-test walks back through (Q#M2).
source: ChunkSource,
}
/// Origin of one [`RichChunk`] in the shaped (projected) text.
/// Offsets are slice-relative (the same space `projected_rich_chunks`
/// works in); the hit test rebases with the slice's `vstart`.
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
enum ChunkSource {
/// Verbatim source text starting at this slice byte offset.
Source { start: u64 },
/// Injected adornment text (inlay hint) anchored at this slice
/// byte offset. Hits inside it snap to the anchor.
Adornment { anchor: u64 },
}
/// One run of the projected→source hit map (Q#M2), built by
/// [`build_hit_runs`] from the same chunks `reshape` feeds glyphon —
/// so the map and the shaped buffer can never disagree.
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
struct ProjectedRun {
/// Byte offset of this run in the shaped (projected) text.
projected_start: u64,
/// Run length in projected bytes.
len: u64,
source: ChunkSource,
}
/// Build the projected→source run map plus the projected text's line
/// start table (cosmic-text reports hits as line + byte-within-line).
fn build_hit_runs(chunks: &[RichChunk]) -> (Vec<ProjectedRun>, Vec<u64>) {
let mut runs = Vec::with_capacity(chunks.len());
let mut line_starts = vec![0u64];
let mut projected = 0u64;
for chunk in chunks {
let len = chunk.text.len() as u64;
runs.push(ProjectedRun {
projected_start: projected,
len,
source: chunk.source,
});
for (i, b) in chunk.text.bytes().enumerate() {
if b == b'\n' {
line_starts.push(projected + i as u64 + 1);
}
}
projected += len;
}
(runs, line_starts)
}
/// Map a projected byte offset back to a slice-relative source byte
/// (Q#M2). Hits inside an adornment run snap to its anchor; offsets
/// past the last run clamp to its end.
fn projected_to_source(runs: &[ProjectedRun], projected: u64) -> Option<u64> {
if runs.is_empty() {
return None;
}
let idx = runs
.partition_point(|r| r.projected_start <= projected)
.saturating_sub(1);
let run = runs[idx];
let within = projected.saturating_sub(run.projected_start).min(run.len);
match run.source {
ChunkSource::Source { start } => Some(start + within),
ChunkSource::Adornment { anchor } => Some(anchor),
}
}
fn minimap_left(surface_width: u32) -> Option<f32> {
@ -2591,10 +2869,7 @@ fn debug_input() -> bool {
/// has no representation for (the daemon ignores `Key::Unknown`, so
/// there's no value in forwarding them). `translate_key` covers the
/// full editing set; session B1 gates the send on [`is_motion_key`].
fn translate_key(
logical: &Key,
mods: winit::keyboard::ModifiersState,
) -> Option<(ProtocolKey, Modifiers)> {
fn translate_mods(mods: winit::keyboard::ModifiersState) -> Modifiers {
let mut bits = 0u8;
if mods.shift_key() {
bits |= Modifiers::SHIFT.bits();
@ -2608,7 +2883,14 @@ fn translate_key(
if mods.super_key() {
bits |= Modifiers::META.bits();
}
let pmods = Modifiers::from_bits_truncate(bits);
Modifiers::from_bits_truncate(bits)
}
fn translate_key(
logical: &Key,
mods: winit::keyboard::ModifiersState,
) -> Option<(ProtocolKey, Modifiers)> {
let pmods = translate_mods(mods);
let pkey = match logical {
Key::Named(named) => match named {
@ -3248,6 +3530,7 @@ fn projected_rich_chunks(
chunks.push(RichChunk {
text: text[a as usize..b as usize].to_owned(),
color: source_color_at(a, spans, decorations),
source: ChunkSource::Source { start: a },
});
}
}
@ -3261,6 +3544,7 @@ fn projected_rich_chunks(
chunks.push(RichChunk {
text: String::new(),
color: None,
source: ChunkSource::Source { start: 0 },
});
}
chunks
@ -3292,6 +3576,7 @@ fn push_adornments_at(
chunks.push(RichChunk {
text: text.clone(),
color: Some(adornment_text_color(style.fg)),
source: ChunkSource::Adornment { anchor },
});
}
*next += 1;
@ -3854,6 +4139,57 @@ mod tests {
);
}
#[test]
fn hit_runs_map_projected_bytes_back_to_source() {
// Source slice "ab\ncd" with an inlay hint ": i32 " anchored
// at byte 2 (end of "ab"): projected text = "ab: i32 \ncd".
let chunks = vec![
RichChunk {
text: "ab".into(),
color: None,
source: ChunkSource::Source { start: 0 },
},
RichChunk {
text: ": i32 ".into(),
color: None,
source: ChunkSource::Adornment { anchor: 2 },
},
RichChunk {
text: "\ncd".into(),
color: None,
source: ChunkSource::Source { start: 2 },
},
];
let (runs, line_starts) = build_hit_runs(&chunks);
assert_eq!(
line_starts,
vec![0, 9],
"projected line table counts the newline at projected byte 8"
);
// Hits inside source runs map linearly.
assert_eq!(projected_to_source(&runs, 0), Some(0));
assert_eq!(projected_to_source(&runs, 1), Some(1));
assert_eq!(
projected_to_source(&runs, 9),
Some(3),
"projected 'c' (byte 9) maps to source byte 3"
);
// Hits inside the adornment snap to its anchor.
for projected in 2..8 {
assert_eq!(
projected_to_source(&runs, projected),
Some(2),
"adornment hit at projected {projected} snaps to the anchor"
);
}
// Past-the-end hits clamp into the last run.
assert_eq!(projected_to_source(&runs, 999), Some(5));
// Empty map: nothing to hit.
assert_eq!(projected_to_source(&[], 0), None);
}
#[test]
fn optimistic_delete_range_covers_single_codepoints_only() {
let none = Modifiers::NONE;

View File

@ -50,7 +50,7 @@ pub use message::{
DecorationKind, DecorationSegment, FrontendCapabilities, FrontendEvent, GoodbyeReason, Hello,
InlineAdornment, InstanceCapabilities, InstanceIdentity, InstanceMessage, InstanceSignal, Key,
KeyEvent, Modifiers, MouseButton, MouseEvent, MouseKind, NegotiatedCapabilities,
PROTOCOL_VERSION, ResourceBody, SUPPORTED_PROTOCOL_VERSIONS, SelectionSnapshot, StyleSegment,
StyleSpan, is_supported_protocol_version, negotiate_capabilities,
PROTOCOL_VERSION, PointerKind, ResourceBody, SUPPORTED_PROTOCOL_VERSIONS, SelectionSnapshot,
StyleSegment, StyleSpan, is_supported_protocol_version, negotiate_capabilities,
};
pub use transport::{MAX_FRAME_BYTES, TransportError, read_message, write_message};

View File

@ -307,6 +307,51 @@ pub enum FrontendEvent {
/// CRDT generation the frontend computed `visible` against.
generation: u64,
},
/// Q#M1 (protocol v5): a pointer gesture a *semantic* frontend
/// hit-tested locally to a **source byte offset**. The pixel→byte
/// resolution happens entirely frontend-side (the frontend owns
/// layout: fonts, inline adornments, scroll) — consistent with
/// `Viewport`'s no-pixels contract; the instance replays its
/// existing mouse gesture semantics in byte space, so selection
/// behavior stays single-sourced with the grid path.
///
/// Cell-grid frontends keep using [`FrontendEvent::Mouse`]; the
/// two variants are per-session-kind, never mixed. Only sent when
/// the instance's `Hello.protocol_version >= 5` (an older
/// instance cannot decode the variant).
Pointer {
/// Which frontend produced the gesture (untrusted; the
/// instance routes by the authenticated session, matching
/// the `CrdtOp` / `Viewport` source-trust rule).
frontend_id: FrontendId,
/// Buffer the frontend was displaying when it hit-tested.
buffer_id: crate::BufferId,
/// Source byte offset of the hit (frontend-local hit test;
/// adornment runs already snapped to their anchors).
byte: u64,
/// Which gesture step this is.
kind: PointerKind,
/// Modifiers held during the gesture. Carried for future
/// Shift-click extension; the v5 instance ignores them.
mods: Modifiers,
},
}
/// Gesture step for [`FrontendEvent::Pointer`]. Double-click
/// detection is frontend-side (`DoubleDown` instead of a second
/// `Down`): only the frontend knows pixel proximity and its own
/// double-click interval.
#[derive(serde::Serialize, serde::Deserialize, Debug, Clone, Copy, PartialEq, Eq)]
pub enum PointerKind {
/// Primary button pressed at `byte`.
Down,
/// Pointer moved to `byte` with the primary button held.
Drag,
/// Primary button released at `byte`.
Up,
/// Second press at the same hit within the frontend's
/// double-click window — selects the word at `byte`.
DoubleDown,
}
impl FrontendEvent {
@ -322,7 +367,8 @@ impl FrontendEvent {
| Self::FocusLost(frontend_id)
| Self::Detach(frontend_id)
| Self::CrdtOp { frontend_id, .. }
| Self::Viewport { frontend_id, .. } => *frontend_id,
| Self::Viewport { frontend_id, .. }
| Self::Pointer { frontend_id, .. } => *frontend_id,
}
}
}
@ -967,7 +1013,14 @@ pub enum ResourceBody {
/// checks the session's negotiated version and skips the variant for
/// older peers. An old peer would hard-error on decode of an unknown
/// postcard variant; gating prevents that.
pub const PROTOCOL_VERSION: u32 = 4;
///
/// Q#M1 (mouse framing): bumped from 4 to 5 for
/// [`FrontendEvent::Pointer`]. The gate runs in the *frontend* this
/// time (the new variant travels frontend→instance): a semantic
/// frontend sends `Pointer` only when the instance's
/// `Hello.protocol_version >= 5`, because an older instance would
/// hard-error decoding the unknown variant.
pub const PROTOCOL_VERSION: u32 = 5;
/// T M10.5: the set of protocol versions a v1.0 binary accepts on
/// the wire. v0.1 binaries only accepted `[1]`; v1.0 binaries accept
@ -987,7 +1040,11 @@ pub const PROTOCOL_VERSION: u32 = 4;
/// T M11.6: extended to `[1, 2, 3, 4]`. v4 sessions receive
/// `InstanceMessage::DispatchIdle`; older sessions are filtered out
/// of that emission.
pub const SUPPORTED_PROTOCOL_VERSIONS: &[u32] = &[1, 2, 3, 4];
///
/// Mouse framing Q#M1: extended to `[1, 2, 3, 4, 5]`. v5 peers may
/// send `FrontendEvent::Pointer`; the frontend-side gate (see
/// [`PROTOCOL_VERSION`]) keeps the variant off wires negotiated `< 5`.
pub const SUPPORTED_PROTOCOL_VERSIONS: &[u32] = &[1, 2, 3, 4, 5];
/// T M10.5: predicate for the handshake check. Returns `true` if
/// `peer_version` is in [`SUPPORTED_PROTOCOL_VERSIONS`].

View File

@ -186,6 +186,19 @@ mod tests {
round_trip(&h);
}
#[test]
fn pointer_event_round_trips_through_transport() {
// Q#M1 (protocol v5): the byte-position pointer gesture.
let ev = FrontendEvent::Pointer {
frontend_id: FrontendId(7),
buffer_id: crate::BufferId::next(),
byte: 4096,
kind: crate::PointerKind::DoubleDown,
mods: Modifiers::SHIFT,
};
round_trip(&ev);
}
#[test]
fn attach_request_round_trips_through_transport() {
let req = AttachRequest {

View File

@ -1359,6 +1359,25 @@ fn handle_dispatcher_event(
}
}
}
FrontendEvent::Pointer {
buffer_id,
byte,
kind,
mods,
..
} => {
// Mouse framing Q#M1 — a semantic frontend's
// locally hit-tested gesture, in source bytes.
// Routed by the authenticated `source` (the
// client-supplied frontend_id is untrusted — the
// CrdtOp / Viewport source-trust rule). The window
// aligns to the buffer the frontend says it was
// displaying: a click can race a buffer switch.
if semantic_states.contains_key(&source) {
align_semantic_window_to_buffer(editor, source, buffer_id);
editor.dispatch_pointer(source, buffer_id, byte, kind, mods);
}
}
_ => {
let term_size = *term_sizes
.get(&source)
@ -2052,6 +2071,17 @@ fn apply_event(
handle_dispatcher_event. Dropping op."
);
}
FrontendEvent::Pointer { .. } => {
// Mouse framing Q#M1 — only semantic sessions emit
// Pointer, and `handle_dispatcher_event` routes those via
// `apply_semantic_input_event` (with the authenticated
// source). A grid session sending one is a protocol
// violation; drop it like the CrdtOp arm above.
eprintln!(
"pmacs daemon: FrontendEvent::Pointer from a grid session; dropping \
(semantic sessions route via apply_semantic_input_event)"
);
}
}
}

View File

@ -779,6 +779,81 @@ impl EditorState {
&& prev.at.elapsed() <= DOUBLE_CLICK_MAX_DELAY
}
/// Mouse framing Q#M1 — apply a semantic frontend's locally
/// hit-tested pointer gesture (`FrontendEvent::Pointer`) to its
/// window. The byte-space twin of [`Self::dispatch_mouse`]: same
/// gesture semantics, but the position arrives as a source byte
/// offset the frontend resolved against its own layout (fonts,
/// inline adornments, scroll), so no cell geometry is consulted.
///
/// * `Down` places the cursor and anchors a selection there
/// (a following drag grows it).
/// * `Drag` moves the cursor; the anchor stays.
/// * `Up` collapses an empty selection (a click without drag).
/// * `DoubleDown` selects the word at the hit (frontend-side
/// double-click detection — only it knows pixel proximity).
///
/// The hit byte is clamped into the buffer and snapped back to a
/// UTF-8 boundary: the frontend's hit may race an in-flight edit.
/// `mods` is carried for future Shift-click extension and ignored
/// today, matching `dispatch_mouse`.
pub fn dispatch_pointer(
&mut self,
frontend_id: FrontendId,
buffer_id: crate::buffer::BufferId,
byte: u64,
kind: crate::protocol::PointerKind,
_mods: crate::protocol::Modifiers,
) {
use crate::protocol::PointerKind;
let mut core = self.core.borrow_mut();
core.active_frontend = frontend_id;
let Some(win_id) = core.views.get(&frontend_id).map(|v| v.active) else {
return;
};
// The dispatcher aligns the session's window to the declared
// buffer before calling here; re-check defensively (a click
// can race a buffer switch).
if core.windows.get(&win_id).map(|w| w.buffer_id) != Some(buffer_id) {
return;
}
core.set_active_window_id(win_id);
let byte = {
let registry = core.registry.clone();
let reg = registry.borrow();
let Ok(buf) = reg.get(buffer_id) else {
return;
};
snap_to_char_boundary(buf, byte)
};
match kind {
PointerKind::Down => {
let aw = core.active_window_mut();
aw.cursor = byte;
aw.goal_col = None;
core.begin_selection(byte);
}
PointerKind::Drag => {
let aw = core.active_window_mut();
aw.cursor = byte;
aw.goal_col = None;
}
PointerKind::Up => {
if let Some(sel) = core.active_window().selection
&& sel.anchor == core.cursor()
{
core.clear_selection();
}
}
PointerKind::DoubleDown => {
let aw = core.active_window_mut();
aw.cursor = byte;
aw.goal_col = None;
core.select_word_at_cursor();
}
}
}
/// Make `win_id` the active window and place its cursor at the
/// buffer position corresponding to `(local_row, local_col)`,
/// where the coordinates are relative to the window's viewport
@ -1245,6 +1320,24 @@ fn paint_status_line(
/// Rows available for buffer text inside `rect`, after subtracting
/// the per-window mode line (one row).
/// Clamp `pos` into `buf` and walk back to the nearest UTF-8
/// codepoint boundary. Pointer hits arrive from a frontend whose text
/// may be a few unconfirmed edits ahead of or behind the instance, so
/// a raw byte offset can land mid-codepoint; a snapped position is
/// always safe to assign to a window cursor.
fn snap_to_char_boundary(buf: &crate::buffer::Buffer, pos: u64) -> u64 {
let len = buf.len();
let mut pos = pos.min(len);
while pos > 0 && pos < len {
match buf.snapshot_rope().byte_at(pos) {
// UTF-8 continuation byte (0b10xx_xxxx) ⇒ mid-codepoint.
Some(b) if b & 0b1100_0000 == 0b1000_0000 => pos -= 1,
_ => break,
}
}
pos
}
fn inner_rows(rect: &crate::window::Rect) -> u32 {
rect.size.rows.saturating_sub(1)
}
@ -4249,6 +4342,55 @@ mod tests {
assert!(s.core.borrow().active_region().is_none());
}
/// Mouse framing Q#M1 — `dispatch_pointer` replays the mouse
/// gesture semantics in byte space for semantic frontends.
#[test]
fn dispatch_pointer_replays_mouse_semantics_in_byte_space() {
use crate::protocol::{Modifiers as WireMods, PointerKind};
// Bytes: h=0 é=1,2 ' '=3 l=4 l=5 o=6 ' '=7 w=8 ö=9,10 r=11
// l=12 d=13 \n=14; len=15.
let mut s = fresh_with("hé llo wörld\n".as_bytes());
let bid = s.core.borrow().active_buffer_id();
let none = WireMods::NONE;
// Down places the cursor — a mid-codepoint hit (inside 'é')
// snaps back to the boundary — and anchors a selection.
s.dispatch_pointer(FrontendId::LOCAL, bid, 2, PointerKind::Down, none);
assert_eq!(
s.core.borrow().cursor(),
1,
"mid-codepoint hit snaps to the char boundary"
);
// Drag grows the region from the anchor; Up keeps it.
s.dispatch_pointer(FrontendId::LOCAL, bid, 6, PointerKind::Drag, none);
assert_eq!(s.core.borrow().cursor(), 6);
assert_eq!(s.core.borrow().active_region(), Some((1, 6)));
s.dispatch_pointer(FrontendId::LOCAL, bid, 6, PointerKind::Up, none);
assert_eq!(s.core.borrow().active_region(), Some((1, 6)));
// A plain click (Down + Up, no drag) leaves no region.
s.dispatch_pointer(FrontendId::LOCAL, bid, 4, PointerKind::Down, none);
s.dispatch_pointer(FrontendId::LOCAL, bid, 4, PointerKind::Up, none);
assert_eq!(s.core.borrow().cursor(), 4);
assert!(s.core.borrow().active_region().is_none());
// DoubleDown selects the word at the hit ("wörld").
s.dispatch_pointer(FrontendId::LOCAL, bid, 8, PointerKind::DoubleDown, none);
assert_eq!(s.core.borrow().active_region(), Some((8, 14)));
assert_eq!(s.core.borrow().cursor(), 14);
// Past-EOF hits clamp to the buffer length.
s.dispatch_pointer(FrontendId::LOCAL, bid, 999, PointerKind::Down, none);
assert_eq!(s.core.borrow().cursor(), 15);
// A pointer for a buffer the window isn't displaying is
// dropped (click racing a buffer switch).
let other = crate::buffer::BufferId::next();
s.dispatch_pointer(FrontendId::LOCAL, other, 0, PointerKind::Down, none);
assert_eq!(s.core.borrow().cursor(), 15, "mismatched buffer ignored");
}
/// Acceptance bullet 3: mouse events are coalesced at frame
/// boundaries — many drag events between renders all apply, and
/// the cursor ends up at the last position.

View File

@ -1406,11 +1406,17 @@ fn forward_word(buf: &Buffer, mut pos: Position) -> Position {
}
fn word_range_at(buf: &Buffer, pos: Position) -> Option<(Position, Position)> {
let (ch, _) = char_at(buf, pos)?;
let (ch, ch_len) = char_at(buf, pos)?;
if !is_word_char(ch) {
return None;
}
let start = backward_word(buf, pos);
// Walk back from just *past* the char under the cursor, not from
// `pos` itself: `backward_word(pos)` at a word's FIRST character
// sees the non-word char before it, skips it, and crosses into
// the previous word — double-clicking the 'w' of "llo world"
// would select "llo world". From `pos + ch_len` the char behind
// is this word's own first char, so the walk stops at its start.
let start = backward_word(buf, pos.saturating_add(ch_len));
let end = forward_word(buf, pos);
(start < end).then_some((start, end))
}

View File

@ -1683,29 +1683,33 @@ mod tests {
// --- M5.5a handshake & postcard round-trips ---
#[test]
fn protocol_version_is_four_for_dispatch_idle() {
fn protocol_version_is_five_for_pointer_events() {
// Pin the value: T M10.5 bumped 1→2 (v1.0 wire: CrdtOp /
// PresenceUpdate). T M11.1 bumped 2→3 (v1.1 wire: the
// SemanticFrame family + FrontendEvent::Viewport). T M11.6
// bumped 3→4 (DispatchIdle for the optimistic-apply gate).
// The current binary serves v1..=v4 sessions — the slice-
// The mouse framing Q#M1 bumped 4→5 (FrontendEvent::Pointer,
// byte-position gestures from semantic frontends). The
// current binary serves v1..=v5 sessions — the slice-
// membership handshake makes the relaxation symmetric.
assert_eq!(PROTOCOL_VERSION, 4);
assert_eq!(PROTOCOL_VERSION, 5);
}
#[test]
fn supported_protocol_versions_includes_one_through_four() {
fn supported_protocol_versions_includes_one_through_five() {
// T M10.5: v1.0 binaries accept v1+v2. T M11.1: v1.1 binaries
// accept v1+v2+v3. T M11.6: v4 binaries accept v1+v2+v3+v4.
// The check is slice membership, not strict equality, so
// older binaries keep connecting to current binaries
// unchanged. v5+ is rejected until the next bump.
// accept v1+v2+v3. T M11.6: v4 binaries accept v1..=v4. Mouse
// framing Q#M1: v5 binaries accept v1..=v5. The check is
// slice membership, not strict equality, so older binaries
// keep connecting to current binaries unchanged. v6+ is
// rejected until the next bump.
assert!(is_supported_protocol_version(1));
assert!(is_supported_protocol_version(2));
assert!(is_supported_protocol_version(3));
assert!(is_supported_protocol_version(4));
assert!(is_supported_protocol_version(5));
assert!(!is_supported_protocol_version(0));
assert!(!is_supported_protocol_version(5));
assert!(!is_supported_protocol_version(6));
assert!(!is_supported_protocol_version(u32::MAX));
}
@ -2074,10 +2078,11 @@ mod tests {
fn m10_5_handshake_matrix_versions_outside_range_rejected() {
// v1 daemon's strict-equality behavior is documented at the
// v0.1 code level (different binary); the current daemon's
// range check accepts v1/v2/v3/v4 (T M11.1 added v3; T M11.6
// added v4) and rejects v5+ until the next protocol bump.
// range check accepts v1..=v5 (T M11.1 added v3; T M11.6
// added v4; the mouse framing Q#M1 added v5) and rejects v6+
// until the next protocol bump.
assert!(!is_supported_protocol_version(0));
assert!(!is_supported_protocol_version(5));
assert!(!is_supported_protocol_version(6));
assert!(!is_supported_protocol_version(u32::MAX));
}