fix(json-yaml): checkpoint reviewed LSP configuration fixes

Preserve PR #123's unpushed review fixes on a transfer branch: initial
didChangeConfiguration delivery, explicit JSON validation, the pinned
JSON server provider, corrected YAML configuration sections, and
deterministic plus real-provider acceptance coverage. Record the
observed yaml-language-server 1.24.0 standalone smoke and leave the
real YAML-through-pmacs test, rebase, and full gates explicitly pending
for the destination machine.
This commit is contained in:
Levi Neuwirth 2026-07-20 10:53:04 -04:00
parent 9ce6f1abf3
commit 19ad5cc8ac
6 changed files with 326 additions and 77 deletions

View File

@ -200,37 +200,57 @@ pmacs.lsp.config.zig = pmacs.lsp.config.zig or {
args = {},
}
-- JSON via the maintained `vscode-langservers-extracted` bundle
-- (`vscode-json-language-server`, NOT the stale standalone
-- `vscode-json-languageserver` npm package). Schema-driven
-- diagnostics/completion; auto-associates well-known files
-- (`package.json`, `tsconfig.json`) via its built-in schema store. It
-- pulls `workspace/configuration` for the `json` and `http` sections, so
-- both ship present-not-null (empty ⇒ server defaults). The underlying
-- Microsoft server is MIT with no telemetry path; its only outbound
-- behavior is fetching remote `$schema` content, LEFT ENABLED by default
-- (disabling via `handledSchemaProtocols = {"file"}` would break remote
-- schemas without a `vscode/content` implementation). Sections are
-- derived from the server source/docs — the binary is not installed on
-- this build machine to observe live; verify where it is present.
-- JSON via the VS Code JSON server, binary `vscode-json-language-server`.
-- It is a PUSH-model server: it reads config from
-- `workspace/didChangeConfiguration` (the daemon now sends one after
-- `initialized`) and does NOT issue `workspace/configuration` pulls, so
-- without that push these settings would be inert. `json.validate.enable`
-- is set explicitly true — the server treats a MISSING value as false, so
-- an empty `json = {}` would silently disable validation. Schema
-- retrieval performs NETWORK ACCESS for remote `$schema` URLs (left
-- enabled; `handledSchemaProtocols = {"file"}` would disable it but break
-- remote schemas without a `vscode/content` impl). Note: the server does
-- NOT auto-associate `package.json`/`tsconfig.json` — it starts with empty
-- contributions; explicit `$schema` refs or configured `json.schemas` /
-- a `json/schemaAssociations` push (not implemented) are required.
-- Provider: pin `@t1ckbase/vscode-langservers-extracted@2.0.2`
-- (`npm install -g @t1ckbase/vscode-langservers-extracted@2.0.2`).
-- Its published payload bundles the JSON server from VS Code 1.129.0,
-- preserves this command name, and was live-smoked through initialize →
-- config push → invalid-JSON diagnostic → shutdown. The older unscoped
-- package is stale and the current `@zed-industries` payload has a broken
-- JSON launcher; neither is the recommended provider.
pmacs.lsp.config.json = pmacs.lsp.config.json or {
command = "vscode-json-language-server",
args = { "--stdio" },
settings = { json = {}, http = {} },
settings = {
json = { validate = { enable = true } },
http = {},
},
}
-- YAML via Red Hat `yaml-language-server`. It pulls
-- `workspace/configuration` for `yaml`, `http`, and `redhat.telemetry`;
-- all three ship present-not-null, with Red Hat telemetry disabled by
-- default (privacy-respecting; inert if the server never asks). Sections
-- derived from source/docs — verify against an installed binary.
-- YAML via Red Hat `yaml-language-server`. On
-- `workspace/didChangeConfiguration` (now pushed after `initialized`) it
-- reads the `yaml`, `http`, `[yaml]`, `editor`, and `files` sections — all
-- ship present-not-null (empty ⇒ server defaults). SchemaStore / remote
-- schema retrieval performs NETWORK ACCESS by default. The standalone
-- server does not upload telemetry itself — it emits `telemetry/event`
-- notifications to its client, and pmacs has no telemetry uploader, so a
-- `redhat.telemetry` setting would be inert and is not shipped. Sections
-- live-observed with Red Hat `yaml-language-server@1.24.0`: its initial
-- pull requests exactly those five sections, and opening a YAML document
-- requests a second scoped `[yaml]` section. The standalone smoke reached
-- a real syntax diagnostic and clean shutdown; the PATH-gated
-- pmacs-through-server acceptance remains a transfer task.
pmacs.lsp.config.yaml = pmacs.lsp.config.yaml or {
command = "yaml-language-server",
args = { "--stdio" },
settings = {
yaml = {},
http = {},
redhat = { telemetry = { enabled = false } },
["[yaml]"] = {},
editor = {},
files = {},
},
}

View File

@ -325,13 +325,15 @@ grammars — HTML/CSS/GraphQL/SQL (`<script>`/`<style>`, JS/TS template
literals, doc-comment code); modeline detection as a 5th layer
(`-*- mode: … -*-` / `# vim: ft=…`);
byte-accurate multibyte cursor placement in `move_active_cursor_to`
(still steps one codepoint per LSP byte column). **JSON + YAML grammars
LANDED** (grammar-gap style, `tree-sitter-json`/`-yaml`; LSP configs
`vscode-json-language-server` / `yaml-language-server`; YAML `---` and
TOML `+++` markdown frontmatter now highlight via the #122 engine). A
full Jupyter `.ipynb` setup (reader → editable → kernel execution) has
both grammar prerequisites in place now (JSON + injections); what remains
is the reader → editable → kernel arc itself, NOT a one-shot.
(still steps one codepoint per LSP byte column). **JSON + YAML PR #123
OPEN** (grammar-gap style, `tree-sitter-json`/`-yaml`; LSP configs
`vscode-json-language-server` with provider pin
`@t1ckbase/vscode-langservers-extracted@2.0.2`, plus
`yaml-language-server`). Review fixes are preserved on
`json-yaml-handoff-2026-07-20`; the real YAML-through-pmacs smoke,
rebase, and full gates remain. Once merged, YAML `---` and TOML `+++`
markdown frontmatter highlight via the #122 engine and the Jupyter
reader → editable → kernel arc has both grammar prerequisites.
GPU: auto-reconnect after daemon restart, splits/multi-buffer, gutter
riders (whitespace guides, folding, git markers).
Themes (full list in theme-faces framing rev 9 "Deferred (named)"):

View File

@ -1,5 +1,13 @@
# JSON + YAML grammars — framing (side quest, highlight family)
**Revision 2 — 2026-07-20. Status: PR #123 open; review-fix
checkpoint preserved on branch `json-yaml-handoff-2026-07-20`.**
The public PR branch still pointed to the original implementation
`4be2a65` when this checkpoint was created. JSON provider resolution
and its real pmacs smoke are complete; the YAML 1.24.0 standalone
protocol smoke is complete; the real YAML-through-pmacs acceptance,
rebase onto current main, and full gates remain.
**Intent.** Add `tree-sitter-json` and `tree-sitter-yaml` grammars (plus
their language servers) to the bundle. Two config formats that pmacs
currently renders as plain text, and — the reason this is the natural
@ -76,30 +84,51 @@ Root kinds (pinned by the ABI test): json `document`, yaml `stream`.
### Q#JY2 — LSP configs: `vscode-json-language-server` + `yaml-language-server`
- **json:** `vscode-json-language-server --stdio` from the **maintained
`vscode-langservers-extracted` bundle** — NOT the stale standalone
`vscode-json-languageserver` npm package. Schema-driven
diagnostics/completion; auto-associates well-known files
(`package.json`, `tsconfig.json`) via its schema store. The underlying
Microsoft JSON server is **MIT-licensed with no telemetry path**; its
only relevant outbound behavior is fetching remote `$schema` content.
**Remote schema retrieval stays enabled by default** — setting
`handledSchemaProtocols = {"file"}` would disable server-side HTTP but
then remote schemas fail unless pmacs implements the `vscode/content`
request (out of scope), so we leave it on and document the behavior.
- **yaml:** `yaml-language-server --stdio` (Red Hat).
- **json:** binary `vscode-json-language-server --stdio` (the VS Code
JSON server). It is **push-model**: it reads config from
`workspace/didChangeConfiguration` and does **not** issue
`workspace/configuration` pulls — so pmacs, which previously only
*answered* pulls, must now also **push** a `didChangeConfiguration`
after `initialized` (a general LSP-client fix in `src/lsp.rs`; pull
servers ignore it). `json.validate.enable` is set **explicitly true**
— a missing value reads as false and silently disables validation, so
an empty `json = {}` is wrong. The server does **not** auto-associate
`package.json`/`tsconfig.json` (it starts with empty contributions);
explicit `$schema` refs or configured `json.schemas` / a
`json/schemaAssociations` push (not implemented) are required. Schema
retrieval performs **network access** for remote `$schema` URLs, left
enabled (`handledSchemaProtocols = {"file"}` would disable it but break
remote schemas without a `vscode/content` impl). **Provider:** pin
`@t1ckbase/vscode-langservers-extracted@2.0.2`
(`npm install -g @t1ckbase/vscode-langservers-extracted@2.0.2`). Its
published payload bundles the JSON server from VS Code 1.129.0,
preserves the `vscode-json-language-server` command, and was
live-smoked through initialize → config push → invalid-JSON diagnostic
→ shutdown. The unscoped package is stale; the current
`@zed-industries` payload has a broken JSON launcher, so neither is the
recommended provider.
- **yaml:** `yaml-language-server --stdio` (Red Hat). Its settings handler
reads the sections **`yaml`, `http`, `[yaml]`, `editor`, `files`** (via
`didChangeConfiguration` / pulls) — all ship present-not-null. It does
**not** upload telemetry itself (it emits `telemetry/event` to the
client; pmacs has no uploader), so a `redhat.telemetry` setting is inert
and is not shipped. SchemaStore / remote schema retrieval performs
**network access** by default.
Both servers stay **external** (installed by the user), adding **no
licensing payload** to pmacs; if either is ever bundled, retain its MIT +
dependency notices. Both **pull `workspace/configuration`** (the
mechanism that bit CMake in #117), so the config ships **non-nil empty
settings** matching the taplo/cmake precedent. **Implementation verifies
the exact `workspace/configuration` sections each real server requests
and pins those sections in the config + tests** — not merely that a
non-null settings table exists (if a server is not installed on the
build machine, the sections are derived from its source/docs and the
gap is noted, per the basedpyright machine-caveat precedent). Servers
activate only if installed; the grammar is the always-on value.
dependency notices. The exact sections are **pinned in the config + a
test** (not merely "some non-nil table exists"). The pinned JSON
provider was installed into an isolated temporary prefix and
live-smoked through pmacs. Red Hat `yaml-language-server@1.24.0` was
also installed in an isolated prefix and live-smoked over stdio: its
initial configuration pull was exactly `yaml`, `http`, `[yaml]`,
`editor`, `files`; opening the document caused a second scoped
`[yaml]` pull; invalid YAML produced a parser diagnostic; shutdown was
clean. The remaining gap is the same provider through pmacs.
Config-push delivery is also proven deterministically through the fake
server's config sink. Servers activate only if installed; the grammar
is the always-on value.
### Q#JY3 — Filetype fallback + alias entries
@ -125,9 +154,11 @@ fences resolve through the #122 engine. Acceptance proves both end to end
languages — verified by a build; the ABI test is the runtime pin.
2. The frontmatter/fence synergy needs zero engine changes — it falls out
of #122 + the markdown injection query.
3. The LSP `workspace/configuration` shape is the only real unknown;
empty-but-present settings + implementation-time verification against
the real servers is the mitigation (the #117 CMake lesson).
3. The two configuration models are now observed: JSON consumes the
pushed full settings object; YAML 1.24.0 pulls the five documented
sections plus a document-scoped `[yaml]` request. The remaining bet
is that pmacs answers the real YAML server correctly end to end,
which the pending PATH-gated acceptance must prove.
## Deferred (named)
@ -159,17 +190,31 @@ fences resolve through the #122 engine. Acceptance proves both end to end
highlights; **the headline synergy with #122**.
8. `json_fence_injects_in_markdown` — a ` ```json ` fence yields a `json`
child layer.
9. LSP config tests (m4-style): `pmacs.lsp.config.json`/`.yaml` present
with the expected command/args (json uses
`vscode-json-language-server`), and the settings table carries
**exactly the `workspace/configuration` sections each server
requests** — pinned, not merely asserted non-nil.
9. `m4_json_yaml_lsp_configs_pin_command_and_sections` — the configs
pin the binary + `json.validate.enable = true` + the exact section
sets (json: `json`,`http`; yaml: `yaml`,`http`,`[yaml]`,`editor`,
`files`; no inert `redhat.telemetry`) — pinned, not merely non-nil.
10. `m4_5_initial_config_pushed_via_did_change_configuration` — the
daemon PUSHES `workspace/didChangeConfiguration` after `initialized`
(the push-model delivery path), verified through the fake server's
config sink. Without it, push-only servers' settings are inert.
11. `m4_real_json_provider_receives_config_and_reports_diagnostics`
PATH-gated live smoke for the pinned provider: initialize through
pmacs, receive the pushed default config, open invalid JSON, and
publish a syntax diagnostic. Skips when the binary is absent.
12. `m4_real_yaml_provider_pulls_config_and_reports_diagnostics`
**pending on the destination machine**. PATH-gated live smoke for
Red Hat `yaml-language-server@1.24.0`: auto-attach through pmacs,
disable SchemaStore and Kubernetes CRD catalog network access for
determinism, reach initialized, open invalid YAML, publish a
diagnostic, and remain alive.
## Risks / interactions
- **LSP `workspace/configuration`** (Q#JY2) — the real risk; verified
against the servers at implementation time, empty-present settings as
the safe default.
- **LSP configuration** (Q#JY2) — JSON push and YAML standalone pulls
are observed. The real YAML-through-pmacs path is still pending and
is an explicit transfer blocker, not implied complete by source
inspection.
- **Themes / injections** — untouched. This is pure grammar+detection
addition; it consumes the #122 engine, doesn't change it. No protocol
bump.

View File

@ -414,6 +414,26 @@ fn main() {
});
write_frame(&mut stdout, &resp);
}
("workspace/didChangeConfiguration", _) => {
// Record the pushed `settings` so a test can assert the
// daemon delivered configuration after `initialized` (the
// push-model config-delivery path push-only servers like the
// VS Code JSON server rely on). One JSON line per push.
if let Ok(sink) = std::env::var("PMACS_FAKE_LSP_CONFIG_SINK") {
use std::io::Write as _;
if let Ok(mut f) = std::fs::OpenOptions::new()
.create(true)
.append(true)
.open(&sink)
{
let settings = params
.get("settings")
.cloned()
.unwrap_or(serde_json::Value::Null);
let _ = writeln!(f, "{settings}");
}
}
}
("textDocument/didOpen" | "textDocument/didChange", _) => {
let uri = params
.get("textDocument")

View File

@ -2497,6 +2497,25 @@ impl LspManager {
}
}
/// Push the client's configured `settings` via
/// `workspace/didChangeConfiguration` immediately after `initialized`.
/// Push-model servers — notably the VS Code JSON server, which listens
/// for this notification and does NOT issue `workspace/configuration`
/// pulls — only learn their config this way; pull-model servers
/// (pyright, clangd, gopls) ignore it and pull instead, so it is safe
/// to send unconditionally. No-op when no `settings` are configured.
fn push_initial_configuration(&self, sid: LspServerId) {
if let Some(client) = self.clients.get(&sid)
&& let Some(settings) = client.spec.settings.clone()
{
let cfg = make_notification(
"workspace/didChangeConfiguration",
json!({ "settings": settings }),
);
let _ = send_frame_to(&self.supervisor, client, &cfg);
}
}
fn handle_response(
&mut self,
sid: LspServerId,
@ -2552,6 +2571,9 @@ impl LspManager {
if let Some(client) = self.clients.get(&sid) {
let _ = send_frame_to(&self.supervisor, client, &body);
}
// Push the configured settings right after `initialized`
// (before any deferred `didOpen`).
self.push_initial_configuration(sid);
// T M4.5 Option B: honour the server's negotiated
// `general.positionEncoding`. Absent ⇒ LSP spec default
// (UTF-16). We advertised `["utf-8","utf-16"]`, so a

View File

@ -6290,36 +6290,40 @@ fn m4_gap_grammars_align_with_lsp_configs() {
}
}
/// JSON/YAML LSP configs pin the exact server binary and the
/// `workspace/configuration` sections each server pulls (framing Q#JY2).
/// The servers are not installed on the build machine, so the section set
/// is derived from server source/docs and pinned here as the config
/// contract — a machine with the binaries should confirm them live. The
/// point is to pin the sections, not merely that some settings table
/// exists.
/// JSON/YAML LSP configs pin the server commands and the settings shape
/// each consumes (framing Q#JY2): JSON receives a pushed full object;
/// YAML pulls five named sections. The pinned JSON provider has a separate
/// PATH-gated live smoke below. YAML 1.24.0's five-section pull was
/// observed over a standalone protocol smoke; its PATH-gated pmacs
/// integration test remains a transfer task. The point is to pin the
/// contract, not merely assert that some settings table exists.
#[test]
fn m4_json_yaml_lsp_configs_pin_command_and_sections() {
use pmacs::editor::EditorState;
let s = EditorState::new();
let lua = s.lua_host.lua();
// json: the maintained extracted-bundle binary (NOT the stale
// standalone `vscode-json-languageserver`), `--stdio`, and the `json`
// + `http` workspace-config sections present (empty ⇒ server defaults;
// remote `$schema` fetch left enabled — no `handledSchemaProtocols`).
// json: the `@t1ckbase/vscode-langservers-extracted@2.0.2` binary
// (NOT the stale standalone `vscode-json-languageserver`), `--stdio`,
// and the `json` + `http` workspace-config sections present. The
// provider preserves this stable command name; its exact pin and live
// handshake evidence are documented beside the default config.
let json_command: String = lua
.load("return pmacs.lsp.config.json.command")
.eval()
.unwrap();
assert_eq!(
json_command, "vscode-json-language-server",
"json uses the extracted-bundle binary, not the stale standalone"
"json uses the pinned T1ckbase provider's stable command name"
);
// The JSON server is push-model (reads didChangeConfiguration, no
// pulls), so `json.validate.enable` must be EXPLICITLY true — a missing
// value reads as false and disables validation. Remote schemas left on.
let json_ok: bool = lua
.load(
"local c = pmacs.lsp.config.json
return c.args[1] == '--stdio'
and c.settings.json ~= nil
and c.settings.json.validate.enable == true
and c.settings.http ~= nil
and c.settings.handledSchemaProtocols == nil",
)
@ -6327,11 +6331,14 @@ fn m4_json_yaml_lsp_configs_pin_command_and_sections() {
.unwrap();
assert!(
json_ok,
"json config: --stdio, json + http sections present, remote schemas left on"
"json config: --stdio, json.validate.enable=true, http present, remote schemas on"
);
// yaml: `yaml-language-server --stdio`; `yaml` + `http` +
// `redhat.telemetry` sections, Red Hat telemetry disabled by default.
// yaml: `yaml-language-server --stdio`. Its settings handler reads the
// `yaml`, `http`, `[yaml]`, `editor`, and `files` sections — pin all
// five, and confirm the inert `redhat.telemetry` is NOT shipped (the
// standalone server emits telemetry events to the client; it does not
// upload, and pmacs has no uploader).
let yaml_command: String = lua
.load("return pmacs.lsp.config.yaml.command")
.eval()
@ -6346,16 +6353,149 @@ fn m4_json_yaml_lsp_configs_pin_command_and_sections() {
return c.args[1] == '--stdio'
and c.settings.yaml ~= nil
and c.settings.http ~= nil
and c.settings.redhat.telemetry.enabled == false",
and c.settings['[yaml]'] ~= nil
and c.settings.editor ~= nil
and c.settings.files ~= nil
and c.settings.redhat == nil",
)
.eval()
.unwrap();
assert!(
yaml_ok,
"yaml config: --stdio, yaml + http + redhat.telemetry sections, telemetry off"
"yaml config: --stdio, the five pulled sections present, no inert redhat.telemetry"
);
}
/// Round-1 finding (P1): the daemon must PUSH configuration via
/// `workspace/didChangeConfiguration` after `initialized`. Push-model
/// servers — notably the VS Code JSON server — never issue
/// `workspace/configuration` pulls, so without the push their `settings`
/// (including `json.validate.enable`) are inert. Verified through the fake
/// server's config sink: the settings the daemon sends are recorded and
/// inspected, proving delivery end to end.
#[test]
fn m4_5_initial_config_pushed_via_did_change_configuration() {
use pmacs::editor::EditorState;
let mut state = EditorState::new();
let fake = fake_lsp_path();
let dir = tempfile::tempdir().expect("tempdir");
let sink = dir.path().join("config.jsonl");
let file = dir.path().join("probe.rs");
std::fs::write(&file, "fn main() {}\n").expect("write");
let sink_disp = sink.display().to_string();
let file_disp = file.display().to_string();
// Point rust at the fake server WITH a settings table and route the
// config sink into the spawned process, then open the file (auto-attach
// → initialize → initialized → the config push).
state
.lua_host
.lua()
.load(format!(
"pmacs.lsp.config.rust = {{
command = '{fake}',
env = {{ PMACS_FAKE_LSP_CONFIG_SINK = '{sink_disp}' }},
settings = {{ rust = {{ probe = true }} }},
}}
pmacs.buffer.find_or_open('{file_disp}')"
))
.exec()
.expect("configure + open");
assert!(
pump_lua_flag(
&mut state,
"(function() for _,r in ipairs(pmacs.lsp.list()) do \
if r.state and r.state.kind=='initialized' then return true end \
end return false end)()",
5,
),
"fake never initialized"
);
// A few more ticks for the push + the server's sink write to land.
let sink_probe = sink.clone();
pump_async(&mut state, move |_| {
std::fs::read_to_string(&sink_probe).is_ok_and(|s| s.contains("probe"))
});
let recorded = std::fs::read_to_string(&sink).unwrap_or_else(|e| {
panic!("config sink not written ({e}); the didChangeConfiguration push did not arrive")
});
assert!(
recorded.contains("\"probe\":true"),
"the daemon pushed the configured settings after initialized: {recorded}"
);
}
/// PATH-gated provider smoke: drive a real `vscode-json-language-server`
/// through pmacs's default JSON config, including the post-initialize
/// `didChangeConfiguration` push, and require a syntax diagnostic for an
/// invalid document. The reviewed provider is
/// `@t1ckbase/vscode-langservers-extracted@2.0.2`; CI skips cleanly when
/// no compatible binary is installed.
#[test]
fn m4_real_json_provider_receives_config_and_reports_diagnostics() {
use pmacs::editor::EditorState;
let Ok(command) = which_binary("vscode-json-language-server") else {
eprintln!("vscode-json-language-server not on PATH; skipping");
return;
};
let command = command.display().to_string();
let dir = tempfile::tempdir().expect("tempdir");
let file = std::fs::canonicalize(dir.path())
.expect("canonicalize")
.join("invalid.json");
std::fs::write(&file, b"{\"broken\": }\n").expect("write invalid json");
let file_disp = file.display().to_string();
let uri = format!("file://{file_disp}");
let mut state = EditorState::new();
state
.lua_host
.lua()
.load(format!(
"pmacs.lsp.config.json.command = '{command}'
pmacs.buffer.find_or_open('{file_disp}')"
))
.exec()
.expect("configure real JSON server + open file");
assert!(
pump_lua_flag(
&mut state,
"(function() for _,r in ipairs(pmacs.lsp.list()) do \
if r.state and r.state.kind=='initialized' then return true end \
end return false end)()",
30,
),
"real JSON server never reached initialized"
);
let deadline = Instant::now() + Duration::from_secs(10);
let mut got_diagnostic = false;
while Instant::now() < deadline {
state.tick_processes();
state.tick_lsp();
state.tick_async();
got_diagnostic = state
.lua_host
.lua()
.load(format!("return pmacs.diag.count('{uri}') > 0"))
.eval()
.unwrap_or(false);
if got_diagnostic {
break;
}
std::thread::sleep(Duration::from_millis(10));
}
assert!(
got_diagnostic,
"real JSON server produced no diagnostic; config delivery or validation is broken"
);
assert_no_lsp_crash(&mut state, "real JSON server");
}
/// Typing-perf: the default bundle coalesces full-document
/// `didChange` notifications instead of sending one per keystroke
/// (each send copies the whole buffer several times and writes