pmacs/docs/mcp-for-package-authors.md

MCP for package authors

This guide is for package authors who want to integrate Model Context Protocol (MCP) servers into pmacs. It covers the pmacs.mcp.* API, the architectural pattern for AI-assistance packages, and the disciplines distilled from M9.5 – M9.8's audit work.

The general package-publishing mechanics are at docs/package-author-guide.md. This document is the inverse: how to write a package that uses an MCP server.

Five fixture packages live under tests/fixtures/ as worked examples:

• pmacs-mcp-resources/ — resources
• pmacs-mcp-tools/ — tools-as-commands
• pmacs-mcp-prompts/ — prompts-as-result-buffers
• pmacs-mcp-ai/ — AI-assistance composing the above

---

1. Why MCP for package authors

The architectural claim (spec/pmacs-spec.tex, §sec:m9-ai) is:

AI is a transport binding, not a feature.

In practical terms: pmacs has no built-in "ask Claude" command, no hard-coded model API, no Anthropic-specific or OpenAI-specific code anywhere in the editor. Instead, pmacs ships an MCP transport layer, and AI features are built as packages on top.

This means:

1. Your package speaks MCP, not a model API. The model behind the configured server is interchangeable. Re-pointing your package at a different MCP server changes which model serves the prompts; your code is unchanged.
2. Your package composes with other MCP packages. The transport layer is shared; subscriptions, caching, cancellation are uniform. Two packages talking to the same server share a single process.
3. The user's API keys live in the MCP server's environment, not yours. Your package never sees credentials. The server process inherits them; pmacs's spawn API takes a command and optional env.

The recommendation: write against the MCP layer, not against any specific model API. If your package wants to talk to Claude, spawn an MCP server that talks to Claude. If your package wants to talk to GPT, spawn an MCP server that talks to GPT. Your package code is the same.

---

2. The transport: pmacs.mcp.*

The full public API surface is six Lua functions plus userdata methods. Everything else MCP-related is reachable from these.

pmacs.mcp.spawn { ... } → McpServerIdLua

Spawn an MCP server as a child process and start the initialize handshake.

local server = pmacs.mcp.spawn {
  label   = "my-server",       -- string. Used as buffer-name prefix and roster key.
  command = "/usr/local/bin/mcp-claude",
  args    = { "--config", "/path/to/config.toml" },  -- optional
  env     = { ANTHROPIC_API_KEY = os.getenv("ANTHROPIC_API_KEY") },  -- optional
  restart = "OnCrash",         -- "OnCrash" (default) | "Always" | "Never"
}

Spawning is asynchronous: spawn returns immediately with a handle. The server may take some time to initialize. Use pmacs.mcp.list() to observe state transitions.

pmacs.mcp.list() → array

Returns the current server roster:

for _, row in ipairs(pmacs.mcp.list()) do
  print(row.label, row.id, row.state.kind)
  -- row.state.kind: "spawning" | "initializing" | "initialized" | "crashed" | "exited"
end

Use state.kind == "initialized" as the gate for sending requests. Sending to a non-initialized server raises not ready for requests.

pmacs.mcp.read_resource(server, uri) → handle

Read an MCP resource. Returns an awaitable handle:

pmacs.async(function()
  local body = pmacs.mcp.read_resource(server, "file:///etc/config.toml"):await()
  -- body = { contents = [ { uri, text? | blob?, mimeType? }, ... ] }
end)

The handle is cache-aware: a settled response is returned from cache for subsequent calls with the same (server, uri) until invalidation. Concurrent calls during an in-flight request share the awaitable. Cancellation of an awaiter is independent — the wire request is only cancelled when all awaiters cancel.

Invalidation triggers:

• notifications/resources/updated (per-uri) — invalidates that uri.
• notifications/resources/list_changed — invalidates all resources for that server.

You don't typically call on_notification for these — the cache listens internally.

pmacs.mcp.invoke_tool(server, name, args) → handle

Invoke an MCP tool. Returns an awaitable handle:

pmacs.async(function()
  local result = pmacs.mcp.invoke_tool(server, "search", { query = "foo" }):await()
  -- result = { content = [ { type, text? | image? | resource? }, ... ],
  --            isError = bool, _meta? }
  if result.isError then
    -- semantic failure: the tool ran but reported a failure
  end
end)

Three failure modes to distinguish:

1. Success: isError = false, content has the result.
2. Semantic failure: isError = true, content describes the failure ("file not found", "permission denied"). The tool ran; the operation failed.
3. Transport / protocol failure: the :await() call raises. Either the server is gone, the request was cancelled, the server returned a JSON-RPC error (unknown tool, invalid args).

Don't conflate (2) and (3). Semantic failures are results; transport failures are exceptions.

Tool calls are not cached client-side — the server may have side effects, and v0.1 doesn't read the MCP idempotency hint. Cancellation is via :cancel() on the handle.

pmacs.mcp.get_prompt(server, name, args) → handle

Get an MCP prompt response. Returns an awaitable handle:

pmacs.async(function()
  local response = pmacs.mcp.get_prompt(server, "review_function", {
    language = "rust",
    file_path = "src/main.rs",
    source = "fn main() { ... }",
  }):await()
  -- response = { description?, _meta?, messages = [ { role, content }, ... ] }
end)

Required arguments are validated by the server — missing them raises a JSON-RPC error.

response._meta.format carries a content-type hint when the server supports it: "text" (default), "code" (with _meta.language for syntax highlighting), "markdown". Unknown formats fall back to text. The pmacs-mcp-prompts.render function (see §6) reads these hints and routes the buffer through the appropriate highlight pipeline.

args may be a Lua table with structured values — arrays, nested objects. The wire shape is JSON. M9.8's pmacs-mcp-ai uses this to send a structured files: [{path, content}, ...] array for its project-context prompt. Don't separator-encode JSON into strings; let the marshaler handle it.

pmacs.mcp.on_notification(method, fn) → token

Subscribe to MCP server-to-client notifications. The subscription is global per method, not per-server: the handler fn receives (server, params) and is responsible for filtering by server if it cares.

local token = pmacs.mcp.on_notification("notifications/tools/list_changed", function(server, params)
  -- This fires for *any* server's list_changed. Filter if you only
  -- want events from servers your package has registered:
  if not _registered_servers[server:raw()] then return end
  -- re-fetch tools/list and reconcile commands
end)

-- Later:
pmacs.mcp.off_notification("notifications/tools/list_changed", token)

Multiple subscriptions to the same method fire in registration order. A throwing callback doesn't break the dispatcher — the error is logged via pmacs.error and the next callback fires.

The dispatcher under the hood is a single per-tick drain regardless of how many packages have registered. The Rust side is told to queue notifications for method on the first subscription and stop on the last unsubscription, so there's no idle-cost when no package cares about a method.

Subscription discipline: balance subscribes with cancels. M9.6's per-server-refcount finding showed how easily a re-register flow leaks subscriptions: if your package's register(server) calls on_notification unconditionally, registering N servers leaks N-1 subscriptions for the same method. Use a per-server refcount — add the subscription on the first server registered, drop it on the last server unregistered.

---

3. Server lifecycle in your package

The shape M9.6/M9.7/M9.8 settled on:

local M = {}

local _registered_servers = {}  -- per-server state, keyed by server:raw()

function M.register(server)
  local key = server:raw()
  if _registered_servers[key] then return end  -- idempotent
  _registered_servers[key] = {
    -- ... per-server state: subscriptions, label, etc.
  }
  -- subscribe, fetch initial state, define commands, etc.
end

function M.unregister(server)
  local key = server:raw()
  local state = _registered_servers[key]
  if state == nil then return end
  -- unsubscribe, drop commands, etc.
  _registered_servers[key] = nil
end

Why server:raw() and not tostring(server): M9.6 finding 2. The :raw() userdata method returns the canonical underlying id as a string; tostring formats it for display and may not be stable across pmacs versions.

Server-gone teardown

When the server crashes or exits, in-flight requests fail with unknown server or not ready for requests. Detect these in your dispatch path:

local function looks_like_server_gone(err)
  local s = type(err) == "table" and tostring(err.message or "") or tostring(err)
  return s:find("unknown server", 1, true) ~= nil
      or s:find("not ready for requests", 1, true) ~= nil
end

When this fires, tear down your per-server state (drop commands, clear caches) so the next user invocation surfaces a useful "reconfigure" message rather than the same dead-server error on every retry. M9.6 finding 5.

---

4. Pattern: tools as commands

Pattern from pmacs-mcp-tools. Each tool advertised by the server becomes a command at <label>-<tool-name>. The list of advertised tools is fetched via the generic pmacs.mcp.send_request seam — there's no dedicated pmacs.mcp.list_tools in v0.1 because every server's tools/list shape is identical and the generic seam handles it cleanly:

function M.register(server)
  pmacs.async(function()
    local result = pmacs.mcp.send_request(server, "tools/list", {}):await()
    -- result.tools is the array of tool entries: { name, description?, inputSchema? }
    for _, tool in ipairs(result.tools or {}) do
      define_command_for(server, tool)
    end
  end)
end

pmacs.mcp.send_request(server, method, params) returns the same awaitable handle shape as read_resource / invoke_tool / get_prompt and is the bottom-rung public seam: any MCP request the spec defines is reachable through it. The pmacs-mcp-tools/init.lua fixture is the worked example.

(Whether tools/list and prompts/list deserve dedicated typed surfaces is an audit consideration; v0.2 territory if real package authors find the generic seam awkward.)

Reconciliation on list_changed

Tools can change at runtime. Subscribe (global per-method; filter by server inside the handler):

pmacs.mcp.on_notification("notifications/tools/list_changed", function(server, params)
  if not _registered_servers[server:raw()] then return end
  -- Re-fetch tools/list, diff against current commands, add/remove.
end)

Compute a schema hash per tool to detect schema changes (not just name changes). Add commands for new tools, remove for gone tools, redefine for changed-schema tools. M9.6's package shows the canonical implementation.

Cross-source command collisions

Two servers advertising a tool with the same name can both want to register <label>-<tool-name> if the labels collide, or different name shapes can collide with builtins. Always check:

if pmacs.command.exists(name) then
  -- skip + warn, don't abort the rest of the registration
else
  pmacs.command.define { name = name, ... }
end

M9.6 finding 6.

---

5. Pattern: prompts as result buffers

Pattern from pmacs-mcp-prompts. Each prompt becomes a command that prompts for required args, calls get_prompt, renders the response into a *mcp:<label>:<prompt>* buffer.

The package exposes a public function that v0.2+ packages should compose with, not duplicate:

local mcp_prompts = require("pmacs-mcp-prompts")

-- After calling pmacs.mcp.get_prompt and awaiting:
mcp_prompts.render(server_label, prompt_name, response)

render handles:

• buffer creation / reuse (keyed by (label, prompt))
• read-only intercept (so the user can't accidentally edit the result)
• format dispatch via _meta.format (text / code / markdown)
• syntax highlighting attach (for code / markdown formats)
• cursor / region / scroll reset on re-paint

If your package wants different rendering — multi-turn conversation history, inline image rendering, etc. — you write your own. The composition story is opt-in: M9.8 chose to compose so that re-invoking the same prompt from M9.7's auto- registered command and M9.8's ai.ask-about-X lands in the same buffer. Your package may have different ergonomic goals.

---

6. Pattern: AI assistance composing the above

Pattern from pmacs-mcp-ai. The full M9.8 example is 247 lines of code (461 total with comments). Three commands, tree-sitter context selection, project-buffer collection, structured-arg prompts.

The architectural commitment, recorded in init.lua's header:

-- Architectural commitment (the M9.8 ship gate):
--
--   * Zero direct calls into the Rust core. Everything reaches the
--     Rust side through the public Lua surface (`pmacs.mcp.*`,
--     `pmacs.parse.*`, `pmacs.command.*`, etc.).
--   * Zero model-specific code. The package speaks MCP; the model
--     behind the configured server is interchangeable.

The configure { server_label, prompts = { ... } } shape is the key:

local ai = require("pmacs-mcp-ai")
ai.configure {
  server_label = "claude-mcp",
  prompts = {
    fn      = "review_function",
    project = "review_project",
    ask     = "ask_freeform",
  },
}

server_label is the only model-specific input. Re-configure to a different server_label (with the same prompt names served by a different MCP server) and the same commands route to the new server. Zero code changes.

This is the recommendation in concrete form: structure your package's API so that the model is a configuration knob, not a code path.

---

7. Disciplines from the M9 audits

The audit findings from M9.6 – M9.8 distill into a small set of disciplines worth applying up-front in every new MCP package:

notify() should hit both status and error

Status messages are overwritten by the next status message. Errors persist in *pmacs-error*. Use both:

local function notify(msg)
  pmacs.editor.set_status(msg)
  if pmacs.error then
    pmacs.error("my-package: " .. msg)
  end
end

M9.6 finding 10.

Server-gone clears your local state

See §3.

Cross-source DuplicateName via pmacs.command.exists

See §4.

Subscription refcount

If your register flow subscribes to notifications, balance subscribes with cancels. Use a per-server refcount, not "always subscribe."

M9.6 finding 3.

Required-arg-order is identity, not order

When validating prompts/tools args, treat the arguments field as a set of required names. The advertised order is for UI prompting; required-ness is per-arg, not per-position.

M9.6 finding 4.

Buffer state keyed by tostring(buf), not userdata

When tracking per-buffer state in a Lua table:

-- WRONG: silently fails on every re-lookup
local _state = {}
_state[buf] = { ... }

-- RIGHT: stable per underlying BufferId
local _state = {}
_state[tostring(buf)] = { ... }

pmacs.buffer.list() and pmacs.window.buffer() return fresh userdata wrappings on every call — a userdata-keyed lookup only finds the first wrapping ever inserted. tostring(buf) is stable per underlying id, same convention builtin/runtime/syntax.lua already uses for highlighted_buffers.

M9.8 amendment to M9.7 audit. The bug went undetected through M9.7's full acceptance suite because the suite only checked buffer count, not body update.

Test seams use _underscore_prefix

Functions exposed on your module table for tests but not part of your stable public API should be _prefix-named:

function M._find_enclosing_function(buf, byte_pos)
  -- Test seam (unstable). Public API uses the higher-level
  -- ai.ask-about-function command.
end

This is an idiom, not enforced — but it makes the audit's "public API surface" math unambiguous. The M9.6 audit's _render_schema_doc, M9.7's _format_messages, M9.8's _collect_project_files all follow this pattern.

---

8. The five fixture packages as worked examples

Package	Demonstrates
pmacs-mcp-resources	Resource read with cache awareness; notifications/resources/updated consumption
pmacs-mcp-tools	Tool-call dispatch; tools-as-commands reconciliation on list_changed; M9.6's twelve audit findings disposed
pmacs-mcp-prompts	Prompt-as-result-buffer rendering; format-hint dispatch; tree-sitter highlight attach for code/markdown
pmacs-mcp-ai	AI-assistance composing the above; tree-sitter context selection; structured-arg prompts; server pluggability

Each ships under tests/fixtures/ because they're audit fixtures. The packages compile and run as real packages — the fixture location is just where they live in-tree. v0.2+ may move them to builtin/packages/ once the audit pipeline is settled.

The corresponding audit docs are M9.5-AUDIT.md ... M9.9-AUDIT.md. Each documents the per-package architectural decisions, surface-area math, and audit-finding disposition.

---

9. Where things go wrong

Common pitfalls:

1. Sending requests before state.kind == "initialized". pmacs.mcp.spawn returns before the handshake completes. Gate on the roster row's state.kind, or use the wait-and-poll pattern from tests/m9_8_acceptance.rs's spawn_initialized_server.

2. Not distinguishing isError from JSON-RPC error. A tool reporting isError = true is a normal result. Don't pcall the await and treat all failures equally — the user wants different UX for "the tool said no" vs "the server fell over."

3. Caching tool results client-side. Don't. v0.1's tool layer doesn't, and you shouldn't either — tools may have side effects, and the MCP idempotency hint isn't surfaced yet.

4. Subscribing per-call instead of per-server. If your command subscribes to list_changed every invocation, you'll accumulate subscriptions. Subscribe at register time, unsubscribe at unregister time, refcount across servers.

5. Reaching past pmacs.mcp.* to a transport detail. If you find yourself wanting pmacs.mcp._send_raw_jsonrpc or similar, stop and ask whether your use case warrants a public API addition. The audit pattern is: surface real use cases, then promote when a second consumer materializes (the "promote-on-second-consumer" discipline). Don't reach around.

---

10. Versioning and stability

The pmacs.mcp.* Lua API is the stable surface. Six functions plus userdata methods. The shape is locked for v1.0; additions are backwards-compatible.

The fixture packages are fixtures: their public shape is stable enough for examples to keep working, but minor versions may add fields to response shapes (e.g., M9.7 added _meta.format honoring; M9.8 amendment promoted M.render). Treat their documented public functions as semver-respecting; treat their _underscore_prefix test seams as unstable.

The MCP protocol itself versions independently. pmacs's transport layer reads protocolVersion from the initialize result and rejects unsupported versions (M9 Pass-2 finding 3). Your package doesn't need to know about protocol versions — pmacs handles that.

---

See also

• docs/package-author-guide.md — general package mechanics (manifest, addresses, lockfiles, audit lint)
• TRANSITION-M9.md — M9 milestone summary, deferred items, audit-finding accumulation
• spec/pmacs-spec.tex §sec:m9-ai — the architectural claim AI is a transport binding, not a feature
• tests/fixtures/pmacs-mcp-{resources,tools,prompts,ai}/init.lua — the four worked examples