27 KiB

Raw Blame History

Package author's guide

This guide is for authors who want to publish a pmacs package. It covers the manifest format, the address schemes pmacs accepts, the audit-lint rule set every package is expected to follow, and the mechanics of distributing a package via Git.

The user-facing side (how to install a package) is at docs/packages.md. This document is the inverse: how to publish one.

A reference implementation lives at builtin/packages/repl/ --- pmacs's own bundled REPL package, the dogfood case for everything below.

1. Package layout

A package is a directory tree with at minimum a manifest and an entry module:

my-package/
├── pmacs.toml          # required: manifest
└── init.lua            # required: entry module (path = manifest's `entry` field)

Larger packages add submodule files and supporting directories:

my-package/
├── pmacs.toml
├── init.lua
├── core.lua            # exported submodule
├── ui.lua              # exported submodule
├── internal/
│   └── helpers.lua     # NOT exported; only reachable from within the package
└── README.md

The pmacs loader resolves require("my-package") to init.lua and require("my-package.core") to core.lua --- if and only if my-package.core appears in the manifest's exports list. Anything not exported is private to the package, even if other packages can technically reach it on disk.

2. Manifest format (`pmacs.toml`)

The manifest is a TOML file at the package root. v1.0 fields:

name = "my-package"
version = "1.2.3"
summary = "One-line description of what the package does."
pmacs_required = ">= 0.1.0, < 1.0.0"
entry = "init.lua"
exports = ["my-package", "my-package.core", "my-package.ui"]

# Optional --- omit if empty.
[[dependencies]]
address = "github:other/utility-package"
version = "^0.4.0"

# Optional --- omit if none.
[[conflicts]]
address = "github:competing/repl"
version = "*"

Field reference

Field	Type	Required	Notes
`name`	string	✓	Lowercase, hyphen-separated. Optional `namespace/name` form (one `/`). Must start with a-z; `[a-z0-9-]` only.
`version`	semver string	✓	Strict semver: `MAJOR.MINOR.PATCH`.
`summary`	string	✓	One-line description.
`pmacs_required`	version-req	✓	Range of pmacs versions this package supports.
`entry`	path string	✓	Relative to package root. Conventionally `init.lua`.
`exports`	list of strings	✓	Public Lua module names other packages may `require`. The package's basename should be in the list if you want `require("<name>")` to work; submodules outside the list cannot be required by other packages.
`dependencies`	list of tables	optional	Each entry has `address` (see §3) and `version` (a version-req). Defaults to empty.
`conflicts`	list of tables	optional	Same shape as `dependencies`. Use sparingly --- the resolver fails the install if any conflicting package is also being installed.

The manifest parser rejects unknown fields silently (forward compatibility): a v1.0 binary reading a v1.1 manifest with a new optional field still loads it.

`exports` and the per-package environment

Every package executes its entry chunk in a per-package _ENV table. Reads of "globals" (pmacs.buffer, string, table, etc.) go through _ENV's metatable's __index = _G, so the standard library and the pmacs API surface remain reachable. Writes to "globals" stay local to the package: assigning to MARKER = "x" inside init.lua does not pollute _G.

Practical implication: a package can declare module-level state without worrying about colliding with another package's module-level state, even if both pick the same name.

Runtime API availability

Package entry chunks run during package load, including audit and headless load paths. Keep top-level code limited to registration and state setup. Surfaces installed by the base Lua host are available there: pmacs.buffer, pmacs.command, pmacs.keymap, pmacs.hook, pmacs.statusline, pmacs.describe, pmacs.help, pmacs.attach, pmacs.now_ms, and the standard Lua libraries.

Editor-state surfaces are available once the editor bridge is installed: command bodies invoked by pmacs, main-thread hooks fired by the editor, edit intercepts, and pmacs.async callbacks resumed by the editor loop can use pmacs.editor, pmacs.window, pmacs.frontend, and pmacs.minibuffer. Top-level package load code that also needs to run in audit/headless contexts should still guard editor-only work or defer it to a command/hook.

All of these callbacks run on the main Lua state. Intercepts have one extra restriction: re-entering a mutation on the same buffer is rejected by the buffer re-entry guard. Use { bypass_intercept = true } for package-owned redraws, not recursive same-buffer edits from inside the intercept body.

3. Address schemes (v1.0)

A package address tells the resolver where to fetch the package from. v1.0 supports the following forms:

Scheme	Example	Resolves to
`github:`	`github:owner/repo`	`https://github.com/owner/repo.git`
`gitlab:`	`gitlab:owner/repo`	`https://gitlab.com/owner/repo.git`
`git:` over HTTPS	`git:https://example.org/repo.git`	`https://example.org/repo.git`
`git:` over SSH	`git:git@example.org:owner/repo.git`	`git@example.org:owner/repo.git`
`git:` over file	`git:file:///srv/git/repo.git`	local path (mostly for testing)

The shorthand schemes (github:, gitlab:) are sugar for the underlying https:// URL. Full URLs work too:

pmacs.packages.install { "git:https://forge.invalid/me/pkg.git", version = "^1" }

Forge aliases (extension path)

github: and gitlab: are baked in. Other forges (Codeberg, Forgejo instances, internal corporate forges) are not. The pmacs extension path for adding a new alias is documented at src/packages/address.rs: a single match arm in Address::parse plus a unit test. PRs welcomed; the criteria are (a) a stable public forge URL pattern, (b) consensus in the issue tracker, and (c) audit-lint coverage of the parser change.

The recommended interim path for one-off forges is to use the full git:https://... form rather than waiting on an alias.

4. Versioning and the lockfile

pmacs is a semver-disciplined ecosystem:

Packages declare version as a strict semver.
Users pin via version constraints (^1.2.3, >=1, <2, ~0.5).
The resolver picks the highest tag matching the constraint set.
The lockfile (pmacs.lock, written next to the user's config or project root) records the exact commit hash, the resolved version, and a content hash (SHA-256 over git archive --format=tar).

When you publish a new version:

Update version in pmacs.toml.
Tag the commit. The tag name must match the version field prefixed with v (i.e., v1.2.3 for version = "1.2.3").
Push the tag to the upstream.

Users who installed your package with version = "^1.0" will see the new tag the next time they run pmacs.packages.update.

5. Audit-lint rule set (T M7.9)

Every package should pass pmacs-audit cleanly. The lint runs declarative tree-sitter queries against your *.lua files and emits findings at three severity levels:

Error --- forbidden patterns. The CLI exits non-zero; CI gates on this.
Warning --- patterns that require capability declaration. Currently always fires for fs / process operations; will gate on a future manifest permissions field.
Info --- patterns that need human classification (currently cross-package dotted requires and private-looking fields read directly from require(...)).

v1.0 rules (16 patterns across 7 spec classes)

The full list lives at audit/audit-rules.scm (the published contract) and src/audit/rules.rs (the metadata table). Summary:

Severity	Class	Rules
Error	private surface	`no-private-surface-require`, `no-private-surface-identifier`
Error	FFI / native loader	`no-ffi-call`, `no-package-loadlib`, `no-package-cpath-mutation`
Error	debug-cancellation	`no-debug-sethook`, `no-debug-setmetatable`
Error	environment escape	`no-rawget-rawset-on-globals`, `no-setfenv-getfenv`
Warning	filesystem mutation	`no-fs-mutation-io-open-write`, `no-fs-mutation-os`
Warning	process spawning	`no-process-spawn-io`, `no-process-spawn-os`, `no-process-spawn-pmacs`
Info	reach-around	`reach-around-require`, `reach-around-require-field`

Common Error rules and their fixes

no-private-surface-* --- a package tried to require a module under pmacs._internal.* or pmacs.core.*, or referenced an identifier prefixed with _pmacs_internal_ or _core_. These surfaces are not API; use the documented pmacs.X.Y namespaces instead.
no-ffi-call --- LuaJIT's FFI escapes the Lua sandbox. If you genuinely need native code, propose the surface as a pmacs API addition rather than reaching past the boundary.
no-debug-sethook --- pmacs uses the debug hook for cooperative C-g cancellation (T M7.8); installing your own hook disables it editor-wide. Use pmacs.async-friendly patterns or schedule work explicitly.
no-rawget-rawset-on-globals --- the per-package _ENV table is the API. Routing through _G defeats the sandboxing.

Running the lint locally

cargo install --git https://git.levineuwirth.org/neuwirth/pmacs \
    --bin pmacs-audit pmacs
pmacs-audit --pretty .

Exit code 1 means at least one Error finding; 2 means an I/O or configuration failure; 0 means clean (Warnings/Info OK).

Sample CI workflows

Drop-in templates for three forges live at audit/ci/:

github-actions.yml
gitlab-ci.yml
forgejo-actions.yml

Each builds pmacs-audit from a pinned pmacs revision, runs it, and uploads the JSON report as a job artifact.

6. Distribution

pmacs has no central registry. Packages are published by tagging a commit in a Git repository. The address scheme picks the forge (or generic Git URL); the resolver's tag enumeration picks the version.

Publishing checklist

Author the package in a fresh repository.
Write pmacs.toml with version = "0.1.0" (or wherever you start).
Author init.lua and any submodules listed in exports.
Run pmacs-audit --pretty . and fix any Error-severity findings.
Commit, tag v0.1.0, push to the forge.

Document the install command in your README:

pmacs.packages.install { "github:you/your-package", version = "^0.1.0" }

For breaking changes: bump MAJOR, document the migration in your CHANGELOG, and consider declaring a [[conflicts]] entry in your new manifest against the old version = "<1.0.0" so users who mass-update don't end up with both shapes resolved at once.

7. Dev-loop APIs (T M8.1)

Iterating on a package without restarting pmacs uses three init-time APIs and one runtime API. Together they let you point the editor at a working tree on disk, edit source files, and reload to observe new behavior.

`pmacs.packages.install_local(path)`

Symlinks a working tree into the install root instead of fetching+extracting from a Git remote.

pmacs.packages.install_local("/srv/dev/your-package")

Init-time-only (like the other pmacs.packages.* install APIs).
Validates the source has a readable pmacs.toml and that the package's pmacs_required matches the running version.
The install dir at <install_root>/<basename> becomes a symlink to your source. Edits to your source files are immediately visible to subsequent loads, with no copy step.
Skips the lockfile. Local installs are explicitly ephemeral and not reproducible across machines. The lockfile records only fetched installs; users sharing a project rely on pmacs.packages.install for that.
If the install path already holds a real directory (a previous fetched install), install_local refuses with a clear error so you don't accidentally clobber an installed tree with manual edits.
Calling install_local twice for the same name swaps the symlink; before the swap, the prior install's on_unload hooks fire (see below). If a hook fails, the swap is aborted with disk unchanged.

`pmacs.packages.reload(name)`

Re-runs the package's chunk against whatever's currently on disk. Returns the new module table.

-- After editing /srv/dev/your-package/init.lua:
local m = pmacs.packages.reload("your-package")

What reload does, in order:

Runs every registered on_unload hook for the package. Hooks fire in registration order; a hook that fails leaves the remaining unrun hooks (and the failed one) in the registry for a retry.
Drops package.loaded[name] (and every package.loaded[name.<sub>] for declared submodule exports) so the next require actually re-runs the chunk.
Drops the cached per-package _ENV table. Globals removed in the new source disappear from the env, instead of lingering from the prior chunk's writes.
Calls require(name) to load the freshly-readable bytes.

reload works against any installed package, not only install_local-installed ones — if you've fetched a package and then edited its on-disk install root, reload picks up the changes too. (Whether that's a good practice is a separate question; install_local is the supported way to author against working trees.)

`pmacs.packages.on_unload(fn)`

Registers a per-package teardown hook. Called from inside your package's chunk:

-- In your package's init.lua:
local worker = pmacs.workers.spawn(...)

pmacs.packages.on_unload(function()
  worker:terminate()
end)

return M

The owning package is recovered from the calling chunk's _ENV; no manual basename argument required.
Calls from non-package code (top-level init.lua, the REPL) error with a pointer at pmacs.hook.add('editor.before-quit', ...) as the right venue for editor-shutdown cleanup that doesn't belong to a single package.
Hooks fire on reload(name) and during an install_local swap that's replacing a prior install at the same name.

Idempotence contract. Hooks must be safe to call more than once. If a hook fails, the next reload (or install_local replacement) re-attempts that exact hook — pmacs doesn't skip past a failed cleanup. A package whose on_unload is worker:terminate() followed by an assertion that the worker is gone will need to handle "worker already terminated" on the retry.

Packages that define commands must unregister them. Any package whose chunk calls pmacs.command.define at top level needs an on_unload hook that hands those slots back, otherwise reload(name) (and install_local replacement) will hit DuplicateName on the second chunk run. The inverse is pmacs.command.unregister(name):

-- In your package's init.lua:
local OWNED = {}
local function define_owned(spec)
  pmacs.command.define(spec)
  OWNED[#OWNED + 1] = spec.name
end

define_owned { name = "mypkg.go", description = "…", fn = function() end }

pmacs.packages.on_unload(function()
  for _, n in ipairs(OWNED) do pmacs.command.unregister(n) end
  OWNED = {}
end)

pmacs.command.unregister(name) returns true if a command was removed and false if name wasn't registered (so the loop above is safe even after a partially-successful prior reload). Unlike install_local, unregister is not init-phase-gated: it has to work whenever define works (parity), and packages need to call it from on_unload hooks that fire on post-init reload(name) calls.

Statusline providers: register for every window, unregister on unload

pmacs.statusline.register installs a live provider and returns an opaque handle. Registration accepts a strict table with only name, side, priority, face, and fn: name is a non-empty display label, side is "left" or "right", priority defaults to 0, face defaults to "ui.modeline" and otherwise must be a ui.modeline.* face, and fn is the callback.

Providers are evaluated once for each rendered window context, not once for the editor's active buffer. Always read the callback's ctx.buffer handle; a split's passive window can display a different buffer:

local segment = pmacs.statusline.register {
  name = "mypkg-buffer",
  side = "left",
  priority = 20,
  face = "ui.modeline.mypkg",
  fn = function(ctx)
    -- ctx.frontend and ctx.window are integer identities.
    -- ctx.buffer is this window's Buffer handle, even when passive.
    local marker = ctx.active and "*" or ""
    return marker .. ctx.buffer:name()
  end,
}

pmacs.packages.on_unload(function()
  pmacs.statusline.unregister(segment) -- idempotent; false if already gone
end)

The callback returns a string, nil, or ""; the latter two mean no segment. Output is one line (the first newline ends it), control characters become spaces, and an over-limit result is omitted as a provider failure. Failures are reported once per provider/window context until that context succeeds or the provider is disabled and re-enabled.

Ordering is deterministic: left providers use priority descending, then registration order; right providers use priority ascending, then registration order. pmacs.statusline.providers() returns fresh metadata tables. set_priority(handle, integer) and set_enabled(handle, boolean) return false for a stale handle and change live output immediately. unregister(handle) is idempotent and returns whether it removed a live provider. Registering in package top-level code without the matching on_unload cleanup leaks the old provider across reload(name).

`pmacs.fs.*` — worker-dispatched filesystem primitives

The four async fs operations packages need without reaching for io.* or os.*. Each returns a Handle (the M3 worker pattern); :await() from inside pmacs.async(...) yields the result.

pmacs.async(function()
  local entries = pmacs.fs.read_dir(path):await()
  -- entries[i] is a table:
  --   { name=, kind=, size=, mtime=, mtime_nsec=, mode=, symlink_target= }
  -- kind is "file" / "dir" / "symlink" / "other".
end)

Function	Shape	Notes
`read_dir(path [, opts])`	array of entry tables	Filesystem iteration order; package owns sort. lstat-based — symlinks reported as `symlink` with `symlink_target` set.
`stat(path [, opts])`	single entry table	Same shape as a `read_dir` entry. lstat-based.
`rename(from, to)`	nil on success	Atomic on the same filesystem; cross-fs returns EXDEV.
`chmod(path, mode)`	nil on success	Follows symlinks (per `chmod(2)`); changes the target's mode, not the link's. Asymmetric with `read_dir`/`stat` which use lstat.
`remove(path)`	nil on success	File or empty dir. Non-empty dirs fail; recurse at the package layer. Symlinks removed as symlinks (target survives).
`watch(path, callback [, opts])`	watcher handle	Polls for file/directory changes. Calls `callback({ kind = "changed" \| "created" \| "removed", path = path, recursive = bool })`. `opts.interval_ms` defaults to 250; `opts.recursive` defaults to false.

opts.supersede = "<key>" chains read ops (read_dir, stat) into the M3 supersede semantics: a later op under the same key cancels the earlier one's :await() with { tag = 'cancelled' }. Mutating ops (rename/chmod/remove) intentionally don't accept opts.supersede — a "cancelled" syscall may have already mutated disk.

watch is intentionally polling-backed. The watcher handle exposes :cancel() and :is_cancelled(). Use package-side coalescing if a burst of filesystem writes should produce one refresh.

Two consequences of the polling design are load-bearing for callers. First, the baseline snapshot is taken asynchronously after watch returns; a change that lands between the watch call and the first completed snapshot is folded into the baseline and never reported. Re-trigger the action if you need a guaranteed first event, rather than assuming watch is armed synchronously. Second, change detection compares a per-entry signature of size, mtime (including nanoseconds), mode, and symlink target — a same-size content rewrite within the filesystem's mtime granularity is not observed. Both are acceptable for the derived-view refresh use case watch targets; neither is suitable as a correctness-critical change feed.

pmacs.async.yield_to_next_tick() may be called inside pmacs.async(function() ... end) when a package needs to resume on the next editor async tick without dispatching a worker job.

pmacs-outline also publishes pmacs.outline.query(buffer, predicate) when the package is loaded. It returns parsed outline entries whose fields match the package parser entries, and is the public way for other packages to inspect outline structure without requiring pmacs-outline.parser directly.

UTF-8 constraint. v0.1's pmacs.fs requires UTF-8 paths and entry names. A directory containing a non-UTF-8 entry surfaces a failed status from :await() with the parent path and offending raw bytes named. Byte-preserving paths are post-v0.1 work.

`pmacs.buffer.*` — buffers, file loading, and cleanup

Packages can create scratch buffers from bytes, load files through the editor's file loader, and observe buffer removal for package-owned state.

Function	Shape	Notes
`create(name)`	buffer handle	Empty clean buffer.
`from_bytes(name, bytes)`	buffer handle	Byte-preserving buffer seeded from a Lua string.
`from_file(path)`	buffer handle	Loads the file using pmacs's normal file loader, creates a clean buffer named by `path`, switches the editor core to it when an editor core is present, and runs `buffer.after-load` if that hook is defined.
`remove(buf)`	nil on success	Removes a buffer and fires removal cleanup.
`on_removed(buf, callback)`	handle	Calls `callback(buf)` after `buf` is removed by `remove` or `kill`. The returned handle has `:remove()` for idempotent unsubscription.

Buffer-local keymaps are pruned automatically when a buffer is removed. Package-local tables that hold per-buffer handles should use on_removed to drop their own state:

local cleanup = pmacs.buffer.on_removed(buf, function(dead)
  handles[tostring(dead)] = nil
end)

-- Later, if the package tears down before the buffer dies:
cleanup:remove()

For generated buffers with intercepts, package-owned writes can skip the intercept chain explicitly:

buf:replace(0, buf:len(), rendered, { bypass_intercept = true })

bypass_intercept applies only to the intercept chain. It does not disable the same-buffer re-entry guard, undo bookkeeping, dirty tracking, view notifications, or CRDT broadcast queueing.

`pmacs.editor.*` — active-window editor state

Editor primitives operate on the active window for the active frontend. Cursor positions are byte offsets; line numbers are 0-based to match cursor_line().

Function	Shape	Notes
`cursor()`	byte offset	Active window cursor.
`cursor_line()`	line index	0-based line containing the cursor.
`cursor_col()`	byte column	0-based byte column within the current line.
`move_to_line(line)`	nil	Moves to the start of `line`; out-of-range values clamp to the last line.
`set_status(message)`	nil	Replaces the status message.

A complete dev-loop example

-- ~/.config/pmacs/init.lua

-- Author your package at /srv/dev/pmacs-mypkg.
pmacs.packages.install_local("/srv/dev/pmacs-mypkg")

-- After editing files in /srv/dev/pmacs-mypkg, evaluate this
-- inside the running pmacs (e.g. from the scratch buffer):
--   pmacs.packages.reload("pmacs-mypkg")
-- The new code runs without restarting pmacs.

7a. Edit interception (`pmacs.buffer.add_intercept`)

For packages that present a buffer as a projection of external state — dired, wdired, magit-style — every user edit needs to be validated, translated to a real-world side effect, or rejected. The edit-intercept chain is the primitive for this.

local handle = pmacs.buffer.add_intercept(buf, function(op)
  -- op.kind is "insert" | "delete" | "replace"
  -- op.bytes (insert/replace only) is the proposed bytes as a Lua string
  -- op.pos (insert) or op.start/op["end"] (delete/replace) are byte positions

  if op.kind == "insert" and op.bytes:find("\n", 1, true) then
    error("newlines not allowed in this view")  -- rejects the edit
  end
  return nil  -- nil = pass through unchanged
end)

-- Later:
pmacs.buffer.remove_intercept(handle)

The intercept body returns:

nil — pass the edit through unchanged. Most common case.
a table of the same kind with new positions — override where the edit lands (bytes are not mutable through the chain).
error(msg) — reject the edit; the user sees msg as the edit's error.

bytes is surfaced as a Lua string (byte-clean: arbitrary 8-bit content round-trips, not just UTF-8). The dired-class wdired layer uses this to validate permission-column edits against the rwx alphabet at intercept time, before any chmod syscall.

Multiple intercepts may be attached to the same buffer; they run in attach order, threading the (possibly position-modified) op through the chain.

Package-owned redraws of derived buffers should prefer { bypass_intercept = true } on insert, delete, or replace instead of maintaining a separate painting boolean around every write.

8. The bundled REPL as a worked example

The REPL in builtin/packages/repl/ is pmacs's own first-party package. It demonstrates:

A real (non-trivial) pmacs.toml with name = "repl", entry = "init.lua", exports = ["repl"].
A package that legitimately spawns processes (the pmacs.process.spawn warning is classified as expected; see tests/m7_11_acceptance.rs).
The same load path third-party packages take: at editor start, the bootstrap registers it in the install roster, and require("repl") resolves through the M7.7 searcher with the manifest's exports whitelist enforced.

If you're stuck modeling something in your own package, comparing against the REPL's source is usually the fastest way to understand the expected shape.

27 KiB Raw Blame History