feat(highlight): filename detection + Dockerfile/Make/CMake grammars

Files identified by their whole basename — Dockerfile, Makefile,
CMakeLists.txt, rc dotfiles — had no detection path (extension-, then
shebang-keyed). Add a filename layer and the three grammars behind it.

- **Grammars** (BUILTIN_LANGUAGES): dockerfile via `tree-sitter-containerfile`
  (the ABI-current grammar; the old `tree-sitter-dockerfile` pins
  `tree-sitter ^0.20` and would fork the graph — containerfile rides
  `tree-sitter-language 0.1`, tree-sitter dev-only, like the others),
  make via `tree-sitter-make`, cmake via `tree-sitter-cmake`. All ship
  self-contained highlights (single fragment). Extensions:
  `.dockerfile`/`.containerfile`, `.mk`/`.make`, `.cmake`.
- **Filename layer**: `pmacs.parse.language_from_filename(name)` backed by
  an extensible `pmacs.parse.filenames` map, wired into the precedence
  chain in both syntax.lua (grammar) and lsp.lua (LSP): grammar-ext →
  filetype map → filename → shebang. A recognized extension still wins;
  the basename map only fires when the extension misses. Seeds the three
  filenames plus shell rc dotfiles (`.bashrc`/`.zshrc`/`PKGBUILD`/… →
  bash) — highlighting them against the grammar shipped in #115.
- **LSP**: `config.dockerfile` (docker-langserver --stdio) and
  `config.cmake` (cmake-language-server). Make has no server, so no
  `config.make` — grammar highlight only. Extension filetype fallbacks
  added for id stability.

Bite-verified acceptance:
- filename_grammars_load_and_parse — each grammar's ABI is accepted by
  the tree-sitter 0.26 core and parses a representative snippet without
  error (dockerfile/cmake root at source_file, make at makefile).
- builtin_languages_include_dockerfile_make_cmake /
  language_for_path_resolves_dockerfile_make_cmake_extensions — entry
  presence and extension detection.
- m4_filename_map_resolves_special_files — the basename map (incl. path
  form and dotfiles→bash), config.dockerfile/cmake commands, and no
  config.make. Bite-verified against pre-feature syntax.lua.
- m4_filename_extensionless_dockerfile_highlights — an extensionless
  `Dockerfile` resolves to dockerfile for LSP and gets a dockerfile parse
  tree; reachable only via the filename map. Bite-verified.

Gates: fmt; clippy -D warnings; test --lib; --features crdt;
m4_acceptance --skip basedpyright; GPU; full workspace sweep;
git diff --check.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YJ9FQ832QwftJXCD9LeFan
This commit is contained in:
Levi Neuwirth 2026-07-14 16:40:50 +01:00
parent 13dbadd0b6
commit 7646dda583
6 changed files with 348 additions and 18 deletions

33
Cargo.lock generated
View File

@ -2552,9 +2552,12 @@ dependencies = [
"tree-sitter",
"tree-sitter-bash",
"tree-sitter-c",
"tree-sitter-cmake",
"tree-sitter-containerfile",
"tree-sitter-cpp",
"tree-sitter-cuda",
"tree-sitter-lua",
"tree-sitter-make",
"tree-sitter-md",
"tree-sitter-rust",
"unicode-width",
@ -3737,6 +3740,26 @@ dependencies = [
"tree-sitter-language",
]
[[package]]
name = "tree-sitter-cmake"
version = "0.7.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "164e0c4f4236ec5ceff14824a5528615cf462e100467e49826442ff57d327061"
dependencies = [
"cc",
"tree-sitter-language",
]
[[package]]
name = "tree-sitter-containerfile"
version = "0.9.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f4811d55a5a2c32bb024b441c32b7417c3ec1af1a080a25fa20d321627b65a2b"
dependencies = [
"cc",
"tree-sitter-language",
]
[[package]]
name = "tree-sitter-cpp"
version = "0.23.4"
@ -3773,6 +3796,16 @@ dependencies = [
"tree-sitter-language",
]
[[package]]
name = "tree-sitter-make"
version = "1.1.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "c5998dc7cbcbdab19fae8aefef982bf2d6544513d8d2e69cc44aec4c63810104"
dependencies = [
"cc",
"tree-sitter-language",
]
[[package]]
name = "tree-sitter-md"
version = "0.5.3"

View File

@ -167,6 +167,20 @@ tree-sitter-cuda = "0.21"
# bash-language-server (shellcheck refuses zsh, so `.zsh` diagnostics
# may be sparse; highlighting is unaffected).
tree-sitter-bash = "0.25"
# Filename-identified languages (no useful extension): Dockerfile, Make,
# CMake. Highlighting for files detected by BASENAME (`Dockerfile`,
# `Makefile`, `CMakeLists.txt`) via the new filename map, plus their
# extensions (`.dockerfile`/`.mk`/`.cmake`). All three ride
# `tree-sitter-language 0.1` with `tree-sitter` dev-only (no second core
# in the graph) and export `LANGUAGE`/`HIGHLIGHTS_QUERY`.
# * Dockerfile: `tree-sitter-containerfile` — the maintained,
# ABI-current grammar (the older `tree-sitter-dockerfile` is pinned to
# `tree-sitter ^0.20` and would fork the graph). Covers Containerfile.
# * Make / CMake serve `dockerfile`/`cmake` LSP servers too (see lsp.lua);
# Make has no language server.
tree-sitter-containerfile = "0.9"
tree-sitter-make = "1.1"
tree-sitter-cmake = "0.7"
# T M9.7: markdown grammar so prompt result buffers with
# `_meta.format = "markdown"` get structured highlighting through the
# same M4 path as rust/lua — no special-case painter in Lua.

View File

@ -156,6 +156,29 @@ pmacs.lsp.config.bash = pmacs.lsp.config.bash or {
args = { "start" },
}
-- Dockerfile via docker-langserver (dockerfile-language-server-nodejs).
-- `--stdio` is the LSP transport. No project config. The `dockerfile`
-- language id is set from the bundled grammar / filename map, so a file
-- named `Dockerfile` (no extension) attaches. Users override from
-- init.lua before a Dockerfile opens. Make has no language server, so
-- there is deliberately no `config.make`.
pmacs.lsp.config.dockerfile = pmacs.lsp.config.dockerfile or {
command = "docker-langserver",
args = { "--stdio" },
}
-- CMake via cmake-language-server (the Python server). It speaks LSP over
-- stdio with no transport flag and pulls configuration via
-- `workspace/configuration` under the `cmake` section (empty ⇒ defaults,
-- present not null — same rationale as gopls). It reads
-- `build/compile_commands.json` for the project model. Users populate
-- `settings.cmake` (buildDirectory, …) from init.lua.
pmacs.lsp.config.cmake = pmacs.lsp.config.cmake or {
command = "cmake-language-server",
args = {},
settings = { cmake = {} },
}
-- TOML via taplo. `taplo lsp stdio` serves LSP over stdio. taplo
-- pulls configuration via `workspace/configuration` under the
-- `taplo` section (empty ⇒ defaults, present not null); a project
@ -224,6 +247,17 @@ pmacs.lsp.filetypes.lua = pmacs.lsp.filetypes.lua or "lua"
for _, ext in ipairs({ "sh", "bash", "zsh", "ksh", "ash", "bats" }) do
pmacs.lsp.filetypes[ext] = pmacs.lsp.filetypes[ext] or "bash"
end
-- Dockerfile / Make / CMake. Bundled grammars resolve these via
-- `language_for_path` (and filename map for extensionless files); these
-- extension entries are the LSP-only fallback that keeps the id stable if
-- a grammar is dropped. `.mk`/`.make` map to `make`, which has no server
-- (grammar highlight only); `.dockerfile`/`.cmake` attach their servers.
pmacs.lsp.filetypes.dockerfile = pmacs.lsp.filetypes.dockerfile or "dockerfile"
pmacs.lsp.filetypes.containerfile = pmacs.lsp.filetypes.containerfile or "dockerfile"
pmacs.lsp.filetypes.cmake = pmacs.lsp.filetypes.cmake or "cmake"
for _, ext in ipairs({ "mk", "make" }) do
pmacs.lsp.filetypes[ext] = pmacs.lsp.filetypes[ext] or "make"
end
-- TOML (taplo).
pmacs.lsp.filetypes.toml = pmacs.lsp.filetypes.toml or "toml"
-- Zig (zls). `.zon` is Zig Object Notation, handled by the same server.
@ -373,15 +407,18 @@ local function buffer_language(buf)
local ok, path = pcall(function() return buf and buf:path() end)
if not ok or not path then return nil end
-- Grammar-backed detection first (keeps rust/.rs etc. exactly as
-- before); fall back to the LSP-only filetype map so languages
-- with a server but no tree-sitter grammar (Python) still attach;
-- finally, for an extensionless file, sniff a `#!interp` shebang so
-- e.g. an extensionless `#!/bin/sh` script still attaches its server.
-- before); fall back to the LSP-only filetype map so languages with a
-- server but no tree-sitter grammar (Python) still attach; then the
-- basename map for filename-identified files (`Dockerfile`,
-- `CMakeLists.txt`); finally, for an extensionless file, sniff a
-- `#!interp` shebang so e.g. `#!/bin/sh` still attaches its server.
local lang = pmacs.parse.language_for_path(path)
if lang then return lang end
local ext = path:match("%.([%w_]+)$")
local by_ext = ext and pmacs.lsp.filetypes[ext]
if by_ext then return by_ext end
local by_name = pmacs.parse.language_from_filename(path)
if by_name then return by_name end
return pmacs.parse.language_from_shebang(buf)
end
-- Public: the per-buffer language chain. Auto-pairing resolves

View File

@ -147,20 +147,55 @@ function pmacs.parse.language_from_shebang(buf)
return pmacs.parse.shebangs[base]
end
-- Filename → language detection --------------------------------------------
--
-- Some files are identified by their whole BASENAME, not an extension:
-- `Dockerfile`, `Makefile`, `CMakeLists.txt`, and rc dotfiles like
-- `.bashrc`/`PKGBUILD` (highlighted by the already-bundled bash grammar).
-- Consulted after extension detection (a recognized extension still wins)
-- and before the shebang — a `Makefile` never carries a shebang, and the
-- basename is the more reliable signal. User-extensible from init.lua,
-- e.g. `pmacs.parse.filenames["Vagrantfile"] = "ruby"`.
pmacs.parse.filenames = pmacs.parse.filenames or {
["Dockerfile"] = "dockerfile",
["Containerfile"] = "dockerfile",
["Makefile"] = "make",
["makefile"] = "make",
["GNUmakefile"] = "make",
["BSDmakefile"] = "make",
["CMakeLists.txt"] = "cmake",
-- Shell rc / config files → the bundled bash grammar.
[".bashrc"] = "bash",
[".bash_profile"] = "bash",
[".bash_logout"] = "bash",
[".profile"] = "bash",
[".zshrc"] = "bash",
[".zprofile"] = "bash",
[".zshenv"] = "bash",
["PKGBUILD"] = "bash",
}
-- Language for a path's basename, or nil. `name` may be a full path.
function pmacs.parse.language_from_filename(name)
if not name then return nil end
local base = name:match("([^/]+)$") or name
return pmacs.parse.filenames[base]
end
-- Set of buffer ids that already have a highlight overlay
-- attached, keyed by raw id (number). A buffer that opens, gets
-- highlights, gets killed, and is reopened needs a fresh overlay
-- attach; the kill path clears the entry below if/when it lands.
local highlighted_buffers = {}
-- Filetype-aware language resolution for the active buffer, in
-- precedence order: grammar extension → LSP filetype map → shebang. The
-- shebang is consulted ONLY when the extension is unrecognized (a known
-- non-grammar extension like `.py` must not fall through to a stray
-- `#!/bin/sh` and be misparsed as bash). Keyed on `buf:name()` for the
-- extension parts (matching the historical behavior — path-less buffers
-- that resolve a grammar by name keep working); the shebang reads buffer
-- content directly.
-- Filetype-aware language resolution for the active buffer, in precedence
-- order: grammar extension → LSP filetype map → filename → shebang. A
-- recognized extension is authoritative (a `.py` must not fall through to
-- a stray `#!/bin/sh` and be misparsed as bash); the basename map handles
-- extensionless `Dockerfile`/`Makefile`/rc-dotfiles, and only then does
-- the shebang (buffer content) get a look. Keyed on `buf:name()` for the
-- path parts (matching the historical behavior — path-less buffers that
-- resolve a grammar by name keep working).
local function resolve_active_language(buf)
local name = buf:name()
if name then
@ -168,9 +203,9 @@ local function resolve_active_language(buf)
if grammar then return grammar end
local ext = name:match("%.([%w_]+)$")
local by_ext = ext and pmacs.lsp and pmacs.lsp.filetypes and pmacs.lsp.filetypes[ext]
-- A recognized (even non-grammar) extension is authoritative; do not
-- consult the shebang for it.
if by_ext then return by_ext end
local by_name = pmacs.parse.language_from_filename(name)
if by_name then return by_name end
end
return pmacs.parse.language_from_shebang(buf)
end

View File

@ -451,16 +451,42 @@ pub const BUILTIN_LANGUAGES: &[LanguageEntry] = &[
// `.ash` are close-enough dialects and `.bats` is bash. None collide
// with an entry above. Because the language name is `bash` — matching
// the `pmacs.lsp.config.bash` key — opening any of these also
// auto-attaches bash-language-server. Extensionless shebang scripts
// (`#!/bin/sh`) and rc dotfiles (`.bashrc`) are NOT covered: detection
// is extension-keyed, and shebang/filename sniffing is a separate
// (deferred) feature.
// auto-attaches bash-language-server. Extensionless shell scripts are
// resolved by shebang, and rc dotfiles (`.bashrc`, `PKGBUILD`) by the
// filename map — both in `builtin/runtime/syntax.lua`.
LanguageEntry {
name: "bash",
extensions: &["sh", "bash", "zsh", "ksh", "ash", "bats"],
loader: || tree_sitter_bash::LANGUAGE.into(),
highlights_query: &[tree_sitter_bash::HIGHLIGHT_QUERY],
},
// Filename-identified languages. These files usually have no useful
// extension (`Dockerfile`, `Makefile`, `CMakeLists.txt`), so the bulk
// of detection is the filename map in `syntax.lua`; the extensions
// here catch the `.dockerfile`/`.mk`/`.cmake` variants. All three ship
// self-contained highlights (no `; inherits:`), so single fragments.
//
// Dockerfile uses the `tree-sitter-containerfile` crate (the
// ABI-current grammar; also covers Containerfile); its root node is
// `source_file`. Make roots at `makefile`, CMake at `source_file`.
LanguageEntry {
name: "dockerfile",
extensions: &["dockerfile", "containerfile"],
loader: || tree_sitter_containerfile::LANGUAGE.into(),
highlights_query: &[tree_sitter_containerfile::HIGHLIGHTS_QUERY],
},
LanguageEntry {
name: "make",
extensions: &["mk", "make"],
loader: || tree_sitter_make::LANGUAGE.into(),
highlights_query: &[tree_sitter_make::HIGHLIGHTS_QUERY],
},
LanguageEntry {
name: "cmake",
extensions: &["cmake"],
loader: || tree_sitter_cmake::LANGUAGE.into(),
highlights_query: &[tree_sitter_cmake::HIGHLIGHTS_QUERY],
},
];
/// Registry that the Lua surface ([`crate::lua_bindings::install_parse`])
@ -1060,6 +1086,91 @@ mod tests {
}
}
#[test]
fn builtin_languages_include_dockerfile_make_cmake() {
for (name, exts) in [
("dockerfile", &["dockerfile", "containerfile"][..]),
("make", &["mk", "make"][..]),
("cmake", &["cmake"][..]),
] {
let entry = BUILTIN_LANGUAGES
.iter()
.find(|l| l.name == name)
.unwrap_or_else(|| panic!("`{name}` language entry must be present"));
for ext in exts {
assert!(entry.extensions.contains(ext), "`{name}` claims `.{ext}`");
}
assert!(
entry.highlights_query.iter().any(|q| !q.is_empty()),
"`{name}` ships a highlights query"
);
}
}
#[test]
fn filename_grammars_load_and_parse() {
// ABI acceptance: each 0.x/1.x grammar must be accepted by our
// tree-sitter 0.26 core (set_language succeeds at runtime) and
// parse a representative snippet without error, at its own root.
let reg = SyntaxRegistry::new();
let cases: &[(&str, &str, &[u8])] = &[
(
"dockerfile",
"source_file",
b"FROM alpine:3\nRUN apk add curl\nCMD [\"sh\"]\n",
),
(
"make",
"makefile",
b"all: build\n\tcc -o app main.c\n.PHONY: all\n",
),
(
"cmake",
"source_file",
b"cmake_minimum_required(VERSION 3.10)\nproject(demo)\n",
),
];
for (lang, root_kind, src) in cases {
let language = reg
.language(lang)
.unwrap_or_else(|| panic!("`{lang}` loads from BUILTIN_LANGUAGES"));
let mut buf = fresh_buffer(&format!("probe.{lang}"));
buf.apply_edit(EditOp::Insert { pos: 0, bytes: src })
.unwrap();
let view = ParseView::new(&buf, language, (*lang).to_owned());
let handle = view.handle();
let _vid = buf.attach_view(Box::new(view));
let bundle = parse_synchronously(&handle);
assert_eq!(
bundle.tree.root_node().kind(),
*root_kind,
"`{lang}` roots at `{root_kind}`"
);
assert!(
!bundle.tree.root_node().has_error(),
"`{lang}` parses its snippet without error"
);
}
}
#[test]
fn language_for_path_resolves_dockerfile_make_cmake_extensions() {
let reg = SyntaxRegistry::new();
for (path, lang) in [
("app.dockerfile", "dockerfile"),
("svc.containerfile", "dockerfile"),
("rules.mk", "make"),
("common.make", "make"),
("toolchain.cmake", "cmake"),
] {
assert_eq!(
reg.language_name_for_path(path).as_deref(),
Some(lang),
"{path} resolves to {lang}"
);
}
}
#[test]
fn byte_to_point_handles_first_line() {
let src = b"hello world";

View File

@ -6127,6 +6127,106 @@ fn m4_shebang_edit_keeps_pinned_grammar() {
assert_eq!(errs, 0, "reparse with the pinned grammar reports no error");
}
/// Filename detection: `pmacs.parse.language_from_filename` maps a
/// basename (Dockerfile / Makefile / CMakeLists.txt / rc dotfiles) to a
/// language, resolving a full path too, and returns nil for a plain file.
/// The default bundle also wires the dockerfile and cmake LSP configs;
/// Make has no server, so `config.make` is absent.
#[test]
fn m4_filename_map_resolves_special_files() {
use pmacs::editor::EditorState;
let s = EditorState::new();
let probe: mlua::Table = s
.lua_host
.lua()
.load(
r"
local f = pmacs.parse.language_from_filename
local out = {}
out.dockerfile = f('Dockerfile')
out.containerfile = f('Containerfile')
out.make = f('Makefile')
out.gnumake = f('GNUmakefile')
out.cmake = f('CMakeLists.txt')
out.bashrc = f('.bashrc')
out.pkgbuild = f('PKGBUILD')
out.pathform = f('/home/x/proj/Dockerfile')
out.plain_is_nil = f('notes.txt') == nil
out.cfg_docker = pmacs.lsp.config.dockerfile and pmacs.lsp.config.dockerfile.command
out.cfg_cmake = pmacs.lsp.config.cmake and pmacs.lsp.config.cmake.command
out.has_make_cfg = pmacs.lsp.config.make ~= nil
return out
",
)
.eval()
.expect("probe filename map");
assert_eq!(probe.get::<String>("dockerfile").unwrap(), "dockerfile");
assert_eq!(probe.get::<String>("containerfile").unwrap(), "dockerfile");
assert_eq!(probe.get::<String>("make").unwrap(), "make");
assert_eq!(probe.get::<String>("gnumake").unwrap(), "make");
assert_eq!(probe.get::<String>("cmake").unwrap(), "cmake");
assert_eq!(probe.get::<String>("bashrc").unwrap(), "bash");
assert_eq!(probe.get::<String>("pkgbuild").unwrap(), "bash");
assert_eq!(
probe.get::<String>("pathform").unwrap(),
"dockerfile",
"basename is extracted from a full path"
);
assert!(probe.get::<bool>("plain_is_nil").unwrap());
assert_eq!(
probe.get::<String>("cfg_docker").unwrap(),
"docker-langserver"
);
assert_eq!(
probe.get::<String>("cfg_cmake").unwrap(),
"cmake-language-server"
);
assert!(
!probe.get::<bool>("has_make_cfg").unwrap(),
"Make has no language server, so no config.make"
);
}
/// End-to-end: opening an extensionless `Dockerfile` attaches the
/// dockerfile grammar (a settled parse tree) and resolves `dockerfile`
/// for LSP — reachable only via the filename map, since the file has no
/// extension and no shebang. `pmacs.lsp.config` is emptied first so
/// docker-langserver is not spawned; grammar detection is independent.
#[test]
fn m4_filename_extensionless_dockerfile_highlights() {
use pmacs::editor::EditorState;
let mut s = EditorState::new();
let dir = tempfile::tempdir().expect("tempdir");
let f = dir.path().join("Dockerfile"); // no extension
std::fs::write(&f, b"FROM alpine:3\nRUN apk add curl\n").expect("write");
let f_disp = f.display();
s.lua_host
.lua()
.load(format!(
"pmacs.lsp.config = {{}}
pmacs.buffer.find_or_open('{f_disp}')"
))
.exec()
.expect("open Dockerfile");
let lsp_lang: Option<String> = s
.lua_host
.lua()
.load("return pmacs.lsp.active_buffer_language()")
.eval()
.expect("lsp language");
assert_eq!(
lsp_lang.as_deref(),
Some("dockerfile"),
"extensionless Dockerfile resolves to dockerfile for LSP"
);
pump_async(&mut s, |st| current_tree_language(st).is_some());
assert_eq!(
current_tree_language(&s).as_deref(),
Some("dockerfile"),
"extensionless Dockerfile gets a dockerfile parse tree"
);
}
/// Typing-perf: the default bundle coalesces full-document
/// `didChange` notifications instead of sending one per keystroke
/// (each send copies the whole buffer several times and writes