neuwirth
/
pmacs
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

feat(latex): bundle LaTeX grammar + reconcile highlights + tests 

Register a `latex` entry in BUILTIN_LANGUAGES (.tex/.latex/.sty/.cls) backed by
codebook-tree-sitter-latex 0.6.1 — the linkable republish of latex-lsp's grammar
over the tree-sitter-language shim (the squatted `tree-sitter-latex` 0.1.0 ships
no scanner.c and cannot link). The single `extensions` field wires the whole
detection chain ahead of the LSP filetype map, so no Lua edit is needed.

The crate exports no query constants, so highlighting is driven by the in-repo
overlay builtin/queries/latex/highlights.scm — the first such overlay,
include_str!'d as LATEX_HIGHLIGHTS (the audit-rules.scm precedent). This commit
reconciles the vendored nvim-treesitter query (previous commit) onto pmacs'
recognized capture set:
  * strip @spell/@nospell — meaningless to pmacs, and clobber-risk on a
    multi-capture node;
  * remove the 8 #eq?/#any-of?/#lua-match? patterns — pmacs evaluates only
    `#is? local`, so unevaluated they would over-match every generic command
    (as conditional/emphasis) and every line comment (as a magic directive);
  * remap fall-through captures: @module->keyword, @label->type,
    @markup.heading*->keyword.control, @markup.link*->constant,
    @markup.math->string;
  * fix node-name drift: this grammar cut uses curly_group_label(_list) for the
    label commands where newer latex-lsp unified them onto curly_group_text.

Tests (framing acceptance): table guard; load-and-parse including a verbatim
environment (exercises the external scanner the broken crate lacked);
highlights-resolve (doubles as the grammar/query node-name compatibility gate);
and extension resolution.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
Levi Neuwirth 2026-07-23 12:04:49 -04:00
parent 09a1901458
commit f11d625fb0
4 changed files with 256 additions and 138 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

11
Cargo.lock generated
View File

@ -423,6 +423,16 @@ dependencies = [
"thiserror 2.0.18",
]
[[package]]
name = "codebook-tree-sitter-latex"
version = "0.6.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "a3ab764ede0961a43ee3121cd8a71dd01d355f836181ebf2ede04f4c24636ca0"
dependencies = [
"cc",
"tree-sitter-language",
]
[[package]]
name = "codespan-reporting"
version = "0.13.1"
@ -2528,6 +2538,7 @@ checksum = "b4596b6d070b27117e987119b4dac604f3c58cfb0b191112e24771b2faeac1a6"
name = "pmacs"
version = "1.0.0"
dependencies = [
"codebook-tree-sitter-latex",
"crossbeam",
"crossterm",
"loro",

16
Cargo.toml
View File

@ -217,6 +217,22 @@ tree-sitter-yaml = "0.7"
# alternative names `tree-sitter-markdown*` are forks of varying
# maintenance status — pick the upstream one.
tree-sitter-md = "0.5"
# LaTeX / TeX (`.tex`/`.latex`/`.sty`/`.cls`) — the "basic LaTeX mode"
# foothold and the tier-1 detection substrate for inline math
# (`docs/inline-math-framing.md`; lane framing
# `docs/latex-grammar-math-substrate-framing.md`). The canonical crate name
# `tree-sitter-latex` is squatted by a broken 0.1.0 repackage that ships no
# `scanner.c` for its declared external tokens and therefore CANNOT LINK
# (undefined `tree_sitter_latex_external_scanner_*`). `codebook-tree-sitter-latex`
# is a linkable republish of the real, maintained `latex-lsp/tree-sitter-latex`
# (MIT, texlab's author) modernized to the `LanguageFn` shim: it ships
# `scanner.c`, exports `LANGUAGE` over `tree-sitter-language 0.1` (tree-sitter
# dev-only, shared ABI crate — no second core in the graph), and its
# `grammar.js`/`scanner.c` are byte-identical to upstream (provenance diff
# discharged in the framing). It exports NO query constants, so highlighting
# uses the in-repo overlay `builtin/queries/latex/highlights.scm` (the first
# such overlay; `include_str!`'d in `crate::syntax::BUILTIN_LANGUAGES`).
codebook-tree-sitter-latex = "0.6"
# T M4.4 process supervisor: signal sending without `unsafe`. Keep
# the feature surface tight to keep build time low. `poll` feeds the
# compile-mode group readers (cancellable poll-based reads, Q#CM3).

232
builtin/queries/latex/highlights.scm
View File

@ -1,23 +1,41 @@
; LaTeX highlights — vendored from nvim-treesitter (Apache-2.0) and reconciled
; onto pmacs' recognized capture set (src/highlight.rs; framing Q#LX2 / F4).
;
; Reconciliation vs the verbatim upstream (previous commit):
; * @spell / @nospell markers stripped — meaningless to pmacs, and a
; same-node @nospell risks clobbering the real capture.
; * The 8 predicate-gated patterns (#eq? / #any-of? / #lua-match?) removed:
; pmacs evaluates only `#is? local` predicates, so unevaluated they would
; over-match every generic command (as \if-conditional / emphasis) and
; every line_comment (as a magic directive). Re-instating them is deferred
; to predicate support. This also drops the only @markup.italic/@markup.strong
; uses, so those need no remap.
; * Fall-through captures remapped onto the recognized set:
; @module -> @keyword (\begin/\end/sectioning commands)
; @label -> @type (environment / theorem names)
; @markup.heading -> @keyword.control (section-title text)
; @markup.link* -> @constant (labels, refs, citations, urls)
; @markup.math -> @string (inline/displayed formulas)
; Grammar node names are UNCHANGED from upstream — Query::new compiling this
; against the bundled grammar is the node-name compatibility gate (acceptance 3).
; General syntax
(command_name) @function @nospell
(command_name) @function
(caption
command: _ @function)
; Turn spelling on for text
(text) @spell
; \text, \intertext, \shortintertext, ...
(text_mode
command: _ @function @nospell
command: _ @function
content: (curly_group
(_) @none @spell))
(_) @none))
; Variables, parameters
(placeholder) @variable
(key_value_pair
key: (_) @variable.parameter @nospell
key: (_) @variable.parameter
value: (_))
(curly_group_spec
@ -51,200 +69,167 @@
; General environments
(begin
command: _ @module
command: _ @keyword
name: (curly_group_text
(text) @label @nospell))
(text) @type))
(end
command: _ @module
command: _ @keyword
name: (curly_group_text
(text) @label @nospell))
(text) @type))
; Definitions and references
(new_command_definition
command: _ @function.macro @nospell)
command: _ @function.macro)
(old_command_definition
command: _ @function.macro @nospell)
command: _ @function.macro)
(let_command_definition
command: _ @function.macro @nospell)
command: _ @function.macro)
(environment_definition
command: _ @function.macro @nospell
command: _ @function.macro
name: (curly_group_text
(_) @label @nospell))
(_) @type))
(theorem_definition
command: _ @function.macro @nospell
command: _ @function.macro
name: (curly_group_text_list
(_) @label @nospell))
(_) @type))
(paired_delimiter_definition
command: _ @function.macro @nospell
command: _ @function.macro
declaration: (curly_group_command_name
(_) @function))
; NOTE: this grammar cut (codebook 0.6.1, ~Dec 2025) uses distinct
; `curly_group_label`/`curly_group_label_list` nodes for label commands, where
; newer latex-lsp (which nvim's query targets) unified them onto
; `curly_group_text`. Kept grammar-accurate here (acceptance 3 is the gate).
(label_definition
command: _ @function.macro
name: (curly_group_text
(_) @markup.link @nospell))
name: (curly_group_label
(_) @constant))
(label_reference_range
command: _ @function.macro
from: (curly_group_text
(_) @markup.link)
to: (curly_group_text
(_) @markup.link))
from: (curly_group_label
(_) @constant)
to: (curly_group_label
(_) @constant))
(label_reference
command: _ @function.macro
names: (curly_group_text_list
(_) @markup.link))
names: (curly_group_label_list
(_) @constant))
(label_number
command: _ @function.macro
name: (curly_group_text
(_) @markup.link)
number: (_) @markup.link)
name: (curly_group_label
(_) @constant)
number: (_) @constant)
(citation
command: _ @function.macro @nospell
keys: (curly_group_text_list) @markup.link @nospell)
command: _ @function.macro
keys: (curly_group_text_list) @constant)
((hyperlink
command: _ @function @nospell
(hyperlink
command: _ @function
uri: (curly_group_uri
(_) @markup.link.url @nospell)) @_hyperlink
(#set! @_hyperlink url @markup.link.url))
(_) @constant))
(glossary_entry_definition
command: _ @function.macro @nospell
command: _ @function.macro
name: (curly_group_text
(_) @markup.link @nospell))
(_) @constant))
(glossary_entry_reference
command: _ @function.macro
name: (curly_group_text
(_) @markup.link))
(_) @constant))
(acronym_definition
command: _ @function.macro @nospell
command: _ @function.macro
name: (curly_group_text
(_) @markup.link @nospell))
(_) @constant))
(acronym_reference
command: _ @function.macro
name: (curly_group_text
(_) @markup.link))
(_) @constant))
(color_definition
command: _ @function.macro
name: (curly_group_text
(_) @markup.link))
(_) @constant))
(color_reference
command: _ @function.macro
name: (curly_group_text
(_) @markup.link)?)
(_) @constant)?)
; Sectioning
(title_declaration
command: _ @module
command: _ @keyword
options: (brack_group
(_) @markup.heading.1)?
(_) @keyword.control)?
text: (curly_group
(_) @markup.heading.1))
(_) @keyword.control))
(author_declaration
command: _ @module
command: _ @keyword
authors: (curly_group_author_list
(author)+ @markup.heading.1))
(author)+ @keyword.control))
(chapter
command: _ @module
command: _ @keyword
toc: (brack_group
(_) @markup.heading.2)?
(_) @keyword.control)?
text: (curly_group
(_) @markup.heading.2))
(_) @keyword.control))
(part
command: _ @module
command: _ @keyword
toc: (brack_group
(_) @markup.heading.2)?
(_) @keyword.control)?
text: (curly_group
(_) @markup.heading.2))
(_) @keyword.control))
(section
command: _ @module
command: _ @keyword
toc: (brack_group
(_) @markup.heading.3)?
(_) @keyword.control)?
text: (curly_group
(_) @markup.heading.3))
(_) @keyword.control))
(subsection
command: _ @module
command: _ @keyword
toc: (brack_group
(_) @markup.heading.4)?
(_) @keyword.control)?
text: (curly_group
(_) @markup.heading.4))
(_) @keyword.control))
(subsubsection
command: _ @module
command: _ @keyword
toc: (brack_group
(_) @markup.heading.5)?
(_) @keyword.control)?
text: (curly_group
(_) @markup.heading.5))
(_) @keyword.control))
(paragraph
command: _ @module
command: _ @keyword
toc: (brack_group
(_) @markup.heading.6)?
(_) @keyword.control)?
text: (curly_group
(_) @markup.heading.6))
(_) @keyword.control))
(subparagraph
command: _ @module
command: _ @keyword
toc: (brack_group
(_) @markup.heading.6)?
(_) @keyword.control)?
text: (curly_group
(_) @markup.heading.6))
; Beamer frames
(generic_environment
(begin
name: (curly_group_text
(text) @label)
(#any-of? @label "frame"))
.
(curly_group
(_) @markup.heading))
((generic_command
command: (command_name) @_name
arg: (curly_group
(_) @markup.heading))
(#eq? @_name "\\frametitle"))
((generic_command
command: (command_name) @_name
arg: (curly_group
(_) @markup.italic))
(#any-of? @_name "\\emph" "\\textit" "\\mathit"))
((generic_command
command: (command_name) @_name
arg: (curly_group
(_) @markup.strong))
(#any-of? @_name "\\textbf" "\\mathbf"))
(generic_command
(command_name) @keyword.conditional
(#lua-match? @keyword.conditional "^\\if[a-zA-Z@]+$"))
(generic_command
(command_name) @keyword.conditional
(#any-of? @keyword.conditional "\\fi" "\\else"))
(_) @keyword.control))
; File inclusion commands
(class_include
@ -296,47 +281,18 @@
command: _ @keyword.import
paths: (curly_group_path_list) @string)
; Turn spelling off for whole nodes
[
(label_reference)
(label_reference_range)
(label_number)
(glossary_entry_reference)
(acronym_reference)
(color_definition)
(color_reference)
(class_include)
(package_include)
(latex_include)
(verbatim_include)
(import_include)
(bibstyle_include)
(bibtex_include)
(biblatex_include)
(graphics_include)
(svg_include)
(inkscape_include)
(tikz_library_import)
] @nospell
; Math
[
(displayed_equation)
(inline_formula)
] @markup.math @nospell
] @string
(math_environment
(_) @markup.math)
(_) @string)
; Comments
[
(line_comment)
(block_comment)
(comment_environment)
] @comment @spell
((line_comment) @keyword.directive @nospell
(#lua-match? @keyword.directive "^%% !TeX"))
((line_comment) @keyword.directive @nospell
(#lua-match? @keyword.directive "^%%&"))
] @comment

135
src/syntax.rs
View File

@ -1093,8 +1093,32 @@ pub const BUILTIN_LANGUAGES: &[LanguageEntry] = &[
locals_query: &[],
injections_query: &[],
},
// LaTeX / TeX. The grammar crate exports no query constants (unlike every
// entry above), so the highlights query is the in-repo overlay
// `builtin/queries/latex/highlights.scm`, `include_str!`'d as
// `LATEX_HIGHLIGHTS` below — the first such overlay in the tree (framing
// Q#LX2; the `audit-rules.scm` include is the precedent). Locals and
// injections are empty for v0; `(math_environment) @math` injection
// detection is deferred to the inline-math arc.
LanguageEntry {
name: "latex",
extensions: &["tex", "latex", "sty", "cls"],
loader: || codebook_tree_sitter_latex::LANGUAGE.into(),
highlights_query: &[LATEX_HIGHLIGHTS],
locals_query: &[],
injections_query: &[],
},
];
/// LaTeX highlights overlay (framing Q#LX2). The chosen grammar crate
/// (`codebook-tree-sitter-latex`) ships no query constants, so — unlike every
/// other [`BUILTIN_LANGUAGES`] entry, which references a crate-exported
/// `HIGHLIGHTS_QUERY` — LaTeX highlighting is driven by this vendored query,
/// reconciled onto pmacs' recognized capture set (`crate::highlight`). The
/// `include_str!` path mirrors the sole prior `.scm` precedent,
/// `crate::audit`'s `audit-rules.scm`.
const LATEX_HIGHLIGHTS: &str = include_str!("../builtin/queries/latex/highlights.scm");
/// Registry that the Lua surface ([`crate::lua_bindings::install_parse`])
/// reads to map language names to grammars and buffer ids to attached
/// [`ParseView`] handles. Held by `Rc<...>` --- main-thread state, no
@ -2237,6 +2261,117 @@ mod tests {
);
}
#[test]
fn builtin_languages_include_latex() {
// The LaTeX entry claims its four extensions and — uniquely among
// BUILTIN_LANGUAGES — drives highlighting from the in-repo overlay
// `builtin/queries/latex/highlights.scm` (`LATEX_HIGHLIGHTS`), because
// the grammar crate exports no query constant (framing Q#LX2).
let latex = BUILTIN_LANGUAGES
.iter()
.find(|l| l.name == "latex")
.expect("`latex` language entry must be present");
for ext in ["tex", "latex", "sty", "cls"] {
assert!(latex.extensions.contains(&ext), "`latex` claims `.{ext}`");
}
assert!(
latex.highlights_query.contains(&LATEX_HIGHLIGHTS),
"`latex` carries the in-repo highlights overlay"
);
assert!(
!LATEX_HIGHLIGHTS.trim().is_empty(),
"the vendored LaTeX highlights overlay is non-empty"
);
}
#[test]
fn latex_grammar_loads_and_parses() {
// ABI acceptance: `codebook-tree-sitter-latex` (LanguageFn over
// `tree-sitter-language 0.1`) must be accepted by our `tree-sitter`
// 0.26 core. The `verbatim` environment exercises the grammar's
// external scanner (`scanner.c`) — the exact surface the squatted,
// scanner-less `tree-sitter-latex` 0.1.0 crate lacked — so an
// error-free parse proves the linkable republish is wired, not a
// partial grammar.
let reg = SyntaxRegistry::new();
let language = reg
.language("latex")
.expect("`latex` language loads from BUILTIN_LANGUAGES");
let mut buf = fresh_buffer("paper.tex");
buf.apply_edit(EditOp::Insert {
pos: 0,
bytes: b"\\documentclass{article}\n\
\\begin{document}\n\
Hello $x^2$ and text.\n\
\\begin{verbatim}\n\
raw $ text\n\
\\end{verbatim}\n\
\\end{document}\n",
})
.unwrap();
let view = ParseView::new(&buf, language, "latex".to_owned());
let handle = view.handle();
let _vid = buf.attach_view(Box::new(view));
let bundle = parse_synchronously(&handle);
assert_eq!(
bundle.root_tree().root_node().kind(),
"source_file",
"LaTeX grammar roots at source_file"
);
assert!(
!bundle.root_tree().root_node().has_error(),
"LaTeX grammar parses a document with a verbatim environment \
(external scanner) without error"
);
}
#[test]
fn latex_highlights_resolve() {
// The vendored overlay must COMPILE against the bundled grammar —
// simultaneously the grammar/query node-name compatibility gate
// (framing Q#LX2): a query referencing a node the grammar version
// lacks fails here. Assert the reconciled captures are the ones pmacs
// recognizes, and that no upstream fall-through capture survived.
let reg = SyntaxRegistry::new();
let query = reg
.highlights_query("latex")
.expect("latex highlights compile against the grammar");
let names = query.capture_names();
for expected in ["function", "keyword", "comment"] {
assert!(
names.contains(&expected),
"reconciled query carries the recognized `@{expected}` capture; got {names:?}"
);
}
for stray in [
"module",
"label",
"markup.heading",
"markup.link",
"markup.math",
] {
assert!(
!names.contains(&stray),
"reconciliation removed the fall-through `@{stray}` capture; got {names:?}"
);
}
}
#[test]
fn language_for_path_resolves_latex_extensions() {
// `.tex`/`.latex`/`.sty`/`.cls` all resolve to the LaTeX grammar via
// the same extension path as every bundled language — the single
// `extensions` field wires detection ahead of the LSP filetype map.
let reg = SyntaxRegistry::new();
for path in ["paper.tex", "slides.latex", "mypkg.sty", "myclass.cls"] {
assert_eq!(
reg.language_name_for_path(path).as_deref(),
Some("latex"),
"{path} resolves to latex"
);
}
}
#[test]
fn builtin_languages_include_bash() {
// Regression guard: the bash entry claims the wider shell family