ci: give the M10 perf suites a job (they were dark for two reasons)
`m10_2_perf` and `m10_11_perf` were dark for TWO independent reasons,
and only one of them is this lane's headline subject:
1. They are `crdt`-gated, and nothing in the workflow enabled the
feature, so they were never compiled.
2. Even setting `crdt` aside, no job named them. Grepping ci.yml for
`--test` yielded exactly four suites: acceptance, m4_acceptance,
m5_perf_acceptance, m6_perf_acceptance. Their `#[ignore]` is
deliberate; their absence from every workflow was not.
Fixing both here is a deliberate scope call, taken with the second cause
named rather than absorbed: leaving it would ship a lane headlined "the
dark tests now run" with 7 still dark, and the remedy is one job block,
not a second investigation.
The two suites are NOT the same kind of thing, and the job says so
because a future reader will otherwise assume "perf gates" means
"budgets":
* m10_11_perf asserts one budget — cross-frontend propagation p99
under 50ms. Measured 1.47ms locally, a ~34x margin, so a red here
is a regression rather than runner noise.
* m10_2_perf asserts NOTHING. Six benches that print throughput
numbers, the baselines M10.2's 391x unicode finding and v0.2+
optimization work compare against.
Including a suite that cannot fail a budget needs its own
justification, so the workflow states it: `run_workload` drives 30
seconds of randomized mixed edits against both the v0.1 and CRDT buffer
paths, and nothing else in the corpus exercises a sustained randomized
CRDT workload. Its value here is soak and panic detection, not gating.
The comment explicitly warns against "fixing" it later by adding
throughput assertions, which is how perf jobs become flaky on shared
runners.
Per framing Q#CC3 and §1.3a, `tests/m10_10_perf.rs` is deliberately NOT
touched: it is a CI-default regression tripwire with generous bounds,
not a bench, and adding `#[ignore]` to give it a job would have shipped
a coverage reduction inside a coverage lane. It is recovered by the
plain crdt leg in the next commit instead.
Observed execution, local release build 2026-08-01: m10_2_perf 79s
(6 passed), m10_11_perf 5s (1 passed). The 25-minute ceiling matches
m5-perf-gates and is set by the cold-cache release build, not the ~85s
of tests — the reasoning the file already applies to its other perf
jobs.
Verified: ci.yml parses, 9 jobs, every one carrying timeout-minutes.
Both suites pass locally under the exact commands the job runs.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
parent
7a9cf5b812
commit
06abbacc70
|
|
@ -319,3 +319,57 @@ jobs:
|
|||
PMACS_M6_CANCEL_TRIALS: "30"
|
||||
PMACS_M6_CANCEL_MAX_DELAY_MS: "500"
|
||||
run: cargo test --release --test m6_perf_acceptance -- --ignored --nocapture --test-threads=1
|
||||
|
||||
# m10-perf-gates: the M10 suites, which were dark for TWO independent
|
||||
# reasons. Both are fixed here.
|
||||
#
|
||||
# 1. They are `crdt`-gated, and nothing in this workflow enabled the
|
||||
# feature, so they were never compiled. That is this lane's subject.
|
||||
# 2. Even setting `crdt` aside, NO job named them. Grepping this file
|
||||
# for `--test` before this job existed yielded exactly four suites:
|
||||
# acceptance, m4_acceptance, m5_perf_acceptance, m6_perf_acceptance.
|
||||
# Their `#[ignore]` is deliberate; their absence from CI was not.
|
||||
#
|
||||
# luajit-only, for the same reason m5-perf-gates and m6-perf-gates are:
|
||||
# the measured paths (CRDT buffer mutation, socket round-trips) do not
|
||||
# enter the Lua VM, so matrixing over flavors doubles cost for no
|
||||
# signal.
|
||||
#
|
||||
# WHAT EACH SUITE ACTUALLY GATES — these differ, and the difference
|
||||
# matters for how a red run is read:
|
||||
#
|
||||
# * m10_11_perf asserts ONE budget: cross-frontend propagation p99
|
||||
# under 50ms. Observed 1.47ms locally (2026-08-01), a ~34x margin,
|
||||
# so a red here is a real regression rather than runner noise.
|
||||
#
|
||||
# * m10_2_perf asserts NOTHING. It is six measurement benches that
|
||||
# print throughput numbers — the baselines M10.2's 391x unicode
|
||||
# finding and v0.2+ optimization work compare against. It cannot
|
||||
# fail a budget because it has none.
|
||||
#
|
||||
# It is here anyway, and NOT as a perf gate: `run_workload` drives
|
||||
# 30 seconds of randomized mixed edits against both the v0.1 and
|
||||
# CRDT buffer paths, and nothing else in the corpus exercises a
|
||||
# sustained randomized CRDT workload. Its CI value is soak and
|
||||
# panic detection. Do not "fix" a future silent run by adding
|
||||
# budget assertions — the numbers are deliberately reported, not
|
||||
# enforced, and asserting throughput on shared runners is how perf
|
||||
# jobs become flaky.
|
||||
#
|
||||
# OBSERVED EXECUTION (local, 2026-08-01, release): m10_2_perf 79s
|
||||
# (6 tests), m10_11_perf 5s (1 test) — about 85s combined. The ceiling
|
||||
# is 25 rather than something near that, because a perf job's cost is
|
||||
# dominated by its COLD-CACHE RELEASE BUILD, not its tests; this is
|
||||
# the same reasoning that gives m5-perf-gates 25.
|
||||
m10-perf-gates:
|
||||
name: M10 Perf Gates (crdt)
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 25
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
- uses: dtolnay/rust-toolchain@stable
|
||||
- uses: Swatinem/rust-cache@v2
|
||||
- name: CRDT buffer throughput baselines (soak; asserts nothing)
|
||||
run: cargo test --release --features crdt --test m10_2_perf -- --ignored --nocapture
|
||||
- name: cross-frontend propagation p99 over a real socket
|
||||
run: cargo test --release --features crdt --test m10_11_perf -- --ignored --nocapture
|
||||
|
|
|
|||
Loading…
Reference in New Issue