661 lines
38 KiB
Markdown
661 lines
38 KiB
Markdown
# Changelog
|
|
|
|
All notable changes to NeuroPose are recorded in this file.
|
|
|
|
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
|
|
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
|
|
|
|
## [Unreleased]
|
|
|
|
This section covers the ground-up rewrite of NeuroPose. The entries
|
|
below describe the difference between the previous internal prototype
|
|
and the state of the repository at the first tagged release, and will
|
|
be split into per-release sections once tagging begins.
|
|
|
|
### Added
|
|
|
|
#### Package structure and tooling
|
|
|
|
- `src/neuropose/` package layout with `py.typed` marker, MIT `LICENSE`,
|
|
policy-enforcing `.gitignore`, pinned Python 3.11 (`.python-version`),
|
|
and `pyproject.toml` with full project metadata, classifiers, and
|
|
URL pointers. The runtime TensorFlow dependency is pinned to
|
|
`tensorflow>=2.16,<2.19` — see *Changed* below for the rationale.
|
|
`psutil>=5.9` is a runtime dependency used by the estimator's
|
|
always-on `PerformanceMetrics` collection to sample peak RSS.
|
|
- `[project.optional-dependencies].analysis` extra for fastdtw, scipy,
|
|
scikit-learn, and sktime — install via `pip install neuropose[analysis]`.
|
|
- `[project.optional-dependencies].metal` extra pulling
|
|
`tensorflow-metal>=1.2,<2` under `sys_platform == 'darwin' and
|
|
platform_machine == 'arm64'` environment markers. Opt-in only via
|
|
`pip install 'neuropose[metal]'` or `uv sync --extra metal`; silently
|
|
no-op on every non-Apple-Silicon platform. The Metal path is **not**
|
|
exercised in CI and is documented as experimental in
|
|
`docs/getting-started.md` — users enabling it are expected to
|
|
spot-check numerics against the CPU path before trusting results
|
|
downstream.
|
|
- `[dependency-groups].dev` (PEP 735) with the full dev + docs + analyzer
|
|
toolchain: pytest, pytest-cov, ruff, pyright, pre-commit,
|
|
mkdocs-material, mkdocstrings, fastdtw, and scipy. `uv sync --group dev`
|
|
gives contributors everything needed to run the whole suite.
|
|
- `AUTHORS.md`, `CITATION.cff` (with a MeTRAbs upstream `references:`
|
|
entry), and a MIT-licensed `LICENSE` with an explicit MeTRAbs
|
|
attribution paragraph.
|
|
- Pre-commit configuration (`.pre-commit-config.yaml`) running ruff,
|
|
ruff-format, gitleaks (secret scanning), a 500 KB-limit
|
|
large-files hook, end-of-file fixers, trailing-whitespace fixers,
|
|
and YAML/TOML/JSON validators. Pyright is deliberately **not** in
|
|
pre-commit — it runs in CI only, so pre-commit stays fast.
|
|
- Ruff configuration in `pyproject.toml` with a deliberately broad
|
|
rule selection (pycodestyle, pyflakes, isort, bugbear, pyupgrade,
|
|
simplify, ruff-specific, pep8-naming, comprehensions, pathlib,
|
|
pytest-style, tidy-imports, numpy-specific, pydocstyle with numpy
|
|
convention). Per-file ignores for tests and private modules.
|
|
- Pyright configuration in `standard` mode (not `strict` — TF/OpenCV
|
|
stubs would otherwise drown the signal). Unknown-type reports are
|
|
explicitly silenced until the TensorFlow version pin is settled.
|
|
- Pytest configuration with strict markers, an opt-in `slow` marker,
|
|
and a `--runslow` CLI flag implemented in
|
|
`tests/conftest.py::pytest_collection_modifyitems` so integration
|
|
tests stay out of the default run.
|
|
|
|
#### CI / infrastructure
|
|
|
|
- GitHub Actions workflow `.github/workflows/ci.yml` running three
|
|
parallel jobs — **lint** (ruff), **typecheck** (pyright), and
|
|
**test** (pytest) — on every push and PR to `main`. Uses `uv` with
|
|
a pinned version (`0.9.16`) and cache-enabled setup for fast reruns.
|
|
Concurrency control cancels superseded runs on the same branch.
|
|
- GitHub Actions workflow `.github/workflows/docs.yml` that builds the
|
|
mkdocs-material site on every relevant push and uploads the rendered
|
|
site as a 14-day workflow artifact. GitHub Pages deployment is
|
|
intentionally not wired up yet; the workflow header comment
|
|
describes what to add when the repo flips public.
|
|
|
|
#### Runtime modules
|
|
|
|
- **`neuropose.config`** — `Settings` class built on
|
|
`pydantic-settings`. Field-level validation for `device`,
|
|
`poll_interval_seconds`, and `default_fov_degrees`; explicit
|
|
`from_yaml()` classmethod (no implicit config-file discovery); XDG
|
|
defaults for `data_dir` and `model_cache_dir` (`~/.local/share/neuropose/…`)
|
|
so runtime data never lives inside the repository; `ensure_dirs()`
|
|
as an explicit method so construction remains filesystem-side-effect-free.
|
|
- **`neuropose.io`** — validated prediction schemas:
|
|
`FramePrediction` (frozen), `VideoMetadata` (frame count, fps,
|
|
width, height), `VideoPredictions` (metadata envelope + frames
|
|
mapping + optional `segmentations` field), `JobResults`,
|
|
`JobStatus` enum, `JobStatusEntry` (with a structured `error`
|
|
field plus optional live-progress fields — `current_video`,
|
|
`frames_processed`, `frames_total`, `videos_completed`,
|
|
`videos_total`, `percent_complete`, `last_update` — populated by
|
|
the interfacer during inference and consumed by
|
|
`neuropose.monitor`), and `StatusFile`. Legacy status files
|
|
written before the progress fields existed still load cleanly
|
|
because every new field is optional with a `None` default. Performance schema: frozen
|
|
`PerformanceMetrics` carrying per-call timings
|
|
(`model_load_seconds`, `total_seconds`, `per_frame_latencies_ms`),
|
|
`peak_rss_mb`, `active_device`, `tensorflow_metal_active`, and
|
|
`tensorflow_version`; `BenchmarkResult` pairing a discarded
|
|
`warmup_pass` with `measured_passes` and a `BenchmarkAggregate`
|
|
(mean / p50 / p95 / p99 per-frame latency, mean throughput, max
|
|
peak RSS); optional `CpuComparisonResult` nested inside
|
|
`BenchmarkResult` for `--compare-cpu` runs, carrying both
|
|
device aggregates, the throughput speedup, and the
|
|
maximum-element-wise `poses3d` divergence in millimetres.
|
|
Segmentation schema: frozen `Segment` windows (`start`, `end`,
|
|
`peak`), `SegmentationConfig` (with a `method` version literal,
|
|
e.g. `valley_to_valley_v1`), a discriminated `ExtractorSpec`
|
|
union over `JointAxisExtractor`, `JointPairDistanceExtractor`,
|
|
`JointSpeedExtractor`, and `JointAngleExtractor`, and
|
|
`Segmentation` pairing a config with its segments so on-disk
|
|
results are self-describing. Load and save helpers with an atomic
|
|
tmp-file-then-rename pattern for every state file.
|
|
`load_benchmark_result` / `save_benchmark_result` follow the same
|
|
atomic pattern. `load_status` is deliberately crash-resilient:
|
|
missing, corrupt, or non-mapping JSON returns an empty
|
|
`StatusFile` rather than raising. Legacy predictions files
|
|
without the `segmentations` field deserialize cleanly to an
|
|
empty mapping.
|
|
- **`neuropose.estimator`** — `Estimator` class that streams frames
|
|
directly from OpenCV into the model, with no intermediate write-to-
|
|
disk-then-read-back-as-PNG round trip. Returns a typed
|
|
`ProcessVideoResult` containing a validated `VideoPredictions`
|
|
object and an always-populated `PerformanceMetrics` bundle (per-
|
|
frame latency in ms, total wall clock, peak RSS via `psutil`,
|
|
active TF device string, `tensorflow-metal` detection, TF
|
|
version, and model load time when the caller went through
|
|
`load_model()`). Does not touch the filesystem. Constructor
|
|
accepts an injected model for testability; `load_model()`
|
|
delegates to `neuropose._model.load_metrabs_model()`. Typed
|
|
exception hierarchy: `EstimatorError`, `ModelNotLoadedError`,
|
|
`VideoDecodeError`. Optional per-frame `progress` callback for
|
|
long videos. Frame identifier convention is `frame_000000`
|
|
(six-digit zero-pad, no extension — no file is implied).
|
|
- **`neuropose.visualize`** — `visualize_predictions()` for per-frame
|
|
2D + 3D overlay rendering. `matplotlib.use("Agg")` is called inside
|
|
the function rather than at module import, so `import neuropose.visualize`
|
|
has no global side effect. Explicit deep-copy of `poses3d` before
|
|
axis rotation to prevent the aliasing bug from the previous
|
|
prototype. Supports `frame_indices` for rendering a subset of
|
|
frames.
|
|
- **`neuropose.interfacer`** — `Interfacer` job-lifecycle daemon with
|
|
dependency-injected `Settings` and `Estimator`. Single-instance
|
|
enforcement via `fcntl.flock` on `data_dir/.neuropose.lock`.
|
|
Crash-recovery `recover_stuck_jobs()` that marks any status entries
|
|
left in `processing` state as failed with an "interrupted"
|
|
message and quarantines their inputs. Graceful shutdown on SIGINT/
|
|
SIGTERM with an interruptible sleep. Structured error fields on
|
|
every failed job. `run_once()` factored out of the main loop so
|
|
tests can drive single iterations without threading. Quarantine
|
|
collision handling (`job_a.1`, `job_a.2`, …) and empty-directory
|
|
silent-skip heuristic (mid-copy directories are not marked failed).
|
|
- **`neuropose._model`** — MeTRAbs model loader. Downloads the pinned
|
|
tarball from the upstream RWTH Aachen URL
|
|
(`metrabs_eff2l_y4_384px_800k_28ds.tar.gz`), verifies its SHA-256
|
|
checksum, atomically extracts to a staging directory and renames
|
|
into place, and loads via `tf.saved_model.load`. Streams the
|
|
download and hash computation in 1 MB chunks so memory is flat.
|
|
One automatic retry on SHA-256 mismatch (in case the previous
|
|
download was truncated). Post-load interface check for
|
|
`detect_poses`, `per_skeleton_joint_names`, and
|
|
`per_skeleton_joint_edges`.
|
|
- **`neuropose.monitor`** — localhost HTTP status dashboard. A small
|
|
`http.server`-based HTTP server (pure stdlib, zero new runtime
|
|
dependencies) that serves a plain HTML page at `GET /` with an
|
|
auto-refresh meta tag, one row per tracked job, a
|
|
`<progress>` bar, and a stale-entry warning badge for
|
|
`processing` jobs whose `last_update` has not ticked in 60 s.
|
|
`GET /status.json` returns the raw validated `StatusFile` as JSON
|
|
for `curl`/scripted pipelines; `?job=<name>` filters to a single
|
|
entry. `GET /health` is a simple liveness probe. Binds to
|
|
`127.0.0.1:8765` by default — loopback-only, with an explicit
|
|
`--host` override required to expose externally. Every request
|
|
re-reads `status.json`, so the monitor has no in-memory cache, no
|
|
sync protocol with the daemon, and stays useful even if the
|
|
daemon is down (last-known state surfaced with the stale badge).
|
|
- **Progress checkpointing in the interfacer.** `Interfacer` now
|
|
updates the currently-running job's `JobStatusEntry` every
|
|
`settings.status_checkpoint_every_frames` frames (default 30, a
|
|
new `Settings` field) during inference via the estimator's
|
|
`progress` callback. Each checkpoint rewrites `status.json`
|
|
atomically through the existing `save_status` helper; writes are
|
|
best-effort and I/O failures are logged without interrupting
|
|
inference. `_run_job_inner` seeds a "videos_total=N" checkpoint
|
|
before calling the estimator so the monitor shows the job's
|
|
scope from the first poll. Checkpoint cadence is knob-exposed for
|
|
operators who want to tune the smoothness-vs-write-rate trade-off.
|
|
- **`neuropose.ingest`** — zip-archive intake utility. `ingest_zip()`
|
|
extracts a zip of videos into one job directory per video under
|
|
`$data_dir/in/`, with validation-before-write (path-traversal and
|
|
absolute-path members rejected, oversize archives rejected at the
|
|
20 GB-uncompressed cap), zip-internal and external collision
|
|
detection reported in one shot, non-video members silently
|
|
skipped (`.DS_Store`, `README.md`, etc.), and per-job atomic
|
|
placement via a staging directory + `os.rename`. Nested paths are
|
|
flattened into job names by joining components with underscores
|
|
and sanitising unsafe characters — `patient_001/trial_01.mp4`
|
|
becomes job `patient_001_trial_01`, preserving disambiguation
|
|
against a sibling `patient_002/trial_01.mp4`. Typed exception
|
|
hierarchy: `IngestError`, `ArchiveInvalidError`,
|
|
`ArchiveEmptyError`, `ArchiveTooLargeError`, `JobCollisionError`
|
|
(with a `.collisions` list of offending names). The running
|
|
daemon needs no changes — ingested job dirs are picked up on the
|
|
next poll.
|
|
- **`neuropose.migrations`** — schema-migration infrastructure for
|
|
the three top-level serialised payloads (`VideoPredictions`,
|
|
`JobResults`, `BenchmarkResult`). Every payload carries a
|
|
`schema_version` field defaulting to `CURRENT_VERSION`; on load,
|
|
the raw JSON dict is passed through `migrate_video_predictions` /
|
|
`migrate_job_results` / `migrate_benchmark_result` *before*
|
|
pydantic validation so files written by older NeuroPose versions
|
|
upgrade transparently. One shared `CURRENT_VERSION` counter;
|
|
per-schema migration registries populated via
|
|
`register_video_predictions_migration(from_version)` and
|
|
`register_benchmark_result_migration(from_version)` decorators.
|
|
`JobResults` is a `RootModel` with no envelope of its own, so its
|
|
migration runs per-entry across the root mapping. The driver raises
|
|
`FutureSchemaError` for payloads newer than the current build
|
|
(clear upgrade-NeuroPose message), `MigrationNotFoundError` for
|
|
missing chain links (indicates a `CURRENT_VERSION` bump that forgot
|
|
its migration), and logs at INFO on each version advance. Currently
|
|
at `CURRENT_VERSION = 2`, with registered v1 → v2 migrations for
|
|
`VideoPredictions` and `BenchmarkResult` that add the optional
|
|
`provenance` field.
|
|
- **`neuropose.analyzer.features.procrustes_align`** — Kabsch
|
|
rigid-alignment helper for pose sequences, plus a
|
|
`ProcrustesMode` literal (`"per_frame"` | `"per_sequence"`) and a
|
|
frozen `AlignmentDiagnostics` dataclass (`rotation_deg`,
|
|
`rotation_deg_max`, `translation`, `translation_max`, `scale`,
|
|
plus the mode that produced them). Per-sequence mode fits one
|
|
rigid transform across the whole trial; per-frame fits an
|
|
independent transform per frame. Optional `scale=True` fits a
|
|
uniform scale factor for cross-subject comparisons. Wired into
|
|
every DTW entry point in `neuropose.analyzer.dtw` via a new
|
|
keyword-only `align: AlignMode = "none"` parameter — `"none"`
|
|
preserves the 0.1 raw-coordinate behaviour, while
|
|
`"procrustes_per_frame"` and `"procrustes_per_sequence"` route
|
|
inputs through `procrustes_align` before DTW runs so the returned
|
|
distance is rotation- and translation-invariant. Paper C's
|
|
pipeline is expected to set `align="procrustes_per_sequence"`;
|
|
see `TECHNICAL.md` Phase 0.
|
|
- **`neuropose.analyzer.dtw.Representation`** and
|
|
**`neuropose.analyzer.dtw.NanPolicy`** — two new Literal types
|
|
exposing orthogonal DTW preprocessing knobs on every entry point.
|
|
`representation` (on `dtw_all` and `dtw_per_joint`) switches the
|
|
per-frame feature vector between `"coords"` (the 0.1 default) and
|
|
`"angles"`, which runs `extract_joint_angles` on the supplied
|
|
`angle_triplets` first — yielding distances that are translation-,
|
|
rotation-, and scale-invariant by construction, and directly
|
|
interpretable in clinical terms. `nan_policy` (on all three entry
|
|
points) selects `"propagate"` (surface fastdtw's ValueError on
|
|
NaN — the default), `"interpolate"` (linear fill per feature
|
|
column), or `"drop"` (remove NaN frames before DTW); the
|
|
policy is applied consistently whether NaN originated from the
|
|
angles pipeline or from corrupted upstream coordinates.
|
|
`dtw_relation` stays a standalone convenience entry point for
|
|
two-joint displacement DTW; users who prefer a unified API can
|
|
express the same computation via `dtw_all` with an appropriate
|
|
pair of angle triplets or run `dtw_relation` directly.
|
|
- **`neuropose.analyzer.pipeline`** (schemas) — declarative
|
|
analysis-pipeline configuration and output artifact, parseable from
|
|
YAML or JSON via pydantic. `AnalysisConfig` captures a full
|
|
experiment: inputs (primary + optional reference predictions
|
|
files), preprocessing (person index, with room to grow),
|
|
optional segmentation (`gait_cycles` / `gait_cycles_bilateral` /
|
|
`extractor` discriminated union), and a required analysis stage
|
|
(`dtw` / `stats` / `none` discriminated union). `AnalysisReport`
|
|
is the runtime output: carries the originating config, a
|
|
`Provenance` envelope with `analysis_config` populated, per-input
|
|
summaries, produced segmentations, and an analysis-result payload
|
|
that mirrors the stage choice (`DtwResults`, `StatsResults`, or
|
|
`NoResults`). Cross-field invariants — `method="dtw_relation"`
|
|
requires `joint_i`/`joint_j`, `representation="angles"` requires
|
|
non-empty `angle_triplets`, `analysis.kind="dtw"` requires
|
|
`inputs.reference`, `analysis.kind="stats"` refuses a reference —
|
|
are enforced at parse time via `model_validator` so typos fail in
|
|
milliseconds instead of after a multi-minute predictions load.
|
|
`AnalysisReport` carries a `schema_version` field defaulting to
|
|
`CURRENT_VERSION = 2`, with a new
|
|
`register_analysis_report_migration` decorator and
|
|
`migrate_analysis_report` driver in `neuropose.migrations` ready
|
|
for future schema changes. `run_analysis(config)` loads the named
|
|
predictions files, applies the configured segmentation, dispatches
|
|
to the selected analysis kind (DTW, stats, or none), and emits a
|
|
fully populated `AnalysisReport` whose `Provenance` inherits the
|
|
inference-time envelope from the primary input with
|
|
`analysis_config` stamped in, so the report is self-describing
|
|
even if the source YAML is lost. For DTW runs with segmentation,
|
|
segments are paired one-to-one by index across primary and
|
|
reference, truncating to `min(len_primary, len_reference)`;
|
|
bilateral segmentations emit per-side distances under
|
|
`"left_heel_strikes[i]"` / `"right_heel_strikes[i]"` labels.
|
|
`load_config(path)` parses YAML, `save_report(path, report)`
|
|
writes atomically, and `load_report(path)` rehydrates via the
|
|
migration chain. Wired to the CLI as `neuropose analyze --config
|
|
<yaml> [--output <json>]` — replaces the placeholder stub that
|
|
previously returned `EXIT_PENDING`. The CLI surfaces schema
|
|
violations and YAML parse errors as `EXIT_USAGE=2` with a clear
|
|
message pointing at the offending file, prints a one-line summary
|
|
of the run (segmentation counts, analysis kind, per-segment
|
|
distance count + mean for DTW), and supports `--output`/`-o` to
|
|
override the report path declared in the config (useful for
|
|
sweeping a single config over multiple input pairs from a shell
|
|
loop). Ships three example configs under `examples/analysis/`:
|
|
`minimal.yaml` (smallest working DTW pipeline), `paper_c_headline.yaml`
|
|
(representative Paper C config with bilateral gait-cycle
|
|
segmentation, per-sequence Procrustes, and joint-angle DTW on
|
|
knee/hip triplets), and `per_joint_debug.yaml` (per-joint DTW
|
|
breakdown for diagnosing which joint drives an unexpected
|
|
distance). An integration suite exercises each example against
|
|
synthetic predictions so schema drift between the YAMLs and the
|
|
executor fails CI, not silently at run time. Documented in
|
|
`docs/api/pipeline.md`.
|
|
- **`neuropose.analyzer.segment.segment_gait_cycles`** and
|
|
**`segment_gait_cycles_bilateral`** — clinical convenience
|
|
wrappers over `segment_predictions` that pre-fill a `joint_axis`
|
|
extractor with gait-appropriate defaults (`joint="rhee"`,
|
|
`axis="y"`, `min_cycle_seconds=0.4`). The single-side entry point
|
|
accepts any berkeley_mhad_43 joint name and any spatial axis as a
|
|
string literal `"x" | "y" | "z"`, plus an `invert` flag for
|
|
recordings whose vertical axis runs opposite to MeTRAbs's
|
|
Y-down world-coordinate convention. The bilateral wrapper runs
|
|
the detection on both `lhee` and `rhee` and returns the two
|
|
results under `"left_heel_strikes"` / `"right_heel_strikes"`
|
|
keys — shape-compatible with `VideoPredictions.segmentations` so
|
|
the dict can be merged in directly. Degrades gracefully on
|
|
pathological gaits (shuffling, walker-assisted) by returning an
|
|
empty segments list rather than raising. Closes the gait-cycle
|
|
segmentation item in `TECHNICAL.md` Phase 0.
|
|
- **`neuropose.io.Provenance`** — reproducibility envelope for every
|
|
inference run. Populated automatically by `Estimator.process_video`
|
|
when the model was loaded via `load_model` (the production path)
|
|
and attached to the output `VideoPredictions`; propagates from
|
|
there into `JobResults` (per-video) and `BenchmarkResult` (via the
|
|
benchmark loop). Captures the MeTRAbs artifact SHA-256 and
|
|
filename, `tensorflow` / `tensorflow-metal` / `numpy` /
|
|
`neuropose` / Python versions, and reserved slots for a `seed`,
|
|
`deterministic` flag (Track 2), and `analysis_config` (Phase 0
|
|
YAML pipeline). `None` on the injected-model test path where
|
|
NeuroPose has no way to fingerprint the supplied artifact. Frozen
|
|
pydantic model with `extra="forbid"` and
|
|
`protected_namespaces=()` so the `model_*` field names do not
|
|
collide with pydantic v2's internal namespace. `_model.load_metrabs_model`
|
|
now returns a `LoadedModel` dataclass bundling the TF handle with
|
|
the pinned SHA and filename so the estimator can build the
|
|
`Provenance` without re-hashing the tarball.
|
|
- **`neuropose.reset`** — pipeline-wide reset utility for the
|
|
benchmark / iteration loop. `find_neuropose_processes()` scans the
|
|
OS process table (via `psutil`) for running `neuropose watch` and
|
|
`neuropose serve` instances and classifies each as `daemon` or
|
|
`monitor`. `terminate_processes()` SIGINTs them, polls for graceful
|
|
exit up to a configurable grace period, and optionally escalates
|
|
to SIGKILL with `force_kill=True`. `wipe_state()` removes the
|
|
contents of `$data_dir/in/`, `$data_dir/out/` (including
|
|
`status.json`), `$data_dir/failed/` (unless `keep_failed=True`),
|
|
the `.neuropose.lock` file, and any leftover `.ingest_<uuid>/`
|
|
staging dirs from interrupted ingests; container directories
|
|
themselves are preserved so the daemon does not need to recreate
|
|
them on next startup. `reset_pipeline()` composes the three with
|
|
one safety guard: if any process survives termination, the wipe
|
|
phase is skipped and the returned `ResetReport` flags
|
|
`wipe_skipped_due_to_survivors`, because removing `$data_dir`
|
|
out from under an active daemon would corrupt its in-flight
|
|
writes. Surfaced as `neuropose reset` in the CLI with
|
|
`--yes/-y`, `--keep-failed`, `--force-kill`, `--grace-seconds`,
|
|
and `--dry-run/-n` flags; the command always prints a preview
|
|
before prompting (skipped under `--yes`) and returns
|
|
`EXIT_USAGE=2` when survivors block the wipe.
|
|
- **`neuropose.benchmark`** — multi-pass inference benchmarking for
|
|
a single video. `run_benchmark()` runs `process_video` N times
|
|
(default 5), always discards the first pass as warmup (graph
|
|
compilation, file-system cache warmup), and aggregates the
|
|
remaining `PerformanceMetrics` into a `BenchmarkAggregate` with
|
|
mean / p50 / p95 / p99 per-frame latency, mean throughput, and
|
|
max peak RSS. `capture_reference=True` additionally preserves the
|
|
last measured pass's `VideoPredictions` in memory so the
|
|
`--compare-cpu` CLI flow can diff the `poses3d` arrays between a
|
|
GPU and CPU run. `compute_poses3d_divergence()` computes the
|
|
maximum element-wise absolute difference (in millimetres) between
|
|
two prediction sets, skipping frames with mismatched detection
|
|
counts and surfacing the `frame_count_compared` so callers can
|
|
tell if the number is trustworthy. `format_benchmark_report()`
|
|
renders a human-readable summary for CLI stdout.
|
|
- **`neuropose.analyzer`** — post-processing subpackage with lazy
|
|
imports for the heavy dependencies:
|
|
- `analyzer.dtw` — three DTW entry points (`dtw_all`,
|
|
`dtw_per_joint`, `dtw_relation`) over fastdtw, with a frozen
|
|
`DTWResult` dataclass and three orthogonal preprocessing knobs
|
|
(`align`, `representation`, `nan_policy`). See `RESEARCH.md`
|
|
for the ongoing
|
|
methodology investigation.
|
|
- `analyzer.features` — `predictions_to_numpy`,
|
|
`normalize_pose_sequence` (uniform and axis-wise),
|
|
`pad_sequences` (edge-padding), `procrustes_align` (Kabsch
|
|
rigid alignment, per-frame or per-sequence, optional uniform
|
|
scaling), `extract_joint_angles` (NaN on degenerate vectors),
|
|
`extract_feature_statistics` (`FeatureStatistics` frozen
|
|
dataclass), and a `find_peaks` thin wrapper around
|
|
`scipy.signal.find_peaks`.
|
|
- `analyzer.segment` — repetition segmentation for trials in
|
|
which a subject performs the same movement several times. A
|
|
three-layer API: `segment_by_peaks` (pure 1D
|
|
valley-to-valley peak detection on a generic signal),
|
|
`segment_predictions` (top-level entry point taking a
|
|
`VideoPredictions` plus an `ExtractorSpec`, converting
|
|
time-based parameters to frame counts via `metadata.fps`), and
|
|
`slice_predictions` (split a `VideoPredictions` into one per
|
|
detected repetition with re-keyed frame names and a rewritten
|
|
`frame_count`). Gait-specific convenience wrappers
|
|
`segment_gait_cycles` (single heel) and
|
|
`segment_gait_cycles_bilateral` (both heels, returning a dict
|
|
keyed by `"left_heel_strikes"` / `"right_heel_strikes"`) sit
|
|
above `segment_predictions` with clinical defaults. Ships four extractor factories —
|
|
`joint_axis`, `joint_pair_distance`, `joint_speed`, and
|
|
`joint_angle` — plus a `JOINT_NAMES` constant for the
|
|
berkeley_mhad_43 skeleton with a `joint_index(name)` lookup,
|
|
so post-processing callers can resolve `"rwri"` → integer
|
|
without loading the MeTRAbs SavedModel. A matching integration
|
|
test (`tests/integration/test_joint_names_drift.py`, marked
|
|
`slow`) loads the real model and asserts the constant still
|
|
matches, so any upstream skeleton drift fails CI.
|
|
- **`neuropose.cli`** — Typer-based command-line interface with
|
|
eight subcommands: `watch` (run the daemon), `process <video>`
|
|
(run the estimator on a single video), `ingest <archive>` (unzip
|
|
a video archive into per-video job directories under
|
|
`$data_dir/in/` with validation-before-write and atomic
|
|
placement; `--force` overwrites collisions, otherwise the whole
|
|
operation refuses if any target name already exists),
|
|
`serve` (start the localhost HTTP monitor at `127.0.0.1:8765`
|
|
by default — `--host` and `--port` are the two overrides;
|
|
KeyboardInterrupt exits with the standard shell-interruption
|
|
code and an `OSError` at bind time is translated to a clean
|
|
usage error with the bind target in the message),
|
|
`reset` (stop the daemon and monitor, then wipe pipeline state
|
|
for a clean restart — wraps `neuropose.reset` with a confirmation
|
|
prompt, `--dry-run` preview, `--keep-failed` to preserve the
|
|
forensic quarantine, `--force-kill` to escalate to SIGKILL after
|
|
the SIGINT grace period, and `--grace-seconds` to tune the wait;
|
|
refuses to wipe state while any process survives termination so
|
|
active writes cannot be corrupted),
|
|
`segment <results>` (post-hoc repetition segmentation — loads a
|
|
JobResults or a single VideoPredictions, runs
|
|
`neuropose.analyzer.segment.segment_predictions` with the chosen
|
|
extractor and thresholds, and atomically writes the file back
|
|
with the new segmentation attached under `--name`),
|
|
`benchmark <video>` (multi-pass inference benchmark — runs
|
|
`--repeats N` passes with a discarded first pass and
|
|
`--warmup-frames M` excluded from the head of each measured
|
|
pass, reports aggregates to stdout, and optionally writes a
|
|
structured `BenchmarkResult` to `--output`. Supports
|
|
`--compare-cpu` which spawns a `--force-cpu` subprocess, diffs
|
|
the resulting `poses3d` arrays, and reports throughput speedup
|
|
and max divergence in mm — the missing Apple Silicon numerical
|
|
verification answer from `RESEARCH.md`), and
|
|
`analyze --config <yaml>` (run the declarative analysis
|
|
pipeline — see the dedicated entry above for scope). The
|
|
`segment` subcommand accepts
|
|
joint specifiers as either berkeley_mhad_43 names (`lwri`,
|
|
`rwri`, …) or integer indices, and refuses to overwrite an
|
|
existing segmentation of the same name without `--force`.
|
|
Global options `--config/-c`, `--verbose/-v`, `--quiet/-q`,
|
|
`--version`. Structured error handling turns expected exceptions
|
|
(`FileNotFoundError` on config, `ValidationError`,
|
|
`AlreadyRunningError`, `NotImplementedError`,
|
|
`KeyboardInterrupt`) into clear stderr messages and distinct
|
|
exit codes (`EXIT_OK=0`, `EXIT_USAGE=2`, `EXIT_PENDING=3`,
|
|
`EXIT_INTERRUPTED=130`). The CLI entry point is wired in
|
|
`[project.scripts]` as `neuropose = "neuropose.cli:run"`.
|
|
|
|
#### Documentation
|
|
|
|
- **mkdocs-material documentation site** under `docs/` with the full
|
|
theme configuration (light/dark toggle, tabs navigation, search),
|
|
`mkdocstrings` Python handler set to numpy docstring style with
|
|
source links, and a `pymdownx` extension set for admonitions,
|
|
tabbed content, collapsible details, and syntax-highlighted code
|
|
blocks. Nav: Home → Getting Started → Architecture → API Reference
|
|
(auto-generated from module docstrings) → Development → Deployment.
|
|
- Prose documentation pages: `docs/index.md` (public landing page),
|
|
`docs/getting-started.md` (install, CLI, output schema, Python API,
|
|
visualization, troubleshooting), `docs/architecture.md` (three-stage
|
|
pipeline, data flow, runtime directory layout, design principles),
|
|
`docs/development.md` (contributor setup, tests, lint/type,
|
|
commit hygiene, release process stub), and `docs/deployment.md`
|
|
(systemd user unit, Docker pointer, GPU notes, backup guidance).
|
|
- API reference stubs `docs/api/{config,estimator,interfacer,io,visualize}.md`
|
|
— each is a two-line file containing a `:::` mkdocstrings directive,
|
|
so the API documentation is generated from the source docstrings
|
|
at build time and cannot drift out of sync.
|
|
- `RESEARCH.md` at the repo root: a living R&D log for DTW
|
|
methodology alternatives and MeTRAbs self-hosting / fine-tuning
|
|
plans. Not user-facing documentation; not linked from the mkdocs
|
|
nav.
|
|
|
|
#### Tests
|
|
|
|
- `tests/unit/` covering configuration (defaults, validation, YAML
|
|
loading, env overrides, `ensure_dirs`), IO schema and helpers
|
|
(roundtrip, atomic save, frozen-model guarantees, corruption
|
|
tolerance), the estimator (construction, model-guard, process path
|
|
with fake MeTRAbs model, error paths), the visualize module
|
|
(smoke tests + an anti-regression check for the audit §6 aliasing
|
|
bug), the interfacer (construction, discovery, process-job happy
|
|
and failure paths, stuck-job recovery, lock, run_once,
|
|
interruptible sleep), the CLI (top-level options, config handling,
|
|
each subcommand's error path), the analyzer DTW helpers, and the
|
|
analyzer features helpers.
|
|
- `tests/conftest.py` with an autouse `_isolate_environment` fixture
|
|
that redirects `$HOME` and `$XDG_DATA_HOME` at a per-test temp
|
|
directory so no test can accidentally write to the developer's real
|
|
machine, and clears any `NEUROPOSE_*` env vars. Adds a
|
|
`synthetic_video` fixture (cv2-generated 5-frame MJPG AVI sized
|
|
for most unit tests) and a `fake_metrabs_model` fixture.
|
|
- `tests/integration/test_estimator_smoke.py` — end-to-end model
|
|
loader + estimator smoke test against the real MeTRAbs tarball,
|
|
marked `@pytest.mark.slow`, skipped by default, opt-in via
|
|
`--runslow`. Uses a session-scoped model cache so the download
|
|
happens at most once per run.
|
|
|
|
#### Operations
|
|
|
|
- `Dockerfile` — CPU image based on `python:3.11-slim-bookworm`.
|
|
Installs the package with the `analysis` extra, runs as non-root
|
|
user `neuropose` (UID 1000), exposes `/data` as a volume, sets
|
|
`NEUROPOSE_DATA_DIR` and `NEUROPOSE_MODEL_CACHE_DIR` to point at
|
|
the mounted volume, and uses `ENTRYPOINT ["neuropose"]` with
|
|
`CMD ["watch"]` so the default is the daemon and overrides are
|
|
ergonomic.
|
|
- `.dockerignore` that aggressively excludes developer tooling,
|
|
caches, tests, documentation sources, research notes, and
|
|
ancillary scripts from the build context.
|
|
- `scripts/download_model.py` — standalone pre-warm script that
|
|
invokes `load_metrabs_model()` with an optional `--cache-dir`
|
|
override. Useful for seeding a deployment's cache before cutting
|
|
off network access.
|
|
|
|
### Changed
|
|
|
|
- **Relicensed from AGPL-3.0 (used in the prior internal prototype)
|
|
to MIT.** The prior license was copied from precedent rather than
|
|
chosen deliberately; the MIT relicense better matches both the
|
|
project's "research software others can build on" intent and the
|
|
upstream MeTRAbs license.
|
|
- Reorganised from the prior `backend/` + runtime-data layout into
|
|
a `src/neuropose/` Python package. Runtime data now lives outside
|
|
the repository by default (under `$XDG_DATA_HOME/neuropose/`) so
|
|
subject-identifying inputs cannot accidentally end up in a
|
|
`git add`.
|
|
- Frame identifier convention changed from `frame_0000.png` (old,
|
|
misleading — no PNG file exists) to `frame_000000` (six-digit
|
|
zero-pad, no extension, pure identifier).
|
|
- Estimator API: `process_video()` now returns a typed
|
|
`ProcessVideoResult` containing a validated `VideoPredictions`
|
|
object, instead of a stringly-typed dict with `results_path` and
|
|
`frame_count`. The estimator no longer owns filesystem
|
|
destinations — the caller decides where to save.
|
|
- `VideoPredictions` schema now carries a `VideoMetadata` envelope
|
|
(frame count, fps, width, height) alongside the per-frame
|
|
predictions. Downstream analysis can convert frame indices to
|
|
real time without needing access to the original video.
|
|
- Interfacer uses `datetime.now(UTC)` instead of the deprecated
|
|
`datetime.utcnow()`, addresses the "no-videos"-vs-exception-path
|
|
inconsistency (both now quarantine), and persists a structured
|
|
`error` string on every failure for grep-friendly diagnostics.
|
|
- **TensorFlow pin set to `tensorflow>=2.16,<2.19`.** The 2.16
|
|
floor is the first release with native `darwin/arm64` wheels under
|
|
the `tensorflow` package name on PyPI, so a single dependency line
|
|
works across Linux x86_64, Linux arm64, and Apple Silicon macOS
|
|
without platform markers or a separate `tensorflow-macos` package.
|
|
The `<2.19` ceiling is a `tensorflow-metal` compatibility constraint:
|
|
the latest Metal plugin (1.2.0, January 2025) advertises "TF 2.18+"
|
|
but in practice fails on 2.19 and 2.20 with symbol-not-found errors
|
|
and graph-execution `InvalidArgumentError`s
|
|
([tensorflow/tensorflow#84167](https://github.com/tensorflow/tensorflow/issues/84167)).
|
|
Cap is global rather than darwin-only so dependency resolution stays
|
|
identical across platforms. The MeTRAbs SavedModel itself
|
|
(`metrabs_eff2l_y4_384px_800k_28ds`, serialized with TF 2.10) was
|
|
separately verified to load and run `detect_poses` end-to-end on
|
|
TF 2.21 + Keras 3 with no errors and zero custom ops, so the cap is
|
|
purely an external-package constraint and can lift once Apple ships
|
|
a Metal plugin that tracks mainline TensorFlow again. Full probe
|
|
data and op inventory in `RESEARCH.md`.
|
|
- Operating-system classifiers in `pyproject.toml` extended from
|
|
Linux-only to `POSIX` + `POSIX :: Linux` + `MacOS`, reflecting the
|
|
Apple Silicon support that the TF 2.16 floor makes real.
|
|
|
|
### Removed
|
|
|
|
- The previous `backend/analyzer.py` and `backend/validator.py`
|
|
stubs, which were non-functional and had never been run
|
|
successfully. `analyzer.py` is reintroduced as a pure-function
|
|
subpackage (`neuropose.analyzer`) rewritten from the prior
|
|
code's design intent. `validator.py` is reintroduced as a real
|
|
pytest suite (`tests/unit/` and `tests/integration/`).
|
|
- The previous `reconstruct_from_frames` helper on the `Estimator`
|
|
— dead code, broken (dereferenced `self.OUTPUT_PATH`, which did
|
|
not exist), hardcoded 10 fps, never called. ffmpeg is a better
|
|
tool for this and can be invoked directly.
|
|
- The previous `__main__` placeholder (`print("in main"); sys.exit()`)
|
|
on `estimator.py`. The real CLI now lives in `neuropose.cli`.
|
|
- Every file under `docs/` in the previous prototype. All of the
|
|
pydoc-generated HTML, Org-mode sources, and handwritten markdown
|
|
described an older version of the API with methods
|
|
(`bind_and_block`, `construct_paths`, `toggle_visualization`,
|
|
`propagate_fatal_error`, etc.) that no longer exist. The docs are
|
|
now auto-generated from source docstrings via mkdocstrings so
|
|
drift is mechanically impossible.
|
|
- The previous Dockerfile, which referenced a non-existent
|
|
`backend/requirements.txt`, attempted to `COPY ./model /app/model`
|
|
(no such directory), and set `CMD ["uvicorn", "main:app"]` for a
|
|
FastAPI app that never existed.
|
|
- The previous `install/install.sh`, `install/#install.sh#` (an
|
|
Emacs autosave file), `install/install.sh~` (an Emacs backup file),
|
|
and `install/environment.yml`. The conda + `git+https` install
|
|
story is replaced by `uv` + a single `pyproject.toml`.
|
|
- The previous `bit.ly/metrabs_1` URL shortener for the model
|
|
download, replaced by a pinned canonical URL on the upstream
|
|
RWTH Aachen "omnomnom" host, with SHA-256 verification on
|
|
download. See `RESEARCH.md` for the plan to mirror to
|
|
self-hosted storage.
|
|
|
|
### Security
|
|
|
|
- Large-files pre-commit hook (`check-added-large-files` with a
|
|
500 KB limit) blocks accidental commits of subject data or model
|
|
weights.
|
|
- Gitleaks pre-commit hook scans every staged change for secret
|
|
material.
|
|
- Dockerfile runs as a non-root user (UID 1000, `neuropose`) by
|
|
default.
|
|
- Tarfile extraction uses the `filter="data"` option to block path
|
|
traversal and other tar-bomb attacks during MeTRAbs model
|
|
extraction.
|
|
- SHA-256 pinning of the MeTRAbs model artifact. A change to the
|
|
upstream tarball contents fails the checksum verification and
|
|
requires a human-reviewed diff before the new artifact is
|
|
trusted.
|
|
|
|
### Known limitations
|
|
|
|
- Apple Silicon support is established by-construction (TF 2.16+
|
|
publishes native `darwin/arm64` wheels and the MeTRAbs SavedModel
|
|
uses only stock ops verified portable on TF 2.21) but has not yet
|
|
been exercised on real Apple Silicon hardware. A `macos-14` CI
|
|
matrix entry covering the unit tests is the cheapest way to catch
|
|
any regression and is planned as a follow-up.
|
|
- Classification wrappers on top of sktime are deliberately **not**
|
|
included in `neuropose.analyzer` for this release. See `RESEARCH.md`
|
|
for the reasoning and the plan.
|
|
- GPU support in Docker is not yet shipped (`Dockerfile.gpu` is
|
|
planned). The existing `Dockerfile` runs CPU-only.
|
|
- `neuropose analyze` is a CLI stub that exits with a pending
|
|
message. The analyzer subpackage is usable from Python directly;
|
|
the CLI wrapper will follow once the analysis pipeline has a
|
|
concrete shape worth wrapping.
|
|
- The data-handling policy referenced from `docs/deployment.md` and
|
|
`docs/index.md` (`docs/data-policy.md`) is being authored
|
|
separately and is not part of this changelog entry.
|
|
|
|
[Unreleased]: https://git.levineuwirth.org/neuwirth/neuropose/compare/initial...HEAD
|