add example analysis configs and integration suite
Three reference configs under examples/analysis/:
- minimal.yaml: full-trial DTW on raw coordinates, no alignment or
segmentation. Smallest working example; a starting template.
- paper_c_headline.yaml: the representative Paper C pipeline.
Bilateral gait-cycle segmentation, per-sequence Procrustes, and
joint-angle DTW on knee and hip flexion triplets.
- per_joint_debug.yaml: per-joint DTW breakdown for diagnosing
which joint drives an unexpected distance.
tests/integration/test_analyze_examples.py exercises each example
twice: load_config must accept the YAML (catches drift between the
examples and the current schema), and run_analysis must execute the
config end-to-end against synthetic predictions (catches drift
between the examples and the executor). The Paper C example has an
extra guard verifying the knee-flexion triplets haven't been edited
to something unexpected.
Also wires docs/api/pipeline.md into the mkdocs nav so mkdocstrings
surfaces the full schema and executor API.
This commit is contained in:
parent
01b374451f
commit
4f3a6241fb
11
CHANGELOG.md
11
CHANGELOG.md
|
|
@ -301,7 +301,16 @@ be split into per-release sections once tagging begins.
|
|||
distance count + mean for DTW), and supports `--output`/`-o` to
|
||||
override the report path declared in the config (useful for
|
||||
sweeping a single config over multiple input pairs from a shell
|
||||
loop). Example configs land in a follow-up commit.
|
||||
loop). Ships three example configs under `examples/analysis/`:
|
||||
`minimal.yaml` (smallest working DTW pipeline), `paper_c_headline.yaml`
|
||||
(representative Paper C config with bilateral gait-cycle
|
||||
segmentation, per-sequence Procrustes, and joint-angle DTW on
|
||||
knee/hip triplets), and `per_joint_debug.yaml` (per-joint DTW
|
||||
breakdown for diagnosing which joint drives an unexpected
|
||||
distance). An integration suite exercises each example against
|
||||
synthetic predictions so schema drift between the YAMLs and the
|
||||
executor fails CI, not silently at run time. Documented in
|
||||
`docs/api/pipeline.md`.
|
||||
- **`neuropose.analyzer.segment.segment_gait_cycles`** and
|
||||
**`segment_gait_cycles_bilateral`** — clinical convenience
|
||||
wrappers over `segment_predictions` that pre-fill a `joint_axis`
|
||||
|
|
|
|||
|
|
@ -0,0 +1,3 @@
|
|||
# `neuropose.analyzer.pipeline`
|
||||
|
||||
::: neuropose.analyzer.pipeline
|
||||
|
|
@ -0,0 +1,26 @@
|
|||
# Minimal DTW config: full-trial comparison on raw 3D coordinates,
|
||||
# no Procrustes alignment, no segmentation. The simplest working
|
||||
# example; use this as a starting template and add stages as needed.
|
||||
#
|
||||
# Run:
|
||||
# neuropose analyze --config examples/analysis/minimal.yaml \
|
||||
# --output out/minimal_report.json
|
||||
#
|
||||
# (Substitute real paths for `inputs.primary` and `inputs.reference`
|
||||
# before running.)
|
||||
|
||||
config_version: 1
|
||||
|
||||
inputs:
|
||||
primary: data/trial_a.json
|
||||
reference: data/trial_b.json
|
||||
|
||||
analysis:
|
||||
kind: dtw
|
||||
method: dtw_all
|
||||
align: none
|
||||
representation: coords
|
||||
nan_policy: propagate
|
||||
|
||||
output:
|
||||
report: out/minimal_report.json
|
||||
|
|
@ -0,0 +1,48 @@
|
|||
# Paper C headline config: cycle-segmented joint-angle DTW with
|
||||
# per-sequence Procrustes alignment. This is the representative
|
||||
# Paper C pipeline — bilateral gait cycles drive the segmentation
|
||||
# so distances are reported per-stride per-side, and the angle-space
|
||||
# representation makes the distance clinically interpretable
|
||||
# (knee flexion angle, hip extension angle, etc.).
|
||||
#
|
||||
# Joint-triplet indices below target the berkeley_mhad_43 skeleton:
|
||||
# - (27, 31, 32): left hip → left knee → left ankle (left knee flex)
|
||||
# - (35, 39, 40): right hip → right knee → right ankle (right knee flex)
|
||||
# - (34, 27, 31): back hip ← left hip → left knee (left hip flex)
|
||||
# - (34, 35, 39): back hip ← right hip → right knee (right hip flex)
|
||||
#
|
||||
# See neuropose.analyzer.JOINT_NAMES for the full 43-joint table.
|
||||
#
|
||||
# Run:
|
||||
# neuropose analyze --config examples/analysis/paper_c_headline.yaml \
|
||||
# --output out/paper_c_report.json
|
||||
|
||||
config_version: 1
|
||||
|
||||
inputs:
|
||||
primary: data/subject_trial.json
|
||||
reference: data/mocap_reference.json
|
||||
|
||||
preprocessing:
|
||||
person_index: 0
|
||||
|
||||
segmentation:
|
||||
kind: gait_cycles_bilateral
|
||||
axis: y
|
||||
invert: false
|
||||
min_cycle_seconds: 0.4
|
||||
|
||||
analysis:
|
||||
kind: dtw
|
||||
method: dtw_all
|
||||
align: procrustes_per_sequence
|
||||
representation: angles
|
||||
angle_triplets:
|
||||
- [27, 31, 32] # left knee flexion
|
||||
- [35, 39, 40] # right knee flexion
|
||||
- [34, 27, 31] # left hip flexion
|
||||
- [34, 35, 39] # right hip flexion
|
||||
nan_policy: interpolate
|
||||
|
||||
output:
|
||||
report: out/paper_c_report.json
|
||||
|
|
@ -0,0 +1,36 @@
|
|||
# Per-joint debug config: runs dtw_per_joint on raw coordinates so
|
||||
# the resulting report carries a full (segments × joints) distance
|
||||
# breakdown. Useful when one joint is suspected of driving an
|
||||
# otherwise-unexpected DTW distance — the per-joint numbers make it
|
||||
# obvious which joint's trajectory diverges most.
|
||||
#
|
||||
# Raw coordinates (representation: coords) are used rather than
|
||||
# angles because joint-level debugging is most interpretable in the
|
||||
# native measurement space. Procrustes alignment is on so
|
||||
# translation and rotation between trials do not inflate the numbers.
|
||||
#
|
||||
# Run:
|
||||
# neuropose analyze --config examples/analysis/per_joint_debug.yaml \
|
||||
# --output out/per_joint_report.json
|
||||
|
||||
config_version: 1
|
||||
|
||||
inputs:
|
||||
primary: data/trial_a.json
|
||||
reference: data/trial_b.json
|
||||
|
||||
segmentation:
|
||||
kind: gait_cycles
|
||||
joint: rhee
|
||||
axis: y
|
||||
min_cycle_seconds: 0.4
|
||||
|
||||
analysis:
|
||||
kind: dtw
|
||||
method: dtw_per_joint
|
||||
align: procrustes_per_sequence
|
||||
representation: coords
|
||||
nan_policy: propagate
|
||||
|
||||
output:
|
||||
report: out/per_joint_report.json
|
||||
|
|
@ -100,6 +100,7 @@ nav:
|
|||
- neuropose.migrations: api/migrations.md
|
||||
- neuropose.benchmark: api/benchmark.md
|
||||
- neuropose.analyzer.segment: api/segment.md
|
||||
- neuropose.analyzer.pipeline: api/pipeline.md
|
||||
- neuropose.visualize: api/visualize.md
|
||||
- Development: development.md
|
||||
- Deployment: deployment.md
|
||||
|
|
|
|||
|
|
@ -0,0 +1,205 @@
|
|||
"""Example-config sanity integration tests.
|
||||
|
||||
Every YAML config under ``examples/analysis/`` is exercised here in
|
||||
two ways:
|
||||
|
||||
1. ``test_<name>_parses`` — :func:`load_config` accepts the YAML,
|
||||
i.e. the file matches the current :class:`AnalysisConfig` schema
|
||||
including cross-field invariants. Catches silent drift between the
|
||||
example configs and the schema they claim to exercise.
|
||||
2. ``test_<name>_runs`` — the example's pipeline runs end-to-end
|
||||
against synthetic predictions. Paths in the YAML are overwritten
|
||||
with test fixtures before :func:`run_analysis` is invoked;
|
||||
everything else (stages, thresholds, triplets) is used verbatim.
|
||||
|
||||
These tests are deliberately not marked ``slow`` — they use synthetic
|
||||
fixtures and do not touch the MeTRAbs SavedModel, so they run in the
|
||||
default unit-test suite.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import math
|
||||
from pathlib import Path
|
||||
|
||||
import numpy as np
|
||||
import pytest
|
||||
|
||||
from neuropose.analyzer.pipeline import (
|
||||
AnalysisConfig,
|
||||
AnalysisReport,
|
||||
DtwResults,
|
||||
load_config,
|
||||
run_analysis,
|
||||
)
|
||||
from neuropose.analyzer.segment import JOINT_INDEX
|
||||
from neuropose.io import VideoPredictions, save_video_predictions
|
||||
|
||||
EXAMPLES_DIR = Path(__file__).resolve().parents[2] / "examples" / "analysis"
|
||||
|
||||
NUM_JOINTS = 43
|
||||
|
||||
|
||||
def _sinusoid(num_cycles: int, frames_per_cycle: int, amplitude: float = 100.0) -> np.ndarray:
|
||||
total = num_cycles * frames_per_cycle
|
||||
t = np.linspace(0.0, num_cycles * 2.0 * math.pi, total, endpoint=False)
|
||||
return (np.sin(t) * amplitude + amplitude).astype(float)
|
||||
|
||||
|
||||
def _write_trial(
|
||||
path: Path,
|
||||
*,
|
||||
num_cycles: int = 4,
|
||||
frames_per_cycle: int = 30,
|
||||
seed: int = 0,
|
||||
) -> Path:
|
||||
"""Write a synthetic VideoPredictions with every joint oscillating.
|
||||
|
||||
Joint ``0`` gets a reproducible RNG-driven trace so Procrustes has
|
||||
something non-degenerate to align. All other joints get their own
|
||||
phase-shifted sinusoid so joint-angle triplets and per-joint DTW
|
||||
have signal to act on.
|
||||
"""
|
||||
rng = np.random.default_rng(seed)
|
||||
base = _sinusoid(num_cycles, frames_per_cycle)
|
||||
total = base.shape[0]
|
||||
frames: dict[str, dict] = {}
|
||||
for frame_idx in range(total):
|
||||
poses = [[[0.0, 0.0, 0.0] for _ in range(NUM_JOINTS)]]
|
||||
for j in range(NUM_JOINTS):
|
||||
# Unique per-joint position so no triplet is degenerate.
|
||||
phase = rng.uniform(0.0, 2.0 * math.pi)
|
||||
amplitude = 30.0 + 10.0 * (j % 5)
|
||||
offset = float(j) * 15.0
|
||||
poses[0][j][0] = offset + amplitude * math.cos(
|
||||
2.0 * math.pi * frame_idx / frames_per_cycle + phase
|
||||
)
|
||||
poses[0][j][1] = offset * 0.5 + base[frame_idx] + 5.0 * j
|
||||
poses[0][j][2] = 3.0 * j
|
||||
frames[f"frame_{frame_idx:06d}"] = {
|
||||
"boxes": [[0.0, 0.0, 1.0, 1.0, 0.9]],
|
||||
"poses3d": poses,
|
||||
"poses2d": [[[0.0, 0.0]] * NUM_JOINTS],
|
||||
}
|
||||
preds = VideoPredictions.model_validate(
|
||||
{
|
||||
"metadata": {
|
||||
"frame_count": total,
|
||||
"fps": float(frames_per_cycle),
|
||||
"width": 640,
|
||||
"height": 480,
|
||||
},
|
||||
"frames": frames,
|
||||
}
|
||||
)
|
||||
save_video_predictions(path, preds)
|
||||
return path
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def example_fixtures(tmp_path: Path) -> tuple[Path, Path, Path]:
|
||||
"""Return (primary_path, reference_path, report_path) under ``tmp_path``."""
|
||||
primary = _write_trial(tmp_path / "primary.json", seed=1)
|
||||
reference = _write_trial(tmp_path / "reference.json", seed=2)
|
||||
report = tmp_path / "report.json"
|
||||
return primary, reference, report
|
||||
|
||||
|
||||
def _rewrite_paths(
|
||||
example_path: Path, primary: Path, reference: Path, report: Path
|
||||
) -> AnalysisConfig:
|
||||
"""Load an example YAML and rewrite inputs/output paths to fixtures.
|
||||
|
||||
Tests run against synthetic predictions in ``tmp_path``; the
|
||||
example YAML's hardcoded ``data/*.json`` paths would never resolve
|
||||
otherwise.
|
||||
"""
|
||||
config = load_config(example_path)
|
||||
update: dict = {
|
||||
"inputs": config.inputs.model_copy(update={"primary": primary, "reference": reference}),
|
||||
"output": config.output.model_copy(update={"report": report}),
|
||||
}
|
||||
return config.model_copy(update=update)
|
||||
|
||||
|
||||
class TestMinimalExample:
|
||||
def test_minimal_parses(self) -> None:
|
||||
config = load_config(EXAMPLES_DIR / "minimal.yaml")
|
||||
assert isinstance(config, AnalysisConfig)
|
||||
assert config.analysis.kind == "dtw"
|
||||
assert config.segmentation is None
|
||||
|
||||
def test_minimal_runs(self, example_fixtures: tuple[Path, Path, Path]) -> None:
|
||||
primary, reference, report = example_fixtures
|
||||
config = _rewrite_paths(EXAMPLES_DIR / "minimal.yaml", primary, reference, report)
|
||||
result = run_analysis(config)
|
||||
assert isinstance(result, AnalysisReport)
|
||||
assert isinstance(result.results, DtwResults)
|
||||
# Unsegmented → one distance.
|
||||
assert result.results.segment_labels == ["full_trial"]
|
||||
assert len(result.results.distances) == 1
|
||||
|
||||
|
||||
class TestPaperCExample:
|
||||
def test_paper_c_parses(self) -> None:
|
||||
config = load_config(EXAMPLES_DIR / "paper_c_headline.yaml")
|
||||
assert config.analysis.kind == "dtw"
|
||||
assert config.segmentation is not None
|
||||
assert config.segmentation.kind == "gait_cycles_bilateral"
|
||||
# Joint triplets must be in range for berkeley_mhad_43.
|
||||
assert config.analysis.kind == "dtw"
|
||||
angle_triplets = config.analysis.angle_triplets # type: ignore[union-attr]
|
||||
assert angle_triplets is not None
|
||||
for a, b, c in angle_triplets:
|
||||
for idx in (a, b, c):
|
||||
assert 0 <= idx < NUM_JOINTS
|
||||
|
||||
def test_paper_c_runs(self, example_fixtures: tuple[Path, Path, Path]) -> None:
|
||||
primary, reference, report = example_fixtures
|
||||
config = _rewrite_paths(EXAMPLES_DIR / "paper_c_headline.yaml", primary, reference, report)
|
||||
result = run_analysis(config)
|
||||
assert isinstance(result.results, DtwResults)
|
||||
# Bilateral segmentation → distances labelled per side.
|
||||
assert any(lbl.startswith("left_heel_strikes") for lbl in result.results.segment_labels)
|
||||
assert any(lbl.startswith("right_heel_strikes") for lbl in result.results.segment_labels)
|
||||
|
||||
def test_paper_c_uses_documented_knee_triplets(self) -> None:
|
||||
"""The Paper C config must target knee-flexion joint triplets.
|
||||
|
||||
Safety net: if someone edits the YAML and breaks the joint
|
||||
references, this test catches it before the example silently
|
||||
starts computing the wrong angles.
|
||||
"""
|
||||
config = load_config(EXAMPLES_DIR / "paper_c_headline.yaml")
|
||||
assert config.analysis.kind == "dtw"
|
||||
triplets = config.analysis.angle_triplets # type: ignore[union-attr]
|
||||
assert triplets is not None
|
||||
# Left knee flex = hip → knee → ankle.
|
||||
assert (
|
||||
JOINT_INDEX["lhipb"],
|
||||
JOINT_INDEX["lkne"],
|
||||
JOINT_INDEX["lank"],
|
||||
) in triplets or (
|
||||
JOINT_INDEX["lhipf"],
|
||||
JOINT_INDEX["lkne"],
|
||||
JOINT_INDEX["lank"],
|
||||
) in triplets
|
||||
|
||||
|
||||
class TestPerJointDebugExample:
|
||||
def test_per_joint_debug_parses(self) -> None:
|
||||
config = load_config(EXAMPLES_DIR / "per_joint_debug.yaml")
|
||||
assert config.analysis.kind == "dtw"
|
||||
assert config.analysis.method == "dtw_per_joint" # type: ignore[union-attr]
|
||||
|
||||
def test_per_joint_debug_runs(self, example_fixtures: tuple[Path, Path, Path]) -> None:
|
||||
primary, reference, report = example_fixtures
|
||||
config = _rewrite_paths(EXAMPLES_DIR / "per_joint_debug.yaml", primary, reference, report)
|
||||
result = run_analysis(config)
|
||||
assert isinstance(result.results, DtwResults)
|
||||
# dtw_per_joint → per_joint_distances populated.
|
||||
assert result.results.per_joint_distances is not None
|
||||
# Inner length must match num_joints for the coords
|
||||
# representation.
|
||||
for per_seg in result.results.per_joint_distances:
|
||||
assert len(per_seg) == NUM_JOINTS
|
||||
Loading…
Reference in New Issue