neuwirth/levineuwirth.org
neuwirth
/
levineuwirth.org
1
1
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

auto: 2026-08-29T17:32:52Z [skip ci]

This commit is contained in:
Levi Neuwirth 2026-08-29 19:32:52 +02:00
parent 49d6d24fd8
commit b837ac2813
No known key found for this signature in database
1 changed files with 5 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

5
content/work.md
View File

@ -75,6 +75,11 @@ models, execute models against them, debug agent scaffolds when they fail, and
analyze the failures that result. I have led environment design and evaluator analyze the failures that result. I have led environment design and evaluator
calibration; most of it is built in [Harbor](https://github.com/harbor-framework/harbor). calibration; most of it is built in [Harbor](https://github.com/harbor-framework/harbor).
Calibration passes often reveal that a rubric is measuring something other than
what the task was designed to measure: a task-design failure, not a grading one.
I calibrate before a task is scaled, because redesigning a task is cheap and
rescoring a finished run is not.
For a similar methodology on public data, see [The Specification For a similar methodology on public data, see [The Specification
Dilemma](/essays/specification-dilemma/) — pre-registered, matched-pairs, with Dilemma](/essays/specification-dilemma/) — pre-registered, matched-pairs, with
its instrumentation failures reported in full. its instrumentation failures reported in full.