From b837ac2813f077317d122f3eb85fc43deeff3f20 Mon Sep 17 00:00:00 2001 From: Levi Neuwirth Date: Sat, 29 Aug 2026 19:32:52 +0200 Subject: [PATCH] auto: 2026-08-29T17:32:52Z [skip ci] --- content/work.md | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/content/work.md b/content/work.md index ea8389f..93cd436 100644 --- a/content/work.md +++ b/content/work.md @@ -75,6 +75,11 @@ models, execute models against them, debug agent scaffolds when they fail, and analyze the failures that result. I have led environment design and evaluator calibration; most of it is built in [Harbor](https://github.com/harbor-framework/harbor). +Calibration passes often reveal that a rubric is measuring something other than +what the task was designed to measure: a task-design failure, not a grading one. +I calibrate before a task is scaled, because redesigning a task is cheap and +rescoring a finished run is not. + For a similar methodology on public data, see [The Specification Dilemma](/essays/specification-dilemma/) — pre-registered, matched-pairs, with its instrumentation failures reported in full.