Regression

Code has locality. Change a function and the effects are roughly limited to its callers. A prompt has none — change one sentence in a system prompt and you have changed behavior on every input, including the ones that sentence had nothing to do with.

The usual shapes are familiar once you have seen them. Adding "be concise" also truncates the answers that needed length. Adding "always offer next steps" produces next steps on the refusals. Tightening a refusal rule declines adjacent legitimate requests.

None of these appear in the diff, and none of them appear in a headline score if most of your test cases pass regardless. They appear in the list of items that changed verdict between two runs, which is why that list is the thing worth reading after a change rather than the average.

Regressions run in both directions on refusals in particular. Over-refusal and under-refusal alternate, usually because each one gets fixed by loosening or tightening the same rule.