Evaluate a flow with Caliper

A flow that looks good in the test pane can still be wrong on the long tail. Caliper is how you turn "looks good" into a score against cases you trust — and Workbench links straight into it so you don't lose the thread.

1. 1
   Create the eval from your flow

   On the flow's detail page, open the ⋯ menu and choose Create eval in Caliper. It drops you into Caliper's new-eval wizard with this flow already set as the target — Caliper will run the flow itself on each item. Pick a dataset and a rubric, and save.

2. 2
   Run it and read the score

   Run the eval. Caliper feeds each dataset item through your flow and scores every output against every rubric criterion with an LLM judge. You get an overall score plus a per-item, per-criterion heatmap — where it's weak, not just that it scored 0.78.

3. 3
   Watch it from Workbench

   Back in the flow's dev drawer, the Evals tab lists every Caliper eval bound to this flow with its latest score (green / amber / rose) and run counts. A Run now button kicks a fresh eval run and shows progress in-row — so you can re-check the flow after an edit without leaving the editor.

Feed good runs back in

Found a run worth keeping — a great answer, or a revealing failure? From the Runs tab, bookmark it to push it into a Caliper dataset. Your eval set grows from real traffic instead of made-up cases.