GET/1.0/caliper/evals/:id

Returns the eval + the dataset items it's bound to. Designed for the CI-scoring loop: one fetch gives the script everything it needs to enumerate items, run inference, and build the submit-run payload — no second roundtrip.

Required scope: caliper:evals:read (scopes reference).

Path parameters

idstringRequired

The eval's id. Get it from the Caliper dashboard URL: caliper.zerowidth.ai/w/<slug>/evals/<id>. The key's workspace must own the eval.

Response (200)

The response carries the eval's own metadata + rubric snapshot, extended with a dataset block that carries the bound dataset's id + items inline.

idstring

The eval's id. Same as the path parameter.

namestring

Human-readable name (e.g. "Refund-tone rubric × support-prompts").

descriptionstring | null

Optional description the author set at creation.

status"ACTIVE" | "ARCHIVED"

ACTIVE evals accept new runs. Archived evals reject new runs with 409 conflict.

visibility"PRIVATE" | "WORKSPACE" | "SHARED"

Within-workspace visibility. Workspace-wide keys can read everything in the workspace, so this field is informational for them rather than an access gate.

datasetIdstring

The bound dataset's id. Same as dataset.id below — included at the top level so the eval object stands alone.

rubricIdstring

The live rubric's id (the one the snapshot was taken from). Not the snapshot — see rubricSnapshot for the frozen criteria this eval scores against.

rubricSnapshotobject

Frozen-at-create-time copy of the rubric. The shape is { rubricId, name, description, criteria: { id, name, scaleMin, scaleMax, description? }[] }. The LLM judge reads this snapshot — not the live rubric — so editing the rubric later won't change how this eval scores.

taskConfigobject

The "what's being tested" config. Discriminated by kind:

{ kind: "workbench_flow", flowId, flowRevisionId } — Caliper runs inference itself. You don't need this API for these.
{ kind: "external", label } — Caliper doesn't run inference; you submit actuals via the submit-run endpoint. The API surface exists for this case.

label is a free-text identifier for what produces the actuals (e.g. "CI", "deployed-prod") — purely human-readable, not enforced.

totalRunsinteger

How many runs have been submitted against this eval so far.

completedRunsinteger

How many of those reached DONE.

latestScorenumber | null

Most-recent DONE run's overallScore (normalized 0–1), or null if no run has completed yet. Useful for "is this passing?" at-a-glance status checks.

datasetobject

The bound dataset's id + name + items inline.

dataset.id — string
dataset.name — string
dataset.items — array. Each item is discriminated by kind:
- { kind: "qa", id, input, expectedOutput? } — single Q&A pair. Feed input into your model; expectedOutput is the reference for scoring.
- { kind: "chat", id, messages, expectedResponse? } — multi-turn input. messages is the transcript to feed; expectedResponse is the reference reply.
- { kind: "raw", id, payload, metadata? } — opaque payload. The script knows how to interpret payload based on the dataset's documented schema.

createdAtstring

ISO-8601 timestamp.

updatedAtstring

ISO-8601 timestamp.

Example

curl https://api.zerowidth.ai/1.0/caliper/evals/eval_8a2c91... \
  -H "Authorization: Bearer zw_live_abc12345_..."

{
  "id": "eval_8a2c91...",
  "name": "Refund-tone rubric × support-prompts",
  "description": "Tracks tone + accuracy on our top-100 refund prompts.",
  "status": "ACTIVE",
  "visibility": "WORKSPACE",
  "datasetId": "ds_3f12ab...",
  "rubricId": "rub_77c4de...",
  "rubricSnapshot": {
    "rubricId": "rub_77c4de...",
    "name": "Tone + accuracy",
    "description": null,
    "criteria": [
      { "id": "crit_a", "name": "Accuracy", "scaleMin": 1, "scaleMax": 5 },
      { "id": "crit_b", "name": "Tone",     "scaleMin": 1, "scaleMax": 5 }
    ]
  },
  "taskConfig": { "kind": "external", "label": "CI" },
  "totalRuns": 7,
  "completedRuns": 6,
  "latestScore": 0.82,
  "dataset": {
    "id": "ds_3f12ab...",
    "name": "Support prompts (top 100)",
    "items": [
      {
        "kind": "qa",
        "id": "item_001",
        "input": "What's your refund policy?",
        "expectedOutput": "30 days from purchase, full refund."
      },
      {
        "kind": "qa",
        "id": "item_002",
        "input": "Can I return a digital download?",
        "expectedOutput": "Digital downloads are not refundable."
      }
    ]
  },
  "createdAt": "2026-05-19T18:42:11.000Z",
  "updatedAt": "2026-05-20T09:15:03.000Z"
}

Errors

Endpoint-specific error codes on top of the shared error envelope:

Status	`code`	When
`404`	`not_found`	The eval id doesn't exist in the key's workspace, or it's not visible to the key. The response is the same either way.
`403`	`forbidden`	A Personal key was used. Mint a Workspace-wide key with the `ci_evals` preset and retry.

Auth + scope codes (401, 403 scope_missing) come from the shared envelope.

Troubleshooting

403 forbidden — personal key used

A Personal key was used. This endpoint accepts only Workspace-wide keys — mint one with the ci_evals preset.

404 not_found

The eval id doesn't exist in the key's workspace, or isn't visible to it. Grab the id from the Caliper dashboard URL (caliper.zerowidth.ai/w/<slug>/evals/<id>), and confirm the key belongs to that workspace.

4 min read