Skip to main content
ZeroWidthdocs
Get started

Reviews

A review is what happens when you put a dataset together with a rubric and a group of raters. Each rater scores each item against each criterion; Caliper aggregates the result.

Reviews have their own Reviews tab in the Caliper header — a gallery of every review in the workspace, with search, tags and bulk actions like the other galleries. A dataset's page links to the reviews that use it.

Creating a review

  1. Open the Reviews tab and click New review

    A three-step wizard opens.

  2. Pick the dataset and the rubric

    Any dataset and rubric you can see in the workspace. Caliper snapshots the rubric onto the review at this moment — future edits to the live rubric don't change this review's results.

  3. Name it and choose who can see it

    A name, a description, and a visibility setting. Then it's live and ready for raters.

The rating form

When a rater opens a review (via invite or share link), they land on a focused per-item form:

  • One item displayed at a time, in the dataset's native shape (raw text, Q&A, or a chat transcript).
  • The review's snapshot rubric below — every criterion, with its scale and description visible.
  • Next / Finish buttons that advance the rater forward; the review's progress bar updates as ratings land.

Raters can stop and resume — their progress sticks per item. They don't need to finish in one session.

Reading results

The review overview page is the home for results. It has four parts:

  1. Primary actions

    Review (or View ratings once the review is complete), Share, Invite. Same row on every review so the next action is always one click away.

  2. Highlights band

    Three callouts: lowest-scored item overall, items with highest rater disagreement, and worst-by-criterion. Tells you where to look first.

  3. Per-item breakdown

    Sortable table — every dataset item is a row, with mean / min / stddev / rating count / raters columns. Sort by mean to see the best and worst items; sort by stddev to find the items raters disagreed about.

  4. Raters strip

    Avatars and names of everyone who's contributed.

Status

Reviews are either In progress or Complete. Complete just means the review owner closed it off — there's no automatic transition. Closing is final for new ratings: a complete review is read-only for raters, so close it once everyone has had their say.

What persists, what doesn't

Where to go next

3 min read