Turn a spreadsheet of prompts and responses into evals

Not everyone has a tracing platform. A CSV of prompts and responses is a perfectly good starting point, and it's the format most teams actually have when they first ask what's going wrong.

Drop your traces here

JSONL · JSON · NDJSON · CSV · TSV · TXT · GZ · ZIP

How your columns are read

  1. 1Wide format: one row per exchange — an input column (prompt, input, question, query…) and an output column (response, output, completion, answer…).
  2. 2Long format: one row per message — a conversation id (conversation_id, session_id, thread_id), a role (or from), and content. Rows are grouped and ordered for you.
  3. 3Ground truth: expected, ideal, or reference columns are kept, and label, score, correct, or passed become the labels that unlock judge-accuracy numbers.
  4. 4JSON in a cell: if a cell holds a message array, it's parsed as one — a chat log wearing a CSV costume still gets full tool-call handling.
  5. 5Tabs and semicolons work too, and Excel users can save as CSV.

What you get back

A self-contained HTML report: the failure modes discovered in your own traces (not a generic checklist), a heatmap of which traces hit which mode, the exact quoted evidence behind every verdict, and the judge's own reliability numbers. Plus downloadable evals — a promptfoo config, an OpenAI Evals data file, and a portable judge suite — so the analysis turns into a regression suite you keep.

Related