Benchmarking

AI agent benchmarking, answer by answer

A benchmark space brings together your questions, a single set of expected answers and as many result datasets as agents or systems you want to compare.

01

Automatic background runs

Each question is sent to the chosen Dust agent. Answer, response time and credits fill in line by line, with adjustable parallelism and retry of failures.

02

AI evaluation, human control

Correct, Partial or Incorrect verdict with a reason. Only meaning counts. A second look re-checks doubtful cases and every verdict can be overridden by hand.

03

Question complexity

Each question gets a Simple, Medium or Complex level, shared by every dataset: you see exactly where your RAG agents fall short.

04

Compare test sets across platforms

Pit test sets from Dust and results imported from other platforms against each other, on the same questions. Strict and weighted scores, speed, cost, and a filter on differences.

05

Detailed reports and ranking

Generate a detailed report ranking every test set, with summary and charts, enriched with your own recommendations, regenerable and exportable to PDF. Everything is kept in history.

06

Annotate results and differences

Annotate a question, a result or a gap between platforms, flag it for review and filter annotated lines in tables and CSV exports.

Your AI agents deserve serious measurement

Connect your Dust workspace and get a first inspection and a first benchmark within minutes.