← Back to blog
AI Evaluation

How Many Test Items Does an AI Eval Need?

5 min read

Most eval sets are sized by what was easy to collect: a few hundred prompts someone wrote, a benchmark's test split, whatever the labeling budget covered. The right size depends on what the eval has to answer. "How accurate is this model?" and "Is the new version better than the old one?" are different questions, and the second usually needs several times more items.

Question 1: how accurate is the model?

An accuracy score measured on a sample of items is an estimate, and its margin of error shrinks only with the square root of the number of items. For a model that gets about 80% right, the 95% margin of error is roughly:

  • 100 items: ±7.8 points (the true accuracy could be anywhere from about 71% to 87%)
  • 200 items: ±5.5 points
  • 500 items: ±3.5 points
  • 1,000 items: ±2.5 points
  • 2,000 items: ±1.8 points

To plan, work backwards: pick the margin you can live with and solve for the number of items, n = 1.96² × p(1 − p) ÷ margin², where p is the accuracy you expect. For accuracy near 80%, a ±5-point margin takes 246 items, ±3 points takes 683, and ±2 points takes 1,537. Near 90% you need fewer (139, 385 and 865), because scores close to the ceiling vary less. If you have no idea what to expect, plan for 50%, the worst case: 385, 1,068 and 2,401 items.

The practical consequence: on a 200-item eval, a reported 80% is consistent with a true accuracy anywhere from about 74% to 85%. Reporting it as 80.5% implies a precision the eval doesn't have.

Question 2: is model A better than model B?

Comparing two models is where most evals come up short, because the comparison depends on a different number: how often the two models disagree. When both models answer the same items, every item they both get right, or both get wrong, says nothing about which is better. The comparison rests entirely on the items where one was right and the other wrong. (That's why the comparison should be paired: run both models on the same items and compare them item by item. Our guide to comparing two models walks through the test.)

So the number of items depends on two inputs: the smallest accuracy gap you need to detect, and the share of items on which the models will disagree. For an 80% chance of detecting the gap at the usual 5% significance level (Connor, 1987):

  • 5-point gap, models disagree on 15% of items: about 470 items
  • 3-point gap, 10% disagreement: about 870 items
  • 3-point gap, 20% disagreement: about 1,740 items
  • 2-point gap, 10% disagreement: about 1,960 items
  • 2-point gap, 5% disagreement: about 980 items
  • 1-point gap, 5% disagreement: about 3,900 items

We checked each of these by simulation, using the same test as our free model comparison calculator. Every one came out at 80% power, give or take a point.

Two patterns stand out. Small gaps are expensive: halving the gap you want to detect roughly quadruples the items. And more disagreement means more items for the same gap, not fewer: when two models disagree often, a small net difference is harder to tell apart from chance.

To estimate the disagreement rate, run both models on a pilot of a few hundred items and count. The accuracy gap on a pilot is too noisy to plan with, but the disagreement rate is estimated well enough; to be safe, plan with the upper end of its likely range.

Pairing is also what keeps these numbers manageable. If the two models were scored on different item sets, detecting 80% versus 83% would take about 2,600 items per model. Scoring both on the same items is the cheapest way to make an eval more sensitive.

When your items aren't independent

All of these numbers assume every item is a separate, independent test. Eval sets often aren't: five questions about the same document, several paraphrases of one prompt, turns from one conversation. Items in a group tend to succeed or fail together, so they carry less information than the same number of independent items.

The standard correction is the design effect, 1 + (m − 1) × ICC, where m is the number of items per group and the ICC (intraclass correlation) measures how alike items in the same group are. Take 500 items built from 100 documents with 5 questions each, and a modest ICC of 0.2: the design effect is 1.8, so the eval carries about as much information as 280 independent items, and the ±3.5-point margin grows to ±4.7. Either collect more groups (more documents beats more questions per document) or report clustered standard errors, as Miller (2024) recommends for language model evals.

Other things that change the count

  • Sampling randomness. If outputs are generated at a temperature above zero, the same item can pass on one run and fail on the next. Running each item several times reduces that noise, but it doesn't replace more items: repeated runs of one item are a cluster too.
  • Subgroups. If you need a reliable score for each category (each language, product area or customer segment), each category needs enough items on its own. Five categories at ±5 points means about 250 items each, not 250 in total.
  • Many comparisons. Testing ten prompt variants against a baseline raises the chance of a false win. Budget more items, or hold out a fresh set to confirm the winner.

A short planning checklist

  • Write down the question the eval must answer: one model's accuracy, or a difference between models.
  • Pick the margin of error, or the smallest gap, that would change a decision.
  • For comparisons, estimate the disagreement rate from a small pilot.
  • If items share a source, count the groups, not just the items.
  • Report a margin of error or confidence interval next to every score.

If you're sizing an eval that has to hold up to a customer, an auditor or your own leadership, this is the kind of evaluation design we help AI teams with.

Related reading

Sources

  • Connor, R. J. (1987). Sample size for testing differences in proportions for the paired-sample design. Biometrics, 43(1), 207–211.
  • Miller, E. (2024). Adding error bars to evals: A statistical approach to language model evaluations. arXiv:2411.00640.
Get new posts by email

One email whenever we publish something new. No spam, unsubscribe anytime.

Check your inbox — click the confirmation link to finish subscribing.

{{ subscribeErrorMsg }}