What Belongs in a Statistical Analysis Plan?

A statistical analysis plan (SAP) is a document that specifies how you'll analyze your data, written and finalized before you see the outcomes — ideally before data collection ends, or before unblinding in a trial. Its purpose is narrow but important: it draws a hard line between analyses you planned and analyses you decided on after seeing what the data looked like, which is exactly the distinction a skeptical reviewer, funder, or reader wants to be able to trust.

What a solid SAP specifies in advance

• Primary and secondary outcomes, named specifically — not "we'll look at academic outcomes" but the exact measure and how it's scored.
• The primary analysis model, including which covariates are included and why, and what the unit of analysis is.
• How missing data will be handled — which method, and under what assumption.
• Any subgroup or sensitivity analyses, specified as planned rather than discovered — with an explicit note that anything not listed here is exploratory if it appears in the results.
• How multiplicity will be handled if there are multiple outcomes or subgroups being tested, so a real correction is applied rather than decided after seeing which comparisons came out significant.
• Stopping rules or interim analysis plans, if applicable — particularly relevant for trials with a data safety monitoring component.

Why it matters even for a paper, not just a registered trial

SAPs are standard practice in clinical trials, but the same logic applies to any funded study or evaluation where credibility depends on the reader believing the reported analysis wasn't chosen after the fact. A pre-specified analysis plan is one of the simplest things you can point to when a reviewer asks whether a result is the product of the design or the product of trying several models until one looked good.

A practical note on timing

The SAP doesn't have to be a long document, and it can be refined up until the point you see outcome data — the requirement is that the key decisions are locked in before you know how they'll turn out, not that the plan is perfect on the first draft.

If you're writing an analysis plan for a grant, trial registration, or IRB submission, this is a document we write and review constantly.

One email whenever we publish something new. No spam, unsubscribe anytime.