When Should You Involve a Statistician? (Earlier Than You Think)

The most common way we meet a new client is: the data collection is finished, the deadline is close, and something about the analysis isn't cooperating. Sometimes that's a quick fix. Often, though, the real problem was decided months earlier — during design — and by the time it shows up in the analysis, it can't be undone with a better model. It can only be worked around, disclosed as a limitation, or in the worst case, it sinks the study's ability to answer its own question.

The decisions that are cheap to fix before data collection and expensive after

• Sample size and power. Underpowered studies can't be rescued by a fancier model after the fact. If you find out post hoc that you needed 300 clusters and you have 40, no analysis choice changes that.
• What gets measured, and how. A construct measured with a single ad hoc item can't retroactively become a validated scale. If the outcome that matters wasn't measured, or was measured in a way that won't hold up, that's a design problem showing up as an analysis problem.
• Randomization and assignment. Whether you randomize at the individual or cluster level determines which models are even valid later — and it can't be changed retroactively.
• What comparison group exists. Without a plausible comparison or baseline, many analyses can describe what happened but can't support a causal claim about why.

A rough rule of thumb

If a decision would be expensive or impossible to reverse once you've started collecting data — sample size, measurement choice, randomization, what comparison group you'll have — that's a design decision, and it's worth a conversation before you commit to it, not after. If a decision is about how to model data you already have, that's usually fixable at the analysis stage, and later involvement is fine.

What an early conversation actually looks like

It doesn't have to be a large engagement. A short design consult — a few hours, sometimes one call — covering sample size, measurement, and the analysis plan can save far more time and money than it costs, simply by preventing decisions that can't be revisited later.

If you're still designing the study, this is exactly the stage we're built for — before the data locks in decisions you can't take back.

One email whenever we publish something new. No spam, unsubscribe anytime.