Blog topic

Study Design

For researchers, principal investigators, grant teams, and organizations planning a study. These articles cover power and sample size, statistical analysis plans, missing data, pilots versus full RCTs, reproducible analysis, and what reviewers and IRBs expect to see — the decisions that are cheap to get right before the data arrive and expensive to fix after.

Browse topics

All articles Study Design Program Evaluation AI Evaluation Statistical Consulting Data Literacy

Featured

{{ p.readingTime }} min read

Study Design

The Beatles Song That Made People Younger

4 min read

A 2011 study made a Beatles song lower people's reported age by 1.5 years (p=.04). A real case study in p-hacking, and how ordinary choices can fake a result.

Read post →

Study Design

Building Publication-Ready Statistical Figures

2 min read

A figure that misrepresents your own data's uncertainty undermines a paper more than an awkward color choice ever will. Here are the rules that actually affect whether a figure is trustworthy, not just tidy.

Read post →

Study Design

Why Bayesian Methods Are Often the Better Fit for Small Samples

2 min read

Standard frequentist tests lean on large-sample approximations that a 15-person pilot doesn't have. Bayesian methods don't need that assumption — here's what changes, and what doesn't.

Read post →

Study Design

What an IRB Wants to See in Your Methods Section

2 min read

IRB reviewers aren't grading your statistics — they're checking whether the risks are justified by the design. Here's what a methods section needs to make that case clearly.

Read post →

Study Design

Writing Reproducible R Code for Academic Research

2 min read

A model that only runs on your machine, in your session, with your undocumented fixes, isn't reproducible — and journals and funders are increasingly asking for proof that it is. Here's what actually makes R code reproducible.

Read post →

Study Design

When Does a Pilot Study Need to Become a Full RCT?

2 min read

There's no headcount threshold. The moment anyone starts treating a pilot's numbers as evidence the intervention worked, the question has outgrown the design meant to answer it — here's how to tell, and what the next stage actually requires.

Read post →

Study Design

When Is a Sample Too Small to Bother Analyzing?

3 min read

There's no universal cutoff, but there are principled ways to tell before you run a single test — and a few situations where a small sample is still worth analyzing honestly.

Read post →

Study Design

The Preregistration That Didn't Change a Thing

4 min read

A 2024 analysis of nearly 16,000 test statistics found that preregistration alone doesn't reduce p-hacking. What actually worked was far more specific than a checkbox.

Read post →

Statistical ConsultingStudy Design

When Should You Involve a Statistician? (Earlier Than You Think)

2 min read

Most researchers call a statistician after the data is collected. By then, the decisions that would have mattered most — sample size, measurement, randomization — are already locked in.

Read post →

Study Design

How to Respond When a Reviewer Challenges Your Statistical Model

2 min read

A reviewer questioning your model isn't automatically a rejection. It's a request for evidence — and how you respond matters more than whether the original model was perfect.

Read post →

Study Design

Statistical Power for Cluster-Randomized Trials: Why More Participants Isn't Enough

2 min read

Randomizing schools, clinics, or teams instead of individuals means your effective sample size is much smaller than your headcount suggests — and a power analysis that ignores this will underestimate what you need.

Read post →

Study Design

FIML vs. Multiple Imputation: Which Missing-Data Method Should You Use?

2 min read

Both handle missing data under the same core assumption, but they work differently and fit different situations. Here's how to decide which one your analysis actually needs.

Read post →

Study Design

What Belongs in a Statistical Analysis Plan?

2 min read

A statistical analysis plan written before you see the data is one of the cheapest ways to protect a study's credibility — and reviewers, funders, and IRBs increasingly expect to see one.

Read post →

Study Design

The Antidepressant That Worked 94% of the Time — On Paper

4 min read

Published trials made a class of antidepressants look almost uniformly effective. The FDA had the trials that never got published, and the real picture was a coin flip. A case study in publication bias, and why the literature you can read is not the literature that was run.

Read post →

Study DesignProgram Evaluation

The Program That Looked Like It Worked — Until Someone Checked

3 min read

Scared Straight felt like it worked: kids were shaken, parents saw a difference, the numbers looked good. The best evidence says it made things worse. Here's why a good story keeps fooling careful people, and what to ask for instead.

Read post →

{{ p.readingTime }} min read

Read post →

Planning a study?

We help research teams with design, power analysis, and analysis plans before data collection starts.

Book a free consult Support for researchers

← Back to all articles

Get new posts by email

One email whenever we publish something new. No spam, unsubscribe anytime.

Check your inbox — click the confirmation link to finish subscribing.

Study Design

Browse topics

Featured

{{ p.title }}

{{ listHeading }}

{{ p.title }}