Blog topic

Study Design

For researchers, principal investigators, grant teams, and organizations planning a study. These articles cover power and sample size, statistical analysis plans, missing data, pilots versus full RCTs, reproducible analysis, and what reviewers and IRBs expect to see — the decisions that are cheap to get right before the data arrive and expensive to fix after.

Browse topics

All articles Statistical Analysis Survey Design & Measurement Study Design Program Evaluation AI Evaluation Statistical Consulting Data Literacy

Featured

{{ p.readingTime }} min read

Data LiteracyStudy Design

The Oscar Winners Who Couldn't Have Died Young

4 min read

Oscar winners outlived their peers by nearly four years, a famous 2001 study found. Most of that gap came from when the clock started.

Read post →

Study Design

How to Choose an Effect Size for a Power Analysis

3 min read

Start with a difference that matters, examine what is plausible, and show how uncertainty changes the sample-size requirement.

Read post →

Study Design

How to Plan a Feasibility Study: Outcomes and Progression Criteria

3 min read

A feasibility study should resolve uncertainties about doing the next study. Define the operational questions, measurements, and progression decisions before starting.

Read post →

Study Design

Which Reporting Guideline Does Your Study Need?

5 min read

CONSORT, SPIRIT, JARS-Quant, SAMPL, TRIPOD+AI, BARG and ADEMP: what each reporting guideline is for, what they share, and how to stack them.

Read post →

Program EvaluationStudy Design

How Much Attrition Is Too Much?

3 min read

Losing a third of your participants can be fine. Losing a tenth can sink the evaluation. What matters is who left, and whether they left evenly.

Read post →

Study DesignAI EvaluationSurvey Design & Measurement

How Do You Measure Inter-Rater Reliability?

3 min read

Percent agreement and correlation can look excellent when raters barely agree. How to choose between Cohen's kappa, Krippendorff's alpha, and the ICC.

Read post →

Statistical ConsultingStudy Design

Should Your Statistician Be a Co-Author?

3 min read

Paying a statistician doesn't make them an author, and neither does running your numbers. Here's where the line actually falls, and how to settle it early.

Read post →

Study Design

How to Write a Power Analysis for a Grant

3 min read

Reviewers don't just want your sample size, they want to rebuild it. What a grant's power analysis must include, and where most of them fall apart.

Read post →

Study Design

The Beatles Song That Made People Younger

4 min read

A 2011 study made a Beatles song lower people's reported age by 1.5 years (p=.04). A real case study in p-hacking, and how ordinary choices can fake a result.

Read post →

Study Design

Building Publication-Ready Statistical Figures

2 min read

A figure that misrepresents your own data's uncertainty undermines a paper more than an awkward color choice ever will. Here are the rules that actually affect whether a figure is trustworthy, not just tidy.

Read post →

Study DesignStatistical Analysis

Why Bayesian Methods Are Often the Better Fit for Small Samples

2 min read

Standard frequentist tests lean on large-sample approximations that a 15-person pilot doesn't have. Bayesian methods don't need that assumption — here's what changes, and what doesn't.

Read post →

Study Design

What an IRB Wants to See in Your Methods Section

2 min read

IRB reviewers aren't grading your statistics — they're checking whether the risks are justified by the design. Here's what a methods section needs to make that case clearly.

Read post →

Study Design

Writing Reproducible R Code for Academic Research

2 min read

A model that only runs on your machine, in your session, with your undocumented fixes, isn't reproducible — and journals and funders are increasingly asking for proof that it is. Here's what actually makes R code reproducible.

Read post →

Study Design

When Does a Pilot Study Need to Become a Full RCT?

2 min read

There's no headcount threshold. The moment anyone starts treating a pilot's numbers as evidence the intervention worked, the question has outgrown the design meant to answer it — here's how to tell, and what the next stage actually requires.

Read post →

Study Design

When Is a Sample Too Small to Bother Analyzing?

3 min read

There's no universal cutoff, but there are principled ways to tell before you run a single test — and a few situations where a small sample is still worth analyzing honestly.

Read post →

Study Design

The Preregistration That Didn't Change a Thing

4 min read

A 2024 analysis of nearly 16,000 test statistics found that preregistration alone doesn't reduce p-hacking. What actually worked was far more specific than a checkbox.

Read post →

Statistical ConsultingStudy Design

When Should You Involve a Statistician? (Earlier Than You Think)

2 min read

Most researchers call a statistician after the data is collected. By then, the decisions that would have mattered most — sample size, measurement, randomization — are already locked in.

Read post →

Study DesignStatistical Analysis

How to Respond When a Reviewer Challenges Your Statistical Model

2 min read

A reviewer questioning your model isn't automatically a rejection. It's a request for evidence — and how you respond matters more than whether the original model was perfect.

Read post →

Study Design

Statistical Power for Cluster-Randomized Trials: Why More Participants Isn't Enough

2 min read

Randomizing schools, clinics, or teams instead of individuals means your effective sample size is much smaller than your headcount suggests — and a power analysis that ignores this will underestimate what you need.

Read post →

Study DesignStatistical Analysis

FIML vs. Multiple Imputation: Which Missing-Data Method Should You Use?

2 min read

Both handle missing data under the same core assumption, but they work differently and fit different situations. Here's how to decide which one your analysis actually needs.

Read post →

Study DesignStatistical Analysis

What Belongs in a Statistical Analysis Plan?

2 min read

A statistical analysis plan written before you see the data is one of the cheapest ways to protect a study's credibility — and reviewers, funders, and IRBs increasingly expect to see one.

Read post →

Study Design

The Antidepressant That Worked 94% of the Time — On Paper

4 min read

Published trials made a class of antidepressants look almost uniformly effective. The FDA had the trials that never got published, and the real picture was a coin flip. A case study in publication bias, and why the literature you can read is not the literature that was run.

Read post →

Study DesignProgram Evaluation

The Program That Looked Like It Worked — Until Someone Checked

3 min read

Scared Straight felt like it worked: kids were shaken, parents saw a difference, the numbers looked good. The best evidence says it made things worse. Here's why a good story keeps fooling careful people, and what to ask for instead.

Read post →

{{ p.readingTime }} min read

Read post →

Planning a study?

We help research teams with design, power analysis, and analysis plans before data collection starts.

Book a free consult Support for researchers

← Back to all articles

Get new posts by email

One email whenever we publish something new. No spam, unsubscribe anytime.

Check your inbox — click the confirmation link to finish subscribing.

Study Design

Browse topics

Featured

{{ p.title }}

{{ listHeading }}

{{ p.title }}