For a randomized study with a continuous outcome measured before the intervention and at one follow-up, baseline-adjusted follow-up analysis—often called ANCOVA—is commonly a useful starting point. With additional visits or clustering, a longitudinal or multilevel model may be more appropriate.
For a nonrandomized comparison, model choice is only part of the problem. Baseline differences, selection, outside events, and missing outcomes affect what you can claim. Statistical adjustment does not, by itself, turn a comparison into a randomized study.
Write the target comparison before running tests: for example, the difference in mean follow-up scores between assigned groups, accounting for baseline scores. Record whether assignment was randomized, who was eligible, when the baseline occurred, and whether the comparison group received other services.
Use the analysis-planning worksheet to identify the outcome, groups, observation schedule, clustering, and missing-data plan. Check the existing protocol or analysis plan before selecting a different method.
A significant pre/post change in one group and a nonsignificant change in the other does not establish that the changes differ. The between-group comparison must be estimated directly. Two p-values are not a test of their difference.
For a simple change-score analysis, calculate each participant's follow-up minus baseline score and compare those changes between groups. Report the estimated difference and its uncertainty, not just which group improved significantly.
The following numbers are invented to demonstrate arithmetic, not results from a client or fitted dataset. Imagine a randomized workshop study with these group means:
The unadjusted follow-up difference is 58 − 58 = 0 points. The difference in mean changes is 8 − 3 = 5 points. These are different summaries of the same hypothetical group means.
Now suppose a fitted ANCOVA had a common within-group baseline slope of 0.6. That slope is assumed solely for this illustration; it cannot be calculated from the group means above. The adjusted group difference would be:
Follow-up difference − slope × baseline difference = 0 − 0.6 × (50 − 55) = 3 points.
The three-point adjusted comparison differs from both the raw follow-up difference and the five-point change comparison. A change-score comparison is equivalent to fixing the baseline coefficient at 1 when written as a follow-up model. ANCOVA estimates that relationship rather than imposing it.
None of these arithmetic summaries provides a confidence interval or a p-value. Those require participant-level information and the fitted model's uncertainty.
A basic specification models follow-up score using group and baseline score. Examine whether the baseline relationship is adequately represented and whether a common slope is appropriate. If the group effect depends on baseline, specify that interaction and report contrasts at meaningful baseline values or an appropriate average contrast.
In randomized studies, pre-intervention prognostic adjustment can improve precision. Choose adjustment variables in advance rather than deciding based on which baseline differences happen to be significant. Avoid routine adjustment for variables measured after assignment that the intervention may have affected.
For a nonrandomized study, consider how people entered each group and which variables may confound the comparison. Change scores and ANCOVA can give different answers when groups differ at baseline. Neither is a universal correction for selection.
With several follow-ups, define whether you want a contrast at a particular visit, a trajectory, or an average over a specified period. A longitudinal model can account for within-person dependence while representing those time points. The mixed-model guide shows why a group-by-time coefficient is not automatically the same as a final-visit group difference.
If assignment or sampling is clustered by school, clinic, or site, the analysis must address that dependence. Treating every participant as independent can misstate uncertainty.
Report how many participants have baseline and follow-up information in each group. A basic complete-case ANCOVA excludes records missing required variables; its validity depends on the missingness mechanism. Longitudinal likelihood methods and multiple imputation also require assumptions and appropriate model specification.
Do not assign zero change to everyone missing follow-up or assume that carrying the last score forward removes bias. Plan the primary approach and sensitivity analyses around plausible reasons for missingness.
For nonrandomized evaluations, a difference-in-differences approach needs a credible parallel-trends argument and attention to other threats. One pre-intervention measurement cannot establish that assumption. See difference-in-differences versus pre/post comparisons for the design question behind the model.
State the design, analysis population, model, adjustment variables, group and observation counts, target comparison, uncertainty, and missing-data assumptions. Explain whether the finding supports a causal interpretation and what could change the conclusion.
If you need help matching the analysis to your design, DASS provides research analysis and methods support. A blank data dictionary, assignment description, and analysis question are useful starting materials; identifying participant data is not needed for an initial inquiry.
One email whenever we publish something new. No spam, unsubscribe anytime.
{{ subscribeErrorMsg }}