How to Respond When a Reviewer Challenges Your Statistical Model

"Reviewer 2" critiquing a model is one of the most common reasons researchers call us, and it's rarely because the original analysis was wrong. More often, the model was defensible but under-explained, or a genuinely reasonable alternative wasn't ruled out in the manuscript. Either way, the response matters as much as the original analysis — a well-handled revision often strengthens a paper more than a clean first submission does.

First, identify what kind of challenge it actually is

• "Why this model and not that one?" — a specification question. The fix is usually to run the alternative and report both, showing the conclusion is (or isn't) sensitive to the choice.
• "You're missing a confound." — an omitted-variable concern. If the variable is available, add it and report the change. If it isn't, say so explicitly and discuss the likely direction of bias rather than ignoring the point.
• "Your assumptions don't hold." — a diagnostics question. This is answered with evidence (residual plots, robustness checks), not argument.
• "I don't believe this effect is real." — often really a request for a sensitivity analysis: does the result survive a different model, a different sample restriction, a different way of handling missing data?

The response pattern that works

Concede what's genuinely correct, run the analysis the reviewer is implicitly asking for, and report the result plainly — even when it doesn't fully agree with you. A response that runs the requested robustness check and reports "the effect is somewhat smaller under this specification, but the direction and significance hold" reads as more credible than a response that argues the original model was fine. Reviewers are typically not trying to kill the paper; they're trying to find out whether the result is fragile. Show them, don't tell them.

When to push back instead of conceding

Not every critique is correct. If a reviewer is factually wrong about what a method assumes, or is asking for something inappropriate to your design (e.g., a fixed-effects model that would absorb the very variation you're testing), it's appropriate to explain why — briefly, with citations, and without getting defensive. The goal either way is the same: give the editor evidence to evaluate, not just an assertion that you're right.

If you're facing a revise-and-resubmit and want a second set of eyes on the model itself, this is the kind of work we do constantly.

One email whenever we publish something new. No spam, unsubscribe anytime.