A surprising share of the "the model isn't working" calls we get turn out to be data problems that surfaced downstream, dressed up as modeling problems. A short, disciplined audit before any modeling starts catches most of these early, when they're cheap to fix, instead of after you've built an analysis on top of them.
The checklist
- Is there a codebook or data dictionary? If nobody can say with confidence what every variable means and how it was coded, that has to be resolved before analysis, not discovered during it.
- What does the missingness actually look like? Not just "how much is missing," but whether it's missing at random or concentrated in specific groups, waves, or conditions — which determines what you're allowed to do about it.
- Are there duplicate records? Especially in merged or multi-source data, the same unit can enter more than once and silently inflate your sample size.
- Have outliers been looked at, not just removed? A value that looks extreme might be a data-entry error, a genuinely unusual but valid case, or a sign the variable was coded differently for a subset of records. Each implies a different fix.
- Is coding consistent across waves, sites, or collectors? The same variable name doesn't guarantee the same coding scheme if data was collected in stages or by different teams.
- Does the achieved sample match the intended population? Compare who you meant to sample against who actually responded — attrition and non-response rarely happen randomly.
- Is the unit of analysis actually clear? Especially with nested or longitudinal data, confirm what one row represents before writing a single line of model code.
Why this is worth doing even under a deadline
It's tempting to skip straight to modeling when time is short, but an hour spent auditing data almost always saves more than an hour later — because the alternative is discovering a coding inconsistency or a duplicate-records problem after the results are already written up, at which point it's not a data-cleaning task anymore, it's a credibility problem.
If you'd rather have a second set of eyes run this audit before you build on the data, that's exactly where we start every analysis engagement.