DASS Blog Statistics, study design, and evaluation guides from Data Analysis & Statistical Solutions. https://statisticalsolutions.org/blog 2026-10-19T12:00:00Z DASS — Data Analysis & Statistical Solutions The Hot Hand That Was Real All Along https://statisticalsolutions.org/blog/the-hot-hand-that-was-real-all-along 2026-10-19T12:00:00Z The hot hand fallacy, explained: why the 1985 study missed real streaks, and how to check any streak or sequence analysis for selection bias. When AI-Written Code Runs, What Has It Proved? https://statisticalsolutions.org/blog/when-ai-written-code-runs 2026-10-18T12:00:00Z How to validate data analyses written by coding agents: state the estimand, assert join counts, recover known parameters, and know what a pass can't prove. Read NIH Critiques Before Revising Methods https://statisticalsolutions.org/blog/read-nih-critiques-before-revising-methods 2026-10-18T12:00:00Z How to read an NIH summary statement, including 'competitive but not discussed,' and turn reviewer critiques into evidence-based methods decisions. How Measurement Error Shrinks Regression Slopes https://statisticalsolutions.org/blog/measurement-error-in-regression 2026-10-18T12:00:00Z See how predictor measurement error shrinks a regression slope, with a verified R and Python simulation, clear assumptions, and options for better measurement. The Women Who Were Healthy Before the Hormones https://statisticalsolutions.org/blog/the-women-who-were-healthy-before-the-hormones 2026-10-17T12:00:00Z Healthy user bias explained through hormone therapy: why observational studies showed heart protection a randomized trial did not, and how to avoid it. Do You Need a Multiple Comparisons Correction? https://statisticalsolutions.org/blog/do-you-need-a-multiple-comparisons-correction 2026-10-17T12:00:00Z When to correct for multiple comparisons, when you don't need to, and how to choose between Bonferroni, Holm, Benjamini–Hochberg and gatekeeping. The Oscar Winners Who Couldn't Have Died Young https://statisticalsolutions.org/blog/the-oscar-winners-who-couldnt-have-died-young 2026-10-12T12:00:00Z Immortal time bias explained through the Oscar longevity study: how counting years before an event inflates survival gaps, and how to fix it. How to Choose an Effect Size for a Power Analysis https://statisticalsolutions.org/blog/how-to-choose-effect-size-for-power-analysis 2026-10-10T12:00:00Z Choose a defensible effect size for a power analysis using meaningful differences, prior evidence, uncertainty, and sensitivity scenarios. How to Plan a Feasibility Study: Outcomes and Progression Criteria https://statisticalsolutions.org/blog/how-to-plan-a-feasibility-study 2026-10-10T12:00:00Z Plan a feasibility study around recruitment, retention, delivery, measurement, and prespecified progression criteria for the next study. How to Choose a Statistical Consultant for Your Research Project https://statisticalsolutions.org/blog/how-to-choose-a-statistical-consultant 2026-10-10T12:00:00Z Choose a statistical consultant by checking relevant experience, proposed deliverables, reproducibility, communication, and fit for your research project. What Should a Statistical Consulting Proposal Include? https://statisticalsolutions.org/blog/what-should-a-statistical-consulting-proposal-include 2026-10-10T12:00:00Z A practical statistical consulting proposal checklist covering scope, deliverables, data readiness, revisions, fees, timelines, and reproducibility. How to Interpret Logistic Regression Results https://statisticalsolutions.org/blog/how-to-interpret-logistic-regression-results 2026-10-10T12:00:00Z Interpret logistic regression coefficients, odds ratios, and predicted probabilities, with a worked example and practical reporting checklist. How to Interpret Interaction Effects in Regression https://statisticalsolutions.org/blog/how-to-interpret-interaction-effects-in-regression 2026-10-10T12:00:00Z Interpret regression interactions with a worked example, simple slopes, predicted outcomes, uncertainty, and guidance for clear reporting. Cronbach’s Alpha vs. Omega: What Should You Report? https://statisticalsolutions.org/blog/cronbachs-alpha-vs-omega 2026-10-10T12:00:00Z Cronbach’s alpha vs. McDonald’s omega: choose and report reliability estimates for questionnaire scores without treating a cutoff as validation. EFA vs. CFA: Which Does Your Questionnaire Need? https://statisticalsolutions.org/blog/efa-vs-cfa-questionnaire 2026-10-10T12:00:00Z EFA vs. CFA for questionnaires: when to explore or test factor structure, how to plan the analysis, and what to report. Can You Compare Survey Scores Across Different Groups? https://statisticalsolutions.org/blog/comparing-survey-scores-across-groups 2026-10-10T12:00:00Z Can you compare survey scores across groups? Learn what measurement invariance addresses and how to plan defensible group and time comparisons. Mixed Models vs. Repeated-Measures ANOVA: Which Should You Use? https://statisticalsolutions.org/blog/mixed-models-vs-repeated-measures-anova 2026-10-10T12:00:00Z Mixed models vs. repeated-measures ANOVA: choose an approach for repeated observations, missing visits, unequal timing, and your intended comparison. How to Analyze Pre/Post Data With a Comparison Group https://statisticalsolutions.org/blog/how-to-analyze-pre-post-data-with-a-comparison-group 2026-10-10T12:00:00Z Analyze pre/post data with a comparison group: understand change scores, ANCOVA, longitudinal models, missing data, and limits on causal interpretation. How to Validate a Questionnaire for Your Organization https://statisticalsolutions.org/blog/how-to-validate-a-questionnaire 2026-10-10T12:00:00Z How to validate questionnaire scores for your organization: define the intended use, review items, test understanding, and plan measurement evidence. Can You Change a Validated Questionnaire? https://statisticalsolutions.org/blog/can-you-change-a-validated-questionnaire 2026-10-10T12:00:00Z Can you shorten, translate, or adapt a validated questionnaire? Learn what changes need review and how to plan evidence for the revised scores. How Much Does a Nonprofit Program Evaluation Cost? https://statisticalsolutions.org/blog/how-much-does-nonprofit-program-evaluation-cost 2026-10-10T12:00:00Z What drives nonprofit program evaluation costs, what to include in your budget, and how to compare evaluator proposals for the evidence you need. How to Write a Nonprofit Evaluation Plan—with an Example https://statisticalsolutions.org/blog/how-to-write-nonprofit-evaluation-plan 2026-10-10T12:00:00Z Write a nonprofit evaluation plan with clear questions, measures, responsibilities, and reporting. Includes a free example and printable template. The Erasers That Raised Atlanta's Test Scores https://statisticalsolutions.org/blog/the-erasers-that-raised-atlantas-test-scores 2026-10-10T12:00:00Z Goodhart's law and Campbell's law through Atlanta's test-cheating scandal: why a metric that becomes a target stops measuring, and how to guard against it. An Active AI Tutor Isn't Yet an Effective One https://statisticalsolutions.org/blog/an-active-ai-tutor-isnt-yet-an-effective-one 2026-10-07T12:00:00Z How to evaluate an AI tutor pilot: separate usage from learning, use an outcome students complete without the tutor, and plan the comparison first. Which Reporting Guideline Does Your Study Need? https://statisticalsolutions.org/blog/which-reporting-guideline-does-your-study-need 2026-10-07T12:00:00Z Which reporting guideline fits your study? CONSORT 2025, SPIRIT 2025, JARS-Quant, SAMPL, TRIPOD+AI, BARG and ADEMP compared, and how to combine them. The Poll of 2.4 Million That Picked the Loser https://statisticalsolutions.org/blog/the-poll-of-2-4-million-that-picked-the-loser 2026-10-06T12:00:00Z How the 1936 Literary Digest poll got 2.4 million answers and the wrong winner: a case study in nonresponse bias and why sample size can't fix it. How Much Attrition Is Too Much? https://statisticalsolutions.org/blog/how-much-attrition-is-too-much 2026-10-06T12:00:00Z How much attrition is too much in a program evaluation: why differential dropout matters more than the overall rate, and how to handle and report it. The Best Schools Were Small. So Were the Worst. https://statisticalsolutions.org/blog/the-best-schools-were-small-so-were-the-worst 2026-10-05T12:00:00Z Why small schools, counties, and eval slices top and bottom every ranking: the law of small numbers, the Gates small-schools case, and how to check. How Do You Measure Inter-Rater Reliability? https://statisticalsolutions.org/blog/how-to-measure-inter-rater-reliability 2026-10-05T12:00:00Z How to measure inter-rater reliability: when to use Cohen's kappa, weighted kappa, Krippendorff's alpha, or an ICC, and the mistakes that undermine each. The 27 Programs That Failed Together https://statisticalsolutions.org/blog/the-27-programs-that-failed-together 2026-10-05T12:00:00Z Condorcet's jury theorem says majority votes get more accurate as voters are added, but only if their errors are independent. A 1986 software test shows why. Should Your Statistician Be a Co-Author? https://statisticalsolutions.org/blog/should-your-statistician-be-a-co-author 2026-10-05T12:00:00Z When a statistical consultant should be a co-author vs. acknowledged: the ICMJE criteria, clear cases on each side, and how to agree on authorship early. AI Can Think for You, But It Can't Understand for You https://statisticalsolutions.org/blog/ai-can-think-for-you-but-it-cannot-understand-for-you 2026-09-30T12:00:00Z AI can produce the reasoning behind an analysis, but understanding it is still your job. What the research shows, and a simple test for keeping understanding. The Bitter Lesson Has a Second Half https://statisticalsolutions.org/blog/the-bitter-lesson-has-a-second-half 2026-09-29T12:00:00Z Sutton's Bitter Lesson says compute beats human knowledge. But someone must still decide what counts as correct. Why AI evaluation doesn't scale with compute. The Test Questions That Were Failing the AI https://statisticalsolutions.org/blog/the-test-questions-that-were-failing-the-ai 2026-09-29T12:00:00Z AI benchmarks contain broken questions that distort model rankings. How item analysis from psychometrics finds them, and what to check in your own evals. The AI Users Who Scored 13 and Guessed 17 https://statisticalsolutions.org/blog/the-ai-users-who-scored-13-and-guessed-17 2026-09-29T12:00:00Z People using ChatGPT on LSAT problems scored about 13 of 20 and believed they got 17. Why AI answers feel like your own knowledge, and a four-question check. The Asylum Judges Who Couldn't Say Yes Twice https://statisticalsolutions.org/blog/the-asylum-judges-who-couldnt-say-yes-twice 2026-09-29T12:00:00Z How the gambler's fallacy skewed 150,000 asylum rulings, and how to check whether sequential ratings in your research, evaluation or AI eval are biased too. How to Write a Power Analysis for a Grant https://statisticalsolutions.org/blog/how-to-write-a-power-analysis-for-a-grant 2026-09-29T12:00:00Z How to write the power analysis in a grant proposal: what to include, where to get the effect size, attrition, and the mistakes reviewers flag most. The Serial Numbers That Counted Germany's Tanks https://statisticalsolutions.org/blog/the-serial-numbers-that-counted-germanys-tanks 2026-09-29T12:00:00Z The German tank problem explained through the real WWII case: how serial numbers from captured tanks beat intelligence estimates, and when the method works. Does Your Evaluation Need a Logic Model? https://statisticalsolutions.org/blog/does-your-program-evaluation-need-a-logic-model 2026-09-29T12:00:00Z What a logic model is, how evaluators use it to choose outcomes and explain null results, and the signs yours won't help your program evaluation. The Checks That Were Just Under $100,000 https://statisticalsolutions.org/blog/the-checks-that-were-just-under-100000 2026-09-29T12:00:00Z Benford's law explained through a real $2 million check fraud: why honest data starts with 1 so often, and how digit tests flag invented numbers. How to Choose Outcomes for a Program Evaluation https://statisticalsolutions.org/blog/how-to-choose-outcomes-for-a-program-evaluation 2026-09-29T12:00:00Z How to choose outcome measures for a nonprofit program evaluation: pick one primary outcome from your claim and avoid measures that can't move or be gamed. The Number That Convicted an Innocent Mother https://statisticalsolutions.org/blog/the-number-that-convicted-an-innocent-mother 2026-09-29T12:00:00Z The prosecutor's fallacy, explained through the Sally Clark case: why the chance of the evidence isn't the chance of guilt, and how to spot it in your data. How Many Test Items Does an AI Eval Need? https://statisticalsolutions.org/blog/how-many-test-items-does-an-ai-eval-need 2026-09-29T12:00:00Z How many items an AI eval needs depends on its question: one model's margin of error, or a gap between two models. Worked numbers, checked in R. How to Tell If One LLM Is Really Better Than Another https://statisticalsolutions.org/blog/how-to-tell-if-one-llm-is-really-better-than-another 2026-09-29T12:00:00Z A 3-point eval gain can be luck. How to test whether one LLM or prompt is really better: the paired test, a worked example, and common false wins. How to Vet an AI Vendor's Accuracy Claims https://statisticalsolutions.org/blog/how-to-vet-an-ai-vendors-accuracy-claims 2026-09-29T12:00:00Z A vendor's "95% accuracy" claim is only as good as the test set, scoring method, and baseline behind it. Ask for those before you believe it. The Answer a Thousand PhDs Called Wrong https://statisticalsolutions.org/blog/the-answer-a-thousand-phds-called-wrong 2026-09-29T12:00:00Z In 1990, Marilyn vos Savant gave the correct answer to a probability puzzle. Ten thousand readers, nearly a thousand with PhDs, insisted she was wrong. What Makes an AI Eval Defensible to Auditors? https://statisticalsolutions.org/blog/what-makes-an-ai-eval-defensible-to-auditors 2026-09-29T12:00:00Z A defensible AI eval documents its test data, scoring method, and failure cases so it holds up to outside scrutiny, not just a good accuracy score. The Test GPT-3 Had Already Studied For https://statisticalsolutions.org/blog/the-test-gpt-3-had-already-studied-for 2026-09-29T12:00:00Z In 2020, OpenAI checked whether GPT-3's benchmarks had leaked into its training data. For most, it didn't matter — for two, it clearly did. How to Respond to a Funder's Evidence Request https://statisticalsolutions.org/blog/how-to-respond-to-a-funders-evidence-request 2026-09-29T12:00:00Z The right response depends on which tier of evidence you actually have — outputs, a pre/post number, or a real comparison. Here's how to answer honestly. The Beatles Song That Made People Younger https://statisticalsolutions.org/blog/the-beatles-song-that-made-people-younger 2026-09-29T12:00:00Z A 2011 study made a Beatles song lower people's reported age by 1.5 years (p=.04). A real case study in p-hacking, and how ordinary choices can fake a result.