Part of our Statistical paradoxes series · The German tank problem
In early 1943, a small team of economists at the Economic Warfare Division of the American Embassy in London was given a question that sounded like it belonged to spies: how many tanks is Germany building each month? The Allies had plenty of answers already. Conventional intelligence estimates put German tank output at well over a thousand a month at times. If that was true, it changed everything from bombing priorities to the timing of an invasion of Europe.
Richard Ruggles and Henry Brodie took a different approach. Instead of asking people how many tanks they thought existed, they looked at the tanks the Allies had already captured or destroyed. German manufacturing was orderly, and its parts were stamped with serial numbers: gearboxes, chassis, engines, even the road wheels. Those numbers ran in sequences. Once you have a handful of numbers drawn from a sequence, you can estimate how long the sequence is. The team started with tires, then moved on to tanks, trucks, guns, and eventually flying bombs and rockets. After the war, when Germany's own production records became available, the two sets of numbers could finally be compared. For June 1940, intelligence had estimated 1,000 tanks; the serial-number method said 169; German records said 122. For August 1942, intelligence said 1,550; the statisticians said 327; the records said 342. Ruggles and Brodie wrote the whole thing up in the Journal of the American Statistical Association in 1947, and the puzzle has been called the German tank problem ever since.
Here is the idea with small, illustrative numbers. Say tanks are numbered 1, 2, 3 and so on up to some unknown total, and you capture five of them with serial numbers 14, 22, 37, 51 and 60. What's your best guess for the total?
The total can't be less than 60, because you've seen a 60. But it's probably a little more, because it would be a coincidence if you happened to capture the very last tank off the line. How much more? Your five numbers split the range from 1 to 60 into gaps, and the average gap is roughly 60 divided by 5, or 12. It's reasonable to assume there's a gap about that size sitting above your highest number too. The standard estimate works out to the largest number, plus the largest number divided by the sample size, minus one: 60 + 12 − 1 = 71.
That's the whole trick. Nothing about it needs a spy, a defector or a lucky photograph. It only needs two things to be true: the numbers really were assigned in sequence, and the tanks you captured are roughly a random draw from everything that was built. When those assumptions hold, a small sample can beat a room full of experienced analysts.
And it's worth asking why the analysts were so far off. The error wasn't random: in both of those months it pointed the same way, toward a bigger, scarier enemy. Estimates built from impressions and scattered reports tend to drift toward whatever the people making them already fear or expect, and nobody's impression comes with a margin of error. The serial numbers had no opinion about the war.
When someone hands you a big number about something nobody has counted, ask one question: what is this estimate actually built on? If the answer is expert judgment, reports or a general impression, treat it as a hypothesis. If it's built from a sample with a known relationship to the whole, and it comes with a range, you have something you can use. The best estimate in the room is rarely the one from the most confident person. It's the one whose method you can check.
If you need to estimate something you can't count directly, such as program reach, a hidden population or how often a model really fails, and want a number that will hold up to a skeptical reviewer, talk to us.
One email whenever we publish something new. No spam, unsubscribe anytime.
{{ subscribeErrorMsg }}