Base Rate Fallacy
Easy Definition of Base Rate Fallacy: Don't think "99% accurate" means a 1% failure rate. There's far more to think about before you can work out the failure rate. This idea is linked to the Base Rate Fallacy.
Geeky Definition of Base Rate Fallacy: The Base Rate Fallacy is an error in reasoning which occurs when someone reaches a conclusion that fails to account for an earlier premise – usually a base rate, a probability or some other statistic.
(Help! I'm still confused. Look at the examples below and then read the definitions above again. Hopefully, they will become clearer.)
An Example of Base Rate Fallacy
This machine is useless because it's only 99% accurateImagine you have a machine that can detect whether coins are real or fake.
Your machine is pretty good at this. When it checks a coin, it only gets it wrong 1% of the time. 99% of the time it makes the right decision.
Of course, there are two types of error the machine can make:
- It can say that a false coin is real (a false positive).
- It can say that a real coin is false (a false negative).
If you ran the machine until it spurted out a coin it believed to be a fake, what would be the chance of that coin actually being a fake? If you said 99%, that's almost certainly wrong. It could be much lower. The truth is you just don't know. Here's why:
Imagine your machine checked 10,010 coins, 10 of which were fake.
In checking the 10,000 real coins, the machine would wrongly identify 1% of them as fakes. In other words, it would get 100 coins wrong, and place them in its fake pile. Let's imagine the machine found all 10 of the fake coins, and correctly put them in the fake pile. We would now have 110 coins in the fake pile, but only 10 of them would be actual fakes. In this example, the chance of any coin being an actual fake would be less than 10%, even though your machine is "99% accurate".
If you originally thought 99%, your answer was derived through fallacious reasoning because an earlier premise (the tiny probability of the coins being fake in the first place, i.e., the base rate) was not taken into account.
A Practical Application for Base Rate Fallacy
Know what those cameras will do for youDoes knowing about Base Rate Fallacy have a practical application? Well, it could. Face-detection software is starting to appear in all sorts of places, and you might think it's the silver bullet to your particular problem. It could well be, but don't forget it depends on what you're testing for and the percentage of those things among everything else you're testing.
For example, if your shop buys a "99% accurate" face-detection camera to spot known shoplifters, your store detectives might find themselves following hundreds of people around the store who have no intention of shoplifting. The usefulness of such a system depends as much on the proportion of shoplifters in our society as it does on the accuracy of the system.
What about cameras to spot terrorists? What proportion of the population are terrorists? Minuscule. There would be thousands of false arrests if face-detection cameras that weren't 100% accurate were deployed at places like airports. Of course, this doesn't mean the cameras wouldn't be useful. It just means complementary procedures would need to be introduced. And that would be useful to know before spending millions on equipping an airport with such a capability.
Another Practical Application for Base Rate Fallacy
Give them 33% and tell them it's 50%Lots of food companies exploit the Base Rate Fallacy on their packaging. When something says "50% extra free," only a third (33%) of what you're looking at is free. If you think half of what you're looking at is free, then you've committed the Base Rate Fallacy.
For example, when you buy six cans of Coke labelled "50% extra free," only two of the cans are free, not three. (It's because the original pack had four cans, and 50% of the original amount is two cans.)
If you thought three of the cans were free, then you failed to account for an earlier premise (i.e., there were four cans originally), and you committed the Base Rate Fallacy.
Summary of Base Rate Fallacy: If you think someone has taken a fact at face value without factoring in a key supporting premise (like a probability or a base rate), tell him he has committed the Base Rate Fallacy.