What is Survivorship Bias?
“Twenty years ago,” says the inspirational speaker, “I was sitting in that audience where you are now. And look at me today. That’s proof that you can make it too if you don’t give up hope.”
This speaker is either suffering from survivorship bias, or cynically exploiting it in their audience. Presumably there were thousands of other people sitting in the audience twenty years ago, and only one of them is standing on the stage today. For any given person in the audience today, it is more likely that they will end up like one of the forgotten thousands than like the one success story.
Survivorship bias means considering only the examples which made it through some kind of filter or selection process, and ignoring the ones which didn’t, and then drawing unjustified conclusions or generalizations from the “survivors”.
Examples of Survivorship Bias
Success in Business or Entertainment
If you want to start a billion-dollar business, it’s not enough to look at Google or Amazon and copy what they did. Amazon did certain things and succeeded big-time, but many other companies you’ve never heard of did the same things and failed, or perhaps achieved very modest success. That’s why you’ve never heard of them. You’re getting a false picture if you only consider the success stories. Similarly, if you want to be a star in music, movies or sport, it’s not enough to read the biography of your favorite star and copy what they did.
Music, Movies and Books
The curmudgeon who says the music, movies or books of today aren’t as good as the ones of the distant past is exhibiting survivorship bias. We see all the media of today, good and bad; but we only see the best of the media of yesterday, because the lower-quality examples have been justly forgotten.
Military researchers in World War II studied the patterns of damage on aircraft caused by enemy fire, with a view to adding extra reinforcement to the areas of aircraft that were typically damaged. But a statistician called Abraham Wald pointed out that the patterns of damage they were seeing indicated the places where an aircraft could take damage and still survive. The aircraft that got hit in more critical areas never made it home to be studied. The original researchers were exhibiting survivorship bias, by only considering the aircraft that survived combat. Wald suggested that, instead, aircraft should be reinforced in the areas where returning aircraft tended not to be damaged. Assuming that enemy fire was approximately likely to hit an aircraft anywhere, Wald was able to infer that the places where returning aircraft didn’t show damage were the places where an aircraft would be destroyed if hit, so those were the places which needed extra reinforcement.
Stock Market Scammers
Here is a strategy some scammers use. Pick a stock, and email half of a large mailing list with a prediction that the stock will go up tomorrow, and the other half with a prediction that it will go down tomorrow. If it goes up, stop emailing the second half of the list, and split the first half of the list in half again, and email one half with a prediction that a stock will go up and the other half with a prediction that it will go down. Repeat every day for a week.
By the end of a week, the scammer will have a group of people, 1/128 the size of the original mailing list, who have received seven correct predictions in a row by pure chance. It doesn’t matter that the other 127/128 of the list have received at least one incorrect prediction; they have ceased to be the target of the scam. The “lucky” 1/128 who have received seven correct predictions, if they are not savvy, will exhibit survivorship bias by not realizing they are part of a small group that has survived seven binary filters, and will think the series of seven correct predictions means that the scammer has some special and valuable insight into the stock market. Then they may be persuaded to give the scammer lots of money to subscribe to future updates or buy his book.
Survivorship Bias as a Logical Fallacy
Survivorship bias can arise from the logical fallacy of affirming the consequent.
The probability of event A given event B isn’t the same as the probability of event B given event A. If 10% of CEOs are women, that doesn’t mean 10% of women are CEOs. To make this more obvious, consider men: maybe 90% of CEOs are men, but that definitely doesn’t mean 90% of men are CEOs!
Similarly, maybe you’ve noticed that 90% of world-famous guitarists practiced for two hours every day, so you think practicing for two hours every day is a good route to becoming a famous guitarist. But the converse doesn’t hold: it’s not true that 90% of aspiring guitarists who practice for two hours every day go on to become world-famous. Maybe only 1% do, or even fewer.