why base rates matter

Here are three recent scientific findings you may or may not have heard about:

1. The use of stimulant medications commonly prescribed for ADHD is associated with a nearly 8-fold increase in the likelihood of dying suddenly among children aged 7 – 19.

2. Gum disease increases the risk of head and neck cancer quite dramatically: for every millimeter of alveolar bone loss (i.e., loss of the bone that surrounds the roots of your teeth), there is a 400% increase in the risk of cancer (note: article requires paid access).

3. People who talk on a cell phone while driving are 1.3 times more likely to have an accident than people who drive without any distractions.

At a cursory glance, all three of these stories seem like pretty bad news. And they are. But one of them is actually much worse than the others. Your job is to decide which one; take a moment to think about it, then read on.

If you’re like most people, you probably picked either the first or the second story. After all, it’s pretty terrible to think of children dying suddenly, or of getting cancer of the head and neck. Sudden death implies death for certain, and cancer implies death with a high probability. Most of us generally don’t see death as a good thing, so we want to avoid those outcomes. Car accidents aren’t anyone’s idea of a good time, of course; but at least most car accidents aren’t fatal. And then there’s the matter of the differing odds to consider: in the first story, the negative outcome is 8 times as likely, and in the second, it’s 4 times as likely, but in the third story, it’s only 1.3 times as likely. Surely then, it’s more important to avoid taking stimulant drugs and to brush and floss regularly than to worry about talking on a cell phone!

Well, as you might have guessed from the fact that I started the previous paragraph with “if you’re like most people…”, the truth is actually somewhat counterintuitive. The fact of the matter is that, even if the above stories are completely true (and as far as I know, they are, pending further research), turning off your cell phone when you drive is probably a much, much better way to minimize your chance of dying early than swearing off stimulants or practicing great oral hygiene (though the latter is still important!). The reason is that the information I gave you in the three stories above neglects what’s probably the most important piece of of all to consider: the base rate (or frequency) of each event occurring.

Let’s add some context to each of the three stories. Take the first one. It’s true (at least based on one preliminary study) that kids who take stimulant medications are much more likely to die suddenly than kids who don’t. But the critical thing to consider is the base rate of sudden death. You probably won’t be surprised to hear that the odds of dying suddenly are incredibly low when you’re 7 – 19 years old. It’s unclear exactly how low they are, but consider that the study that reported this finding scoured state databases between the years of 1985 and 1996 and still only came up with 564 cases of sudden death. That’s a tiny, tiny, tiny fraction of the number of kids who make it past 19 years of age in good health. Suppose we say that the probability of sudden death for a kid in this age range is 0.0001% per year. An eightfold increase would mean that the average kid goes from a one in a million chance to just under a one in a hundred thousand chance of dying per year. And of course, it’s not average kids for whom stimulant medications are prescribed; usually, there’s a condition (e.g., ADHD) that the drugs are intended to alleviate. When you weigh the increase in the negligible likelihood of sudden death against the very sizeable benefits conferred by stimulant medications, it’s clear that this finding isn’t really cause for alarm. As John Grohol notes, “The finding is of greater interest in trying to understand why it’s occurring at all, not for anyone to make a treatment decision based upon it.”

What about the second story? Well, you can probably already see where this is going. Head and neck cancer is quite rare, accounting for fewer than 50,000 new cases per year in the United States. In other words, approximately one in every 6000 people will develop head and neck cancer. This of course includes both people who have good oral hygiene and people who don’t, so the reality is that, even if you have terrible oral hygiene and rampant gum disease, you’re very unlikely to ever develop head and neck cancer. Conversely, there are other factors that present even greater risk factors for head and neck cancer than gum disease (e.g., smoking). This isn’t to say that you shouldn’t brush your teeth, of course; there are plenty of other good reasons to take good care of your gums. It’s just to say that you shouldn’t lose any sleep over the prospect of developing head and neck cancer because of your gums. In the grand scheme of things, there are any number of other things you should be much more concerned about.

One of the things you should be much more concerned about, actually, is your risk of having a car accident while talking on your cell phone. Unlike sudden death in children and head and neck cancers in adults, the odds of dying in a car accident are not very small. Worldwide, approximately 2% of deaths every year are caused by road accidents. And that’s to say nothing about serious injuries sustained in non-fatal accidents. Put simply, a 1.3-fold increase in the likelihood of enduring car accidents is not trivial. If we do a back-of-the-envelope calculation and assume that the odds of dying in a car accident increase by the same proportion (i.e., that drivers on cell phones don’t have more serious accidents than drivers off cellphones–which is debatable), it turns out that you can reduce your overall odds of dying in any given year by about 0.6% just by not talking on your cell phone while driving. Admittedly, that’s a very loose estimate that’s based on questionable data and many simplifying assumptions. And it’s not like it’s a dramatic reduction by any stretch (which only goes to further illustrate the importance of considering base rates). But the point is, there are probably relatively few lifestyle change you could make this year that would require so little effort for such a large benefit. So take your ADHD meds, brush your teeth regularly, and don’t talk on your cell phone while driving.

For a nice overview of empirical data on the base rate fallacy, see this article in BBS. For more blogospheric bloviation on base rates, see here, here, and here.