Are Substantial Improvements in Air Passenger Security Readily Available?
In my vigorous debate with Michael Heath (both here and elsewhere) about the value and legitimacy of the Transportation Security Agency’s new backscatter scanners and the enhanced pat downs for those who opt-out of the scanner, he has argued that we need to focus on the marginal increase in security this technique can provide. Michael notes that he doesn’t have good data on how much improvement these methods will produce. Despite this, he is willing to at least provisionally employ them, a point which I harshly criticize* given the civil liberties invasions the methods entail. Before even beginning to entertain whether surrendering such liberties is an acceptable trade-off, I’d have to have some some persuasive data that the gains in security are substantial. Here I present a thumbnail sketch of the numbers on passenger airline terrorist incidents to argue that substantial increases in security may not even be available.
Michael employs a useful term from his background in the manufacturing sector, “defect rate,” which I like and will use here, defining a defect rate as the number of flights in the U.S. that experience an attempted terrorist attack. Notice that I could use the number of flights that experience a successful terrorist attack, which since 9/11 would be zero (try getting a marginal improvement on a zero defect rate!) because we can argue that current screening methods forced would-be terrorists to use materials and methods that were awkward enough that they gave passengers time to react. There’s a legitimate argument–and personally I do make that argument–that this is a satisfactory outcome. By that token we could actually even argue that pre-9/11 screening methods were sufficient, and the actual policy problem was not in our pre-flight security procedures but in our response procedures which encouraged passengers to sit tight and not take matters into their own hands. I have also made that argument and still stand by it.
But I want to make a conservative argument here, skewing the data away from my personal outcome and toward a conclusion in support of employing these enhanced screening methods. Because if they cannot be supported using data skewed in their favor, then they simply are not justified. So for the sake of argument here, let’s treat 9/11 as a screening failure–a defect–and consider the defect rate since then under the post 9/11 security procedures.
First we need to establish the baseline; how many flights since 9/11? Data from the Bureau of Transportation Statistics shows 81,052,327 from the beginning of 2002 through August of 2010. This baseline begins as about a 5% undercount since it leaves out post 9/11 flights in 2001 and is just out of date enough to not include almost 3 months of 2010, about 6 of 112 post-9/11 months to date. But let’s discount it even more by knocking and additional 10% of the counted number, bringing our baseline down to 72,947,000 and since I like to play with round numbers for mathematical simplicity, I’ll round that down even further, to 70,000,000 flights since 9/11.
How many actual attempts to commit a terrorist attack on or with a plane have there been since 9/11? I know of Richard Reid, the shoe-bomber, and Umar Farouk Abdulmutallab, the underwear-bomber. But neither actually got on board their aircraft at a U.S. airport. Looking at Wikipedia’s rather impressive record of terrorist incidents, I see not one other incident of an attempted attack on or with a commercial airliner in the U.S.. So it appears we have a defect rate that is 0 per 70 million. But again, let’s be conservative. Let’s say there has actually been one every single month since 9/11, or 112 incidents so far. That gives us a defect rate of 112 per 70,000,000, or 0.0000016, or 1.6 per million.
Analysis Assuming a 90% Improvement in Defect Rate
1.6 per million, which is unrealistically high because I’ve cooked the books, still sounds like a pretty good defect rate. But let’s say the enhanced screening process is 90% effective. I pulled that number out of my…err, hat, thinking that a) a 90% improvement from any single process change would normally be considered phenomenally successful; b) it’s surely an overestimate, since we’re certainly already at the point of diminishing marginal gains from security procedures; and c) it’s widely acknowledged that they don’t detect everything. A 90% improvement from a defect rate of 1.6 per million would leave us at .16 per million, or 0.00000016. That is, we’ve added one more zero to the right of the decimal point, where we already had five, moving us from thousandths of a percent to millionths of a percent.
Let’s view it another way. Say you fly once a week for ten years, which would be 520 flights. Based on my numbers–purposefully skewed to make flying seem more dangerous than it is–your odds of being in an attempted terrorist attack would be 0.000832, or eight hundredths of one percent over the course of ten years. That is, you have a 99.9168% chance of never being in an attempted terrorist attack on a plane, despite flying weekly for ten years. If we implement an enhanced security measure that gives us a 90% improvement, then your odds over that ten year period in which you fly once a month improve to 0.0000832 or eight thousands of one percent. That is, you improve to a 99.99168% chance of never being in an attempted terrorist attack.
To reiterate, my numbers, cooked to be favorable toward implementing the enhanced security measures, suggest that a person who flies weekly for ten years would increase their odds of not being in an attempted terrorist attack from 99.91% to 99.99%.
How many people would willingly subject themselves to gross indignities (on a regular basis) to improve their ten-year safety rate from 99.92% to 99.99%? And I’m not really even talking about safety, since I’m only considering attempted terrorism, so that to talk seriously about odds of harm or death we’d have to discount even further.
It seems to me that anyone who would feel unsafe in an activity that has a 99.9% safety rate over 520 trials is clinically paranoid. If you’re worried about that kind of safety rate you will never drive, never take a shower, and never eat peanuts or popcorn.
Let’s do one more quick analysis, playing with that 99.9168% chance of never encountering an attempted terrorist attack if you fly weekly for ten years and we don’t implement enhanced screening methods. As noted, that’s a 0.000832 chance of encountering an attempt. Let’s assume each attempt is actually successful, so that if you encounter an attempt you will die. Let’s further assume that you fly weekly not just for 10 years, but for 80 years–from birth to (hopefully natural) death. To get from 10 years to 80, obviously we just multiply by 8, so we also multiply 0.000832 by 9 which gives us 0.006656, or 6.6 tenths of one percent chance of dying.
I want to emphasize that. With my wildly inflated assumptions of risk, not implementing security procedures would give you a six tenths of one percent chance of dying in a terrorist incident if you flew weekly for 80 years, or over 41,000 times in your life. Let’s compare that to actual odds of dying as
calculated by the National Safety Council provides that information. A 6.6 tenths of one percent chance is about 1 in 150. That’s less than the odds of accidental poisoning or exposure to noxious substances, and less than the odds of dying from self-inflicted harm. The only difference is that those are real odds, and mine come from inflating the risks by several orders of magnitude.
There is, I think, no serious argument that the backscatter scanners can reduce the defect rate or improve our safety rate to any substantial degree, because we’re already doing a pretty damn good job. To get to even the tiny marginal improvements produced by my analysis I had to count the shoe and underwear bombers (although they didn’t originate from U.S. airports) and add 110 more wholly mythical terrorist attempts. If we look at the real numbers, the defect rate in U.S. airports in the 9 years since 9/11 appears to be 0 in 80 million.
Of course there is room for critique here. We could focus on number of actual people trying to get on the airlines as the baseline. The problem is, there’s just no available evidence for that. Anyway, the point I’m trying to make is about overall safety, which is a combination of there actually being only a small number of such attempts–which is important to the analysis because it is itself a factor suggesting we should not over-react–and how many of those few attempts get through. By focusing on overall safety, I think it’s clear that the safety gains available are vanishingly small. To be sure, there will eventually be another successful terrorist attempt on or with a passenger aircraft. But that will happen with or without these improved measures, and when you’re talking about events of such rarity that their frequency is less than 1 in 80 million, you’d be hard-pressed to prove that any individual security measure was an important factor in reducing that frequency.
And what price are we willing to pay for mythical improvements? Michael and I both agree (I think) that there is no sense in pushing for a zero-defects rate. Put another way, “the zero-risk society is idiotic. We can never get to zero-risk,and two bad consequences result from the effort to do so. First, because increasing marginal costs and decreasing marginal benefits is one of few iron-clad laws of social science, each additional improvement in safety costs us yet more for less gain. It’s not just the financial costs of implementation that a benefit-cost analysis would count, but the loss in civil liberties, the loss in time from having to get to airports earlier, and other such relative intangibles. Second, because we cannot achieve true zero-risk, there is always a push for “just this one thing more,” (which, it turns out, is always just a prelude to yet another, “just this one thing more”). Some degree of risk will always exist, and someone can always hypothesize one more thing that might help. As Richard Forno asks, “Where does it end?” And as he implicitly answers, there is no end. So let’s draw the line here, right now, where we can demonstrate that the harm to civil liberties and personal dignity is substantial, and the gains to be made are insubstantial. And if we do insist on having improved scanner technology, let’s follow the Dutch model, which allegedly is both technically superior, and which displays spotted contraband on a generic image of a human body, rather than displaying the person’s actual body. But let’s not engage in the clearly fallacious argument that we have to surrender our basic dignity for safety. That really is letting the terrorists win.
* I want to note, however, that while our debate on this issue is somewhat fierce, I like and respect Michael. As much as one can actually say they have friendships with people they’ve never actually met face-to-face, I consider him a friend. My critique of him is limited to this particular issue.