Monday, January 11, 2010

Evidence-based airport security?

Backscatter scanners cost $150,000-$180,000 a piece. Congress has appropriated funding for 450 scanners to be placed in the US airports. By my simple back-of-the-envelope calculation we are talking $67,500,000-$81,000,000 of our taxpayer dollars. How do we wrap our brains around the value proposition of this investment?

Well, in order to do this we need to know something about what outcome we are trying to affect and the impact of this machine on this outcome of interest. Seems like the most sensible outcome of interest is deaths averted due to airplane-related terrorist activity. According to Nate Silver at fivethirtyeight.com, in the first decade of the 21st century, factoring out the 9/11 fatalities, there were about 200 deaths related to violent incidents on board of commercial aircraft in the entire decade. So, assuming that the scanners' effectiveness is 100% (that is that it can prevent any violent act aboard an airplane that would result in any number of deaths), the cost to avert one death is $337,500-$405,000 over 10 years (this is not taking into account either inflation or discounting for future events).

If we had a medical technology with the same cost-effectiveness profile, would we think it reasonable to pay for it? More importantly, would we get congressional appropriations to pay for it even in the absence of any effectiveness data? What we know and what we think we know about this are widely divergent. What we know is that the scans can detect certain culprits of potential acts of terrorism. What we do not know is whether this level of detection will indeed result in aversion of death. After all, had we not detected prior to the Detroit flight that Umar Farouk Abdumutallab was a potential threat to US citizens? We had, but the information was not acted upon. Similarly with the scanners, they may have efficacy in detecting the threat, but how this information is utilized is sure to impact their effectiveness as far as the endpoint of interest: death prevention. So, reaching for a new expensive and potentially more invasive technology in this case is a bit like blaming one's failure to treat a cancer detected on a mammogram on unavailability of the more sensitive MRI technology.

There seem to many points of faulty logic in this undertaking of heightened airport security. Not only do we need to re-examine critically our assumptions, but also we have to assess soberly whether the investments we have made to date are in the right places. Remember the cold war? It was all about getting a leg up in the nuclear arms race. "You build a better missile, we will build a missile shield. Then you will build an even better missile that will penetrate the shield." And so on, and so on. It is much like laws, upon which clever corporate lawyers rely for loop holes in favor of their clients' misdeeds.

As much as anyone, I would like international terrorism, just like ventilator-associated pneumonia and other hospital-acquired complications, to become a "zero event". Unfortunately, I am all too keenly aware that wishing something to be true only makes it so in fairy tales. The dearth of evidence to support many of the expensive anti-terrorism interventions is concerning. Poor logic, erroneous assumptions and unjustified inferences have been driving our decisions for too long. The public should demand the same level of evidentiary support for astronomical "anti-terrorism" appropriations that we do for healthcare.

3 comments:

  1. "The public should demand the same level of evidentiary support for astronomical "anti-terrorism" appropriations that we do for healthcare. "

    Unfortunately, I think we already do, and it's a low level indeed. We have backed ourselves into a no-risk corner, where nothing bad is allowed to happen and if it does, it's always somebody's fault and has a monetary value. I think we have to learn that life has risks and dangers and no amount of money is going to make that fact go away (though it might make a lot of people rich, and lots more very poor).

    ReplyDelete
  2. Why factor out 9/11?? That was an 'airplane-related terrorist incident', surely?

    Adjust your numbers by an order of magnitude and $4,000 a year might seem more palatable.

    ReplyDelete
  3. Thank you for the insightful post. The fact of the matter is that the amount of money spent to "potentially" avert a death from terrorism is completely out of synch with any cost-benefit analysis of what we would be willing to pay to avert death from any disease.

    I suspect that your numbers would be relevant and unacceptably astronomical EVEN IF September 11th were included in your calculations. Let's not forget that the calculations above are for 450 airports. However, there are currently over 14,900 airports in the US (> 5000 of which accomodate large planes). If the intent is to save human lives by avoiding an aircraft being used as a weapon; then the 100% effectiveness that you rightfully set (which, of course is highly unlikely) would only be possible if ALL airports had such scanners.

    As you have said. There are "too many points of faulty logic in this undertaking of heightened airport security".

    I'm frankly amazed that so little analysis is taking place with regard to unintended consequences of proposed actions. It's like Iraq all over again. (But of course, that's an entirely different issue, isn't it).

    Thank you for the post.

    ReplyDelete