Via Schneier, a paper by William Press which demonstrates that “strong profiling” is no better than completely random sampling. Here’s the abstract:
The use of profiling by ethnicity or nationality to trigger secondary security screening is a controversial social and political issue. Overlooked is the question of whether such actuarial methods are in fact mathematically justified, even under the most idealized assumptions of completely accurate prior probabilities, and secondary screenings concentrated on the highest-probablity individuals. We show here that strong profiling (defined as screening at least in proportion to prior probability) is no more efficient than uniform random sampling of the entire population, because resources are wasted on the repeated screening of higher probability, but innocent, individuals. A mathematically optimal strategy would be ”square-root biased sampling,” the geometric mean between strong profiling and uniform sampling, with secondary screenings distributed broadly, although not uniformly, over the population. Square-root biased sampling is a general idea that can be applied whenever a ”bell-ringer” event must be found by sampling with replacement, but can be recognized (either with certainty, or with some probability) when seen.
Based on a fairly quick reading, the paper seems to do what it says on the tin. The optimal strategy might, in plain English, be called “weak profiling”; you do screen individuals with a higher probability of being terrorists (or whatever) more frequently than low-probability individuals, but not in proportion with their probability (and, as a corollary, people who are very unlikely to be terrorists are screened more frequently than they would be under strong profiling). It seems counter-intuitive, but you can’t argue with the math.
Also, note the point about this being true even assuming that the probabilities we assign to people are completely accurate. I haven’t tried to prove it, but I’d guess that under more realistic assumptions strong profiling fares even worse.
