The Annual Misuse of Hate Crime Statistics

NOTE: SIGNIFICANT UPDATES BELOW THE JUMP.

Every year around this time, the FBI publishes its statutorily-mandated annual report on hate crime statistics.  Like clockwork, every year that report gets misused no matter what the FBI does to discourage that misuse (previous examples of misuse here).  This year is no exception, as several prominent liberal sites have picked up on the lede that this year’s report shows a “sharp increase” in anti-gay hate crime while also noting that race-based hate crimes barely decreased at all. 

The problem is that these particular FBI statistics are virtually useless for evaluating year-to-year trends – always have been, always will be.  This year, the FBI itself went out of its way to warn against such readings, stating “our Uniform Crime Reporting (UCR) Program doesn’t report trends in hate crime stats—yearly increases or decreases often occur because the number of agencies who report to us varies from year to year.”

Yet in reporting an 11 percent increase in hate crimes against gays while lamenting a mere 1% decrease in race-based hate crimes, Think Progress and Feministing ignore this important disclaimer.  This failure is significant because several hundred more law enforcement agencies participated in this year’s survey than last year’s survey: last year’s survey had the participation of 13,241 agencies, this year’s of 13,690.  Of those agencies, 2025 in 2007 and 2145 in 2008 actually reported any hate crimes.  This discrepancy in reporting agencies alone makes a worthwhile one-to-one, year-to-year comparison very difficult to make.  At a minimum, for the purposes of this year’s numbers, the discrepancy in reporting agencies accounts for somewhere between 1/3 and 1/2 of the apparent increase in hate crimes against gays.

Moreover, the composition of the agencies participating in the report changes from year to year, and varies greatly from state to state.  For instance, in 2007, only 18 law enforcement agencies in the state of Georgia, covering a population of 680,591, participated in the report; in 2008, this number actually decreased to 6 law enforcement agencies covering 554,193 people.  Conversely, in 2007, North Carolina had 369 reporting agencies covering 7,421,293 people; in 2008, it had 495 reporting agencies covering 9,128,679 people.  The statistics, however, do not account for these variations in state by state, year-to-year reporting making year-to-year comparisons (in any direction) highly suspect.

Another reason why these statistics are particularly meaningless (for purposes of a year-to-year comparison at least) is that the statistics do not account for differences in state laws or prosecutorial attitudes and priorities.  This is especially important when discussing anti-gay hate crime because of the fact that sexual orientation is still a relatively new addition to hate crimes laws such that the willingness (and, for that matter, ability) of prosecutors to charge a defendant with an anti-gay hate crime – or of law enforcement officers to report an anti-gay hate crime – is likely to be increasing significantly every year.  There may, therefore, be an increase in the number of reported anti-gay hate crimes even as there may (or may not) be a decrease in the number of actual anti-gay hate crimes.  Indeed, as this map shows, 19 states still do not recognize anti-gay hate crimes as hate crimes at all – even for reporting purposes.  This means that even small local changes in state-level prosecutorial priorities are likely to have an outsized effect on the national statistics for anti-gay hate crimes. 

This discrepancy in law enforcement priorities is readily apparent when you compare the statistics for a state like socially liberal Vermont with a state with a history of open racism like Mississippi.  In 2007, Vermont had reports from 80 agencies representing 610,907 people, with a total of 21 hate crimes reported by those agencies.  Mississippi, by contrast, had reports from 58 agencies representing 704,703 people, with a total of exactly zero – yes, zero – hate crimes reported.  This year, Missippi had reports from two more agencies than last year, resulting in one agency reporting 4 hate crimes; Vermont, with two additional agencies reporting but with slightly fewer people represented in the report, still reported 20 hate crimes from 12 different agencies.  It seems implausible, to say the least, that hippie-dippie Vermont’s actual hate crime rate was more than 200% higher than Mississippi’s in 2008 – and infinitely higher in 2007.  Far more likely is that Vermont has tougher hate crimes laws than Mississippi and is thus significantly more likely to report hate crimes than Mississippi.

Brian Levin at Huffington Post does a good job explaining why such caution must drawn about drawing inferences from these statistics – although he still notes the overall slight decline in hate crimes, writing:

Three of the five states with the highest proportion of African-Americans, who account for 35% of all reported hate crimes nationally in 2007, barely reported hate crime at all that year:

1. Mississippi, 38% African-American, 0 hate crimes

2. Louisiana, 32% African-American, 31 hate crimes

3. Georgia, 31% African-American, 13 hate crimes 

Contrast these totals with that of South Carolina, a state in the same region with similar demographics and a total population almost the same as Louisiana and half that of Georgia’s. South Carolina has over one million African-American residents comprising 29% of the state’s total population. The state reported 127 hate crimes in 2007. In 2007 Boston, a city with a dedicated police hate crime unit and a population of 500,000, counted more hate crime cases than Georgia, Alabama, Arkansas, Louisiana, and Mississippi combined with a total population of about 24.4 million.

The point here is that it is extraordinarily difficult – if not impossible – to draw any kind of conclusions about trends in hate crime on the basis of the FBI’s annual reports, unless there are truly massive swings in a data set or unless a trend continues for a number of years without any plausible explanation related to changes in law enforcement priorities or state-level hate crimes legislation. 

Because of the severe limitations of this annual report, I am indeed skeptical whether it has any value whatsoever.  At a minimum, however, interpretations of the data in these reports need to be limited to assessments of where law enforcement priorities lie and of what forms of hate crimes are, relatively speaking, the most common.

UPDATE: Also spotted misusing the statistics: Raw Story and, to a lesser extent, NPR.  All via memeorandum.

UPDATE II: Via greginak in comments, Think Progress has updated the previously mentioned post to make note of the FBI’s disclaimer against using these statistics to make year-to-year comparisons.  Think Progress’ update still suggests that the data may be evidence of an increase in hate crimes against gays or just  result of the difference in reporting agencies, however, rather than noting that there is no basis for a statistical comparison at all.  I’m not saying that there hasn’t been an increase in anti-gay hate crime, by the way – just that the hate crimes statistics provide no evidence that there has been such an increase and, to my knowledge, there is not currently any conceivably reliable statistical evidence to suggest either an increase or a decrease in anti-gay hate crime.

UPDATE III: Crikey.  It looks like there’s been a host of misreporting on this topic by old media outlets, including the Associated Press (though it at least bothered to include a warning about the data quality from the aforementioned Brian Levin).  The Detroit Free Press uses the deeply misleading headline “Michigan 4th in nation in hate crimes.”  The Jewish Telegraphic Agency reports that “Hate crimes hit seven-year high.”  USA Today reports that “Hate crimes against blacks, religious groups rise.” CBS News reports “Hate Crimes Up in 2008” – and doesn’t even bother to mention the FBI’s disclaimer or Levin’s warning, as do some of the other stories.  The list could go on.  Of the old media outlets I’ve seen report on this, only CNN used a headline that was not absurdly misleading, proclaiming (accurately and unsensationally) “FBI cites thousands of hate crimes in ’08.”