Mini-Throughput: What the Data Say About Guns

As expected, the recent spate of mass shootings has prompted intense debate about gun control. I’m not going to wade too deep into this because, increasingly, the debate over guns is less about crime, murder and suicide than it is about culture. This occurred to me recently when I proposed a compromise on the subject of assault weapons: that they be brought into the same legal regime that governs concealed weapons: legal but with a permit required. What surprised me was that both sides hated this idea. The anti-ASW side wanted them banned period; the pro-ASW side didn’t period. Meeting halfway was unthinkable.

When Republicans politicians send out Christmas cards with their kids holding rifles, that’s not about laws. When a Republican runs for governor on the platform of “Jesus, Guns and Babies”, she’s not opening a debate on gun control. What they are doing is a kind of virtue-signaling, showing that they are “one of you” with “you” being defined as gun owners. And when Democratic politicians focus on never-never bans of weapons that are responsible for only a few percentage points of gun violence, that’s not really about the gun either. It’s virtue signaling that they are one of the “you” that want “military style” guns to disappear.

However, I thought I would turn toward the debate over guns as a scientist and see what the data actually say about gun control, gun ownership and gun deaths. A lot of other people have done this, but usually canted toward one particular side or another. I’m not unbiased. I have a fairly strong pro-second amendment stance. I am extremely dubious of gun control both as a matter of civil liberty and as a solution to the problem of gun violence. My position is not on the extreme edge — I support background checks, red flag laws and child safety measures. I am open to the idea of raising the age at which guns can be purchased or having an extra measure on so-called assault rifles.

But I’m going to try to be as objective as possible, to look at the data as disapassionately as I can. I don’t think the zealots will be persuaded — anti-gun folks will still to ban them no matter what the data say and militia-types will oppose any gun control no matter what the data say. To them, these are moral issues, not pragmatic ones. But there is a large squishy middle on this debate. If you’re looking for someone to confirm your priors on this, you’re in the wrong place. If you’re looking for someone to claim they have the absolute answers either way, you’re in the wrong place. The data on this subject are … murky at best. The science, such as it is, is not nearly as conclusive or as sweeping as either side would like. But if you’re looking for some skepticism on big sweeping claims about the link — or lack thereof — between gun control and gun violence — I’m hoping I can help generate a little more light and a little less noise.¹

No one is ever all right or all wrong. No piece of data ever ends an argument. Policy debates are more like basketball games. Some arguments are three-pointers. Some are foul shots. And some are complete bricks. My purpose here is not to give you the final score because what’s a three-pointer to someone may just be a foul shot to you. But I’m hoping to identify which arguments are bricks and shouldn’t be part of the debate.

Claim: Other countries don’t have this level of violence.

What motivated this post was a strange thread by Andrew Follett which tried to argue that the US is not an outlier on gun violence and that we have been lied to by the media. How does he claim this? By noting that out of 97 countries surveyed in a recent study, the US is 64th in mass shootings and 65th in murder rate.

The problem is that while this claim is literally true it’s … a really strange thing to be boasting about. It’s the equivalent of a UNC coach boasting about beating a 16-seed in the tournament. Almost all of the countries that have higher murder rates than us are far less wealthy and far less developed. Countries with comparable murder rates to our 6.3 per 100,000 are Bolivia, Ukraine, Cabo Verde, Haiti and Afghanistan. We trail countries like Iran (2.5), Rwanda (2.6) and Ghana (2.1). Our peer countries have murder rates that are a fraction of ours: Canada is at 2.0. France at 1.2, the UK at 1.2, Switzerland at 0.5.

This comparison is even more egregious when you talk about mass shootings. Mass shootings are, by their nature, rare. And so they are subject to small number stats. Follett makes a big deal of France having had more mass shootings than the US per capita, but the data he cites were compiled immediately after the Charlie Hebdo massacre and before the recent uptick in mass shootings within the United States.

I know this is stating the bleeding obvious. But sometimes we need to start with the obvious. Yes, the US is an outlier from similar countries in both overall gun violence and mass shootings. We have a problem.

Claim: This disparity is because the US has so many guns.

The connection between gun ownership and gun deaths is a lot trickier to pick out than most people think. There’s a graph that people like to circulate that shows that a country’s gun death rate closely tracks its gun ownership rate, shown below.

This graph is misleading, however. It includes both suicide and murders, but only if those are committed with a gun. This a problem because some countries, like South Korea, have astronomical suicide rates but very few of those suicides involve a gun. If you plot overall suicide rate against gun ownership, there is no connection whatsoever. Suicide is far more a function of culture and circumstance than it is of gun ownership. That being said, there are reason to believe that limiting access to guns — particularly for adolescents — could drop the suicide rate. Israel, for example, saw a big drop in suicides after restricting soldiers’ access to weapons. Suicide tends to be impulsive and guns tend to make that impulse final.

The connection between gun ownership and murder is also unclear because, again, you can’t just ignore murders when they don’t involve a gun. In the United States, two-thirds of murders involve a gun. In other countries, only about one-third do. If you plot total murder rate again gun ownership, the correlation shown in the graph above completely disappears. What you end up with is a bunch of countries with low murder rates and low gun ownerships rates that are not correlated at all … and the United States being a massive outlier on both axes.

Connecting a cloud of random data to one outlier is not analysis; it’s bias confirmation. If you can drop literally one data point and your thesis collapses, it’s not supported by the evidence. I’m not saying gun ownership isn’t a big part of the problem; I’m saying that this is very difficult to argue by comparing the US to other countries. Some countries have lot of guns and few murders. Others have few guns and lots of murders. There does not seem to be a connection between the two unless you include the United States. We are an outlier in every sense.

Claim: The violent places in United States all have strict gun control laws. Continuing with Follet’s thread, he breaks out the tired trope that violence mostly occurs in Democratically-controlled cities with strict gun control laws. He contrasts this against Wyoming which has high gun ownership and a low homicide rate.

This is frankly, garbage. First of all, guns can move. The data say so. At least half the guns used to commit crimes in Chicago, for example, come from outside the state entirely.

Second, if you look at things on a state level, the states with the highest murder rates are almost all red states. Louisiana, Missouri, Mississippi, Arkansas, South Carolina, Alabama and Tennessee lead the pack before Illinois shows up. California is 27th on the list. New York is 32nd. Just look at this map and tell me if anything jumps out at you.

But it might actually be worse than that. The CDC, noting that murder is generally committed by younger people, has an age-adjusted table. On this table, Wyoming is actually worse than New York. Because comparing a sparsely populated rural state with no major cities to a state with several big cities isn’t exactly fair.

Now does this mean that gun ownership is driving murder rates? Not necessarily. The state-to-state correlation of gun ownership to murder rate is as murky as the country-to-country data, with no obvious correlations jumping out until you start adjusting for confounding factors. It is broadly true that gun violence tends to happen in cities while most legal gun owners don’t live in cities. And I agree with Follett that cracking down on straw purchases and other illegal sales would be something both sides could agree on and might do some good. But the gun control advocates would argue — not unreasonably — that this is a game of whack-a-mole. There are a lot of avenues for guns to find their ways into the wrong hands.

Claim: Mass shooting are happening more often. One of the problems in discussing mass shootings is that the definition of what constitutes a mass shooting is slippery. You may have seen the claim that there have 27 shootings at schools just this year. Or some of your friends may have posted a long list of schools on Facebook where shootings supposedly occurred. These claims are very misleading because the database they use includes any shooting near a school in which someone is wounded. So if two drug dealers shoot it out in parking lost next to school at midnight in July, that constitutes a “school shooting”. You may also have heard that there have been over 200 mass shootings in the US just this year. But this is also a bit misleading. This comes from the Gun Violence Archive, which classifies any shooting in which four people are injured or killed as a mass shooting, thus including hundreds of incidents that are just ordinary gun violence, mostly related to the drug trade. That’s not what most people think of when the phrase “mass shooting” pops up.

While I’ve been critical of the Mother Jones database on mass shootings in the past, specifically that their pre-2000 data is almost certainly incomplete, it uses a definition of mass shooting that is closer to what most of us think of as a mass shooting. They define it as four or more people killed in a rampage where the shooting is not related to a robbery or gang violence. And their data shows that mass shootings are increasing, both in frequency and lethality. They increased a little in the 2000’s and early 2010’s, but have spiked dramatically in the last five years. So while mass shootings — at least as conventionally understood — are still rare and constitute less only a few percentage points of gun violence, they are, in fact, increasing, even using the strictest criteria.

Claim: Mass shootings dramatically increased since the assault weapons ban expired. The graphic I’ve seen is that they dropped 43% after the ASW ban was implemented in 1994 and increased 239% since it expired in 2004. Joe Biden has made this claim and Politifact has rated it as mostly true.

I disagree.

The paper on which this claim is based is behind a firewall but does a lot of running of trend lines through very noisy data, combining 15 years of post-ban data into one group and deciding that the recent spike reflects a trend. It assumes that the databases they use accurately reflect the number of mass shootings in the 80’s and 90’s, which they don’t. For example, their graph appears to show no mass shootings from 1994 to 1997. This despite the fact that the Pearl High School Shooting, the Fairchild AFB shooting, the Phelon Company shooting and several other shootings occurred during this time. Several of these are in the Mother Jones database (Pearl isn’t because only three people were killed). It also classifies mass shootings as using assault weapons if any semi-automatic weapons were used, which means that the Virginia Tech killing, which involved handguns, is classified as an assault weapons killing. Columbine, which happened while the assault weapons ban was in place and did not use assault weapons, is also classified as using assault weapons.

The bulk of research, as noted in the RAND metastudy — more on that in a minute — is inconclusive at best. This has been the general conclusion of researchers on the subject of the assault weapons ban.²

Claim: All the studies show gun control works. This phrasing should immediately trigger your radar. “All the studies” never show anything. There are many many studies that show that gun control measures, however defined, work. But there are almost as many that show that they don’t. This field is difficult to study and subject to large biases and complications because gun violence is not monocausal.

The only way to sort through the noise is to do a meta-study, to combine all the research and see what patterns merge. The National Research Council did this in 2004 and concluded … that more research was needed. There was no clear data for or against. The RAND corporation did another meta-study in 2018. Their conclusion was that there was strong evidence to support background checks, waiting periods, child safety protections, the repeal of Stand Your Grand laws and denying weapons to domestic abusers as ways to bring down gun deaths. Other policies were not clearly supported.

Claim: But a study in Connecticut showed a gun registration law made homicides plunge. See above. You can always find one study of one state in one particular period of time that will give you the conclusion you want. But other states have not had similar results. The Connecticut study found that homicides dropped 40% … relative to a synthetic Connecticut: a hypothetical state cobbled together from other states that supposedly represents what Connecticut would have been like without the gun control law. There have been significant and valid criticisms of this method. I would not base policy on it.

Claim: More guns means less crime. This is the claim advanced by John Lott’s 1998 book and often touted by pro-gun groups. This claims is now 24 years old and not one but two major meta-studies have failed to find the same result.

Claim: Guns are used defensively 2.5 million times a year. This claim comes from the 1994 Kleck survey, which polled 5000 people to see how many had used guns defensively in the previous year and extrapolated that to the entire nation. This survey has been a source of intense debate over methodology and replication. Other surveys have estimated the number of defensive uses as anywhere from 100,000 cases a year to five million, meaning the stat is almost meaningless. The problem with this area of research is that, because police reports are rarely filed when a gun is used defensively, there is no objective information. You’re relying almost entirely on what people say in response to surveys. And what they can say can reflect reality. Or it can reflect what they think happened or how they rationalize what happened or can be entirely fictional. Just look at the Kyle Rittenhouse trial: most legal analysts and the jury concluded that he used his gun in self defense. But they also noted that he helped create the dangerous situation in the first place. Is that a clear and unequivocal defensive use of a weapon? Or do we need to qualify it by the circumstances? I certainly know how he would describes it in a phone survey.

As it happens, I think the number is probably closer to Kleck’s estimate than the lowball estimates. But I would say this number is far too uncertain to be used to inform public debate.

Claim: Gun ownership is relatively new thing. Americans owned very few guns until the 20th century.. This claim is at least partially based on a now-discredited book.

Claim: Australia banned many guns and their homicide rate went down

This claim is literally true. In the wake of the Port Arthur massacre, Australia enacted tight restrictions on gun ownership and bought back over 600,000 weapons. In the 25 years since, Australia’s murder rate was cut in half and there has been only one mass shooting.

However, this isn’t as meaningful as it looks. Australia has a 15th of our population. Even if they had as many mass shooting as the United States, you would expect maybe one a decade. Furthermore, Australia’s drop in murder rate was part of a global trend. In the United States, the murder rate over that period of time dropped almost 40%. The UK saw a 60% drop. France saw a 40% drop. Almost every developed country in the world saw huge drops in crime starting in the early- to mid-90’s.

No one knows why this happened. There are many many theories. I am leaning more and more toward the lead hypothesis — that removing lead from our air, food, water and fuel resulted in a generation with less brain damage and better impulse control. But years of research have yet to reach a firm conclusion.

The point here is that Australia did see a drop after their gun buyback.³ But they were not unusual in that regard and not an outlier in that regard. This isn’t anywhere close to the slam dunk people seem to think it is.

———————————————-

This is just a partial list of some of the more prominent claims. But I wrote to illustrate just how tricky this debate is and how dishonestly some of the representation of the data are. It’s not just that simple. It never is.

As stated above, I am broadly on board with the policies identified by the Rand Corporation as effective. The bulk of the evidence persuades me that these would help. But there is little evidence that they would be a panacea or drop us down to levels of violence compatible with Western Europe. As I stated in my last piece on the subject, whatever is causing this has much deeper roots than gun access.

1. I’m also not going to rate claims as “true” or “false” or “four Pinocchios” or “three Baron Munchausens” because it’s rare that claims are made in this context that are absolutely false. There are vast differences in interpretation, significant problems in methodology and often unclear implications for policy.
2. I will note that this is typical of Politifact. The one study that confirms their biases is mentioned first while the more numerous and comprehensive studies are mentioned as an afterthought.
3. Although compliance with said buyback was, at best, 50%, and probably lower.