Mini-Throughput: Masks, Redux

With COVID-19 numbers beginning to increase and growing concerns that new variants may cause a wave of infections this winter, the pushback against potential COVID interventions is already beginning. Among the talking points of the pro-virus faction is the claim that masks did nothing to stop or slow the pandemic, based on Bret Stephens February summary of a Cochrane Library review published earlier this year. In Stephens’ article, the lead author, Tom Jefferson¹ claims that they’ve proven that masks don’t work.

“There is just no evidence that they” — masks — “make any difference,” he told the journalist Maryanne Demasi. “Full stop.”

But, wait, hold on. What about N-95 masks, as opposed to lower-quality surgical or cloth masks?

“Makes no difference — none of it,” said Jefferson.

What about the studies that initially persuaded policymakers to impose mask mandates?

“They were convinced by nonrandomized studies, flawed observational studies.”

What about the utility of masks in conjunction with other preventive measures, such as hand hygiene, physical distancing or air filtration?

“There’s no evidence that many of these things make any difference.”

So that’s it, right? We did all that masking for nothing?

Not so fast.

First, let’s talk about Tom Jefferson. While he does have the academic credentials, he also has a tendency to associate with anti-vax morons, has disputed the utility of the flu vaccine and is associated with the Brownstone Institute, the primary COVID denial thinktank. This doesn’t make his research wrong, obviously, but it does place into a context. And the context is that he’s appears to be overstating the results of his work.

Let’s consider his public statements against what the review actually says:

Jefferson:

“There is just no evidence that they” — masks — “make any difference,” he told the journalist Maryanne Demasi. “Full stop.”

The study:

Wearing [medical/surgical] masks in the community probably makes little or no difference to the outcome of influenza‐like illness (ILI)/COVID‐19 like illness compared to not wearing masks (risk ratio (RR) 0.95, 95% confidence interval (CI) 0.84 to 1.09; 9 trials, 276,917 participants; moderate‐certainty evidence. Wearing masks in the community probably makes little or no difference to the outcome of laboratory‐confirmed influenza/SARS‐CoV‐2 compared to not wearing masks (RR 1.01, 95% CI 0.72 to 1.42; 6 trials, 13,919 participants; moderate‐certainty evidence). Harms were rarely measured and poorly reported (very low‐certainty evidence).

In other words, there was little evidence that surgical masks worked, but the uncertainties were high. They could be anywhere from a 16% reduction in COVID risk to a 9% increase. That actually suggest to me a slight beneficial effect but an uncertain one. And they find no evidence of harm.

Jefferson:

But, wait, hold on. What about N-95 masks, as opposed to lower-quality surgical or cloth masks?

“Makes no difference — none of it,” said Jefferson.

The study:

We are very uncertain on the effects of N95/P2 respirators compared with medical/surgical masks on the outcome of clinical respiratory illness (RR 0.70, 95% CI 0.45 to 1.10; 3 trials, 7779 participants; very low‐certainty evidence). N95/P2 respirators compared with medical/surgical masks may be effective for ILI (RR 0.82, 95% CI 0.66 to 1.03; 5 trials, 8407 participants; low‐certainty evidence). Evidence is limited by imprecision and heterogeneity for these subjective outcomes.

Now this actually suggests that N95’s did help. The uncertainty is high, ranging from a 30% benefit to a 10% deteriment. So it’s statistically consistent with no benefit. But the uncertainties are heavily on the benefit side of the equation, which suggests a benefit but does not prove one.

Jefferson again:

What about the utility of masks in conjunction with other preventive measures, such as hand hygiene, physical distancing or air filtration?

“There’s no evidence that many of these things make any difference.”

And the study again:

Comparing hand hygiene interventions with controls (i.e. no intervention), there was a 14% relative reduction in the number of people with ARIs in the hand hygiene group (RR 0.86, 95% CI 0.81 to 0.90; 9 trials, 52,105 participants; moderate‐certainty evidence), suggesting a probable benefit. In absolute terms this benefit would result in a reduction from 380 events per 1000 people to 327 per 1000 people (95% CI 308 to 342). When considering the more strictly defined outcomes of ILI and laboratory‐confirmed influenza, the estimates of effect for ILI (RR 0.94, 95% CI 0.81 to 1.09; 11 trials, 34,503 participants; low‐certainty evidence), and laboratory‐confirmed influenza (RR 0.91, 95% CI 0.63 to 1.30; 8 trials, 8332 participants; low‐certainty evidence), suggest the intervention made little or no difference. We pooled 19 trials (71, 210 participants) for the composite outcome of ARI or ILI or influenza, with each study only contributing once and the most comprehensive outcome reported. Pooled data showed that hand hygiene may be beneficial with an 11% relative reduction of respiratory illness (RR 0.89, 95% CI 0.83 to 0.94; low‐certainty evidence), but with high heterogeneity. In absolute terms this benefit would result in a reduction from 200 events per 1000 people to 178 per 1000 people (95% CI 166 to 188). Few trials measured and reported harms (very low‐certainty evidence).

To summarize that block of text: hand-washing helped. There’s evidence of that. It falls apart when you break down the sample further, but it is suggestive.

Those word salads may sound uncertain to you because … they are. Like all sound studies, this one is very careful about stating the limits of their results and the possible confounding factors that may be throwing them off. But look again at Jefferson’s statements. Note his certainty. His simplicity. None of it works, he says. No question. No doubt. By contrast, this is what the study says:

Our confidence in these results is generally low to moderate for the subjective outcomes related to respiratory illness, but moderate for the more precisely defined laboratory‐confirmed respiratory virus infection, related to masks and N95/P2 respirators. The results might change when further evidence becomes available. Relatively low numbers of people followed the guidance about wearing masks or about hand hygiene, which may have affected the results of the studies.

These caveats are necessary because the Cochrane study mashes together different kinds of studies into one big glop of science.

The analysis is flawed because it compares apples to oranges. The paper mixes together studies that were conducted in different environments with different transmission risks. It also combines studies where masks were worn part of the time with studies where masks are worn all the time. And it blends studies that looked at Covid-19 with studies that looked at influenza.

If apples work and oranges don’t, but your analysis mixes them together, you may come to the false conclusion that apples don’t work. Out of the 78 papers analyzed in the review, only two actually studied masking during the Covid-19 pandemic. And both of those found that masks did protect wearers from Covid-19. But these studies are drowned out by the greater number of studies on influenza included, where the benefit of masking is harder to detect because it’s a far less contagious virus than Covid-19.

Note that this was written in February, when the study first came out. It’s not a new study. This tends to be a tactic of the COVID skeptics: tout a study, wait for the furor to die down, then tout it again. But Tran goes into the weeds here, showing that while the study is interesting, it is far from conclusive.²

Early in the pandemic, Starslate Codex joked that it’s difficult to do double-blind control studies of masks for the same reason it’s difficult to do double-blind control studies of parachutes. There’s no such thing as placebo mask, with Lana Del Ray-sized exemptions. Any mask breaks up airflow and therefore might do something to slow the spread. Community-based studies have been more suggestive but, again, it’s really hard to measure compliance and other factors can have a huge impact.

Jefferson and others have suggested that what we should have done was had some states masks and other not. But not only would this be unethical, it still wouldn’t tell us anything because masks are not the only thing that impacts COVID infection rates. If you don’t believe me, check out Jefferon’s own study, which talks about this. If California had fewer cases than New York, was it because of masks? Or lockdowns? Or the weather? Or the differing strains? Or some combination of all of the above plus factors we don’t know about? Absolutely.

I have and probably will never understand the hostility to masks in the skeptic crowd. There isn’t a huge amount of evidence that they do harm (source: Jefferson’s study) and some evidence that they can help on a community level (source: Jefferson’s study). I wore them because my wife is immune-compromised. I also wore them when teaching because I didn’t want to potentially spray COVID all over my students. I don’t like masks but the hysteria over them seems fantastically out of proportion.

I’m also not sure that we need to bring back mask mandates under the present circumstances. Almost everyone has had COVID or been vaccinated. Very few people have no resistance. So whatever impact they had in the early stages — and it doesn’t appear to have been huge — would be even lower now. And the justification — keeping our hospitals from being overrun — is lessened. But that may change in a hurry if a particularly vicious variant comes back. And, honestly, slowing the flu might be better justification for masks at this point. No one seems to have noticed that whatever our interventions did for COVID, they massively mitigated flu seasons and may have even yeeted one strain of flu into extinction. We’re kind of overdue for a bad flu season.

Still, I expect this study to be cited a lot, even if better studies supersede it and show that masks work. Because the COVID skeptics use studies the way drunks use lampposts — for support, not illumination. For every ten studies showing Ivermectin doesn’t cure COVID, they’ll site one study that claims it does. For every ten studies showing HCQ doesn’t cure COVID, they’ll site one that claims it does. For every ten studies showing the vaccines are safe, they’ll cite … well, actually they don’t have any studies showing vaccines kill, so they’ll just make them up.

(And, to be fair, the pro-intervention side has a tendency to ignore studies that don’t support their side either. Few have addressed the Cochrane review head on.)

It seems that the battle lines are drawn on this. But, stubborn mule that I am, I will continue to point out the facts, the uncertainties and the caveats. Because they matter. At some level.

---

1. Yes, really.
2. And, I might note, doing a better job of it than Fauci has, who has responded by sputtering about “other studies” instead of addressing the defects of the Cochrane review directly.