Mini-Troughput: The HCQ, Hydroxychloroquine Reloaded

Ball-and-stick model of a hydroxychloroquine molecule

Two weeks ago, I wrote about a massive study of the effects of hydroxychloroquine (HCQ) on victims of COVID-19. The study indicated that not only was HCQ ineffective, it caused heart arrhythmias. At the end, I noted:

So is the debate over? Can we move on from HCQ? Not quite.

First of all, the study is a retrospective study, looking backward at nearly 100,000 cases over the last four months. That’s a massive sample that allows one to correct for potential confounding factors. But it’s not a double-blind trial, so there may be certain biases that can not be avoided. In response to the publication, a group doing a controlled study unblinded some of their data (that is, they let an independent group look up who was getting the actual HCQ and who was getting a placebo). It did not show enough of a safety concern to warrant ending the study.

It’s also worth noting that because this is an unproven therapy, it is usually being used on only the sickest patients (the odd President of the United States aside). It’s possible earlier use of the drug, when the body is not already at war with itself, could help.

Well, this week, things got…a little nuts:

The World Health Organization and a number of national governments have changed their Covid-19 policies and treatments on the basis of flawed data from a little-known US healthcare analytics company, also calling into question the integrity of key studies published in some of the world’s most prestigious medical journals.

A Guardian investigation can reveal the US-based company Surgisphere, whose handful of employees appear to include a science fiction writer and an adult-content model, has provided data for multiple studies on Covid-19 co-authored by its chief executive, but has so far failed to adequately explain its data or methodology.

Data it claims to have legitimately obtained from more than a thousand hospitals worldwide formed the basis of scientific articles that have led to changes in Covid-19 treatment policies in Latin American countries. It was also behind a decision by the WHO and research institutes around the world to halt trials of the controversial drug hydroxychloroquine. On Wednesday, the WHO announced those trials would now resume.

What does all that mean? The Lancet HCQ study, among others, uses data provided by Surgisphere. What is Surgisphere? Well, no one seems to know. It’s a small company with 5 to 11 employees, only one of whom has a science background. For several days, doctors and science bloggers have been asking increasingly pointed questions about Surgisphere. The details are kind of…involved. But the basic thrust is that the data Surgisphere is providing would require a massive infrastructure and extremely sophisticated software. And it’s not clear that they have either of those things. More damning, the Guardian’s Australia arm found errors in the database. They contacted several big hospitals in Australia and none of them had even heard of Surgisphere.

Now it’s certainly possible that we’re badly underestimating Surgisphere’s capabilities and that their data will turn out to be legit. But the concerns raised are so significant that one paper based on their data has been withdrawn and the Lancet has put out an “Expression of Concern” about the HCQ paper¹

If — and I stress that it is still if — Surgisphere’s data turn out to be garbage, I can not overstate what a disservice this is to science. I really hope it turns out to be a big misunderstanding. Not because I “hate Trump” and therefore want HCQ to fail. I want HCQ to succeed because it is a cheap medicine that could save lives. No, I hope this is a misunderstanding because if it’s not, it will feed a thousand conspiracy theories. It will pour rocket fuel on the belief that HCQ is a cure and it is being held back by a vast conspiracy of Democrats, scientists, doctors and Bill Gates. It will make people doubt science at a time when our lives are so critically dependent on it.²

I’ve been saying for some time that, with the COVID crisis, we’re seeing how science works in real time. This is a good thing and a bad thing. Good, in that the public is getting an inside view of how ideas are proposed, tested, criticized, tested again, rejected, revived and debated. It may seem chaotic, but that’s mainly because it’s happening so fast and so publicly³. Usually, the process is a lot less crazy and a hell of a lot more thorough. But this is a somewhat cocaine-binge version of how we do it: constant study, cross-study and debate as we try to reach a rough consensus.

But it’s also bad in the sense that, to some people — especially those of a conspiratorial bent — this makes the science seem less reliable. They see mistakes, errors and, possibly, even fraud. And they assume this means it’s all nonsense.

But if this story is born out, it is not an example of science failing; it’s an example of science working. A seemingly ground-breaking study drew a dramatic conclusion. And people began looking at it. They found some things to be concerned about. If their concerns are born out, it will unravel a critical study in the most important health crisis of our lifetime. But if the study is flawed, that unravelling is a good thing. It’s a very good thing. Because it might turn out that HCQ does save lives. If it does, we need to know that. And if it doesn’t, we need to know a hell of a lot more certainly than we do now.

And even if it turns out that the Surgisphere data are perfectly fine, making sure of that would have been a critical check to perform. We always want to make sure our data are sound, our methods are rational and our conclusions are solid. Even when we’re ultimately right. Especially when we’re ultimately right.

If the Surgisphere data turn out fine, I’ll turn this series into a trilogy to note that. But for now…it’s not looking good. And if it does turn out to be junk, it sets a critical line of research back to…well, not square one. But maybe square two.

Update: The paper has now been retracted.

1. An Expression of Concern is basically a note saying, “uh…give us a minute here.”
2. And indeed, it is taking attention away from the first true double-blind study of HCQ, which shows no positive effect, including as prophylaxis.
3. Indeed, the fast pace is probably why the concerns about Surgisphere were not raised before publication. It didn’t occur to anyone that, given the present crisis, someone might be feeding them junk data.