Mini-Troughput: The HCQ, Hydroxychloroquine Reloaded

Hydroxychloroquine

Ball-and-stick model of a hydroxychloroquine molecule

Two weeks ago, I wrote about a massive study of the effects of hydroxychloroquine (HCQ) on victims of COVID-19. The study indicated that not only was HCQ ineffective, it caused heart arrhythmias. At the end, I noted:

So is the debate over? Can we move on from HCQ? Not quite.

First of all, the study is a retrospective study, looking backward at nearly 100,000 cases over the last four months. That’s a massive sample that allows one to correct for potential confounding factors. But it’s not a double-blind trial, so there may be certain biases that can not be avoided. In response to the publication, a group doing a controlled study unblinded some of their data (that is, they let an independent group look up who was getting the actual HCQ and who was getting a placebo). It did not show enough of a safety concern to warrant ending the study.

It’s also worth noting that because this is an unproven therapy, it is usually being used on only the sickest patients (the odd President of the United States aside). It’s possible earlier use of the drug, when the body is not already at war with itself, could help.

Well, this week, things got…a little nuts:

The World Health Organization and a number of national governments have changed their Covid-19 policies and treatments on the basis of flawed data from a little-known US healthcare analytics company, also calling into question the integrity of key studies published in some of the world’s most prestigious medical journals.

A Guardian investigation can reveal the US-based company Surgisphere, whose handful of employees appear to include a science fiction writer and an adult-content model, has provided data for multiple studies on Covid-19 co-authored by its chief executive, but has so far failed to adequately explain its data or methodology.

Data it claims to have legitimately obtained from more than a thousand hospitals worldwide formed the basis of scientific articles that have led to changes in Covid-19 treatment policies in Latin American countries. It was also behind a decision by the WHO and research institutes around the world to halt trials of the controversial drug hydroxychloroquine. On Wednesday, the WHO announced those trials would now resume.

What does all that mean? The Lancet HCQ study, among others, uses data provided by Surgisphere. What is Surgisphere? Well, no one seems to know. It’s a small company with 5 to 11 employees, only one of whom has a science background. For several days, doctors and science bloggers have been asking increasingly pointed questions about Surgisphere. The details are kind of…involved. But the basic thrust is that the data Surgisphere is providing would require a massive infrastructure and extremely sophisticated software. And it’s not clear that they have either of those things. More damning, the Guardian’s Australia arm found errors in the database. They contacted several big hospitals in Australia and none of them had even heard of Surgisphere.

Now it’s certainly possible that we’re badly underestimating Surgisphere’s capabilities and that their data will turn out to be legit. But the concerns raised are so significant that one paper based on their data has been withdrawn and the Lancet has put out an “Expression of Concern” about the HCQ paper[efn_note]An Expression of Concern is basically a note saying, “uh…give us a minute here.”[/efn_note]

If — and I stress that it is still if — Surgisphere’s data turn out to be garbage, I can not overstate what a disservice this is to science. I really hope it turns out to be a big misunderstanding. Not because I “hate Trump” and therefore want HCQ to fail. I want HCQ to succeed because it is a cheap medicine that could save lives. No, I hope this is a misunderstanding because if it’s not, it will feed a thousand conspiracy theories. It will pour rocket fuel on the belief that HCQ is a cure and it is being held back by a vast conspiracy of Democrats, scientists, doctors and Bill Gates. It will make people doubt science at a time when our lives are so critically dependent on it.[efn_note]And indeed, it is taking attention away from the first true double-blind study of HCQ, which shows no positive effect, including as prophylaxis.[/efn_note]

I’ve been saying for some time that, with the COVID crisis, we’re seeing how science works in real time. This is a good thing and a bad thing. Good, in that the public is getting an inside view of how ideas are proposed, tested, criticized, tested again, rejected, revived and debated. It may seem chaotic, but that’s mainly because it’s happening so fast and so publicly[efn_note]Indeed, the fast pace is probably why the concerns about Surgisphere were not raised before publication. It didn’t occur to anyone that, given the present crisis, someone might be feeding them junk data.[/efn_note]. Usually, the process is a lot less crazy and a hell of a lot more thorough. But this is a somewhat cocaine-binge version of how we do it: constant study, cross-study and debate as we try to reach a rough consensus.

But it’s also bad in the sense that, to some people — especially those of a conspiratorial bent — this makes the science seem less reliable. They see mistakes, errors and, possibly, even fraud. And they assume this means it’s all nonsense.

But if this story is born out, it is not an example of science failing; it’s an example of science working. A seemingly ground-breaking study drew a dramatic conclusion. And people began looking at it. They found some things to be concerned about. If their concerns are born out, it will unravel a critical study in the most important health crisis of our lifetime. But if the study is flawed, that unravelling is a good thing. It’s a very good thing. Because it might turn out that HCQ does save lives. If it does, we need to know that. And if it doesn’t, we need to know a hell of a lot more certainly than we do now.

And even if it turns out that the Surgisphere data are perfectly fine, making sure of that would have been a critical check to perform. We always want to make sure our data are sound, our methods are rational and our conclusions are solid. Even when we’re ultimately right. Especially when we’re ultimately right.

If the Surgisphere data turn out fine, I’ll turn this series into a trilogy to note that. But for now…it’s not looking good. And if it does turn out to be junk, it sets a critical line of research back to…well, not square one. But maybe square two.

Update: The paper has now been retracted.

Michael Siegel

Michael Siegel is an astronomer living in Pennsylvania. He is on Twitter, blogs at his own site, and has written a novel.

Related Post Roulette

15 Responses

  1. Jaybird says:

    Back in my young earth creationist days, one of the things we got into over and over again was stuff like pointing to the actual transcripts of the Scopes Monkey Trial and Inherit the Wind.

    “If this was Science!, then why did Science! need to lie?”, they asked us.

    It’s a good question. Rhetorically, I mean. The answer is somewhat complex and it involves confusing two (or more) groups of people and sweeping all of them together and all kinds of other little tricks. It’s an answer that takes a while to give.

    Anyway, it looks to me that the people who were reporting this sort of thing had goals that were easily gamed by companies inclined to game them.

    • Kazzy says:

      No group is perfect. There will always be errors and bad actors. What matters is how the group as a whole responds to the errors and bad actors.

      Science… to the extent it is a singular group… tends to respond well to both.

      • Philip H says:

        Remember cold fusion?

        • Kazzy says:

          No. Enlighten me.

          • Philip H says:

            In 1989, the scientific world was turned upside down when two researchers announced they had tamed the power of nuclear fusion in a simple electrolysis cell. The excitement quickly died when the scientific community came to a consensus that the findings weren’t real—“cold fusion” became a synonym for junk science. In the quarter-century since, a surprising number of researchers continue to report unexplainable excess heat effects in similar experiments, and several companies have announced plans to commercialize technologies, hoping to revolutionize the energy industry. Yet, no one has delivered on their promises. In the pages that follow, C&EN explores several possible conclusions: The claims are correct, but need more time to develop; those making the claims are committing an elaborate ruse; or it really is junk science that won’t go away.

            https://cen.acs.org/articles/94/i44/Cold-fusion-died-25-years.html

    • Philip H says:

      Back in my young earth creationist days

      Now there’s an opening you don’t see every day.

      • Jaybird says:

        I wrote an essay about it a million years ago.

        It’s one of the things that I experienced as a kid that colors how I see things today.

        If I wanted to spin it positively, I’d say that it’s given me memetic antibodies against a handful of things.

        If I wanted to spin it negatively, I’d say that it’s given me memetic antibodies against things that, seriously, I should actively want as part of my new and improved memetic system.

  2. Aaron David says:

    I absolutly agree that watching SCIENCE!! in real time is quite different than what we see in a film adaptation of a Crichton book. It is slow, it is frustrating, it is two steps forward and three steps back more than half the time. But, I do have one simple questions (that I am not really asking you, but in general)…

    Why? As we watch this whole thing unravel (again, in real time) who whould have done this?And why? This isn’t a question that science (exclamation point or not) can answer. Because, as you say, this drug could have been a real boon to the corona virus fight.

  3. Damon says:

    “The Lancet HCQ study, among others, uses data provided by Surgisphere. What is Surgisphere? Well, no one seems to know. It’s a small company with 5 to 11 employees, only one of whom has a science background.”

    Gee, maybe part of the problem is that the Lancet, allegedly well respected, used data from a tiny company with one scientist. What ever happened to peer review and such? I’d thing the folks at the Lancet would be questioning this small company even before beginning to review the data….in like “how did you come by this data”?

    And while we’re on the subject, doesn’t “science” still have a problem with the reproduceability of studies in all major fields? This incident will only further the nefarious assumptions of the public. Nice job Lancet.

  4. Oscar Gordon says:

    But this is a somewhat cocaine-binge version of how we do it…

    QFT

    To be honest, the problem is not the cocaine binge, it’s the fact that policy is being made and executed based upon the utterances of those currently bingeing *.

    And if that isn’t one for Murali’s Bad Analogy Takes Twitter feed…

  5. George Turner says:

    As the Amazing Randi always said, “Scientists are easier to fool than children.” Unlike suspicious folks at a circus, they don’t go in with the suspicion that the other scientist is a con-artist who has cleverly rigged the experimental equipment, or that the test subjects are in on the act, or that the data is a ruse.

    • Philip H says:

      That’s because 99% of the time the other scientists aren’t rigging equipment or test subjects or willfully rusing with bad data. Our normal professional processes do catch that 1% variance and generally keep it smacked down. Even here, the system mostly worked – the Guardian’s analysis came along side others beginning to ask questions. In prior times, we would have seen the journals eventually publish a retraction because the real scientists would have uncovered it.

      • Damon says:

        Yes, months or years later….and the press coverage of the correction will get 1/10 of the coverage that the original report got. Just like newspaper corrections.

        • Mr.Joe says:

          Let me know when you figure out how to get the press to properly assess… um well anything. Until then remember the press sucks and let the scientists science.

          • Damon says:

            Here’s a start for the news: Report the facts ONLY. Editorials belong on the editorial pages, not within “news reports”.

            The scientists can science all they want….but maybe they should, hmm, be more concerned about the getting the same results from repeating their tests before they go to the press and brag about breakthroughs.

Leave a Reply

Your email address will not be published. Required fields are marked *