On Reinhart and Rogoff

Avatar

Elias Isquith

Elias Isquith is a freelance journalist and blogger. He considers Bob Dylan and Walter Sobchak to be the two great Jewish thinkers of our time; he thinks Kafka was half-right when he said there was hope, "but not for us"; and he can be reached through the twitter via @eliasisquith or via email. The opinions he expresses on the blog and throughout the interwebs are exclusively his own.

Related Post Roulette

11 Responses

  1. Avatar Barry says:

    One of the key points at http://www.nextnewdeal.net/rortybomb/researchers-finally-replicated-reinhart-rogoff-and-there-are-serious-problems#.UW147o4A23o.twitter

    “They find that three main issues stand out. First, Reinhart and Rogoff selectively exclude years of high debt and average growth. Second, they use a debatable method to weight the countries. Third, there also appears to be a coding error that excludes high-debt and average-growth countries. All three bias in favor of their result, and without them you don’t get their controversial result. Let’s investigate further:”
    (bolding is mine)

    So it looks like they selectively trimmed the data.

    Second, their weighting was extreme. There’s a question on how to weight country A for 19 years vs. country B for 1 year. A 19:1 weight really assumes independence between successive years for country A, but treating 19 years as the same as 1 years assumes that there is no year-to-year variation. So on the scale of non-crazy weightings (1:1 or 19:1), they chose an extreme, which favored the same hypothesis.

    At this point their works is really suspect, and not just for innocent Excel errors.Report

  2. Avatar trizzlor says:

    I think it’s important to keep in mind the Statistical Winner’s Curse or regression towards the mean phenomena the often happens in academic research. Imagine there’s some well-known law that many research groups are testing, with hundreds of independent experiments all coming up negative and therefore not being published (we already knew that!). One group, purely by chance, gets a result that has a less than 5% chance of having occurred under the law – statistically significant – and they go ahead and publish this result “refuting” the law. In reality, their result should be adjusted for the other hundreds of attempts that have been made and failed, but because those negative results are unpublished the group has done absolutely nothing wrong in interpreting their findings.

    When you’re thinking about weather Reinhart and Rogoff made these mistakes by accident (or by chance) it’s not enough to say “What’s the chance that a researcher would make these three mistakes” but rather “What’s the chance that out of many researchers studying this, one would make these three mistakes“. As Barry points out up-thread, accidental coding errors happen, but it’s the selective exclusion of data that should have us concerned.Report

    • Avatar Barry says:

      It’s the multiple ‘errors’, all in the same direction.Report

      • Avatar trizzlor says:

        It’s the multiple ‘errors’, all in the same direction.

        But that’s the only way it could go. Think of it as a kind of evolution: hundreds of groups are examining the data and making small errors, with most of them leading to a negative result and no publications; only those groups that accumulate small errors leading to a new finding will publish. So it will invariably look like the published results are biased towards controversy when in reality we just never see the stuff that’s equally biased away from controversy to even it out.Report

  3. Avatar trizzlor says:

    By the way, can we finally bury the idea that Paul Ryan is some kind of numbers wonk when he says things like this about an association:

    Even if high debt did not cause a crisis, however, the Nation would still be in for a long and grinding period of economic decline. A recent study completed by Reinhart and economist Kenneth S. Rogoff of Harvard confirms this common-sense conclusion. The study found conclusive empirical evidence that total debt exceeding 90 percent of the economy has a significant negative effect on economic growth.

    Report

    • Avatar Barry says:

      Paul Ryan was only a numbers guy in the sense that he could spout off numbers. Everything he ever said was shredded the minute anybody got out their calculator.

      Journalists are easy to BS with numbers, and even easier to BS when the elites want them to be.Report

  4. Avatar ktward says:

    I’m not an economist. Not even close. But for whatever my own peculiar OCD reasons, I’m a stickler for both facts and, especially, unbiased analysis– inasmuch either are doable, and equally inasmuch as I’m capable of discernment. Muddy waters, that.

    Anyhoo, I’ve long waded through all manner of internet detritus to find a few expert folks, of varying expertise, who seem to share my own OCD compulsions. Which more or less explains why I read the good folks over at The Monkey Cage, even though, as a rule, I avoid their kind like they were the plague because this stuff is so painfully foreign to me.

    Gelman offers, imo, the sagest advice possible to R & R in this circumstance: admit you f**ked up, move on. Because really, if R & R don’t come clean, what chance do they have of holding on to any legitimacy?

    Then again, if R & R are ultimately influenced by an ideological agenda that leaves them unconcerned with their standing vis legitimacy, especially among their peers. then I suppose we’ll come to learn that soon enough.Report

    • Avatar Kimmi says:

      Yeah. And add to that “admit you fucked up about ‘holding the course'”. We can all be bullheaded (well, maybe me more than most). But you don’t hold onto credibility like that.Report

      • Avatar ktward says:

        It seems to be that cred is the mitigating factor. I mean, you and I don’t care if our opinions carry any weight, but surely R & R care. I mean, it’s how they earn a living. No cred? No living.

        Then again, there’s always Fox News. They’ve a growing list of discredited dunderheads who appear as “experts”.Report

        • Avatar Barry says:

          As you’ve pointed out, right-wingers with opinions favorable to the elites will be taken care of. Also, this is macroeconomics – has any right-wing error *ever* been punished?Report

Leave a Reply

Your email address will not be published. Required fields are marked *