FACT CHECK: Fact checking is bunk

Vikram Bath

Vikram Bath is the pseudonym of a former business school professor living in the United States with his wife, daughter, and dog. (Dog pictured.) His current interests include amateur philosophy of science, business, and economics. Tweet at him at @vikrambath1.

Related Post Roulette

35 Responses

  1. PD Shaw says:

    Couldn’t agree more with the piece. Too little attention is given to whether the “fact” being checked is subject to being falsifiable. Opinions generally are not. Most speech is too inexact to be checked; tweets are worse. Statements regarding the future can be wrong without having been false when made. People can frequently be sloppy, if not false, on tertiary points that don’t really impact the main point.

    There is no value added by media engaging in most of this, but its cheaper than engaging policy analysis. The worst part is fostering the view that one’s political opponents are false, and thus false in all things, which is the ad hominem fallacy.Report

    • greginak in reply to PD Shaw says:

      Yeah, this. To much of what people call lies are actually different opinions. It is possible to check facts but people are typically not talking facts or heavily mixing facts with opinion so a true fact check isn’t possible in most cases.

      It would be good to check fact statements but only if they just focus on the fact part.Report

    • pillsy in reply to PD Shaw says:

      One thing that happens is that they end up doing something like the policy analysis, and then turn around and slap a topline rating of pants-on-firosity on the article that the analysis doesn’t support.

      This seems like it will almost inevitably create some degree of apparent inconsistency, whether due to underlying bias of something else.Report

      • DavidTC in reply to pillsy says:

        Indeed. What I think would be more useful in _normal_ politics is breaking the facts down without ultimately trying to come to an answer. For example, pointing out that Hilter _personally_ lost, but the party, with him as leader, was still in charge, is a reasonable thing to explain to people arguing if Sanders was right or wrong. Just explain it, without trying to decide how ‘correct’ it was. This would, in this case, pretty much prove his point, because the _implication_ of the statement was correct, regardless of how you want to parse the phrasing of his statement.

        Likewise, the _implication_ of Trump’s statement was rather dubious, and explaining ‘Here are the actual unemployment numbers over the years and how we generally measure unemployment, and here’s what Trump measured instead’ would make it clear Trump was rather _misleading_ people regardless of whether his statement was true or not. Don’t decide if it’s true or not, just say ‘Here’s a graph of the normal unemployment numbers over the last 20 years, here’s what also is included in Trump’s number, here’s a graph of _those_ over the last 20 years, etc…’ and it becomes clear that Trump is pointing at an unemployment crisis that really doesn’t exist. But let the reader figure that out.

        That’s actually the important thing. You can lie by telling truths, and you can make statements are, in any real sense, truthful representations of reality but can be _literally_ untruths. ‘I got up early this morning and watch as the sun rose.’, for example, is literally untruth (The sun was not, in fact, moving.), but everyone understands what you _mean_. The question is do people walk away with a reasonable understanding of the point you made, or a complete bullshit understanding? And the papers could ‘fact check’ that just by _giving_ the full details and readers would go, ‘Oh!’

        Granted, I say this is what would be useful in normal politics. Plenty of things that Trump says are just complete, utter lies. It’s not some debatable thing, not something with wording. For example, he oftenj ust makes numbers biggers…he first rounds up, whch is perhaps debatable, and then just…adds numbers. Just adds them each time he says something. 2.2 billion will turn into 3, and then 4, and then 6. Or says something is X when it’s not X. At some point we do need someone to say ‘Yeah, there’s no possible way this statement can be interpreted as true’.

        And we don’t need to confuse that with generally misleading-but-technically statements, even ones that Trump says. The entire concept gets diluted down when the news tries to define ‘wall’ and ‘mostly built’ and other vague things like that.Report

  2. DensityDuck says:

    “fact checking”, like “fake news”, originally meant something specific but has now become just another Term To Describe The Things I Don’t Like.Report

  3. Jaybird says:

    Snopes debunked The Babylon Bee’s report thatU.S. Rep. Ocasio-Cortez Repeatedly Guessed ‘Free’ on TV Show ‘The Price is Right’.

    As it turns out, she did *NOT* do that.

    Check and Mate.Report

    • Oscar Gordon in reply to Jaybird says:

      Man, if you can’t trust The Bee…Report

      • pillsy in reply to Oscar Gordon says:

        Well, evidently some people were circulating it as if it were on the level.

        Snopes accurately describes the Bee as,”an entertainment website that does not publish factual content.”

        I would love to live in a world where such checks were superfluous, but I’m pretty sure that I don’t.Report

      • George Turner in reply to Oscar Gordon says:

        I think the Bee spoke a greater truth, despite being wrong on an unimportant detail. When AOC was growing up, watching The Price is Right, she undoubtedly kept saying “That should be free!”

        So their story is fake, but accurate. *Waves at Dan Rather*

        Snopes also fact checked a Babylon Bee story that said Justin Smollet had been offered a job by CNN because they were impressed by his ability to fabricate a story out of thin air. Snopes rated the story as false.

        Politifact looked at the Babylon Bee story “ISIS Lays Down Arms After Katy Perry’s Impassioned Plea To ‘Like, Just Co-Exist’” and rated it “Pants on Fire.”

        Snopes fact checked an Onion story that said “A judge in Detroit ordered that a white woman should stand trial as an African American instead.” The story was rated false.Report

  4. George Turner says:

    Fact Checking was just another way for wildly biased reporters to point to an “independent” source to justify their own bias. The Southern Poverty Law Center was a similar operation, but far more financially profitable.Report

  5. JoeSal says:

    “The first casualty of culture/political warfare was social truth.”Report

  6. Jaybird says:

    The problem is one of falsifiability.

    An exceptionally good example is Carly Fiorina. (Now, please, keep in mind that I have no love for Carly Fiorina and have gone on record as saying that I would go back to voting for one of the two “real” parties and vote for Biden if he were running against Carly).

    Back in 2015, she said this:

    “I started as a secretary, typing and filing for a nine-person real estate firm. It’s only in this country that you can go from being a secretary to chief executive of the largest tech company in the world, and run for president of the United States. It’s only possible here.”

    Well, Politifact fact checked this statement and gave it Three Pinocchios.

    The WaPo defends giving three of them by saying (I mean, read the whole thing but this paragraph is representative):

    She worked briefly as a secretary in between law school and business school, but she always intended to attend graduate school for her career. She moved up through AT&T with her MBA, and was placed on a fast track to senior management after her company sponsored her to attend one of the most elite mid-career fellowships in the world. Her role as senior executive at Lucent caught the attention of HP recruiters, to become the company’s chief executive.

    Now, compare that to what she said.

    I have absolutely no love for Carly.

    But this example seems so very egregious that I don’t really see any reason to give Politifact any authority when it comes to checking facts at all. If they say something that I agree with I can snort, if they say something I disagree with, I can snarl… but I see no reason to change my mind because *THEY* said something. At best, they might be able to point me to some primary sources so I can decide for myself.

    But I can’t trust them enough to outsource my willingness to determine facts to them. This doesn’t mean that I’m going to say that they’re always wrong, mind… but whether or not the statement is a lie or the truth is something that you’re going to have to determine without Politifact. Politifact isn’t going to be useful one way or the other.

    What got falsified? Politifact did.Report

    • Fish in reply to Jaybird says:

      Also, Carly Fiorina is completely correct when she says that it is only possible to run for president of the United States in this country.Report

    • Saul Degraw in reply to Jaybird says:

      I don’t think there is anything wrong with Washington Post filling out the details here. There is such a thing called the omission of material facts. It might not quite be a lie but it is doing things to make a narrative seem better for you or whatever your ends are. Fiorina’s statement makes it sound like she was the plucky secretary that could through gumption. But if it were merely a stint to get some cash between J.D. and M.B.A that is an omission of material facts.Report

      • DensityDuck in reply to Saul Degraw says:

        There’s “filling out the details” and then there’s saying something that isn’t true, like when you say that someone’s true statement is actually a lie.Report

        • Jaybird in reply to DensityDuck says:

          I mean, I’m down with the argument that says something like “while the statements she made were technically true, the unstated implications are false.”

          Because, you know what? Heck with Carly.

          I also understand that while there might be years where a partisan could argue that a plucky Republican businesswoman only got to where she was because of massive privilege, 2016 was *NOT* one of those years.

          So WaPo did what WaPo felt it had to do.

          Which is fine. Heck, Saul can still use it as a trustworthy source and not feel that he has to do additional research when he sees 1, or 4, or however many Pinocchios.

          But when Politifact says “X” in the future, I have no real reason to believe that they’re not pulling something similar to what they pulled here.

          Which is too bad because having fact-checkers *IS* important.

          Important enough that it’s important to have ones that you can trust rather than have ones that you have to fact-check.Report

          • Vikram Bath in reply to Jaybird says:

            I don’t think it’s fine to call Fiorina a liar based on that. She said she was a secretary, and she was a secretary. Just because she didn’t meet someone’s stereotype of what a secretary should be and for what reasons exactly she should be working doesn’t mean that she wasn’t one. If reality contradicts your stereotypes, it’s not reality that’s wrong, it’s your stereotypesReport

            • Jaybird in reply to Vikram Bath says:

              But the stereotype is a cultural one. If someone says that they started in the mailroom and then became CEO, the assumption is that they did it through moxie and elbow grease.

              If she hadn’t been secretary (perhaps if she had been flipping burgers or folding t-shirts) and gone to school, she still would have ended up as CEO.

              Which would make her (hypothetical) statement that “I started as a burger-flipper, flipping burgers for a greasy spoon. It’s only in this country that you can go from flipping burgers to chief executive of the largest tech company in the world, and run for president of the United States. It’s only possible here.” just as false. Or just as true. Just as Three Pinocchioed.Report

              • PD Shaw in reply to Jaybird says:

                So when Lincoln was described as a railsplitter, not a corporate lawyer, it was all lies? (A point usually raised by followers of Howard Zinn or Southern revisionists, but you won’t see them at the same parties)Report

              • Jaybird in reply to PD Shaw says:

                Failure to fill in details is a tell.

                Apparently.Report

              • Kolohe in reply to Jaybird says:

                I’d only give someone saying something like this grief, if it were say, Don Jr saying “I started mopping floors as a teenager and now I run the whole organization”

                (and frankly, I’d give all the Trump kids and for that matter the elder Trump a bit more credit if any of them had actually done some of the elbow grease physical scut work as a apprenticeship in the family business)Report

    • Pinky in reply to Jaybird says:

      I’d like to see a three-pronged approach:
      Is the statement literally true or false?
      Is the statement applicable to the original topic?
      What is the most charitable reading of the statement within context?

      ETA: I also don’t want to see a subjective Pinocchio count at the end of a detailed examination of an objective claim.Report

    • pillsy in reply to Jaybird says:

      It seems (again) the problem comes down to the need to dole out the Pinocchios, exacerbated by a need to appear unbiased. They supported their wacky decision by pointing at a similarly weird call where they gave Obama the same three Pinnocchios.Report

      • Jaybird in reply to pillsy says:

        The need to appear unbiased didn’t really do them a whole lot of favors in this case.

        Though I’m vaguely impressed with their ability to stand firm in the face of everybody telling them how wrong they got this. (The “we didn’t say she wasn’t a secretary like she said we said! We just gave her statement about how she was a secretary a three!” was a lovely little touch.)Report

    • Mike Schilling in reply to Jaybird says:

      Ny favorite Carly entirely true statement is that she doubled revenue at HP.

      She had HP buy Compaq, which had about the same yearly revenue as HP. Was this a good idea? Probably not, but in the very, vert short term it sure did double revenue.Report

  7. I love this, thanks for writing.

    The fun thing is when (and I have since stopped doing this, since it’s pointless) one sees a friend or family member post something blatantly untrue or misleading, and you yourself debunk it using actual sources, only to see them turn around the next day and post another untrue or misleading story/meme on the same subject.

    At that point, it becomes apparent that some people are actually deliberately posting fake news to support a particular agenda and I am just really not too sure what to DO with that.

    Strange days, or maybe they always were strange and I just never knew before.Report

  8. George Turner says:

    One of the problems with the journalist fact checkers is that they often have an ax to grind or a strongly held bias, especially a strong unconscious bias. This becomes glaringly apparent in elections when they bend into a pretzel to rate their favored candidates as truthful and the opposing candidates as lying.

    That kind of problem shows up in other places, such as a Wiki page for some controversial story or person that’s in the news cycle, with the pros and antis battling it out hour by hour, and the page’s “truth” constantly in flux until someone higher up, a disinterested third party, locks the page.

    Fact checking needs disinterested third parties, and they need to be pedantic nerds whose background is something like real estate law, library science, or baseball stats. Ideally they live in Estonia or Madagascar, to make them even more of an outside observer, but very familiar with US politics in the way that an entymology geek studies an ant colony.

    But nobody in the media seems to employ those.Report

  9. George Turner says:

    Last night I read a comment that was a wonderful fact check of a story that appeared on the website for the local paper in South Pasadena.

    The story was a glowing piece with lots of indulgent photographs of city leaders (the mayor and the city council) greeting their African visitor, the Queen of the Democratic Republic of Congo, Diambi Mukalenga Mukaji Wa Nkashama. The text went into some depth about their backgrounds with a heartfelt bio. (The story’s url was “african-royalty-comes-to-town-drc-queen-visits-south-pasadena”). The story related how the city officials gave the Queen royal treatment.

    Well, the puff piece had only one comment, from a local who pointed out that the Democratic Republic of Congo doesn’t have a queen, is called a “Democratic Republic” for a reason, and to back that up, he’d verified his opinion by contacting the State Department, who confirmed it completely. He also cited the CIA Factbook and another source, and pointed out that the queen only has about a hundred followers on Twitter and LinkedIn.

    It seems the city leaders of South Pasadena got played, and a few minutes ago when I went to check the story (which I’d saved in ‘favorites’) I got a 404 error. ^_^

    I’ll bet the paper got a call from the mayor and other officials, desperate to do some damage control and trying to avoid being the laughing stock of Southern California.Report

    • In 1987, Irwindale gave Al Davis $10 million as a “guarantee against future revenues” because was really, honestly, truly considering moving the Raiders there. It would really have put a town of 1,100 people on the map!

      Getting conned out of some face time with no monetary consequences does not even get into the running for “laughing stock of Southern California”.Report

  10. Agree with most of this but … I still think fact-checking is important. There are claims out there — 300,000 sex slaves for examples — that are complete garbage. I do agree, however, that fact checkers need to limit themselves to very bare facts and be aware of their biases.

    One example I encountered recently was a claim for Harry Reid that the Dem Senate invoke cloture more often on judges than all previous senates combined. It was true, but misleading. Reid was invoking cloture on every single judge, including those that eventually passed by voice vote. And the fact checkers quoted the research paper that had these numbers without noticing that the previous page pointed out that the most filibusters had been made by .. the Democrats under Bush.

    Compromise: keep fact checking, but eliminate the ratings. It’s good to explore the subtleties of a claim without having to jump into “pants on fire” when someone disagrees.Report

  11. Dark Matter says:

    Something to point out is most of these howlers are in the context of highly charged/important emotional situations, i.e. HRC running for Prez against Sanders (several of them), or Trump in general.

    I want this to be true(false), ergo I will look for reasons why it’s true(false).

    Or worse, the truth is less important than the election, if I do a little damage to my rep then I’ll apologize later.Report

  12. Kolohe says:

    This is all great, but if I may

    When Obama made the statement that the border wall was “now basically complete”, Politifact considered that a good thing said by a good person. So, they rated the claim “mostly true.”

    Later, when Donald Trump said he liked the wall, suddenly physical barriers became ineffective racist pipe dreams. Politifact then revised Obama’s statement about the same physical wall to “mostly false.”

    fact check – Politifact originally rated the claim by Obama “Barely True”, not “mostly true”.

    and they changed it to mostly false well before (about 4 years before) Donald Trump finally seriously started to run for President.Report