FACT CHECK: Fact checking is bunk
Politicians lie. Even the good ones that you like. Wouldn’t it be great if we had some systematic way to investigate what politicians and other public figures say, check if they are actually true, and publicize the results? The net result would be to reward truth tellers and punish those who repeat lies.
Fact checking perhaps began with such pure motives, but it is now a mountain of garbage. I will review some of the garbage here.
I.
When you really think about it, true and false really kind of mean the same thing anyway: "Editor's note: This statement was rated Barely True when it was published. On July 27, 2011, we changed the name for the rating to Mostly False." https://t.co/c3HG8GEqyo
— Vikram Bath (@vikrambath1) December 11, 2018
When Obama made the statement that the border wall was “now basically complete”, Politifact considered that a good thing said by a good person. So, they rated the claim “mostly true.”
Later, when Donald Trump said he liked the wall, suddenly physical barriers became ineffective racist pipe dreams. Politifact then revised Obama’s statement about the same physical wall to “mostly false.”
II.
On September 15, 2004, the New York Times ran the headline “Memos on Bush Are Fake But Accurate, Typist Says.”
This was in reference to the clearly faked document that was produced in Microsoft Word. Dan Rather uncritically took them as damning evidence issues with George W. Bush’s service career, and he lost his career over it back when journalists promulgating falsehoods was still considered a no-no.
While this story wasn’t labeled as fact-checking, it’s nevertheless a story where the people who we expected to hold politicians accountable instead ran off uncritically with the first scrap of damaging info because it confirmed what they felt must be true of a political they hated.
III.
During his presidential campaign, Donald Trump claimed the real unemployment rate was 42 percent.
The Washington Post rated this as an egregious lie at Four Pinocchios.
Trump hints at his method in the interview with Time. “I saw a chart the other day, our real unemployment – because you have ninety million people that aren’t working,” he said. “Ninety-three million to be exact. If you start adding it up, our real unemployment rate is 42 percent.”
Trump may have seen a chart, but he misread it. Yes, the BLS shows that there are 93.7 million people “not in the work force,” but the vast majority of those people do not want to work. Most are retired or simply are not interested in working, such as stay-at-home parents.
The Post is correct that these are not how the Bureau of Labor Statistics measures unemployment, but is it an egregious lie to say retired people and stay-at-home parents are unemployed? They are in fact not employed. Trump’s definition of unemployment is certainly not the official one, but he makes clear that he was not referring to the official rate and instead some chart he saw. Additionally, the ratio of people who are not employed to total persons is also an important statistic whether we call it an “unemployment rate” or not. The truth of such a statement is necessarily subjective even if you have all the relevant information.
Nevertheless, all the fact-checkers stood in agreement. Politifact rated a similar claim from Trump that black youth unemployment was 59% as “mostly false.”
IV.
Let’s contrast the treatment of Trump with how Politifact treated Bernie Sanders’s claim that the black youth unemployment rate was 51%.
Sanders said that for African-Americans between the ages of 17 and 20, “the real unemployment rate … is 51 percent.” His terminology was off, but the numbers he used check out, and his general point was correct — that in an apples-to-apples comparison, African-American youth have significantly worse prospects in the job market than either Hispanics or whites do. The statement is accurate but needs clarification or additional information, so we rate it Mostly True.
This time, with a different speaker, we are able to make the same idea up to “mostly true.”
V.
Let’s also look at Hillary Clinton’s lie that the Benghazi attacks were the result of backlash to a controversial anti-Muslim YouTube video. This was rated only two Pinocchios, which is applied to fibs as minor as “Significant omissions and/or exaggerations. Some factual error may be involved but not necessarily.”
The Fact Checker compiled the first detailed timeline showing how administration statements had evolved on the Benghazi incident. For political reasons, it was in the White House’s interest to not portray the attacks as a terrorist incident, especially one that took place on the anniversary of the Sept. 11, 2001, attacks. Instead, the administration kept the focus on what was ultimately proved to be a political red herring — anger in the Arab world over an anti-Muslim video posted on YouTube. With key phrases and message discipline, the administration was able to conflate an attack on the U.S. Embassy in Egypt — which apparently was prompted by the video — with the deadly assault in Benghazi.
Is this truly a simple omission or exaggeration? It feels much closer to a whopper to me than wanting to count retirees as unemployed people.
VI.
Here is an NPR fact check of Hillary Clinton’s statement that gun-makers were “totally free of liability for their behavior.”
It begins very hip and promising:
We want to cut through the spin with a new feature we’re calling “Break It Down.”
I like things broken down. Let’s go!
While the article seems to provide the reader with sufficient information to make their own judgment, we are left with the following short answer:
Clinton is wrong that gun manufacturers have no liability for their products, but she’s right that they have unique protections from lawsuits that most other businesses — and particularly consumer product-makers — do not.
The way this is written, it seems like Clinton was wrong about one part but right about another, so it’s sort of neutral overall. However, the claim that the fact check purported to be actually about from the title was completely false. NPR, however, is giving near equal billing to other statements made in the vicinity of that lie that were not false.
What is the actual standard being applied here for something to be called a lie? Does it require every sentence in a speech need to be false in all aspects? That we can even ask these questions endangers the promise of a fact check giving a fair and complete answer.
VI.
Did Hitler win by election?
Bernie Sanders said so. The Washington Post disagreed awarding four full Pinocchios reserved for “Whoppers.”
But while there were many elections held in Germany in 1932, Hitler was a loser when he appeared on the ballot. He mostly gained power not through the voting booth but through violence, intimidation and deceit. Still, we wavered between Three and Four Pinocchios, given that the Nazis did gain seats in the parliament.
But we ultimately decided that calling it “an election” devalues the traumatic history of the era, and Sanders should clarify the record — especially when legions of his supporters are repeating his flawed history lesson all over the Internet.
Notice that the Post’s fact check not only calls Sanders a liar, but also says he “devalues the traumatic history of the era,” which one might note includes the Holocaust. In other words, Sanders not only lied, but he’s also a terrible human being.
Vox lashed back at the Post’s choice to call Sanders a liar:
Kessler’s argument basically boils down to the fact that when Adolf Hitler personally ran for Reichspresident in 1932, he lost to the incumbent Paul von Hindenburg. But this is obviously not what Sanders was referring to. He was referencing the fact that the Nazi Party, with Hitler as its leader, became the plurality party in the Reichstag, Germany’s lower house of parliament, in July 1932. And though the party lost seats that November, it retained its status as the largest party.
This is commonly referred to as “winning an election.”
[snip]
It is true that winning a plurality of seats does not guarantee that your party will lead the government. But, generally speaking, the party that wins more seats than any other party is the party that won that election, and its leader is the leader who won.
Can fact checkers disagree about what constitutes a whopper? I guess they can and do if there is a contentious primary going on.
If fact checkers can’t agree about such a important historical question that seems like it ought to have a straightforward answer, how can it be rightly called fact-checking at all?
VII.
I still don’t really know whether Politifact thinks there is a complete wall on the border or not. Perhaps fact checking had some primordial origin in a search for an uncomplicated truth. Perhaps every fact checker today still believes in this original ideal and tries to live up to it.
They are failing wildly. Even journalists themselves seem to have given up on the idea of fact checks and are instead cracking jokes about them:
WH press sec says "Pocahontas" is not a racial slur. (Fact check: it is.) https://t.co/tRW5H7ILcn
— Jim Acosta (@Acosta) November 27, 2017
I don’t blame them.
Couldn’t agree more with the piece. Too little attention is given to whether the “fact” being checked is subject to being falsifiable. Opinions generally are not. Most speech is too inexact to be checked; tweets are worse. Statements regarding the future can be wrong without having been false when made. People can frequently be sloppy, if not false, on tertiary points that don’t really impact the main point.
There is no value added by media engaging in most of this, but its cheaper than engaging policy analysis. The worst part is fostering the view that one’s political opponents are false, and thus false in all things, which is the ad hominem fallacy.Report
Yeah, this. To much of what people call lies are actually different opinions. It is possible to check facts but people are typically not talking facts or heavily mixing facts with opinion so a true fact check isn’t possible in most cases.
It would be good to check fact statements but only if they just focus on the fact part.Report
One thing that happens is that they end up doing something like the policy analysis, and then turn around and slap a topline rating of pants-on-firosity on the article that the analysis doesn’t support.
This seems like it will almost inevitably create some degree of apparent inconsistency, whether due to underlying bias of something else.Report
Indeed. What I think would be more useful in _normal_ politics is breaking the facts down without ultimately trying to come to an answer. For example, pointing out that Hilter _personally_ lost, but the party, with him as leader, was still in charge, is a reasonable thing to explain to people arguing if Sanders was right or wrong. Just explain it, without trying to decide how ‘correct’ it was. This would, in this case, pretty much prove his point, because the _implication_ of the statement was correct, regardless of how you want to parse the phrasing of his statement.
Likewise, the _implication_ of Trump’s statement was rather dubious, and explaining ‘Here are the actual unemployment numbers over the years and how we generally measure unemployment, and here’s what Trump measured instead’ would make it clear Trump was rather _misleading_ people regardless of whether his statement was true or not. Don’t decide if it’s true or not, just say ‘Here’s a graph of the normal unemployment numbers over the last 20 years, here’s what also is included in Trump’s number, here’s a graph of _those_ over the last 20 years, etc…’ and it becomes clear that Trump is pointing at an unemployment crisis that really doesn’t exist. But let the reader figure that out.
That’s actually the important thing. You can lie by telling truths, and you can make statements are, in any real sense, truthful representations of reality but can be _literally_ untruths. ‘I got up early this morning and watch as the sun rose.’, for example, is literally untruth (The sun was not, in fact, moving.), but everyone understands what you _mean_. The question is do people walk away with a reasonable understanding of the point you made, or a complete bullshit understanding? And the papers could ‘fact check’ that just by _giving_ the full details and readers would go, ‘Oh!’
Granted, I say this is what would be useful in normal politics. Plenty of things that Trump says are just complete, utter lies. It’s not some debatable thing, not something with wording. For example, he oftenj ust makes numbers biggers…he first rounds up, whch is perhaps debatable, and then just…adds numbers. Just adds them each time he says something. 2.2 billion will turn into 3, and then 4, and then 6. Or says something is X when it’s not X. At some point we do need someone to say ‘Yeah, there’s no possible way this statement can be interpreted as true’.
And we don’t need to confuse that with generally misleading-but-technically statements, even ones that Trump says. The entire concept gets diluted down when the news tries to define ‘wall’ and ‘mostly built’ and other vague things like that.Report
“fact checking”, like “fake news”, originally meant something specific but has now become just another Term To Describe The Things I Don’t Like.Report
Snopes debunked The Babylon Bee’s report thatU.S. Rep. Ocasio-Cortez Repeatedly Guessed ‘Free’ on TV Show ‘The Price is Right’.
As it turns out, she did *NOT* do that.
Check and Mate.Report
Man, if you can’t trust The Bee…Report
Well, evidently some people were circulating it as if it were on the level.
Snopes accurately describes the Bee as,”an entertainment website that does not publish factual content.”
I would love to live in a world where such checks were superfluous, but I’m pretty sure that I don’t.Report
I think the Bee spoke a greater truth, despite being wrong on an unimportant detail. When AOC was growing up, watching The Price is Right, she undoubtedly kept saying “That should be free!”
So their story is fake, but accurate. *Waves at Dan Rather*
Snopes also fact checked a Babylon Bee story that said Justin Smollet had been offered a job by CNN because they were impressed by his ability to fabricate a story out of thin air. Snopes rated the story as false.
Politifact looked at the Babylon Bee story “ISIS Lays Down Arms After Katy Perry’s Impassioned Plea To ‘Like, Just Co-Exist’” and rated it “Pants on Fire.”
Snopes fact checked an Onion story that said “A judge in Detroit ordered that a white woman should stand trial as an African American instead.” The story was rated false.Report
Fact Checking was just another way for wildly biased reporters to point to an “independent” source to justify their own bias. The Southern Poverty Law Center was a similar operation, but far more financially profitable.Report
“The first casualty of culture/political warfare was social truth.”Report
The problem is one of falsifiability.
An exceptionally good example is Carly Fiorina. (Now, please, keep in mind that I have no love for Carly Fiorina and have gone on record as saying that I would go back to voting for one of the two “real” parties and vote for Biden if he were running against Carly).
Back in 2015, she said this:
Well, Politifact fact checked this statement and gave it Three Pinocchios.
The WaPo defends giving three of them by saying (I mean, read the whole thing but this paragraph is representative):
Now, compare that to what she said.
I have absolutely no love for Carly.
But this example seems so very egregious that I don’t really see any reason to give Politifact any authority when it comes to checking facts at all. If they say something that I agree with I can snort, if they say something I disagree with, I can snarl… but I see no reason to change my mind because *THEY* said something. At best, they might be able to point me to some primary sources so I can decide for myself.
But I can’t trust them enough to outsource my willingness to determine facts to them. This doesn’t mean that I’m going to say that they’re always wrong, mind… but whether or not the statement is a lie or the truth is something that you’re going to have to determine without Politifact. Politifact isn’t going to be useful one way or the other.
What got falsified? Politifact did.Report
Also, Carly Fiorina is completely correct when she says that it is only possible to run for president of the United States in this country.Report
Here and Manchuria.Report
Andrés Manuel López Obrador is also a President of the United States, but not in this country.Report
I don’t think there is anything wrong with Washington Post filling out the details here. There is such a thing called the omission of material facts. It might not quite be a lie but it is doing things to make a narrative seem better for you or whatever your ends are. Fiorina’s statement makes it sound like she was the plucky secretary that could through gumption. But if it were merely a stint to get some cash between J.D. and M.B.A that is an omission of material facts.Report
There’s “filling out the details” and then there’s saying something that isn’t true, like when you say that someone’s true statement is actually a lie.Report
I mean, I’m down with the argument that says something like “while the statements she made were technically true, the unstated implications are false.”
Because, you know what? Heck with Carly.
I also understand that while there might be years where a partisan could argue that a plucky Republican businesswoman only got to where she was because of massive privilege, 2016 was *NOT* one of those years.
So WaPo did what WaPo felt it had to do.
Which is fine. Heck, Saul can still use it as a trustworthy source and not feel that he has to do additional research when he sees 1, or 4, or however many Pinocchios.
But when Politifact says “X” in the future, I have no real reason to believe that they’re not pulling something similar to what they pulled here.
Which is too bad because having fact-checkers *IS* important.
Important enough that it’s important to have ones that you can trust rather than have ones that you have to fact-check.Report
I don’t think it’s fine to call Fiorina a liar based on that. She said she was a secretary, and she was a secretary. Just because she didn’t meet someone’s stereotype of what a secretary should be and for what reasons exactly she should be working doesn’t mean that she wasn’t one. If reality contradicts your stereotypes, it’s not reality that’s wrong, it’s your stereotypesReport
But the stereotype is a cultural one. If someone says that they started in the mailroom and then became CEO, the assumption is that they did it through moxie and elbow grease.
If she hadn’t been secretary (perhaps if she had been flipping burgers or folding t-shirts) and gone to school, she still would have ended up as CEO.
Which would make her (hypothetical) statement that “I started as a burger-flipper, flipping burgers for a greasy spoon. It’s only in this country that you can go from flipping burgers to chief executive of the largest tech company in the world, and run for president of the United States. It’s only possible here.” just as false. Or just as true. Just as Three Pinocchioed.Report
So when Lincoln was described as a railsplitter, not a corporate lawyer, it was all lies? (A point usually raised by followers of Howard Zinn or Southern revisionists, but you won’t see them at the same parties)Report
Failure to fill in details is a tell.
Apparently.Report
I’d only give someone saying something like this grief, if it were say, Don Jr saying “I started mopping floors as a teenager and now I run the whole organization”
(and frankly, I’d give all the Trump kids and for that matter the elder Trump a bit more credit if any of them had actually done some of the elbow grease physical scut work as a apprenticeship in the family business)Report
I’d like to see a three-pronged approach:
Is the statement literally true or false?
Is the statement applicable to the original topic?
What is the most charitable reading of the statement within context?
ETA: I also don’t want to see a subjective Pinocchio count at the end of a detailed examination of an objective claim.Report
Yeah, that might be a good way to do it.
Here’s a couple of speeches that are fun to use the three-pronged test against:
Checkers.
Chappaquiddick.
They’re both really, really good speeches.
And that three-pronged approach would tear both to shreds.Report
It seems (again) the problem comes down to the need to dole out the Pinocchios, exacerbated by a need to appear unbiased. They supported their wacky decision by pointing at a similarly weird call where they gave Obama the same three Pinnocchios.Report
The need to appear unbiased didn’t really do them a whole lot of favors in this case.
Though I’m vaguely impressed with their ability to stand firm in the face of everybody telling them how wrong they got this. (The “we didn’t say she wasn’t a secretary like she said we said! We just gave her statement about how she was a secretary a three!” was a lovely little touch.)Report
Ny favorite Carly entirely true statement is that she doubled revenue at HP.
She had HP buy Compaq, which had about the same yearly revenue as HP. Was this a good idea? Probably not, but in the very, vert short term it sure did double revenue.Report
I love this, thanks for writing.
The fun thing is when (and I have since stopped doing this, since it’s pointless) one sees a friend or family member post something blatantly untrue or misleading, and you yourself debunk it using actual sources, only to see them turn around the next day and post another untrue or misleading story/meme on the same subject.
At that point, it becomes apparent that some people are actually deliberately posting fake news to support a particular agenda and I am just really not too sure what to DO with that.
Strange days, or maybe they always were strange and I just never knew before.Report
One of the problems with the journalist fact checkers is that they often have an ax to grind or a strongly held bias, especially a strong unconscious bias. This becomes glaringly apparent in elections when they bend into a pretzel to rate their favored candidates as truthful and the opposing candidates as lying.
That kind of problem shows up in other places, such as a Wiki page for some controversial story or person that’s in the news cycle, with the pros and antis battling it out hour by hour, and the page’s “truth” constantly in flux until someone higher up, a disinterested third party, locks the page.
Fact checking needs disinterested third parties, and they need to be pedantic nerds whose background is something like real estate law, library science, or baseball stats. Ideally they live in Estonia or Madagascar, to make them even more of an outside observer, but very familiar with US politics in the way that an entymology geek studies an ant colony.
But nobody in the media seems to employ those.Report
Last night I read a comment that was a wonderful fact check of a story that appeared on the website for the local paper in South Pasadena.
The story was a glowing piece with lots of indulgent photographs of city leaders (the mayor and the city council) greeting their African visitor, the Queen of the Democratic Republic of Congo, Diambi Mukalenga Mukaji Wa Nkashama. The text went into some depth about their backgrounds with a heartfelt bio. (The story’s url was “african-royalty-comes-to-town-drc-queen-visits-south-pasadena”). The story related how the city officials gave the Queen royal treatment.
Well, the puff piece had only one comment, from a local who pointed out that the Democratic Republic of Congo doesn’t have a queen, is called a “Democratic Republic” for a reason, and to back that up, he’d verified his opinion by contacting the State Department, who confirmed it completely. He also cited the CIA Factbook and another source, and pointed out that the queen only has about a hundred followers on Twitter and LinkedIn.
It seems the city leaders of South Pasadena got played, and a few minutes ago when I went to check the story (which I’d saved in ‘favorites’) I got a 404 error. ^_^
I’ll bet the paper got a call from the mayor and other officials, desperate to do some damage control and trying to avoid being the laughing stock of Southern California.Report
In 1987, Irwindale gave Al Davis $10 million as a “guarantee against future revenues” because was really, honestly, truly considering moving the Raiders there. It would really have put a town of 1,100 people on the map!
Getting conned out of some face time with no monetary consequences does not even get into the running for “laughing stock of Southern California”.Report
Agree with most of this but … I still think fact-checking is important. There are claims out there — 300,000 sex slaves for examples — that are complete garbage. I do agree, however, that fact checkers need to limit themselves to very bare facts and be aware of their biases.
One example I encountered recently was a claim for Harry Reid that the Dem Senate invoke cloture more often on judges than all previous senates combined. It was true, but misleading. Reid was invoking cloture on every single judge, including those that eventually passed by voice vote. And the fact checkers quoted the research paper that had these numbers without noticing that the previous page pointed out that the most filibusters had been made by .. the Democrats under Bush.
Compromise: keep fact checking, but eliminate the ratings. It’s good to explore the subtleties of a claim without having to jump into “pants on fire” when someone disagrees.Report
Something to point out is most of these howlers are in the context of highly charged/important emotional situations, i.e. HRC running for Prez against Sanders (several of them), or Trump in general.
I want this to be true(false), ergo I will look for reasons why it’s true(false).
Or worse, the truth is less important than the election, if I do a little damage to my rep then I’ll apologize later.Report
This is all great, but if I may
fact check – Politifact originally rated the claim by Obama “Barely True”, not “mostly true”.
and they changed it to mostly false well before (about 4 years before) Donald Trump finally seriously started to run for President.Report