PolitiFact as Rumpelstiltskin.

“Round about, round about,
Lo and behold!
Reel away, reel away,
Straw into gold!”

PolitiFact's Nov. 19, 2019 fact check of something President Donald Trump said on the Dan Bongino Show gives us yet another example of a classic fact checker error, the mistake of interpreting ambiguous statements as clear statements.

Here's PolitiFact's presentation of a statement it found worthy of a fact check:

In an interview with conservative show host Dan Bongino, Trump said a false rendition of that call by House Intelligence chairman Adam Schiff, D-Calif., forced him to release the readout of that call.

"They never thought, Dan, that I was going to release that call, and I really had no choice because Adam Schiff made up a call," Trump said Nov. 15. "He said the president said this, and then he made up a call."

The problem with Trump’s statement is that Schiff spoke after the White House released the memo of the phone call, not before.

 Note that PolitiFact finds a timeline problem with Trump's claim.

But also note that Trump makes no clear statement regarding a timeline. If Trump said "I released the transcript after Schiff did his 'parody' version of the telephone call," then he would have established an order of events. Trump's words imply an order of events, but it is not a hard logical implication (A, therefore B).

PolitiFact treats the case exactly like a hard implication.

Here's why that's the wrong approach.

First, significant ambiguity should always slow a fact-checker's progress toward an interpretation.

Second, Trump gave a speech on Sept 24, 2019 that announced the impending release of the transcript (memorandum of telephone conversation). The "transcript" was released on Sept. 25. Schiff gave his "parody" account of the call the next day, on Sept. 26.  And Trump responded to Schiff's "parody" version of his call on Sept. 30 during an event honoring the late Justice Antonin Scalia:

Adam Schiff — representative, congressman — made up what I said.  He actually took words and made it up.  The reason is, when he saw my call to the President of Ukraine, it was so good that he couldn’t quote from it because it — there was nothing done wrong.  It was perfect.

PolitiFact's interpretation asks us to believe that Trump either forgot what he said on Sept. 30 or else deliberately decided to reverse the chronology.

What motive would support that decision? Is one version more politically useful than the other?

It's not uncommon for people to speak of "having no choice" based on an event subsequent to that choice. The speaker means that the choice would have had to take place eventually.

When a source makes two claims touching the same subject and differ in content, the following rule applies: Use the more clear statement to make sense of the less clear statement.

Fact checkers who manufacture certitudes out of equivocal language give fact-checking a bad name.

They are Rumpelstiltskins, trying to spin straw into gold.


Afters

We would draw attention to a parallel highlighted at (Bryan's) Zebra Fact Check last month.

During a podcast interview Hillary Clinton used equivocal language in saying "they" were grooming Democratic Party presidential hopeful Tulsi Gabbard as a third-party candidate to enhance Trump's chances of winning the 2020 election.

No fact checker picked out Clinton's claim for a fact check. And that's appropriate, because the identity of "they" was never perfectly clear. Clinton probably meant the Russians, but "probably" doesn't make it a fact.

In that case, the fact checkers picked on those who interpreted Clinton to mean the Russians were grooming Gabbard (implicitly finding that Clinton's ambiguity clearly meant "Republicans").

Fact checkers have no business doing such things.

Until fact checkers can settle on a consistent approach to their craft, we justifiably view it as a largely subjective enterprise.

We want Bill Adair subjected to the "Flip-O-Meter"

It wasn't that long ago we reported on Bill Adair's article for Columbia Journalism Review declaring "Bias is good" along with a chart indicating fact-check journalism has more opinion to it than either investigative reporting or news analysis.

Yet WRAL, in announcing its new partnership with PolitiFact North Carolina, quoted Adair saying PolitiFact is unbiased:

“What is important about PolitiFact is not just that it’s not biased,” Adair said, “but that we show our work and that we show all of our sources.”

Naturally we cannot allow that to pass. We used WRAL's contact form to reach out to the writer of the article, Ashley Talley.

We pointed out the discrepancy between what Talley reported from Adair and what Adair wrote for Columbia Journalism Review. We suggested somebody should fact check Adair.

Next we'll be contacting Paul Specht of PolitiFact North Carolina over this quotation:

“One thing I love about PolitiFact is that the format is very structured and it's not up to me to decide what is or isn't true,” said Paul Specht, WRAL’s PolitiFact reporter who has been covering local, state and national politics for years. “It's up to me to go do the research and then it's up to the research to tell us what is true.”

We're not sure how that's supposed to square with Adair's declaration from a few years ago that "Lord knows the decision about a Truth-O-Meter rating is entirely subjective."

What changed?

In addition to its "Truth-O-Meter" PolitiFact publishes "Flip-O-Meter" items.

We'd like to see Adair on the Flip-O-Meter.

PolitiFact editor: "It’s important to note that we don’t do a random or scientific sample"

As we have mentioned before, we love it when PolitiFact's movers and shakers do interviews. It nearly guarantees us good material.

PolitiFact Editor Angie Drobnic Holan appeared on Galley by CJR (Columbia Journalism Review) with Mathew Ingram to talk about fact-checking.

During the interview Ingram asked about PolitiFact's process for choosing which facts to check (bold emphasis added):

MI
One question I've been asking many of our interview guests is how they choose which lies or hoaxes or false reports to fact-check when there are just so many of them? And do you worry about the possibility of amplifying a fake news story by fact-checking it? This is a problem Whitney Phillips and Joan Donovan have warned about in interviews I've done with them about this topic.

ADH
Great questions! We use our news judgement to decide what to fact-check, with the main criteria being that it’s a topic in the news and it’s something that would make a regular person say, "Hmmm, I wonder if that’s true." If it sounds wrong, we’re even more eager to do it. It’s important to note that we don’t do a random or scientific sample.

It's important, Holan says, to note that PolitiFact does not do a random or scientific sample when it chooses the topics for its fact check stories.

We agree wholeheartedly with Holan's statement in bold. And that's an understatment. We've been harping for years on PolitiFact's failure to make its non-scientific foundations clear to its audience. And here Holan apparently agrees with us by saying it's important.

How important is it?

PolitiFact's statement of principles says PolitiFact uses news judgment to pick out stories, and also mentions the "Is that true?" standard Holan mentions in the above interview segment. But what you won't find in PolitiFact's statement of principles is any kind of plain admission that its process is neither random nor scientific.

If it's important to note those things, then why doesn't the statement of principles mention it?

At PolitiFact, it's so important to note that its story selection is neither random nor scientific that no example from three pages of Google hits in the politifact.com domain when searching for "random" AND "scientific" has anything to do with PolitiFact's method for story selection.

And despite commenters on PolitiFact's Facebook page commonly interpreting candidate report cards as representative of all of a politician's statements, Holan insists "There's not a lot of reader confusion" about it.
If there's not a lot of reader confusion about it, why say it's important to note that the story selection isn't random or scientific? People supposedly already know that.

We use the tag "There's Not a Lot of Reader Confusion" on occasional stories pointing out that people do suffer confusion about it because PolitiFact doesn't bother to explain it.

Post a chart of collected "Truth-O-Meter" ratings and there's a good chance somebody in the comments will extrapolate the data to apply to all of a politician's statements.

We say it's inexcusable that PolitiFact posts its charts without making their unscientific basis clear to readers.

They just keep right on doing it, even while admitting it's important that people realize a fact about the charts that PolitiFact rarely bothers to explain.

Remember back when it was False to say Nixon was impeached?

I remember reading a story years back about a tire company that enterprisingly tried to drum up business by sending out a team to spread roofing nails on the local roads.

Turns out there's a version of that technique in PolitiFact's fact-checking tool box.

Nixon was Never Impeached

Back on June 13th, 2019 PolitiFact's PunditFact declared it "False" that Nixon was impeached. PunditFact said "Nixon was never officially impeached." We're not sure what would count as "unofficially impeached." We're pretty sure it's the same as saying Nixon was not impeached.

But that was way back in June. Over three months have passed. And it's now sufficiently true that Nixon was impeached so that PolitiFact can spread the idea on Twitter and write an impeachment PolitiSplainer that refers multiple times to the Nixon impeachment.

Nixon was Impeached

Twitter

Johnson, Nixon, Clinton, Trump: We answer nine questions about impeachment. https://t.co/fzgdYA0iqE pic.twitter.com/MLh6cODpuN

— PolitiFact (@PolitiFact) October 12, 2019

Edit: (if embed isn't working use hotlink above)

Is Nixon a good example to include with Johnson and Clinton (let alone Trump) if Nixon wasn't impeached?

More than anything, the procedural details are derived from historical precedent, from the impeachment of President Andrew Johnson in the 1860s to that of President Richard Nixon in the 1970s and President Bill Clinton in the 1990s.

Got it? The impeachment of President Nixon. Because Nixon was impeached, right?

Experts pointed to a variety of differences between the Trump impeachment process and those that went before.

The differences begin with the substance of the charges. All prior presidential impeachments have concerned domestic issues — the aftermath of the Civil War in Johnson’s case, the Watergate burglary and coverup under Nixon, and the Monica Lewinsky affair for Clinton.

Got it? Nixon was impeached over the Watergate burglary. Because Nixon was impeached, right?

The impeachments of both Nixon and Clinton did tend to curb legislative action by soaking up all the attention in Washington, historians say.

Obviously a fact-checker will not refer to "the impeachments of both Nixon and Clinton" if Nixon was not impeached. Therefore, Nixon was impeached. Right?

Some congressional Republicans have openly supported Trump’s assertion that the allegations against Trump are dubious. This contrasts with the Nixon impeachment, when "on both sides there was a pretty universal acknowledgement that the charges being investigated were very important and that it was necessary to get to the bottom of what happened," said Frank O. Bowman III, a University of Missouri law professor and author of the book, "High Crimes and Misdemeanors: A History of Impeachment for the Age of Trump."

Obviously a fact-checker will only draw a parallel to the Nixon impeachment if Nixon was impeached. Therefore Nixon was impeached. Right?

Trump is facing possible impeachment about a year before running for reelection. By contrast, both Nixon and Clinton had already won second terms when they were impeached. (Johnson was such an outcast within his own party that he would have been an extreme longshot to win renomination, historians say.)

Got it? Nixon and Clinton had already won second terms when they were impeached. Because Nixon was impeached, right?

On the eve of impeachment for both Nixon and Clinton, popular support for impeachment was weak — 38% for Nixon and 29% for Clinton, according to a recent Axios analysis. (There was no public opinion polling when Johnson was president.)

Got it? "On the eve of impeachment for both Nixon and Clinton," because a fact checker doesn't refer to the eve of the Nixon impeachment if there was no Nixon impeachment.

Is there a Christmas Eve if there's no Christmas?

That's six times PolitiFact referred to the Nixon impeachment in just one PolitiSplainer article. And about three months after PolitiFact's PunditFact said Nixon was not impeached.

Want a seventh? We've got a seventh:

During Nixon’s impeachment, "people counted on the media to serve as arbiters of truth," he said. "Obviously, we don’t have that now."

 "During Nixon's impeachment" directly implies Nixon was impeached. Seven.

We've been going in order, too.

(Nixon Wasn't Impeached)

But behold! Context at last!

The uncertainty about Senate process stems from the rarity of the process. Nixon resigned before the House could vote to send articles to the Senate, leaving just one precedent -- Clinton’s trial — in the past century and a half.

Admittedly, that's not PolitiFact saying "Nixon was not impeached." On the other hand, it's PolitiFact directly implying Nixon was not impeached. Blink and you might miss it amidst all the talk about the Nixon impeachment.

Can we get to eight after that bothersome bit of context?

Nixon was Impeached, Continued 

We can:

The impeachments of both Nixon and Clinton did tend to curb legislative action by soaking up all the attention in Washington, historians say.

We're curious which historians PolitiFact talked to who explicitly referred to the impeachment of Nixon. There are no quotations in the text of the PolitiSplainer that would support this claim about what historians say.

PolitiFact flirted with nine in the next paragraph. We're capping the count at eight.

In summary, we'll just say this: If there's a sense of "impeachment" that doesn't mean literally getting impeached by Congress and standing trial in the Senate, then Jimmy Kimmel is entitled to that understanding when he says Nixon was the last president to be impeached.

Contrary to PolitiFact's framing, Kimmel was wrong not because Nixon was not impeached. Kimmel was wrong because President William J. Clinton was the last president to be impeached. There was never any need for PunditFact to focus on the fact Nixon wasn't impeached, unless it was to avoid emphasis on Clinton.

This all works out very well for PolitiFact. PolitiFact does what it can to spread the misperception Nixon was impeached. And then it can draw clicks to its PunditFact fact check showing that claim false.

Just like dropping roofing nails on the road.

Fact Check not at PolitiFact Illinois

One of the characteristics of PolitiFact that drags it below its competitors is its penchant for not bothering to fact check what it claims to fact check.

Our example this time comes from PolitiFact Illinois:

From the above, we judge that "Most climate scientists agree" that we have less than a decade to avert a worst case climate change scenario counts as the central claim in need of fact-checking. PolitiFact hints at the confusion it sows in its article by paraphrasing the issue as "Does science say time is running out to stop climate disaster?"

The fact is that time could be running out to stop climate disaster while at the same time (Democrat) Sean Casten's claim could count as entirely false. Casten made a claim about what a majority of scientist believe about a short window of opportunity to avoid a worst-case scenario. And speaking of avoidance, PolitiFact Illinois avoided the meat of Casten's claim in favor of fact-checking its watered-down summary of Casten's claim.

The Proof that Proves Nothing

The key evidence offered in support of Casten was a 2018 report by the United Nations Intergovernmental Panel on Climate Change.

The problem? The report offers no clear evidence showing a majority of climate scientists agree on anything at all, up to and including what Casten claims they believe. In fact, the report only mentions "scientist" or "scientists" once (in the Acknowledgments section):

A special thanks goes to the Chapter Scientists of this Report ...

A fact checker cannot accept that report as evidence of what a majority of scientists believe without strong justification. That justification does not occur in the so-called fact check. PolitiFact Illinois apparently checks the facts using the assumption that the IPCC report would not claim something if a majority of climate scientists did not believe it.

That's not fact-checking.

And More Proof of Nothing

Making this fact-checking farce even more sublime, PolitiFact Illinois correctly found the report does not establish any kind of hard deadline for bending the curve on carbon emissions (bold emphasis added):

Th(e) report said nations must take "unprecedented" actions to reduce emissions, which will need to be on a significantly different trajectory by 2030 in order to avoid more severe impacts from increased warming. However, it did not identify the hard deadline Casten and others have suggested. In part, that’s because serious effects from climate change have already begun.

So PolitiFact did not bother to find out whether a majority of scientists affirm the claim about "less than a decade" (burden of proof, anyone?) and moreover found the "less than a decade" claim was essentially false. We can toss PolitiFact's line about serious effects from climate change already occurring because Casten was talking about a "worst-case scenario."

PolitiFact Illinois rated Casten's claim "Mostly True."

Does that make sense?

Is it any wonder that Independents (nearly half) and Republicans (more than half) think fact checkers favor one side?


Afters

Also worth noting: Where does that "'worst-case scenario'" phrase come from? Does Casten put it inside quotation marks because he is quoting a source? Or is it a scare quote?

We confirmed, at least, that the phrase does not occur in the IPCC report that supposed served as Casten's source.

We will not try to explain PolitiFact Illinois' lack of curiosity on this point.

Let PolitiFact Illinois do that.


Update Sept. 4, 2019: We originally neglected to link to the flawed PolitiFact Illinois "fact check." This update remedies that problem.

PolitiFact founder: "Bias is good"

It was wasn't even a year ago that PolitiFact pompously announced it isn't biased, but now PolitiFact founder Bill Adair has muddied the waters by announcing from his lofty perch at Duke University that bias is good.

Doubtless it is important to make take Adair's words in context.

We'll certainly try.

Here's the Columbia Journalism Review headline:

Op-ed: Bias is good. It just needs a label.

In context so far: Adair appears to say bias is good if the reader understands it (hence the need for a label).

Adair repeated the same point in the article and then used a graphic to spell out what he's saying:

It's hard not to notice that Adair's graphic appears to concede what we have argued for years here at PolitiFact Bias. Fact-checking is not some kind of objective and scientific pursuit even if we set aside the subjective linear-scale truth ratings. Adair understands fact-checking contains more opinion than does "news analysis," with no other form of journalism closer to "opinion."

Unfortunately Adair does little to distinguish the desirable types of bias he's probably talking about--bias toward truth and democracy, for example--from unhealthy cognitive biases. But at least he gives clear guidance that journalists should appropriately label their work.

Now we just need to find the appropriate label at PolitiFact, right?

PolitiFact is not biased -- here’s why

Okay, great. No problem, right?

Seriously, we're not aware of any prominent acknowledgement of bias labeling at PolitiFact.com.

If such a thing existed, perhaps we should expect to find it on PolitiFact's statement of principles. But we get this instead:

Our ethics policy for PolitiFact journalists

PolitiFact seeks to present the true facts, unaffected by agenda or biases. Our journalists set their own opinions aside as they work to uphold principles of independence and fairness.

Anybody see an expression of the idea "bias is good" in there? We don't.

PolitiFact over its history has encouraged readers to take its biased reporting as objective reporting.

It deceived and continues to deceive its readers by the standard Adair advocates.

A PolitiFact gloss on the Michael Brown "murder"

We've been tracking evidence of PolitiFact's look-the-other-way stance on Dem0crats' campaign rhetoric on race. PolitiFact sees no need to issue a "Truth-O-Meter" rating when Democrats call President Trump a racist, for example.

Now, with Democratic presidential candidates like Kamala Harris and Elizabeth Warren asserting that Michael Brown was murdered, again we see PolitiFact reluctant to apply fact-checking to Democratic Party falsehoods.

Instead of issuing a "Truth-O-Meter" rating for either Democratic Party candidate over their Michael Brown statements, PolitiFact published an absurd PolitiSplainer article.

A Fox News article hits most of the points that we would have emphasized:

The fact-checking website PolitiFact again came under fire for alleged political bias Wednesday after it posted a bizarre article that refused to rule on whether Michael Brown was in fact "murdered" by police officer Darren Wilson in Ferguson, Mo. in 2014, as Democratic presidential candidates Kamala Harris and Elizabeth Warren falsely claimed last week.

Indeed, Fox News emphasizes the key expert opinion from the PolitiFact PolitiSplainer:

Jacobson quoted Jean Brown, a communications professor who focuses on "media representations of African Americans," as saying that the entire question of whether Warren and Harris spread a falsehood was nothing more than an "attempt to shift the debate from a discussion about the killing of black and brown people by police."

The Fox article quotes the Washington Examiner's Alex Griswold asking why the expert opinion from Brown was included in the fact check.

We suggest that the quotation represents the reasoning PolitiFact used in deciding not to issue "Truth-O-Meter" ratings for Harris or Warren.

PolitiFact, per the Joe Biden gaffe, seems interested in truth, not facts.

Highlights of PolitiFact's Reddit AMA from August 2, 2019

PolitiFact newbie Daniel Funke, former fact check reporter for the International Fact-Checking Network, represented PolitiFact for a Reddit AMA on Aug. 2, 2019.

We always look forward to public Q&A sessions with PolitiFact staff, for it nearly always provides us with material.

Funke stuck with PolitiFact boilerplate material for the most part, even channeling Bill Adair with his answer about PolitiFact's response to critics who suggest PolitiFact is biased.

Funke's chief error, in our view, was his repetition of a false PolitiFact public talking point:

As far as corrections: We're human beings, so we do make mistakes from time to time. That's why we have a corrections process. You can read our full corrections policy, but the bottom line is that we fix the wrong information and note it. If we give a new rating to a fact-check, we archive the old version so people can see exactly what we changed. Everything that gets a correction or an update gets tagged - see all tagged items.

We've pointed out dozens and dozens of mistakes at PolitiFact, and though we've prompted PolitiFact to fix quite a few mistakes the majority of the time PolitiFact ignores the critique and doesn't bother to fix anything. We tried to get PolitiFact Georgia not to interpret "pistol" as a synonym for "handgun" because revolvers count as handguns but do not count as pistols. No go. The mistake remains enshrined in PolitiFact's "database" of facts. And Funke's recent mistake in using a number PolitiFact found wanting as the deficit figure handed off from Bush to Obama still hasn't been fixed. Nor do we expect PolitiFact to break tradition by fixing it.

PolitiFact fixes mistakes if and only if PolitiFact feels like fixing the mistakes.

So Funke is wrong about the bottom line at PolitiFact. The PolitiFact "database" has more than its share of bad information.

As for archiving the old version of a fact check when the rating changes, contrary to what Funke says readers can't necessarily find the archived version. Here's an example from 2017. The new version contains no link to the old version. A reader would have to figure out how PolitiFact structures its URLs to track down the archived version (assuming there is one).

Finally, Funke repeats the falsehood that "Everything that gets a correction or an update gets tagged," complete with a link to the very incomplete list of corrected items. PolitiFact does not use tags on many of its articles, particularly those that do not feature a rating. Corrections on those articles do not get tagged and do not appear on the list of corrections. Moreover, PolitiFact simply neglects to tag corrected fact checks on occasion.

Apparently it's too much to ask that PolitiFact staffers know what they're talking about when they describe PolitiFact's corrections process.