Highlights of PolitiFact's Reddit AMA from August 2, 2019

PolitiFact newbie Daniel Funke, former fact check reporter for the International Fact-Checking Network, represented PolitiFact for a Reddit AMA on Aug. 2, 2019.

We always look forward to public Q&A sessions with PolitiFact staff, for it nearly always provides us with material.

Funke stuck with PolitiFact boilerplate material for the most part, even channeling Bill Adair with his answer about PolitiFact's response to critics who suggest PolitiFact is biased.

Funke's chief error, in our view, was his repetition of a false PolitiFact public talking point:

As far as corrections: We're human beings, so we do make mistakes from time to time. That's why we have a corrections process. You can read our full corrections policy, but the bottom line is that we fix the wrong information and note it. If we give a new rating to a fact-check, we archive the old version so people can see exactly what we changed. Everything that gets a correction or an update gets tagged - see all tagged items.

We've pointed out dozens and dozens of mistakes at PolitiFact, and though we've prompted PolitiFact to fix quite a few mistakes the majority of the time PolitiFact ignores the critique and doesn't bother to fix anything. We tried to get PolitiFact Georgia not to interpret "pistol" as a synonym for "handgun" because revolvers count as handguns but do not count as pistols. No go. The mistake remains enshrined in PolitiFact's "database" of facts. And Funke's recent mistake in using a number PolitiFact found wanting as the deficit figure handed off from Bush to Obama still hasn't been fixed. Nor do we expect PolitiFact to break tradition by fixing it.

PolitiFact fixes mistakes if and only if PolitiFact feels like fixing the mistakes.

So Funke is wrong about the bottom line at PolitiFact. The PolitiFact "database" has more than its share of bad information.

As for archiving the old version of a fact check when the rating changes, contrary to what Funke says readers can't necessarily find the archived version. Here's an example from 2017. The new version contains no link to the old version. A reader would have to figure out how PolitiFact structures its URLs to track down the archived version (assuming there is one).

Finally, Funke repeats the falsehood that "Everything that gets a correction or an update gets tagged," complete with a link to the very incomplete list of corrected items. PolitiFact does not use tags on many of its articles, particularly those that do not feature a rating. Corrections on those articles do not get tagged and do not appear on the list of corrections. Moreover, PolitiFact simply neglects to tag corrected fact checks on occasion.

Apparently it's too much to ask that PolitiFact staffers know what they're talking about when they describe PolitiFact's corrections process.

The Worst of PolitiFact's April 2, 2019 Reddit AMA

As we mentioned in a Feb. 2, 2019 post, we love it when PolitiFact folks do interviews. It's a near guarantee of generating material worth posting. In celebration of "International Fact-Checking Day," PolitiFact Director Aaron Sharockman and PolitiFact Editor Angie Drobnic Holan conducted a Reddit "Ask Me Anything" event.

I asked PolitiFact to describe why it advocates transparency while keeping the identities and votes of its "Star Chamber" secret. PolitiFact's "Star Chamber" votes on each "Truth-O-Meter" rating. The majority vote rules, though PolitiFact claims it achieves unanimity for most votes. My question wasn't answered (no great surprise there).

Most of the interactions were boilerplate answers to boilerplate questions. But there were a few items of special interest.

Observing the PolitiFact Code?

Card

Though Holan flatly said "Everything that gets a correction or an update gets tagged (see all tagged items)," we were ready with two recent cases contradicting her claim. And we let that cat out of the bag.

Card

PolitiFact makes statements giving readers the impression that it scrupulously follows its code of principles. In fact, PolitiFact loosely follows its code of principles, as in this example. How do Holan and Sharockman not know this?

One of the examples we used was corrected on approximately March 16, 2019. Most of the uncertainly about the correction date comes from PolitiFact Virginia's decision not to mark the date of the correction. As of April 3, 2019 PolitiFact Virginia had not added the "Corrections or Updates" tag and the story did not appear on PolitiFact's supposed list of all of its corrected or updated stories.

Mythical Truth-O-Meter Consistency

One participant asked a question suggesting PolitiFact does not rate statements consistently (suggesting contemporary ratings of Trump make past ratings look far too harsh). Sharockman implied PolitiFact has kept its system consistent over the years:

But beyond the sheer volume [of Trump ratings--ed.], the standards we use to use [sic] to issue our ratings really hasn't [sic] evolved in the 11 years we've been doing this. In that sense, a Pants on Fire in 2009 should still be a Pants on Fire claim today, and vice versa.

There are two big problems with Sharockman's claim. First, PolitiFact itself announced a change to its rating methodology back in 2012.

Second, PolitiFact has admitted that its ratings are pretty much subjective. Sharockman's chosen example, the dividing line between "False" and "Pants on Fire" is perhaps the most sensational example of that subjectivity. How does Sharockman not know that?

The Vast Right-Wing Conspiracy Against PolitiFact?

Someone (not me!) asked "Who fact checks you [PolitiFact--ed.]?"

We found Sharockman's response fascinating and very probably false:

Card

Sharockman's answer paints PolitiFact as the focus of a concentrated group of hostile editors. With all those people combing PolitiFact material for hours on end, it's amazing that PolitiFact makes mistakes so rarely.

Right?

But is there any evidence at all supporting Sharockman's supposition that "a lot of people are reading everything we write looking for mistakes"? We at PolitiFact Bias announced long ago that we could not do a thorough job vetting PolitiFact's body of work:

As PolitiFact expands its state operations, the number of stories it produces far exceeds our capacity to review and correct even just the most egregious examples of journalistic error or bias.  We aim to encourage an army of Davids to counteract the mistakes and bias in PolitiFact's stories.

Who else could Sharockman have had in mind? Media Matters For America? The (defunct) Weekly Standard?

(We asked Sharockman via Twitter the other day whom he had in mind but received no immediate reply)

We suspect Sharockman of Trumpian exaggeration. He knows at least some people look at some of PolitiFact's work for errors. So to convey his point he turns that into "a lot of people" looking at "everything" PolitiFact publishes looking for errors. It's likely the only organization combing over PolitiFact's entire body of work looking for errors is PolitiFact itself.

And look how many times it fails, without swallowing the fiction that this represents the entire number.

***

Holan and Sharockman are politicians advocating for PolitiFact. It appears we cannot trust PolitiFact to hold its own to account.