Thursday, January 28, 2021

PolitiFact's Layers of Editors: Biden says he will ensure enough supply to vaccinate 300 Americans by early fall (Updated)

PolitiFact reported President Biden's plans would ensure covid-19 vaccine supplies large enough to vaccinate 300 Americans by early fall.

The Biden administration has envisioned a more prominent federal role, including setting up 100 vaccine centers across the nation by the end of February. Officials say the biggest roadblock is lack of vaccine supply. Biden announced that his administration aims to purchase enough doses to fully vaccinate 300 Americans by the end of the summer or beginning of the fall.

 That's an average of three vaccines per planned vaccine center, by our reckoning.

Of course, PolitiFact pretty obviously omitted the word "million" that belonged between "300" and "Americans." So why do we even bother to mention such a small error? It's not even an example of bias, is it?

There are two related reasons why.

First, because PolitiFact loves to trumpet its rigorous system for reporting accurately. Did you know that PolitiFact's system calls for three editors to review each fact check?

The reporter who researches and writes the fact-check suggests a rating when they turn in the report to an assigning editor. The editor and reporter review the report together, typically making clarifications and adding additional details. They come to agreement on the rating. Then, the assigning editor brings the rated fact-check to two additional editors.

The three editors and reporter then review the fact-check by discussing the following questions.

We hope we're correctly assuming that two editors aside from the assigning editor would not dream of discussing the critical questions without bothering to read the fact check. Under that assumption, three editors and one writer together failed to keep the fact check from including a figure that was off by a factor of one million.

So our first point consists of the observation that PolitiFact's system may allow for considerable error, even obvious error.

So far as we're concerned, that by itself is no big deal. Everybody makes mistakes. So that brings us to our second reason for writing this up.

Thanks to its performance since its inception, we have a very low expectation that PolitiFact will run a correction or update and include this item on its page of "corrections and updates."

That's troublesome.

The best journalists make thorough corrections a priority. Take a gander at the expectation the International Fact-Checking Network lays down for its verified signatory fact-checking organizations:

6.3 Where credible evidence is provided that the applicant has made a mistake worthy of correction, the applicant makes a correction openly and transparently, seeking as far as possible to ensure that users of the original see the correction and the corrected version.
PolitiFact makes many--perhaps most--of its corrections secretly and without transparency.

PolitiFact has a sort of loophole in its corrections policy allowing that lack of transparency. And we've written about it before.

Here's the loophole (bold emphasis from the original):

Typos, grammatical errors, misspellings – We correct typos, grammatical errors, misspellings, transpositions and other small errors without a mark of correction or tag and as soon as they are brought to our attention.

The section on "Typos, grammatical error, misspellings" sneaks in an extra category that PolitiFact feels needs no notice of correction. That is the "other small errors." We think that most likely PolitiFact will treat reporting 300 instead of 300 million as a small error and insert the correction without any notice at all to its readers. Under that scenario, PolitiFact will make no attempt to correct the error openly and transparently, and will likewise make no attempt to bring the corrected number to the attention of those who read the erroneous reporting in the original version.

If PolitiFact surprises us and issues a correction notice, we expect it to follow the form "We reported the wrong number of vaccines Biden promised in the original version of this story." PolitiFact prefers to report vaguely on its mistakes instead of detailing the exact nature of the error as full transparency would demand.

Doing the right thing isn't hard: "We reported Biden would acquire vaccines for 300 Americans when we intended to specify 300 million Americans." Nobody would have a real problem with that.

Whatever one thinks about the significance of the error, the PolitiFact method of treating substantive errors the same as it does typos, grammatical errors and misspellings does not scrupulously follow the IFCN's requirement on corrections.

Supposedly the IFCN requires its signatory organizations like PolitiFact to "scrupulously" follow its guidance on corrections. But the truth is the IFCN has turned a blind eye on PolitiFact's correction shenanigans for years. They know about it because I've informed them about it periodically. The IFCN has never directly addressed any of the reports I've sent detailing PolitiFact's failures to follow policy despite showing no clear evidence that the reports have in any way affected PolitiFact's signatory status.

If the complaints carry no validity, the IFCN should transparently declare the how and why of its judgments.

So this isn't evidence of bias. It's evidence that PolitiFact tries to trick its readers into believing it strives to meet the highest journalistic standards. It advertises a system with failsafes that would prevent the type of error it made. PolitiFact and the IFCN assure readers that PolitiFact performs corrections to appropriately high standards.

It's deception.

We'll update this post to report on whatever actions PolitiFact takes to correct its mistake. A sufficiently pathetic response may result in yet another complaint filed with the IFCN.

Update Jan 28, 2021: We sent an email pointing out PolitiFact's error about 1 p.m. When we checked at about 3:30 p.m., the error was fixed. PolitiFact also appended a correction with the milquetoast-style wording we expected. Oddly, neither the correction nor the tag denoting the correction show up for the archived version of the story. Only the corrected wording in the story shows up.

Here's the correction notice:
CORRECTION, Jan. 28: We corrected the number of Americans the Biden administration hopes to fully vaccinate by the end of summer or early fall.
On the plus side, PolitiFact corrected the error, posted a correction notice and even placed the "corrections and updates" tag.

On the negative side, the correction falls well short of full transparency. PolitiFact told readers on what specific topic it erred but not the specific error it made. That style of correction hides from readers the magnitude of the error. And that was probably the plan.

No comments:

Post a Comment

Thanks to commenters who refuse to honor various requests from the blog administrators, all comments are now moderated. Pseudonymous commenters who do not choose distinctive pseudonyms will not be published, period. No "Anonymous." No "Unknown." Etc.