Showing posts with label methodological blunders. Show all posts
Showing posts with label methodological blunders. Show all posts

Thursday, March 4, 2021

Editor who won't do corrections makes correction request (Updated)

Here and at Zebra Fact Check we have documented obvious errors pointed out to PolitiFact that PolitiFact simply declines to fix.

How delicious, then, when we heard that PolitiFact Editor-in-Chief Angie Drobnic Holan requested corrections of the conservative publication the Federalist. And royally botched at least one of them.

The Federalist noted PolitiFact had done little to fact check Democratic Party politicians in the new Biden administration, and PolitiFact apparently objected.

The Federalist's Tristan Justice told the story on Feb. 20, 2021:

“Your report is wrong in multiple ways,” Holan wrote in a Friday email. “For one, we published three fact-checks on Feb. 16, so our last check on Biden was the day before your report was published. The link you use in the story shows that. Please issue a correction ASAP.”

Holan followed up two hours later, demanding an update.

The link embedded in the initial post to Biden’s fact-checks on the PolitiFact website, however, which remains unchanged, shows PolitiFact published three fact-checks after the story was published in The Federalist, of statements made on Feb. 16, including Biden’s remarks on vaccines. Contrary to Holan’s assertion, these fact-checks were published on Feb. 17 and 18, following the Federalist article.

The Federalist story included an image backing up Justice's description, which we independently verified. It looks like Holan probably mistook the date the claims were made as the publish date. When PolitiFact revamped its website about a year ago, it moved the claim date up to the top of the article and the publish date down and to the left to accompany the writer's byline.

Old way:

The old layout had the date of the claim in the box up top and the publish date next to the author's byline, down at the bottom of the header section.

New way:


With the new layout the claim date moved all the way up to the top, under the source of the claim. The publish date moved, along with the writer's byline, to the left margin. And shrank.

It looks like Holan believed the claim dates were the publish dates. It's not the kind of mistake an organization wants to see from its editor-in-chief.

We asked Holan for an explanation but received no reply.

It's worth pointing out that Holan claimed there were a number of errors in the Federalist article. We also asked her about those, after we asked Justice, the Federalist writer, about them. If Holan found legitimate problems with the Federalist story then Justice should not have buried the fact.

But as for Justice's allegation that Holan was mistaken with the first part of her correction request, the Internet Archive record from Feb. 17, 2021 shows no fact checks featuring Joe Biden published on Feb. 16, 2021.

Update March 5, 2021: A commenter has correctly noted that this post could benefit from added context. We inserted a short new paragraph in the three slot and added the link to the Federalist story that was always intended to be there. We appreciate comments that help us improve our work.

Wednesday, September 12, 2018

PolitiFact flubs GDP comparison between added debt and cumulative debt

Here at PolitiFact Bias we think big mistakes tell us something about PolitiFact's ideological bias.

If PolitiFact's big mistakes tend to harm Republicans and not Democrats, it's a pretty good sign that PolitiFact leans left. For that reason, much of what we do centers on documenting big mistakes.

Veteran PolitiFact fact checker Louis Jacobson gave us a whopper of a mistake this week in a Sept. 12, 2018 PunditFact fact check.

Before reading the fact check we had a pretty good idea this one was bogus. Note the caveat under the meter telling the reason why Scarborough's true numbers only get by with a "Mostly True" rating: The added debt was not purely the GOP's fault.

We easily found a parallel claim, this one from PolitiFact Virginia but with Trump as the speaker:

Trump's parallel claim was dragged down to "Half True" because there was plenty of blame to share for doubling the debt. In other words it was not purely Obama's fault.


A Meaningless Statistic?

Scarborough's statistic makes less sense than Trump's on closer examination. The point comes through clearly once we see how PolitiFact botched its analysis.

Scarborough said the GOP would create more debt in one year than was generated in America's first 200 years.

After quoting an expert who said percentage of GDP serves as a better measure than nominal dollars, PolitiFact proceeded to explain that testing Scarborough's claim using the percentage of GDP tells essentially the same story.  PolitiFact shared a chart based on data from the executive branch's Office of Management and Budget:



So far so good. The OMB is recognized as a solid source for such data. But then PolitiFact PolitiSplains (bold emphasis added):
The chart does show that, when looking at a percentage of GDP, Scarborough is correct in his comparison. Debt as a percentage of GDP in 2017 was far higher (almost 77 percent)  than it was in 1976 (about 27 percent).
Colossal Blunder Alert!

PolitiFact/PunditFact, intentionally or otherwise, pulled a bait and switch. Scarborough said the GOP would create more debt in one year than was generated in America's first 200 years. As PolitiFact recognized when comparing the nominal dollar figures, that comparison involves the cumulative deficit number for one year (which we call the debt) and comparing it to the non-cumulative deficit number for one year (which we call the deficit). It's a comparison of the debt in 1976, following PolitiFact's methodology for nominal dollars in the first part of the fact check, to the deficit for 2017.

But that's not what PolitiFact did when it tried to test Scarborough using percentage of GDP.

PolitiFact compared the debt in 1976 to the debt in 2017. That's the wrong comparison. PolitiFact needed to substitute the deficit in 2017 as a percentage of GDP for the debt in 2017 as a percentage of GDP. That substitution corresponds to Scarborough's argument.

The deficit in 2017 does not measure out to nearly 77 percent of GDP. Not even close.

The OMB reports the deficit for 2017 was 3.5 percent of GDP. That's less than 27 percent. It's also less than 77 percent.

Using the preferred measure for comparing deficit and debt numbers across time, Scarborough's claim fell flat. And PolitiFact failed to notice.

Testing Scarborough's number correctly as a percentage of GDP illustrates the worthlessless of his statistic. Instead of "Mostly True" PolitiFact could easily have issued a ruling more similar to the one it issued to Republican presidential candidate Mitt Romney when he correctly noted that our armed forces were shorter on ships and planes in 2012 than at times in the past.


Cheer up, PolitiFact. You'll be tarring the conservative Scarborough. So it's not a total loss.