Story Focus Shenanigans

We've pointed out for years that PolitiFact's story focus often determines the "Truth-O-Meter" rating at the end. Story focus shenanigans never go out of style at PolitiFact.

PolitiFact's method allows its fact checkers any number of ways to approach the same claim. A fact checker might focus on what the claimant said was meant. Or the fact checker could focus on how an audience might perceive the claim. One approach might lead to one "Truth-O-Meter" rating and another approach to a different "Truth-O-Meter" rating. There's no good evidence of any objective criteria guiding the process.

That brings us to two timely examples that help illustrate the phenomenon.

"Mostly False" for the Republican

President Biden set policy to allow essential workers who are undocumented to receive coronavirus vaccines. Why is Scalise's statement "Mostly False"? Apparently because American citizens who are not among the first groups eligible for the vaccine are not waiting to get the vaccine:

But, we wondered, does allowing this population access to the vaccine mean they are being invited to step in front of American citizens in the queue?

PolitiFact weasel-words "in the queue" so that Americans in low priority vaccine eligibility groups are not in the queue at all and are thus not skipped over when undocumented immigrants join those in the high priority groups.

You're not waiting for the vaccine if you're not in that narrowly-defined queue. PolitiFact quoted a Scalise spokesperson who explained his meaning. To no avail. Scalise received a "Mostly False" rating even though his statement was literally true taken in context, with "in the queue" encompassing all American citizens awaiting the vaccine.

"Half True" for the Democrat

Hocus-pocus-story-focus.

Pointedly, PolitiFact does not look at all the ways Mr. Biden's claim fails the test of truth. It does mention some of them, but breezes past such technicalities to point out that IF the person making below $15 per hour is the sole breadwinner in a family of four AND/OR lives in an area with high living expenses THEN they would fall below the poverty level.

How many of those earning less than $15 per hour meet those conditions? Well, if that was important then PolitiFact would have given us a number. Obviously it's not important. What we need to know is that under some conditions Biden's statement could be true. Those missing conditions count as missing context and that matches PolitiFact's definition of "Half True"!

Marvel at PolitiFact's rationalization:

A spokesman for Biden said he was referring to a family of four with one full-time income using the federal government’s poverty guideline, an explanation Biden didn’t include in the interview. Using that measurement, that family with a paycheck of $13 an hour would live below the poverty line. At $15 an hour, the same family would clearly be above the poverty line. So Biden was off by about a dollar, using the existing standards.

But experts said wages alone don’t tell the full story about whether a household lives in poverty. Other factors include child care and housing costs, for example, which can vary by geography. Generalizing a "poverty wage" to a specific number ignores the different circumstances that families face. Other experts said the federal definition of a poverty level is out of date and needs changing.

We rate this claim Half True.

You wonder why similar reasoning couldn't justify a "Half True" for Scalise?

Why do you hate science?

True statements ruled "Mostly False"

What happens when PolitiFact finds that a statement is literally true?

That issue was brought up indirectly when Jeff retweeted economist Howard Wall. Wall had tweeted:

.@jamestaranto I was an expert in Politifact once. Sole purpose of the interview was to illicit a few words to show pre-existing conclusion.

— Howard Wall (@GatewayEcon) January 10, 2016

We looked up the story where Wall was quoted as an expert. It was a fact check of Mitt Romney from the 2012 presidential election. The Romney campaign said women had suffered the greatest job losses under Obama, implying Obama's leadership had been bad for women.

PolitiFact ruled the claim "Mostly False."

The Romney campaign pushed back. PolitiFact looked at the issue again and ruled the claim "Mostly False." But at the same time, PolitiFact said "The numbers are accurate but quite misleading."

Don't blame the Romney campaign. It probably operated under the assumption that PolitiFact's definitions for its "Truth-O-Meter" ratings mean something.

Taking PolitiFact's definitions literally, the lowest rating one should receive for a literally true claim is "Mostly True." Once below that level, the definitions start talking about "partially true" statements that give a misleading impression ("Half True") and "The statement contains some element of truth" but ignores facts that could give a different impression ("Mostly False").

What's our point? We've always said PolitiFact's ratings reveal more about PolitiFact than they do about the entities receiving the ratings. It's a scandal that social scientists keep their eyes closed to that. Search PolitiFact's ratings for claims it says are literally true. Note the rating given to the claim. Then take a look at the ideology of the entity making the claim.

There's your evidence of journalistic bias by fact checkers.

This is an important issue. If social scientists aren't looking at it, it suggests they don't care.

Why wouldn't they care?



Jeff Adds: We highlighted a Mark Hemingway critique of PolitiFact's Romney claim back in 2012 that is still worth a read. It would seem little has changed at PolitiFact since then.



Update 0956PST 1/12/2016: Added "Jeff Adds" portion - Jeff