Thursday, June 27, 2019

Selection Bias, Magnified

How PolitiFact uses inconsistent application of principles to help Democrats, starring Beto O'Rourke


PolitiFact Bias has repeatedly pointed out how PolitiFact's selection bias problem serves as a trap for its left-leaning journalists (that likely means somewhere between most and all of them). Left-leaning journalists are likely to fact check suspicious claims that look suspicious to left-leaning journalists.

But beyond that left-leaning journalists may suffer the temptation of looking at statements through a left-leaning lens. Fact-checking a Democrat may lead to confirmation bias favoring the Democrat's statement. The journalist may, perhaps unconsciously, emphasize evidence confirming claims coming from liberal sources. Or cutting the fact-finding process short after finding enough to supposedly confirm what the Democrat said.

When Democratic presidential hopeful Beto O'Rourke claimed to have received more votes than any Democrat in the history of Texas, PolitiFact Texas fact-checked the claim and found it "True."

Note that the fact check was written by long-time PolitiFact staffer Louis Jacobson. PolitiFact National employs Jacobson.

It is literally true that O'Rourke received the most votes for a Democrat ever received in the state of Texas. But literal truth is rarely the benchmark for fact checkers. In this case, we immediately noticed a problem with O'Rourke's claim that typically causes fact-checkers to find fault: As the number of voters in Texas grows, the number of raw votes received shrinks in significance. Measuring the percentage of the total vote (48.3 percent for O'Rourke) or the percentage of registered voters (about 25.6 percent) offers a more complete picture of a candidate's electoral strength in a given state.

For comparison, President Jimmy Carter won Texas in 1976 with 2,082,319 votes. Carter's percentage of the vote was 51.1 percent. His percentage of registered voters was 31.2 percent. It follows that Carter's performance in Texas was stronger than O'Rourke's even though Carter received about half as many votes as O'Rourke received.

We pointed out the problem to a PolitiFact Texas employee on Twitter. PolitiFact elected not to update the story to address O'Rourke's potentially misleading point about his electoral strength.

But it's justified resisting the efforts of conservatives to "work the refs," right? Who would think of trying to put the number of votes in context like we did other than right wing zealots?

Try the BBC, for starters. BBC noted that Hillary Clinton received the most presidential votes in history, then promptly tempered that statement of fact with a caveat:
So the proportion of Clinton votes might be more illuminating than simply how many votes she earned.
Indeed. And even PolitiFact Texas devoted more than one paragraph to the context O'Rourke had left out. Yet PolitiFact had the left-leaning sense not to let that missing information interfere with the "True" rating it bestowed on O'Rourke.
Our ruling

O’Rourke said that in 2018 when he ran for senator, "young voter turnout in early voting was up 500%. We won more votes than any Democrat has in the history of the state of Texas."

His assertion about young voter turnout is backed up by an analysis of state election data by the firm TargetSmart. And he’s correct that no Democrat has ever won more raw votes in a Texas statewide election than he has, an accomplishment achieved through a combination of his own electoral success, a pro-Democratic environment in 2018, and Texas’ rapid population growth in recent years.

We rate his statement True.
PolitiFact does not count the missing information significant, even though it was apparently significant enough to mention in the story.

Partial review of PolitiFact's rating system:
MOSTLY TRUE – The statement is accurate but needs clarification or additional information.

HALF TRUE – The statement is partially accurate but leaves out important details or takes things out of context.
If O'Rourke's statement did not need clarification or additional information, such as the growing number of voters in Texas, then why did PolitiFact provide that clarifying information?

These gray area "coin flips" between ratings offer yet another avenue for left-leaning fact-checkers to express their bias.

PolitiFact has never revealed any mechanism in its methodology that would address this weakness.

2 comments:

  1. That would be like saying the latest movie has grossed the most without taking into account a growing population and the cost of the ticket for the movie

    ReplyDelete
  2. We're here to help:

    https://www.boxofficemojo.com/alltime/adjusted.htm

    ReplyDelete