More notes on PolitiFact's debate night blogging

We don't have time to completely go through PolitiFact's election night blogging, but we'll keep picking out a few gems for comment as the week winds down.

Republican presidential candidate Donald Trump said Democratic presidential candidate Hillary Rodham Clinton wants open borders. A WikiLeaks release offered Trump's claim some support.

Observe how PolitiFact rationalizes calling Trump's claim "Mostly False":

In a brief speech expert from 2013, Clinton purportedly says, "My dream is a hemispheric common market, with open trade and open borders, some time in the future with energy that is as green and sustainable."

But we don’t have more context about what Clinton meant by "open borders" because she has not released the full speech. Her campaign has said she was talking about clean energy across the hemisphere.

We rated Trump’s claim Mostly False.

What other context is necessary to understand Clinton's comment? "Hemispheric common market" is pretty clear. "Open trade" is pretty clear. "Open borders" is pretty clear, particularly in the context of "hemispheric common market" and "open trade."

PolitiFact eventually falls back on "he said, she said" journalism by citing the Clinton campaign's explanation of her remarks: "Her campaign has said she was talking about clean energy across the hemisphere." So it was just about having "open borders" so we could trade clean energy in this hemisphere?

Does that even make any sense?

What kind of clean energy gets traded from one nation to another? Wind? Solar? Clean energy proponents bemoan barriers to investment, but what does "open borders" have to do with that?

PolitiFact is using the abbreviated context as a "get out of jail, free" card for Clinton. The context of her speech provides enough context to find Trump's claim at least "Half True."

HALF TRUE – The statement is partially accurate but leaves out important details or takes things out of context.

In what way does the definition not fit, other than PolitiFact not knowing for sure Trump's statement is only partially accurate, or not knowing the statement was taken out of context?

Forgive us for pretending that PolitiFact's definitions for its ratings are not ultimately subjective.

PunditFact's PolitiMath on the GDP of 29 countries

We do "PolitiFact" stories to examine how PolitiFact's ratings correlate to percentage error. Claims where the ratings seem based purely or mainly on the degree of error serve as the best case studies. PunditFact gives us a great study example with its article on the claim that a boxing match would generate more revenue than the GDP of 29 different countries.

PunditFact ruled that claim "Pants on Fire," finding only six countries with a GDP lower than that predicted for the fight: $400 million.

Jim Lampley's figure of 29 exaggerates PunditFact's total by 383 percent. That substantial error, we suppose, justifies the "Pants on Fire" rating.

On the other hand, PunditFact gave Cokie Roberts a "Half True" rating for a claim she exaggerated by over 9,000 percent. PunditFact gave Roberts credit for her underlying point, that the risk of getting murdered in Honduras is greater than for New York City.

Apparently Lampley has no valid underlying point that the Mayweather-Pacquiao fight would generate a great deal of revenue.

You be the judge.


Update May 3, 2015

While researching and wondering how Lampley ended up with 29 countries producing a GDP under $400 million, we noticed a perhaps-coincidental statistic: The World Bank's 2013 GDP rankings have 29 countries with a GDP above $400 billion.

Lampley's claim may have started with this statistic. After mixing up millions with billions and mistaking the top of the list for the bottom, Lampley's claim makes perfect sense, in a way.


Correction May 4, 2015: Fixed spelling of "Pacquiao."

PunditFact amends pundit's claim about amendments

We've pointed out before how PolitiFact will fault statements made on Twitter for lacking context despite the 140-character limit Twitter imposes.

This week PunditFact played that game with the following tweet from conservative pundit Phil Kerpen:

Amazing fact: Senate has already voted on more amendments in 2015 than Reid allowed ALL YEAR last year.
— Phil Kerpen (@kerpen) January 23, 2015

PunditFact found that the new Republican-controlled Senate has already voted on more amendments in 2015 than Reid allowed in the Democrat-controlled Senate for all of 2014: "On the numbers, that is right."

But PunditFact went on to find fault with Kerpen for leaving out needed context:

On the numbers, that is right. But experts cautioned us that the claim falls more in the interesting factoid category than a sign of a different or more cooperative Senate leadership.

The statement is accurate but needs clarification and additional information. That meets our definition of Mostly True.

We'll spell out the obvious problem with PunditFact's rating: Kerpen's tweet doesn't say anything about different or more cooperative Senate leadership. If Kerpen's not making that argument (we found no evidence he was), then it makes no sense at all to charge him with leaving out information. In effect, PunditFact is amending Kerpen's tweet, giving it context that doesn't exist in the original. Kerpen's statement doesn't need clarification or additional information to qualify as simply "True."

PunditFact's rating offers us a perfect opportunity to point out that if Kerpen's statement isn't simply "True" then there's probably no political claim anywhere that's immune to the type of objection PunditFact used to justify its "Mostly True" rating of Kerpen. A politician could claim the sky is blue and the fact checker could reply that yes, the sky is blue but no thanks to the policies of that politician's party! There are endless ways to rationalize withholding a "True" rating.

This rating convinces us that it would be productive to look at the breakdown between "True" and "Mostly True" ratings to look for a partisan bias. Since there's always context missing from political claims, drawing that line between "True" and "Mostly True" may prove no more objective than the line between "False" and "Pants on Fire."

PolitiFact's coin flips

We've often highlighted the apparent non-objective standards PolitiFact uses to justify its "Truth-O-Meter" ratings. John Kroll, a former staffer at the Cleveland Plain Dealer, PolitiFact's former partner with PolitiFact Ohio, said the choice between one rating and another was often difficult and said the decisions amounted to "coin flips" much of the time.

Heads the liberal wins, tails the Republican loses, at least in the following comparison of PolitiFact's ratings of Stephen Carter (liberal) and Ted Cruz (Republican).

I'll simply reproduce the email PolitiFact Bias editor Jeff D. sent me, reformatted to our standard PFB presentation:

Read the last three paragraphs of each one (emphasis mine):

Carter said that more than 70 percent of American adults have committed a crime that could lead to imprisonment. Based on a strictly technical reading of existing laws, the consensus among the legal experts we reached is that the number is reasonable. Way more than a majority of Americans have done something in their lives that runs afoul of some law that includes jail or prison time as a potential punishment.

That said, experts acknowledged that the likelihood of arrest, prosecution or imprisonment is exceedingly low for many of Americans’ "crimes." 

As such, we rate the claim Mostly True.

Cruz said that "Lorne Michaels could be put in jail under this amendment for making fun of any politician."

Most experts we talked to agreed that the proposed amendment’s language left open the door to that possibility. But many of those same experts emphasized that prosecuting, much less imprisoning, a comedian for purely political speech would run counter to centuries of American tradition, and would face many obstacles at a variety of government levels and run headlong into popular sentiment.

In the big picture, Cruz makes a persuasive case that it’s not a good idea to mess with the First Amendment. Still, his SNL scenario is far-fetched. The claim is partially accurate but leaves out important details, so we rate it Half True.

One wonders if PolitiFact sought the consensus of experts while considering whether blacks were convicted at a higher rate than whites in a recent fact check. Rudy Giuliani received a "False" rating since PolitiFact could locate no official statistics backing his claim. Looks like official statistics aren't really needed if experts think a claim seems reasonable.
 

Jeff Adds: 

Though former Cleveland Plain Dealer (PolitiFact Ohio) editor John Kroll admits PolitiFact's ratings often amount to coin flips, their other journalistic standards are applied with the same consistency. Take for instance their Dec. 2 dodge of the claim Obama's executive order on immigration would create a $3000 incentive to hire undocumented workers:

The claim isn’t so much inaccurate as it is speculative. For that reason, we won’t put this on our Truth-O-Meter.

Was there an unannounced policy change at PolitiFact? Aaron Sharockman was editor on both the Cruz and Carter checks. An unnamed editor signed off on the Incentive claim, adding flip flops to coin flips.

Here's a timeline:

• On Sept. 11, 2014, there was enough established, tangible evidence for something that may or may not happen in the future to say Ted Cruz' prediction was half wrong

• On Dec. 2, 2014, PolitiFact suddenly has a policy against checking speculative claims, but felt compelled enough to spend an entire article Voxsplaining their work to readers.

• On Dec. 8th, 2014, PolitiFact is back in the future-checking business and found enough proof of something that hasn't actually happened yet to definitively determine a liberal's claim is Mostly True.

Remember also that Mitt Romney won the Lie of the Year award for a TV ad that claimed implied Chrysler would be moving Jeep production to China. So in 2012, PolitiFact's most notable falsehood of the year was a campaign ad implying something would happen in the future.

But does Obama's executive order offer a certain economic incentive, as in the Dec. 2 article? Sorry, PolitiFact says it doesn't rate speculative claims.

"The Heart of PolitiFact"

We find it fundamentally dishonest the way PolitiFact treats its trademark "Truth-O-Meter" as though it's some sort of machine that objectively measures the truth content of political statements.

The opposite's true.

Have a look at PolitiFact's "About PolitiFact" page (bold emphasis added):

The heart of PolitiFact is the Truth-O-Meter, which we use to rate factual claims.

The Truth-O-Meter is based on the concept that – especially in politics - truth is not black and white.

PolitiFact writers and editors spend considerable time researching and deliberating on our rulings ...

We then decide which of our six rulings should apply.

There's no machinery, only the machinations of biased reporters and editors.  The "Truth-O-Meter" is just the vehicle for the label they put on their decision.  The eventual rulings are even worse than the "coin flip" described by John Kroll, mentioned in our previous post.  The "Truth-O-Meter" is akin to the "Wheel of Fortune," given the level of subjectivity inherent in the system.

The heart of PolitiFact?