Wednesday, September 28, 2016

PolitiFact's presidential "Pants on Fire" bias

PolitiFact Bias has tracked for years a measure of PolitiFact's bias called the "Pants on Fire" bias. The presidential election gives us a fine opportunity to apply this research approach in a new and timely way.

This measure, based on PolitiFact's data, shows PolitiFact's strong preference for Democrat Hillary Clinton over the Republican candidate Donald Trump. When PolitiFact ruled claims from the candidates as false (either "False" or "Pants on Fire"), Trump was 82 percent more likely than Clinton to receive a "Pants on Fire" rating.

Why does this show a bias at PolitiFact? Because PolitiFact offers no objective means of distinguishing between the two ratings. That suggests the difference between the two ratings is subjective. "Pants on Fire" is an opinion, not a finding of fact.

When journalists call Trump's falsehoods "ridiculous" at a higher rate than Clinton's, with no objective principle guiding their opinions, it serves as an expression of bias.


How does the "Pants on Fire" bias measure work?

Tuesday, September 27, 2016

PolitiFact doubles down on deception

Back in May 2016 we pointed out a particularly deceptive fact check from PolitiFact, calling it "Mostly True" that Donald Trump had hoped for the housing crisis that led to the "Great Recession."

Recycled garbage, courtesy of PolitiFact:

It's no surprise to see PolitiFact's deception recycled, as Democratic presidential candidate Hillary Rodham Clinton used a similar version of the same line during the first presidential debate, inviting PolitiFact to give her another nearly glowing "Mostly True" rating:
In the opening skirmish of the first presidential debate, Hillary Clinton cast her rival as a man who put his own business interests ahead of the welfare of average Americans."Donald was one of the people who rooted for the housing crisis," Clinton said. "He said back in 2006, ‘Gee, I hope it does collapse because then I can go in and buy some and make some money.’["]
As we explained in our earlier post, Clinton was committing a fallacy of equivocation. The deflation of a housing bubble is not the same thing as a "housing crisis." Clinton erases the distinction between the two to create the appearance Trump hoped for bad times for millions of Americans. Clinton surely knows the difference, so her knowing deception would qualify as a "lie" in the worst sense of the word.

Such a lie PolitiFact rates as "Mostly True." More than once.

Our capture of part of a PolitiFact video shows PolitiFact repeating a similar fallacy of equivocation. What is "the situation"? When Trump was speaking in 2006, "the situation" was the potential for a deflation of the housing bubble. The housing market inflates and deflates in a way parallel to the rise and fall of the stock market. Lower prices attract investors in both cases. But in the video, PolitiFact allows "the situation" to refer to the housing crisis, not the mere deflation of the housing bubble.

What good is a fact checker that cannot sniff out so obvious a lie?

Not much, we would say.

Monday, September 26, 2016

Krugman: PolitiFact "overwhelmingly in the ballpark"

Great news for PolitiFact! Luminous lefty economist Paul Krugman has given PolitiFact his sort-of seal of approval (bold emphasis added):
PolitiFact has examined 258 Trump statements and 255 Clinton statements and classified them on a scale ranging from “True” to “Pants on Fire.” One might quibble with some of the judgments, but they’re overwhelmingly in the ballpark.
 At PolitiFact Bias, we wonder what it means for Krugman to declare that PolitiFact's ratings of Clinton and Trump are overwhelmingly in the ballpark.

Did Krugman double-check all of PolitiFact's research on those 513 fact checks and find the vast majority "in the ballpark"?

We tend to doubt it. Who has time to double-check that many fact checks if it's not a full time job?

Did Krugman simply judge the fact checks were "in the ballpark" based on his own vast store of political knowledge?

That option seems more likely. But isn't that type of judgment particularly prone to confirmation bias?

That's an obvious yes, right?

PolitiFact, of course, bills itself as nonpartisan. But what solid evidence do we have of PolitiFact's nonpartisanship? Is it reassuring, for example, that founding PolitiFact editor Bill Adair declines to reveal his politics while having his picture taken in his office with a life-sized cardboard President Obama looking on from the background?

PolitiFact's credibility rests on the plausibility of its claim to nonpartisanship. Krugman endorsements don't help that much. The Krugman endorsement might be expected if PolitiFact leans left, thanks to confirmation bias. We encourage readers to consider what would qualify as good evidence of nonpartisanship and whether PolitiFact delivers the goods.

As for whether PolitiFact is "in the ballpark" with its ratings--that can only come from scientific study designed for that purpose. If Krugman ever produces such a study, we will consider his claims based on the merits.

Until then, pfft. Krugman citing PolitiFact's ratings to bolster a pro-Clinton argument, even to the point of suggesting PolitiFact's ratings will serve as a predictor of the debate results (!) only further discredits Krugman.

Correction Sept. 27, 2016: Added the name of PolitiFact's founding editor, Bill Adair. Adair's name was omitted in the first published version.

Wednesday, September 21, 2016

Unreliable PolitiFact

PolitiFact Editor Angie Drobnic Holan illustrates why we can't have good fact-checking from PolitiFact.

Holan was dispensing her sage advice to debate moderators, suggesting they do not rely on memory for their facts.

It was good advice, but her illustration showcased bad fact-checking:
Moderators who don’t keep research at hand are leaving themselves open to dodged questions and outright bluffing. Check out this exchange between Trump and moderator Becky Quick of CNBC from an October primary debate.

Quick: "You have talked a little bit about Marco Rubio. I think you called him (Facebook founder) Mark Zuckerberg’s personal senator, because he was in favor of the H-1B visa."

Trump: "I never said that. I never said that."

Quick: "So this is an erroneous article the whole way around? … My apologies, I'm sorry."

Trump: "Somebody's really doing some bad fact-checking."

Actually, there was no need for Quick to apologize. She was right; Trump was wrong. Trump’s website has a line about Rubio being Zuckerberg’s personal senator. It’s still there.
Quick and Trump were talking past each other, and Trump was more right than Quick. We presume that Holan had time to think about what she was writing, but she still botched the facts.

1) Quick said Trump had talked a little bit about Rubio. If Quick had said Trump's website had used the line about Rubio being Zuckerberg's personal senator, then she would have been right.

2) Trump was reasonable to assume that when Quick said he had "talked" about Rubio that her example would involve something he talked about, not just something written on his campaign website.

3) If Trump did not talk about Rubio being Zuckerberg's personal senator, then Trump was right.

4) If Trump was right, then Holan was wrong to say Trump was wrong.

5) Quick was wrong, so Holan was wrong to say Quick was right.

It isn't good fact-checking to blithely equivocate between a person literally saying something and making that person responsible for something posted on a campaign website.

Holan should apologize.

The "elite" fact checkers stink at fact checking.

Hat tip to PolitiFact for providing us with a steady stream of illustrative material.

A vintage PolitiFact "gotcha" fact check

When was PolitiFact anything other than a horrible, left-leaning fact check organization?

We don't buy it.

Take this example from 2007, which caught our eye while we reviewed PolitiFact's preposterous ruling of "Mostly False" for the idea that France and Germany thought Iraq had WMD.

When Republican Fred Thompson was running for president in 2007, he argued that the "Iraq Study Group" said Iraq was planning to get its nuclear program up and running again despite sanctions.

PolitiFact to the rescue! "False," screamed the trademarked "Truth-O-Meter."

According to PolitiFact, the "Iraq Study Group" (put together by Congress) had made no such finding. So Thompson's claim was false.

Hilariously, unless you were Fred Thompson, PolitiFact bothered to note that the "Iraq Survey Group," the CIA group tasked in Iraq with assessing Iraq's WMD capability, had made a claim that pretty much matched what Thompson said:
So we find Thompson's claim to be False.

It's possible that Thompson was referring to the Iraq Survey Group, a CIA panel that was formed to investigate whether there were weapons of mass destruction or the intent to produce WMDs in Saddam Hussein's Iraq. The report found that Saddam did not produce or possess any weapons of mass destruction for more than a decade before the U.S.-led invasion, but that he "aspired to develop a nuclear capability — in an incremental fashion, irrespective of international pressure and the resulting economic risks."

PolitiFact eventually got around to writing up a statement of principles, which it published in February 2011. We would highlight PolitiFact's declaration about "gotcha" journalism.
Is the statement significant? We avoid minor "gotchas"’ on claims that obviously represent a slip of the tongue.
Too late for Fred Thompson, unfortunately. Thompson's stuck with that undeserved "False" on his record. Maybe PolitiFact went with it because it was a major "gotcha"?

This is the type of fact check that signaled early on to us that something was wrong with the new fact checker in town.

The worst part? PolitiFact isn't getting better.

Tuesday, September 20, 2016

Extremely deceptive abortion ad looks "Half True" to PolitiFact

In arguing that PolitiFact displays a liberal bias, we suggest that big mistakes harming conservatives or helping liberals/progressives potentially make a strong argument for PolitiFact's liberal bias. If PolitiFact's biggest blunders harm conservatives or help liberals, it strengthens our case against PolitiFact.

PolitiFact veterans Angie Drobnic Holan and Louis Jacobson, editing and writing for upstart franchise PolitiFact Nevada, give an absolutely sensational example supporting our case.

The political arm of the nation's largest abortion provider, Planned Parenthood, ran an ad saying Republican candidate Joe Heck voted to "criminalize abortion for rape victims."

The ad's vagueness misleads its audience in two main ways, suggesting:
  • Rape victims risk criminal charges for seeking an abortion
  • Criminal charges would apply for any abortion sought by a rape victim, regardless of the number of weeks elapsed since the pregnancy started
PolitiFact's research confirmed that the bill Heck voted for would not result in criminal charges for rape victims. The penalties were reserved for abortion providers.

PolitiFact completely overlooked the other main deception. We would call the second deception the main one. PolitiFact mentioned in its fact check that the abortion bill Heck supported would ban abortion after 20 weeks in nearly all cases, including for rape victims. But PolitiFact penalized Planned Parenthood Votes not a bit for allowing the ad's audience to think Heck was criminalizing abortion for all rape victims.

Note the resounding silence in PolitiFact's summary paragraphs, echoing the silence in the rest of its story:
The Planned Parenthood Votes ad said that "Joe Heck voted to criminalize abortion for rape victims."

The group has a point that a bill Heck voted for and co-sponsored would have criminalized medical professionals from performing abortions after 20 weeks for rape victims who are not at risk of death or significant physical health complications due to pregnancy, at least in Washington, D.C. This would have eliminated all legal abortions for women in that category.

However, the ad blurs the issue of whether medical professionals or the women themselves would be at risk of prosecution. Only medical professionals would face legal consequences under the bill, but the ad’s imagery implies otherwise, using only women as visuals. On balance, we rate the ad Half True.
For some reason, it did not seem important to PolitiFact to point out that abortions after 20 weeks occur rarely. A fact check cited the Guttmacher Institute in saying abortions after 20 weeks account for 1.2 percent of all U.S. abortions. Unless we assume that rape victims tend to wait longer for their abortions than other women, the statistic means that the law would affect very few rape victims.

How does this not draw the attention of a fact checker? It's like meeting Cyrano face to face and failing to notice his nose.

We suggest that mistakes like this favoring a cause dear to the political left make a good evidence of PolitiFact's liberal bias. This is the kind of mistake you look for from a fact checker that has a liberal bias.

This is one of many we've found from PolitiFact. But it's an especially obvious one.


We contacted the writer and editor of the fact check to point out the highly misleading part of the ad they had failed to mention. If we receive any response from the PolitiFact team or a change to the fact check we will update this item.

We note that PolitiFact will not erase the evidence of its bias by changing its fact check days after publishing. A writer got the flawed story past a team of editors. Fixing the story will not change that. If PolitiFact fails to change its story and the rating it gave to Planned Parenthood Votes, it will show something worse than unintentional bias: It will show a lack of integrity.

Monday, September 19, 2016

"Mostly False" that France and Germany thought Iraq had WMD? Seriously?

PolitiFact recently weighed in on French and German intelligence about Iraq's WMD programs. Former Assistant Secretary of Defense Paul Wolfowitz said the French and Germans believed Iraq had WMD.

PolitiFact ruled Wolfowitz's claim "Mostly False."

Some may remember the media routinely reporting that French and German intelligence assessments, along with others, mirrored those of the United States (bold emphasis added):
U.S. government analysts were not alone in these views. In the late spring of 2002 I participated in a Washington meeting about Iraqi WMD. Those present included nearly twenty former inspectors from the United Nations Special Commission (UNSCOM), the force established in 1991 to oversee the elimination of WMD in Iraq. One of the senior people put a question to the group: Did anyone in the room doubt that Iraq was currently operating a secret centrifuge plant? No one did. Three people added that they believed Iraq was also operating a secret calutron plant (a facility for separating uranium isotopes).

Other nations' intelligence services were similarly aligned with U.S. views. Somewhat remarkably, given how adamantly Germany would oppose the war, the German Federal Intelligence Service held the bleakest view of all, arguing that Iraq might be able to build a nuclear weapon within three years. Israel, Russia, Britain, China, and even France held positions similar to that of the United States; France's President Jacques Chirac told Time magazine last February, "There is a problem—the probable possession of weapons of mass destruction by an uncontrollable country, Iraq. The international community is right ... in having decided Iraq should be disarmed." In sum, no one doubted that Iraq had weapons of mass destruction.
Some may remember French President Jacques Chirac, who opposed the invasion, nonetheless admitting the French believed Iraq had WMD:
There is a problem—the probable possession of weapons of mass destruction by an uncontrollable country, Iraq. The international community is right to be disturbed by this situation, and it's right in having decided Iraq should be disarmed.
Chirac's statement came from early 2003, not long before the invasion started.

But it's a trivial matter for PolitiFact to work its way around these inconvenient facts by making its own one-sided case against Wolfowitz's claim.

Why would PolitiFact take that approach? Incompetence? Bias? A little of both?

I'm working on a column for Zebra Fact Check that will expose PolitiFact's fact-checking sins in excruciating detail.