Friday, April 26, 2013

PolitiFact in Mathmagic Land

A reader pointed us to yet another marvelous example that helps show how PolitiFact applies an irregular set of standards.

PolitiFact Georgia investigated Democrat state senator Vincent Fort's charge that Gov. Nathan Deal, a Republican, has appointed blacks to government positions less than 3 percent of the time.  PolitiFact said the actual number was a little over 7 percent.  So the claim was only "Half True":
The senator’s overarching claim that Deal has appointed a relatively low percentage of minorities has merit. But he was wrong by a handful of percentage points. It was based on an incomplete sampling of Deal’s total appointees.

We rate the claim Half True.
The lawmaker, state senator Vincent Fort,  received a "Half True" because of the truth of his underlying point, that the percentage of black appointees was low.  And he was wrong by "only a handful of percentage points."

Compare the treatment Fort received to Grover Norquist's fate at the hands of PolitiFact Virginia back in February.  Norquist said Virginia uses less than 1 percent of its budget surplus on roads.  The actual number was about 7 percent, so Norquist was off by a handful and committed an error of about 86 percent.  Compare that to Fort's error of about 43 percent.  Norquist's rating from PolitiFact Virginia?  "False."  Norquist received no credit at all for an accurate underlying argument that Virginia doesn't spend much of its budget surplus on roads.

Perhaps Fort could claim Deal has appointed zero blacks and still get a "Half True" from PolitiFact because of the accuracy of his same underlying point.  There's no way to know based on PolitiFact's grading system.  It's all up to the subjective impressions of PolitiFact's star chambers.

It's a crazy way to fact check.  Matt Bryant scores a touchdown!  No, it was just a field goal. Half True.

Jeff Adds:

It's hard to avoid the reality that PolitiFact is an editorial site with articles like this. The final rating is based on the editors' arbitrary standard of however much a "handful" is. Not to mention that Fort named a specific figure, 3 percent, that was unarguably wrong. Whether or not Fort's underlying argument is a good one is clearly the stuff of an editorial. His figures were incorrect, and a fact check, by definition, should be limited to that.

Then again, maybe they cut Fort some slack because they assumed he was citing figures from memory?

Saturday, April 20, 2013

Media Trackers: "Arbitrary PolitiFact Opinion Defends DPI, Liberal Bias"

PolitiFact's founding editor Bill Adair may be moving on to Duke University, but as the watchdogs at Media Trackers help show, the remaining editors stand ready to continue Adair's legacy of lamely responding to criticism.

Brian Sikma of Media Trackers pressed PolitiFact Wisconsin editor Greg Borowski for the justification of a fact check of conservative columnist George Will.

Sikma identified a key problem with the rating:
PolitiFact insists that the reason Will is wrong is because DPI merely made the flyers and materials available to educators, and DPI never specifically told students what to do. But is it possible to promote something to educators and expect for that to not be potentially, either overtly or covertly, promoted to the students of those educators?
 Sikma followed up by trying to grill Borowski:
Asked why they rated Will’s statement the way they did, PolitiFact Wisconsin insisted that they did not give Will a “False” rating, only a “Mostly False.” Greg Borowski, editor of PolitiFact Wisconsin, which is housed at the Milwaukee Journal Sentinel, pointed out that the definition of a “Mostly False” rating is “The statement contains an element of truth but ignores critical facts that would give a different impression.”

Asked how Will’s statement, which was also true, didn’t at least merit a “Mostly True” rating, Borowski replied, “I’m not going to be drawn into your little game here.”
 Borowski refuses to let Sikma draw him into the "game" of specifically justifying a PolitiFact rating.

Seriously, how is that a game?  PolitiFact supposedly exists to make these ratings and help readers decide whether they agree with the rating.

Sikma's approach exposes the fact that PolitiFact's "Truth-O-Meter" only plays at objectivity.  If a statement is false, PolitiFact can still justify a rating of "Mostly False" or higher.  If a statement is true, PolitiFact can still find ways to rate it "False" or even "Pants on Fire."  PolitiFact does this by inconsistently applying its set of standards.  The standards assist in the process by including few objective criteria on which to justify ratings.  Subjective and arbitrary ratings result, which may explain why PolitiFact editors get a bit cagey when pressed to justify a rating.

Jeff adds: Sikma highlights a longstanding gripe of ours. PolitiFact likes to present itself as answering some noble journalistic cause and providing an unbiased and objective view. But the reality is their ratings are ordinary commentary typical of any editorial page. There's simply nothing unique about PolitiFact's op-ed's ratings, save for the gimmicky Truth-O-Meter graphic and the conceit that they're immune to partisanship.

Borowski's sensitivity to being questioned betrays the joke. Facts, by definition, are non-partisan. But PolitiFact can't settle for simply publishing facts. That's why Romney's truth that Jeeps will be built in China becomes a Lie of the Year after PolitiFact expounds on what they think Romney was implying. Their Truth-O-Meter and hilariously ambiguous ratings definitions allow the editors to apply their own bias to tell you what the facts mean.

That's not fact-checking. That's political commentary. Sikma touched the right nerve with Borowski, and judging by the snide response it was a sensitive one.

Edit 4-22-13: Corrected spelling of "Sikma" in Jeff adds section

Saturday, April 13, 2013

PolitiFact's comedic tilt

Here are two more examples of PolitiFact's hilarious inability to reach rational conclusions as it goes about its business.

Fred Thompson and the A-Ford-able Care Act

On April 3, a PolitiFact item appeared with the following headline:  "Ex-Sen. Fred Thompson says Obamcare could raise premiums enough to pay for a new Ford Explorer."

That sounds serious.  Ford Explorers are expensive.

But reading just a little deeper into the story, it turns out that Thompson's claim wasn't serious at all.

Here's how PolitiFact reports what Thompson literally said, instead of the attempted paraphrase:
As District Attorney Arthur Branch on Law & Order, former Sen. Fred Thompson was always ready with a wisecrack. In a recent Twitter post, the Tennessee Republican offered a pointed barb about President Barack Obama’s health care law:

"Report: Obamacare could raise ins premiums by 200%. It's the ‘A-Ford-able Care Act’ -- your insurance costs as much as a new Explorer."
First clue:  It's from Twitter, renowned for its 140-character limit.  Second clue:  "Ford" set apart by hyphens in "Affordable."  It's a joke, and PolitiFact hints that it's a joke by referencing the wisecracks Thompson offered while appearing as Arthur Branch on "Law & Order."

Why fact check a joke?

It makes sense to fact check a joke if the joke is supposed to convey a particular truth worth fact checking.  Was Thompson communicating that insurance premiums will cost the same as a new Explorer?  Isn't that line obviously part of the joke?

Thompson's Twitter feed is substantially made up of similar jokes.  He mentions a fact, then riffs on it for laughs.  Have a look at the tweet PolitiFact graded along with the two before and the two after.

It's a pattern with Thompson.  Offer a factoid, crack a joke.  The only facts worth checking are those Thompson expects readers to take seriously, which in this set means the factoid in the first line of each tweet.  PolitiFact, instead of looking at whether insurance premiums could rise as much as 200 percent, looked at the joke end of the statement.  Thompson gets a "False" instead of a "True."  Go figure.

We note that Thompson has written about PolitiFact's focus on his joke.

Dick Cheney and a wobbly President George H. W. Bush?

On April 10, PolitiFact put some bizarre spin on former vice president Dick Cheney's weekend appearance on Fox News, where he offered remembrance of the late Margaret Thatcher.  PolitiFact begins its quotations with the following:
Van Susteren: "But there's that famous quote where, apparently, she told President Bush 41 not to go wobbly."

Cheney: "That's not true."
When I hear the exchange it never occurs to me that Cheney is saying Thatcher never said "Don't go wobbly."  It seems to me that Cheney was telling Van Susteren that President Bush never went wobbly and people should not interpret Thatcher's comment that way.  As Cheney said immediately after:
There was never any doubt about what the president was doing. He didn't need any b[u]cking up.
If Cheney was trying to make the point that Thatcher never said the words attributed to her then he would  more likely say something like "Thatcher never said it.  It's an old wives' tale."

PolitiFact assumes Cheney claimed Thatcher never said the words and rates his supposed claim "False."

These types of PolitiFlubs do not happen rarely.

Jeff adds:

Check out how PolitiFact completely invented a claim that Thompson never made:

Image from PolitiFact

Even if we disregard Thompson's comical context, he never claimed that your premiums would rise by $30,000. That's a total distortion on the part of our Pulitzer winning pals. Thompson's claim, even if taken seriously, was that your insurance policy in total would equal the value of an Explorer, not that its price would rise that much.

And once again, we see PolitiFact playing games with quotation marks:

Here's what Thompson tweeted: "It's the ‘A-Ford-able Care Act’ -- your insurance costs as much as a new Explorer."

Note that PolitiFact added the word premiums, making Thompson's claim seem more outlandish. This, by the way, is exactly what they did to Mitt Romney in their bogus Lie of the Year rating:

PolitiFact took the Romney ad's claim; "sold Chrysler to Italians who are going to build Jeeps in China," which was and is entirely accurate, then added the words at the cost of American jobs, something the Romney ad never claimed, then slapped a Lie of the Year award on their own invention.

Whether it's shoddy work or intentional partisan spin, this type of nonsense should prevent PolitiFact from being taken seriously.


Friday, April 12, 2013

The Houston Chronicle: "Half of Ted Cruz’s political claims are false, PolitiFact reports" and "Ted Cruz v. PolitiFact: Whose pants are on fire?"

Joanna Raines of the Houston Chronicle wrote a story on March 28 attempting to report PolitiFact's assessment of Sen. Ted Cruz (R-Texas).

The headline and text of Raines' story hinted at a fallacious reasoning.

Raines' story was titled "Half of Ted Cruz’s political claims are false, PolitiFact reports."  It's somewhat natural to take PolitiFact's report card-ish stories along those lines, but the fact is that PolitiFact doesn't rate enough statements to draw general conclusions about political figures.  Compounding the problem, PolitiFact offers no hint at all that its system includes controls for selection bias.  Compounding the problem further, PolitiFact often reaches questionable conclusions with its ratings.

Raines published a follow up story on March 29, titled "Ted Cruz v. PolitiFact: Whose pants are on fire?"  That story implicitly acknowledged the last problem on the preceding list.  Raines offered a brief and reasonably fair account of the substance of Cruz's disagreements with PolitiFact.

Unfortunately, the second story contains a strong hint that Raines continues to reason fallaciously:
To navigate these muddy waters, we’re asking for you to weigh in. Cruz’s office has defended itself against PolitiFact’s claims, and we’re going to let you determine who deserves the “pants on fire” rating, Cruz or PolitiFact?

A survey of public opinion serves very poorly as a measure of truth in most instances.  The flawed reasoning goes by the Latin name argumentum ad populum, also known as the fallacious appeal to the people or the appeal to popularity.

Our guide to the truth should remain the state of the evidence, supplemented by consideration of the arguments pro and con.

Raines' first story helped underscore the truth of one of our longstanding criticisms of PolitiFact:  Publish "report cards" for candidates and many readers will assume that the results say something about the candidates unless the fact checkers explain the error behind that assumption.