PolitiFact in Mathmagic Land

A reader pointed us to yet another marvelous example that helps show how PolitiFact applies an irregular set of standards.

PolitiFact Georgia investigated Democrat state senator Vincent Fort's charge that Gov. Nathan Deal, a Republican, has appointed blacks to government positions less than 3 percent of the time.  PolitiFact said the actual number was a little over 7 percent.  So the claim was only "Half True":

The senator’s overarching claim that Deal has appointed a relatively low percentage of minorities has merit. But he was wrong by a handful of percentage points. It was based on an incomplete sampling of Deal’s total appointees.

We rate the claim Half True.

The lawmaker, state senator Vincent Fort,  received a "Half True" because of the truth of his underlying point, that the percentage of black appointees was low.  And he was wrong by "only a handful of percentage points."

Compare the treatment Fort received to Grover Norquist's fate at the hands of PolitiFact Virginia back in February.  Norquist said Virginia uses less than 1 percent of its budget surplus on roads.  The actual number was about 7 percent, so Norquist was off by a handful and committed an error of about 86 percent.  Compare that to Fort's error of about 43 percent.  Norquist's rating from PolitiFact Virginia?  "False."  Norquist received no credit at all for an accurate underlying argument that Virginia doesn't spend much of its budget surplus on roads.

Perhaps Fort could claim Deal has appointed zero blacks and still get a "Half True" from PolitiFact because of the accuracy of his same underlying point.  There's no way to know based on PolitiFact's grading system.  It's all up to the subjective impressions of PolitiFact's star chambers.

It's a crazy way to fact check.  Matt Bryant scores a touchdown!  No, it was just a field goal. Half True.


Jeff Adds:

It's hard to avoid the reality that PolitiFact is an editorial site with articles like this. The final rating is based on the editors' arbitrary standard of however much a "handful" is. Not to mention that Fort named a specific figure, 3 percent, that was unarguably wrong. Whether or not Fort's underlying argument is a good one is clearly the stuff of an editorial. His figures were incorrect, and a fact check, by definition, should be limited to that.

Then again, maybe they cut Fort some slack because they assumed he was citing figures from memory?

Media Trackers: "Arbitrary PolitiFact Opinion Defends DPI, Liberal Bias"

PolitiFact's founding editor Bill Adair may be moving on to Duke University, but as the watchdogs at Media Trackers help show, the remaining editors stand ready to continue Adair's legacy of lamely responding to criticism.

Brian Sikma of Media Trackers pressed PolitiFact Wisconsin editor Greg Borowski for the justification of a fact check of conservative columnist George Will.

Sikma identified a key problem with the rating:

PolitiFact insists that the reason Will is wrong is because DPI merely made the flyers and materials available to educators, and DPI never specifically told students what to do. But is it possible to promote something to educators and expect for that to not be potentially, either overtly or covertly, promoted to the students of those educators?

 Sikma followed up by trying to grill Borowski:

Asked why they rated Will’s statement the way they did, PolitiFact Wisconsin insisted that they did not give Will a “False” rating, only a “Mostly False.” Greg Borowski, editor of PolitiFact Wisconsin, which is housed at the Milwaukee Journal Sentinel, pointed out that the definition of a “Mostly False” rating is “The statement contains an element of truth but ignores critical facts that would give a different impression.”

Asked how Will’s statement, which was also true, didn’t at least merit a “Mostly True” rating, Borowski replied, “I’m not going to be drawn into your little game here.”

 Borowski refuses to let Sikma draw him into the "game" of specifically justifying a PolitiFact rating.

Seriously, how is that a game?  PolitiFact supposedly exists to make these ratings and help readers decide whether they agree with the rating.

Sikma's approach exposes the fact that PolitiFact's "Truth-O-Meter" only plays at objectivity.  If a statement is false, PolitiFact can still justify a rating of "Mostly False" or higher.  If a statement is true, PolitiFact can still find ways to rate it "False" or even "Pants on Fire."  PolitiFact does this by inconsistently applying its set of standards.  The standards assist in the process by including few objective criteria on which to justify ratings.  Subjective and arbitrary ratings result, which may explain why PolitiFact editors get a bit cagey when pressed to justify a rating.


Jeff adds: Sikma highlights a longstanding gripe of ours. PolitiFact likes to present itself as answering some noble journalistic cause and providing an unbiased and objective view. But the reality is their ratings are ordinary commentary typical of any editorial page. There's simply nothing unique about PolitiFact's op-ed's ratings, save for the gimmicky Truth-O-Meter graphic and the conceit that they're immune to partisanship.

Borowski's sensitivity to being questioned betrays the joke. Facts, by definition, are non-partisan. But PolitiFact can't settle for simply publishing facts. That's why Romney's truth that Jeeps will be built in China becomes a Lie of the Year after PolitiFact expounds on what they think Romney was implying. Their Truth-O-Meter and hilariously ambiguous ratings definitions allow the editors to apply their own bias to tell you what the facts mean.

That's not fact-checking. That's political commentary. Sikma touched the right nerve with Borowski, and judging by the snide response it was a sensitive one.



Edit 4-22-13: Corrected spelling of "Sikma" in Jeff adds section

PolitiFact's comedic tilt

Here are two more examples of PolitiFact's hilarious inability to reach rational conclusions as it goes about its business.

Fred Thompson and the A-Ford-able Care Act

On April 3, a PolitiFact item appeared with the following headline:  "Ex-Sen. Fred Thompson says Obamcare could raise premiums enough to pay for a new Ford Explorer."

That sounds serious.  Ford Explorers are expensive.

But reading just a little deeper into the story, it turns out that Thompson's claim wasn't serious at all.

Here's how PolitiFact reports what Thompson literally said, instead of the attempted paraphrase:

As District Attorney Arthur Branch on Law & Order, former Sen. Fred Thompson was always ready with a wisecrack. In a recent Twitter post, the Tennessee Republican offered a pointed barb about President Barack Obama’s health care law:

"Report: Obamacare could raise ins premiums by 200%. It's the ‘A-Ford-able Care Act’ -- your insurance costs as much as a new Explorer."

First clue:  It's from Twitter, renowned for its 140-character limit.  Second clue:  "Ford" set apart by hyphens in "Affordable."  It's a joke, and PolitiFact hints that it's a joke by referencing the wisecracks Thompson offered while appearing as Arthur Branch on "Law & Order."

Why fact check a joke?

It makes sense to fact check a joke if the joke is supposed to convey a particular truth worth fact checking.  Was Thompson communicating that insurance premiums will cost the same as a new Explorer?  Isn't that line obviously part of the joke?

Thompson's Twitter feed is substantially made up of similar jokes.  He mentions a fact, then riffs on it for laughs.  Have a look at the tweet PolitiFact graded along with the two before and the two after.

It's a pattern with Thompson.  Offer a factoid, crack a joke.  The only facts worth checking are those Thompson expects readers to take seriously, which in this set means the factoid in the first line of each tweet.  PolitiFact, instead of looking at whether insurance premiums could rise as much as 200 percent, looked at the joke end of the statement.  Thompson gets a "False" instead of a "True."  Go figure.

We note that Thompson has written about PolitiFact's focus on his joke.

Dick Cheney and a wobbly President George H. W. Bush?

On April 10, PolitiFact put some bizarre spin on former vice president Dick Cheney's weekend appearance on Fox News, where he offered remembrance of the late Margaret Thatcher.  PolitiFact begins its quotations with the following:

Van Susteren: "But there's that famous quote where, apparently, she told President Bush 41 not to go wobbly."

Cheney: "That's not true."

When I hear the exchange it never occurs to me that Cheney is saying Thatcher never said "Don't go wobbly."  It seems to me that Cheney was telling Van Susteren that President Bush never went wobbly and people should not interpret Thatcher's comment that way.  As Cheney said immediately after:

There was never any doubt about what the president was doing. He didn't need any b[u]cking up.

If Cheney was trying to make the point that Thatcher never said the words attributed to her then he would  more likely say something like "Thatcher never said it.  It's an old wives' tale."

PolitiFact assumes Cheney claimed Thatcher never said the words and rates his supposed claim "False."


These types of PolitiFlubs do not happen rarely.



Jeff adds:

Check out how PolitiFact completely invented a claim that Thompson never made:

Image from PolitiFact

Even if we disregard Thompson's comical context, he never claimed that your premiums would rise by $30,000. That's a total distortion on the part of our Pulitzer winning pals. Thompson's claim, even if taken seriously, was that your insurance policy in total would equal the value of an Explorer, not that its price would rise that much.

And once again, we see PolitiFact playing games with quotation marks:

Here's what Thompson tweeted: "It's the ‘A-Ford-able Care Act’ -- your insurance costs as much as a new Explorer."

Note that PolitiFact added the word premiums, making Thompson's claim seem more outlandish. This, by the way, is exactly what they did to Mitt Romney in their bogus Lie of the Year rating:

PolitiFact took the Romney ad's claim; "sold Chrysler to Italians who are going to build Jeeps in China," which was and is entirely accurate, then added the words at the cost of American jobs, something the Romney ad never claimed, then slapped a Lie of the Year award on their own invention.

Whether it's shoddy work or intentional partisan spin, this type of nonsense should prevent PolitiFact from being taken seriously.

 

The Houston Chronicle: "Half of Ted Cruz’s political claims are false, PolitiFact reports" and "Ted Cruz v. PolitiFact: Whose pants are on fire?"

Joanna Raines of the Houston Chronicle wrote a story on March 28 attempting to report PolitiFact's assessment of Sen. Ted Cruz (R-Texas).

The headline and text of Raines' story hinted at a fallacious reasoning.

Raines' story was titled "Half of Ted Cruz’s political claims are false, PolitiFact reports."  It's somewhat natural to take PolitiFact's report card-ish stories along those lines, but the fact is that PolitiFact doesn't rate enough statements to draw general conclusions about political figures.  Compounding the problem, PolitiFact offers no hint at all that its system includes controls for selection bias.  Compounding the problem further, PolitiFact often reaches questionable conclusions with its ratings.

Raines published a follow up story on March 29, titled "Ted Cruz v. PolitiFact: Whose pants are on fire?"  That story implicitly acknowledged the last problem on the preceding list.  Raines offered a brief and reasonably fair account of the substance of Cruz's disagreements with PolitiFact.

Unfortunately, the second story contains a strong hint that Raines continues to reason fallaciously:

To navigate these muddy waters, we’re asking for you to weigh in. Cruz’s office has defended itself against PolitiFact’s claims, and we’re going to let you determine who deserves the “pants on fire” rating, Cruz or PolitiFact?

Ugh.

A survey of public opinion serves very poorly as a measure of truth in most instances.  The flawed reasoning goes by the Latin name argumentum ad populum, also known as the fallacious appeal to the people or the appeal to popularity.

Our guide to the truth should remain the state of the evidence, supplemented by consideration of the arguments pro and con.

Raines' first story helped underscore the truth of one of our longstanding criticisms of PolitiFact:  Publish "report cards" for candidates and many readers will assume that the results say something about the candidates unless the fact checkers explain the error behind that assumption.

Ramesh Ponnuru: "PolitiFact vs. Senator Cruz"

Ramesh Ponnuru of National Review takes note of a recent PolitiFact (Texas) flub:

At CPAC, Senator Cruz (who is, I should note, a friend) said that Democrats had told the Catholic Church, “Change your religious beliefs or we’ll use our power in the federal government to shut down your charities and your hospitals.” PolitiFact Texas is on the case, and it finds that in fact Democrats never used those exact words to describe the import of their policies. Its ruling: Cruz’s statement is “incorrect and ridiculous” and therefore merits a “Pants on Fire” rating.

 Ponnuru explains that Sen. Cruz was correct:

The administration’s policy gives these institutions — that is, tells them they have — two choices. They can comply, even if they think doing so violates their consciences (and they are, presumably, the right judges of that question). Or they can refuse to comply and be driven out of operation.

PolitiFact Texas admitted the possibility that the federal government would levy stiff fines that might make it impossible/impractical for various religious entities and businesses to continue operation.  But supposedly found no evidence of a threat to the Roman Catholic Church:

The Catholic bishops have said that potential accumulated fines resulting from refusals to carry out the contraception mandate will cause some institutions to shut down.

Perhaps. However, such prospects do not reflect a direct threat from Democrats or the government. To the contrary, the administration has moved to widen the mandate’s exemption for religious employers and provide a workaround for those who act as their own insurance providers -- with the goal of allowing affected parties to continue their work without violating or changing their beliefs.

Ponnuru likewise points out the trouble with this line of reasoning from PolitiFact.  Visit National Review Online for the rest of the story.

PolitiFact's partisan poll position

If one aspires to check facts, it doesn't hurt to have some command of logic and an application of logic to poll data.

PolitiFact, unfortunately, provides us a prime example of partisan poll interpretation with a pair of relatively recent fact checks.  Within the past week, PolitiFact ruled President Obama "Mostly True" on the claim that his position on the budget deficit negotiations--using a mix of spending cuts and tax increases to address the problem--has popular support.  In February 2012, PolitiFact revised a ruling on Sen. Marco Rubio (R-Fla.) from "Mostly True" to "Half True."  Rubio claimed a conservative majority in the United States.

PolitiFact's rulings serve as a useful case study showing the pitfalls of misusing poll data.

Where did PolitiFact go wrong on majority support for the president's "balanced approach"?

PolitiFact fell for a fairly obvious piece of misdirection from the White House.

President Obama wants it to look like the public supports his position compared to that of his Republican opposition.  To accomplish that, he gives a murky description.  His actual position would try to address the budget deficit by using spending cuts and "increased revenues" (read as "tax increases") in roughly equal measure.  To the general public, Mr. Obama proclaims he is simply in favor of a "balanced approach," which he hints is defined as a combination of spending cuts and "increased revenues."

The president's rhetoric draws an imbalanced line between his position and the Republican position.

The Republicans originally were unwilling to go along with tax increases.  Increased revenues were fine so long as they occurred because of increased economic activity instead of by changing the tax code.  When the Republicans fared poorly in the 2012 election they adopted a compromise position.  They would go along with increased revenues through the closing of tax loopholes.  When a deal on the "fiscal cliff" was finally struck, the Republicans compromised further, allowing tax rates to increase on the wealthy by allowing some of the Bush tax cuts to lapse in addition to a graduated phase-out of tax deductions.

The Republicans, then, compromised their position and allowed for increased revenues in the form of higher taxes.  But in 2013 the president has continued to say the Repubicans are unwilling to compromise on increased revenues because they will not compromise a second time on increased revenues.  The president identifies the Republican position as refusing all revenue increases.

In reality, the Democratic Party position consists of the approximately 1:1 ratio between spending cuts and revenue increases.  The real-world Republican position has allowed some revenue increases via higher taxation, but refuses a second compromise.

Now suppose we have a person who wants $4 in spending cuts for every $1 increase in revenue.  Does that person agree with the Republicans or with the president?  President Obama's rhetoric suggests the answer:  One agrees with the Republicans if one wants only cuts in spending.  Everyone in the middle ground between only cuts to spending and a 1:1 ratio between spending cuts favors the president's position.  And that's just the way PolitiFact does its fact check.

Where did PolitiFact go wrong on Marco Rubio's conservative majority?

During a Feb. 2012 speech, Rubio said he knew America had a conservative majority because people don't like to wear the label "liberal."

To some extent, Rubio was doing the same thing we looked at just above.  He was defining two groups in a way that would bolster his claim that most Americans are conservative.

In this case, PolitiFact never bothered to use the speaker's measuring stick.  PolitiFact chose its own:  If a person self-identified as a conservative then the person was a conservative.  Not otherwise.  PolitiFact simply ignored Rubio's attempt to claim an expanded middle ground and chose its own measure.

Rubio tried to make the point that conservatives could win elections by appealing to moderates who will not take accept the "liberal" label.  To measure his accuracy, one needs to pressure the moderates to make a choice between liberal and conservative.  Liberal writer Kevin Drum did just that (I've addressed this topic before at Sublime Bloviations).

 

Straw men on the loose

Addressing and defeating a weak version of a person's argument instead of a better version counts as a straw man fallacy.

In our first example, the president used a straw man version of support for Republicans on deficit reduction.  PolitiFact, in testing his claim, bought it.  People only agree with Republicans on deficit reduction if they will not allow for a dime of new revenue.  Otherwise they support the president's "balanced approach."

In the second example, PolitiFact came up with its own straw man version of Rubio's argument.  Rubio said he knew that most Americans are conservatives because they don't like the label "liberal."  PolitiFact did not measure conservatives by dislike of the "liberal" label but by preference for the label "conservative."

Does PolitiFact mislead its readers on purpose?  Probably not.  But the end result is the same either way.

Rand Paul Filibuster: You Should Have Chosen a More Responsible Fact Checker

PolitiFact's partisan posturing is pretzel-like. In this case, Robert Gibbs said something, PolitiFact confirmed he said it, then gave Rand Paul a Half-True for claiming Gibbs said it.  

This so-called fact check illustrates PolitiFacter Louis Jacobson's ability to acknowledge facts, confirm them, then completely ignore them. We've commented that PolitiFact is often an opinion site masquerading as objective fact checkers, but this rating is the stuff of propaganda.

The dust up comes when Paul says he's offended by Gibbs' comments regarding the death of Abdulrahman al-Awlaki, son of terrorist Anwar al-Awlaki. Both were American citizens, and both are dead. The government has confirmed killing one and is suspected of killing the other. Here's Rand Paul's claim (made during his marathon filibuster):

"When the president's spokesperson [Gibbs] was asked about al-Awlaki's son, you know what his response was? This I find particularly callous and particularly troubling. The president's response to the killing of al-Awlaki's son -- he said he should have chosen a more responsible father."

PolitiFact confirmed what Paul said about the incident, quoting Gibbs:

"I would suggest that you should have a far more responsible father if they are truly concerned about the well being of their children."

Most rational people with a functioning auditory system or basic reading comprehension would immediately recognize Paul's assertion as beyond dispute. PolitiFact, on the other hand, defines this as "some support for his claim."

So what are the dastardly details that made Paul so dishonest? PolitiFact explains:

Gibbs did not go out of his way to make this comment; it was elicited by an activist group...

A-HA! You see, Gibbs didn't say what he said because he said it after someone asked him a question! (They always get you that way, don't they?) And the people who asked him had some nefarious motivation behind the question! So, clearly, it's not fair to blame Robert Gibbs for the words that were forcibly extricated from his own mouth. After all, Gibbs is merely a professional spokesman, hand-selected by the president to present Obama's views to the public. How can we expect him to deal with things like "questions"? No True for you, Senator Paul!

PolitiFact goes on to weave a tale of woe for Gibbs, not to mention building a Strawqua Buddah in Paul's honor:

Gibbs did say essentially that. But Paul’s use of Gibbs’ comment leaves out some important context. It was not a carefully crafted statement of White House policy. Rather, it came at the tail end of a contentious exchange with representatives of an activist group, a line of questioning that Gibbs repeatedly tried to brush off.

When did Paul claim (or even suggest) it was a "carefully crafted statement of White House policy"? He didn't. It's fiction invented by PolitiFact. And since when does answering questions "at the end of a contentious exchange" absolve a speaker of his responses? That's hogwash.

But congrats are in order for at least one thing: PolitiFact's invention of the new standard that a speaker is not responsible for his own words if he "repeatedly tries to brush off" a question. Apparently, politicians or spokesmen are no longer accountable for their responses if they try to avoid the topic in the first place. This is what PolitiFact calls "sorting out the truth"?

The final paragraph takes the cake:

So while Paul is correct to say that Gibbs expressed that sentiment, the senator suggests that it’s more of an expression of White House policy...

Paul is correct, but SQUIRREL!

Paul never suggested it was "White House policy," and PolitiFact is being obscenely dishonest by claiming otherwise.


Additional reading: This isn't the first time PolitiFact has burned Paul on a perfectly accurate claim because of what they think he was suggesting. Remember this beauty?


Bryan adds:  

I have some sympathy with PolitiFact drawing a distinction between a statement from Gibbs and a statement from the president.  Paul's full statement equivocates a bit on that point, and Paul's concern about Gibbs' statement is expressed as a concern about the administration.

On the other hand, PolitiFact explains the distinction poorly and puts the "Half True" rating right next to a quotation of Paul that is perfectly accurate.  It's no excuse for Gibbs if he was hounded by journalists.  It isn't truly relevant.

Moreover, if Gibbs gets a break for dealing with hectoring journalists, why doesn't Paul get a break for speaking for hours on end with no prepared script?  Apparently PF didn't even care to investigate the conditions of Paul's speech, if we use its reporting as our gauge.

Jeff responds: Bryan and I went back and forth on this. I think he has a reasonable point that Paul didn't fully distinguish between Gibbs and Obama. I'd be more inclined to accept it if not for a few mitigating factors: 1) Gibbs' professional position as Obama's spokesman 2) The context immediately preceding the conflation ("When the president's spokesperson...") and 3) In the past, PolitiFact has held politicians responsible for claims made by their spokesmen. Why not now?

Regardless, conflating Gibbs and Obama hardly reaches the level of "suggesting" an "expression of White House policy." That's pure fantasy.

Reminder: PolitiFact's standards shift

While doing a triple-decker fact check of PolitiFact and the Violence Against Women Act over at Zebra Fact Check, I stumbled over yet another great example of PolitiFact's shifting standards.

Remember how PolitiFact had to rate Mitt Romney "Pants on Fire" for his claim Jeep would build Jeeps in China because the claim supposedly implied Jeep would move all its manufacturing to China?  Of course.  We all remember that one.

But how about 2008, when then-candidate for president Barack Obama said his newly-chosen running mate, Joe Biden, wrote the VAWA and domestic violence went down dramatically.

PolitiFact found that Obama was accurate about the amount of the decrease, but the story expressed doubts about the implied cause.

Okay, so both of Obama's statements are true. Biden wrote an anti-domestic violence law, and domestic violence rates dropped dramatically.

But did one cause the other? Although Obama doesn't say it directly, that's the clear implication in his statement, and that's where things get a bit murkier. As we discussed back in 2007, when Biden was gunning for the top spot on the Democratic ticket, figuring out the reasons for changes in crime statistics can be tricky.  Multiple factors, including economic prosperity and demographic changes, contributed to an overall decline in violent crime throughout the 1990s and into this decade.

Romney, like Obama, made true statements.  PolitiFact in both cases saved its objections for the implied argument.

After finding a shred of evidence that the VAWA helped decrease domestic violence, PolitiFact offered its ruling on Obama's statement:

A study by the University of Arkansas, for instance, concluded in 2000 that the law's increased funding for civil legal assistance for victims contributed to the decline. Though that study also said economic and demographic factors mattered.

Obama never said the law caused all of the decrease, but he implied it, so we will rate this statement Mostly True.

"Pants on Fire" for Romney.  "Mostly True" for Obama.

Totally fair and unbiased.  Or something.

We do find one mitigating factor in PolitiFact's defense.  This story was from 2008, before 2011 when PolitiFact announced that it would provide greater weight for implied claims of responsibility.  Still, an impressive contrast results between the two rulings.  Romney's "Pants on Fire" ruling helped him capture PolitiFact's ultra-subjective "Lie of the Year" award for 2012, while Obama's "Mostly True" simply helped cement the beltway media impression that Democrats are much more truthful than Republicans.  And Obama gets to keep his "Mostly True" rating in his PolitiFact file to this day.

PolitiFact and fact checking.  Meh.