Offsite Criticism of the Week

While PolitiFact continues its pattern of ducking public criticism, we will continue to model the opposite approach by dealing with a typical offsite criticism we run across (real example from real discussion board):

First, we'll note that generally citing PolitiFact Bias is the wrong way to defuse criticisms that Trump doesn't tell the truth. There are good reasons to dismiss the use of the PolitiFact statistics that some use to "prove" somebody lies more than somebody else. The liberal use of selection bias lays a poor foundation for reliable statistics. We've been pointing that out for years.

As for the criticism:

The first article in the first link says PolitiFact is wrong because it only used the data that was available to make its ruling. Oh the humanity!

No.

We believe the comment was directed at our evaluation of the Loretta Lynch fact check where Lynch said the LGBT community endures most of the statistically recorded hate crimes. It's true, as Dustin Siggins pointed out, that PolitiFact relied on incomplete statistics. In keeping with PolitiFact's common practice, that should have prevented Lynch from getting a rating any better than "Mostly True." Lynch received a "True" rating.

But our central criticism was quite different from what the writer said.

We noted that Lynch said nothing, in context, about using the per capita measure. If that measure isn't used, then PolitiFact's statistics say blacks experience most of the hate crimes recorded in the statistics. So what Lynch said was false. PolitiFact pretended otherwise to give her the "True" rating.

This is typical of the criticism we receive. We're dismissed based on bias (ad hominem fallacy) and on misrepresentations of our work (straw man fallacy).

We'd love to do "Offsite Criticism of the Week" for a criticism that is not fallacious. Comments are open here and on Facebook if anybody knows of one.

A reader's take on our "Pants on Fire" research

Elizabeth MacInnis wrote:

Your premise here is that if there are an equal amount of fact checks and Republicans lie more, then there must be fact-checker bias. It couldn't possibly be that Republicans actually do lie more. That's an unobjective analysis. In my opinion, watching both sides closely in each election, Republicans do lie more, but not for the reason that you think. Democrats typically run on a platform of hope and ideas. It's much more subjective. Republicans tend to run on fear and attacks against other candidates (whether you like this or not, it's true). An attack on someone's record is more likely to be proven true or false. Watching all the debates, I consistently hear Republicans say things like "we've had a job killing president," when in reality (as of now) we've had 72 months of private-sector job growth, a record. With consistent, clearly false statements like this, Republicans are doing it to themselves. I believe we need a balanced system with both parties, but in many Americans' opinions (including mine), Republicans have become more and more outrageous in recent years (and no, that is not to say that Democrats are clear of any wrongdoing). The only way, in my opinion, to tame this is to hold them accountable - in fact-checking and votes. I hope both parties learn this lesson before their groups become too fractured.

Point by point:

"Your premise here is that if there are an equal amount of fact checks and Republicans lie more, then there must be fact-checker bias."

No, that's not our premise. Our premise (the one you seem to be talking about) is that if PolitiFact chose its stories only based on its editorial sense of whether the claim is true, then the results would be proportional. So Republicans could have five times more "false" ratings than Democrats but the distribution curve for both parties should appear similar. We don't think PolitiFact uses only its editorial sense in choosing stories.

The true central premise of the "Pants on Fire" research is that PolitiFact offers no objective means of distinguishing between its ratings of "False" and "Pants on Fire."

"It couldn't possibly be that Republicans actually do lie more. That's an unobjective analysis."

Can anyone explain to me how the "Pants on Fire" rating, with no apparent objective measure undergirding its use, contributes any empirical data toward the notion that Republicans lie more? Isn't that notion a sham?

"In my opinion, watching both sides closely in each election, Republicans do lie more, but not for the reason that you think. Democrats typically run on a platform of hope and ideas. It's much more subjective. Republicans tend to run on fear and attacks against other candidates (whether you like this or not, it's true)."

Summing up, then, MacInnis' opinion is true whether I like it or not? Is there any solid evidence supporting that opinion? Any at all?



(skipping some opinion that doesn't interest me so much)

"Watching all the debates, I consistently hear Republicans say things like "we've had a job killing president," when in reality (as of now) we've had 72 months of private-sector job growth, a record."

Remember when the Obama administration was lauding the helpful effects of the $900 billion stimulus bill? Employment was going down, but PolitiFact accepted arguments that unemployment would be even worse without the stimulus bill. Why is it that PolitiFact gives no consideration at all to a parallel principle with respect to claims of job-killing? That's not consistent, is it? We shouldn't make the mistake of thinking that inconsistent methods lead to good fact-checking, should we?

Elizabeth MacInnis, PolitiFact does fact-checking poorly. Don't be fooled.

PolitiFact's subjective "Pants on Fire" ratings tell us about PolitiFact, not about the entities receiving the subjective ratings.


 

Lost letters: Apology Tour edition

We love getting reader feedback on our posts. And we're much more likely to respond to criticism than is PolitiFact. We found this bit of feedback very recently posted to a message board after somebody mentioned how PolitiFact found claims about an Obama "apology tour" false:

How cute, some nut's blog claiming that conservative-leaning Politifact is biased in favor of Obama. And claims that Obama had an "apology tour" because...far-right nutcase Nile Gardiner of the Heritage Foundation says so

How odd for "conservative-leaning PolitiFact" to dismiss the one conservative view among the four experts whose views it solicited! Especially when Gardiner stood alone among the experts with his experience in international relations.

And, of course, our argument wasn't based solely on PolitiFact arbitrarily ignoring an expert opinion it solicited. We found descriptions of apologies in professional journals and applied the definitions we found to the idea of an Obama "apology tour." Because that's what nutcases do.

Why didn't PolitiFact bother reviewing professional literature in its effort to settle the fact of the matter? Probably the same reason it ignored Gardiner's professional opinion: PolitiFact is conservative-leaning.

It makes complete sense if reality is liberally biased.


Find the full explanation of why it's reasonable to call Obama's world tour an "apology tour" at Zebra Fact Check. 

Addendum:

Jeff thinks it may not be obvious to every visitor that I'm kidding about PolitiFact's conservative bias. This note is intended as a corrective for any who might fall in that group.

Lost letters to PolitiFact Bias

We discovered a lost letter of sorts intended in response to our recent post "Fact-checking while blind, with PolitiMath."

Jon Honeycutt, posting to PolitiFact's Facebook page, wrote that he posted a comment to this site but it never appeared. I posted to Facebook in response to Honeycutt, including the quotation of his criticism in my reply:

Jon Honeycutt (addressing "PolitiFact Bias") wrote:

Hmm, just looked into 'politifact bias', the very first article I read http://www.politifactbias.com/.../fact-checking-while... Claimed that politifact found a 20% difference in the congressional approval rating but still found the meme mostly true. But when you read the actual article they link to, politifact found about a 3% difference. Then when I tried to comment to correct it, my comment never appeared.

Jon, I'm responsible for the article you're talking about. You found no mistake. As I wrote, "percentage error calculations ours." That means PolitiFact didn't bother calculating the error by percentage. The 3 percent different you're talking about is a difference in terms of percentage *points*. It's two different things. We at PolitiFact Bias are much better at those types of calculations than is PolitiFact. You were a bit careless with your interpretation.I have detected no sign of any attempt to comment on that article. Registration is required or else we get anonymous nonsense. I'd have been quite delighted to defend the article against your complaint.

To illustrate the point, consider a factual figure of 10 percent and a mistaken estimate of 15 percent. The difference between the two is 5 percentage points. But the percentage error is 50 percent. That's because the estimate exceeds the true figure by that percentage (15-10=5, 5/10=.5).

http://www.basic-mathematics.com/calculating-percent-error.html

Don't be shy, would-be critics! We're no less than 10 times better than is PolitiFact at responding to criticism, based on past performance. The comments section is open to those who register, and anyone who is a member of Facebook can post to our Facebook page.

PolitiFact Bias is biased

We get stuff like this all the time:

I read a bit of that site, and it happens to have its own biases.

Well, right.  The authors, Jeff D. and Bryan White, both admit to a conservative bias.

So what?

Again, we freely admit to our personal bias on our About page.  We're honest about our political biases.  That's something you won't get from PolitiFact.  PolitiFact's founding editor (now at Duke University and serving PolitiFact in an advisory role) Bill Adair could have a life-sized cutout of President Obama in his office and they'd never willingly tell you about it because it might make them seem less non-partisan. So that makes PolitiFact more objective than PolitiFact Bias.  Seriously?

There are two kinds of bias in journalism. One is bias that's nestled in the brain, and we're all stuck with it to one degree or another, every one of us.  The important bias is the one that comes out in writing and reporting.  When we make our case that PolitiFact is biased, our goal is to make that case without bias.  We're not perfect, but that's the goal.  And if we're not meeting that goal, we'd love to hear about it.  And you know what doesn't help one bit in helping us meet that goal?  It's comments like this:

I read a bit of that site, and it happens to have its own biases.

If there was some specific case mentioned then maybe we could use it to make an improvement.  But comments like the above typically serve as fallacious attacks on our credibility.  We're not wrong because we bring a conservative bias to our work.  If we're wrong on account of bias, it's because we let that bias introduce errors or fallacious thinking into our work.

We love to hear good criticism, whether constructive or not.  The world could do with less criticism that lacks content, focus or specifics.

Letters to PolitiFact: "Losing Credibility"

Alert reader Michael Schmidt shared this letter he wrote to PolitiFacter Louis Jacobson. Mr. Schmidt raises some excellent points and makes a good comparison. We appreciate him granting us permission to post it so you can read it as well. The following is unedited with the exception of formatting:

Your continued statements regarding the following being "half true" impacts your credibility since you treat two individuals in nearly identical situations differently. Here is the statement on Obama:

After the Supreme Court upheld the health care bill he’d signed into law, President Barack Obama applauded the decision in a speech at the White House. In that speech, Obama responded to critics, including Mitt Romney, who say the law could force many Americans off their health care plans.

"If you're one of the more than 250 million Americans who already have health insurance, you will keep your health insurance," Obama said. "This law will only make it more secure and more affordable."

The next statement is regarding Valerie Jarrett:

But critics of the law have been on the attack about what they call Obama’s broken promise. Defending the law, White House senior adviser Valerie Jarrett sent out this message via Twitter on Oct. 28, 2013:

"FACT: Nothing in #Obamacare forces people out of their health plans. No change is required unless insurance companies change existing plans."

Both instances are in response to defending Obamacare. One uses "nothing" while the other states "if you are one of". Your ruling points out that "nothing" is an extreme claim. Obamacare effectively stated "all" of the 250 million people will get to keep their health insurance which appears to be extreme also.

In defense of your ruling for Obama's statement and partly as a critique, you state that Obama "suggests that keeping the insurance you like is guaranteed."  In the statement I read, the statement was explicitly stated versus implied. To soften the statement to "suggests" is a misrepresentation.

I enjoyed the PolitiFact site for a while and I go back from time to time but its apparent bias to a Democratic president and the criticisms I have seen regarding the disparity in rulings between Democrats and Republicans is convincing me that the site is biased and therefore guilty of the same behavior as both politicians and mainstream media outlets (on either side of the political spectrum).

Regards,

Michael Schmidt

Mr. Schmidt repeats a common theme we've talked about here, and that is PolitiFact's inconsistency. Obama makes a definitive statement, and PolitiFact mitigates it by painting it as merely a vague implication. Jarrett does the same but fails to earn PolitiFact's protective nuance allowance. It's true that both Obama and Jarrett are Democrats, but we've pointed out before that inept ratings toward liberals can be indicative of just how incompetent and arbitrary PolitiFact's system is. It's also worth noting that Jarrett is hardly the figurehead of liberalism that Obama is, and throwing her under the bus is a small price to pay for PolitiFact's appearance of neutrality.

Ultimately, PolitiFact's inconsistency overwhelmingly falls against one side of the political aisle, but there's so much inconsistency there's certain to be liberals hit by friendly fire. PolitiFact's ratings are arbitrary, and should be treated as the commentary pieces they are.

Many thanks to Mr. Schmidt for allowing us to share his letter. Of course, if Mr. Jacobson would like to respond to Schmidt's letter we're happy to publish it here with his permission.



Edit 11/10/13 2104PST:  Added links to two PolitiFact articles in both instances of the word "statement" in Mr. Schmidt's letter. The links did not appear in the original letter sent to us, but it's our policy to link to the PolitiFact articles we are critiquing. - Jeff