Left jab: Jared Bernstein and median income in Romney's Massachusetts

Much of the criticism of PolitiFact from liberals is lame.  On occasion, however, a liberal makes a pretty good point, and Jared Bernstein makes one of those in commenting on a PolitiFact rating about median income in Massachusetts while Mitt Romney served as governor.

The claim came from Ed Gillespie, who was defending Romney's record in Massachusetts by pointing out that median income in Massachusetts rose by $5,500.  The problem, as Bernstein pointed out and PolitiFact acknowledged in the story, comes from the fact that family income actually dropped slightly after adjusting for inflation.

Saying that income went up by thousands of dollars while it dropped in terms of real dollars is flatly misleading. Bernstein has a good foundation for a sense of outrage.

But even though Bernstein's basic point is accurate, he oversteps a bit by assigning the statement a rating of "Mostly False."  That's because PolitiFact gives each of the ratings a definition.  Here's the definition of "Mostly False":

MOSTLY FALSE – The statement contains an element of truth but ignores critical facts that would give a different impression.

Gillespie's statement did not simply contain an element of truth.  It was true and misleading.  The "Mostly False" rating serves as an ill fit for claims like Gillespie's because the rating itself is only half true.

When PolitiFact started out, the "Half True" rating appeared to fit statements like Gillespie's:

HALF TRUE – The statement is accurate but leaves out important details or takes things out of context.

Some of PolitiFact's state operations may still have this definition of "Half True" posted on their websites (confirmed for Texas).

PolitiFact's change of the definition of "Half True" (with zero fanfare outside of us) makes it no longer quite fit fundamentally misleading true statements:

HALF TRUE – The statement is partially accurate but leaves out important details or takes things out of context.

Gillespie's statement wasn't partially accurate.  It was accurate but misleading.

A more important point underlies Bernstein's criticism of PolitiFact:  PolitiFact uses an unwieldy rating system.  Many statements do not fit the ratings as PolitiFact defines them, particularly when PolitiFact rates two statements at a time and averages the ratings.

So Bernstein scores a glancing blow against PolitiFact.  It doesn't hurt PolitiFact much but it does expose a critical weakness.

Eric Zorn: "True lies: Media umpires confront the challenge of dishonest facts"

Eric Zorn of the Chicago Tribune produced an excellent column criticizing PolitiFact's (and the Washington Post's) approach to fact checking.

Perhaps its biggest flaw comes from the fact that Zorn is so late to the party, like so many other critics from the ranks of mainstream journalists and the political left. 

The pair of quotations near the end of his column cap it off nicely:

"I've never been able to see an academically defensible way to hand out those kinds of ratings," director Brooks Jackson [of FactCheck.org] told me Thursday. "But I secretly admire the ability of (PolitiFact and The Washington Post) to do that. It engages the readers, and it's fun. But we'll leave questions of 'truth' to the theologians and philosophers."

PolitiFact editor Bill Adair told me that "the Truth-O-Meter is the heart of our organization. We'd never consider getting rid of it, but we always encourage people not to look just at the rating, but also at our reasoning."

The "Truth-O-Meter" misleads PolitiFact's audience, just as Zorn and many others have pointed out.  Yet Adair says "We'd never consider getting rid of it."

Would you buy a used car from that guy?

Zorn makes a good number of important points, so please read and digest the entire column.

Politico: "PolitiFact without the 'Truth-O-Meter'"

Politico's media guy Dylan Byers hits and misses with his "PolitiFact without the 'Truth-O-Meter'" column.

First the miss, occurring in Byers' set up based on last week's dust-up between PolitiFact and Rachel Maddow over a rating of Florida senator Marco Rubio:

PolitiFact, the Tampa Bay Times fact-checking project, has come under fire this week for a ruling that seems to contradict common sense. Yesterday, MSNBC's Rachel Maddow -- PolitiFact's most vocal critic -- went to town on the group for claiming that an assertion made by Florida Senator Marco Rubio was "mostly true" when it was, in fact, false.

Rubio was probably correct, so if Byers intends to say that the statement was false then he needs to do better fact checking himself.  The construction of the sentence allows him to blame it on Maddow, I suppose.

Now the hit:

So, here's a thought. Get rid of the 'Truth-O-Meter.'

Okay, so I was peddling that idea back in 2008, but it's nice to see others picking up on the notion.

Byers scores another hit in supporting his suggestion:

I asked Adair today if PolitiFact would ever consider getting rid of its rulings and just present the facts on their own.

"The Truth-O-Meter is a key part of PolitiFact's work," he said. "We independently research political claims, analyze their overall accuracy and rate them from True to Pants on Fire. The rating allows readers our assessment to see the overall accuracy at a glance; they can read our analysis for more details."

Here is a less generous interpretation of that claim: The "Truth-O-Meter" allows PolitiFact to market its research -- which is painstaking and time-consuming -- to a political discourse that doesn't have time to read its analysis. The "Truth-O-Meter" is what enables pundits to put politicians on the spot by saying, "Ok, but PolitiFact found that that statement was "'mostly false.'" It is what enables political opposition to sound the siren whenever something is ruled "Pants on Fire." And without these convenient rulings, people might stop paying attention.

The "Truth-O-Meter" is a marketing gimmick.  And despite the fact that the meter's design by nature degrades PolitiFact's journalism, PolitiFact is so wedded to it that no divorce is possible.

There's a sense in which PolitiFact's marketing approach is a "savior" to print journalists.  That perception probably helped PolitiFact capture its 2008 Pulitzer Prize.  Hard news reporting was made popular while newspaper circulation numbers steadily declined.

But the false prophecies are getting more difficult to overlook.



Jeff adds: Since when is Rachel Maddow "PolitiFact's most vocal critic"? Perhaps the voices of James Taranto or Mark Hemingway aren't able to break through the echo echo chamber chamber?

Bryan adds:  Maybe she's the "most vocal" critic because her televised messages are audible while most others just write?

Slate: "PolitiFact Weirdly Unable to Discuss Facts"

PolitiFact's recent spat with its liberal readership base has led to the publication of quite a few stories that echo criticisms recurrent in the posts we publish and link at PolitiFact Bias.

Slate's Dave Weigel, famously/formerly of the Journolist, has another such:

After this week, plenty of pundits are well and done with the national version of PolitiFact. The local versions? They're great. I was actually pretty fond of how one of them debunked an ad that misued [sic] one of my quotes, attributing it to a candidate, in 2010. Alas, PolitiFact Editor Bill Adair has committed the main site to a factually dubious "Lie of the Year" claim. PolitiFact claims that it's a "lie" to say that the Path to Prosperity ends Medicare. ActualFacts tell us that this is not a lie.

Adair responds to the critics in the worst possible way.

At a Republican campaign rally a few years ago, I asked one of the attendees how he got his news.

"I listen to Rush and read NewsMax," he said. "And to make sure I'm getting a balanced view, I watch Fox."

We're starting with an anoymous [sic] quote from a straw man that Adair met once?

Weigel continues to expand on Adair's defense, noting that it does nothing to address substantive criticisms.

Adair's response matches the customary pattern at PolitiFact, with the possible exception of the explanation PolitiFact offered after one of its criticisms of Rachel Maddow likewise offended liberal sensibilities.  The sad thing is that it took so long for so many liberals to see it.  Apparently it's easy to overlook the problem so long as conservatives have to deal with the bulk of the harm.

Though we hardly agree with Weigel about the quality of PolitiFact's state franchises (the jury's still out on most of them), his main point is well taken and the post is worth reading.

PolitiFact would gain credibility if it answered substantive criticisms with well-reasoned rebuttals. 

Claiming the critics suffer from some type of echo-chamber syndrome that prevents them from understanding PolitiFact's greatness is not a well reasoned rebuttal.  Rather, it is an ad hominem fallacy.  Readers are not well served with that type of response.

Jeff adds: Weigel continues with a curious new pattern we've noticed with liberal writers describing PolitiFact. What used to be a ubiquitary reference to PolitiFact's Pulitzer (which served to inform the reader of their unquestionable credibility and authority) is no longer worth the extra space to mention.