CJR's interview with Robert Higgs

Columbia Journalism Review has posted online an interview with the editor of PolitiFact Ohio, Robert Higgs.

Though it was tempting to quote the positive, where Higgs expressed reservations about labeling a subject as a "liar," the winner for the PFB spotlight was this cringe-worthy response by Higgs:

What is your audience like? Do you have any sense of how it compares to the audience for the regular political or news coverage?

It is very diverse. I know we have readers who are very conservative, and I know we have readers who are very liberal who read it every day. I get email from all ends of the political spectrum, both criticizing us because they don’t like what we wrote or suggesting items to take a look at. I like that. I get hit from both sides, with people accusing me of having a leftist bias or an obvious conservative bent. It’s refreshing to know we are hitting somewhere down the middle.

Dude.


Afters:

Recommended question for the next Higgs interview:

Mr. Higgs, when PolitiFact Ohio uses the "Half True" grade on the 'Truth-O-Meter,' is it defined as "The statement is partially accurate but leaves out important details or takes things out of context" or is it defined as "The statement is accurate but leaves out important details or takes things out of context"?

Follow up: 

Has that definition ever changed at PolitiFact Ohio?



Jeff adds: This oft repeated notion that if PolitiFact is upsetting both sides, they must be doing something right, is grossly flawed. "If I put this foot in a bucket of ice, and my other foot in a smelting pot, it follows that I'll be perfectly comfortable." If the best self-analysis PolitiFact editors can come up with is a cliche that doesn't work, it's no surprise their fact checks get the same uncritical review.

Politico: "PolitiFact without the 'Truth-O-Meter'"

Politico's media guy Dylan Byers hits and misses with his "PolitiFact without the 'Truth-O-Meter'" column.

First the miss, occurring in Byers' set up based on last week's dust-up between PolitiFact and Rachel Maddow over a rating of Florida senator Marco Rubio:

PolitiFact, the Tampa Bay Times fact-checking project, has come under fire this week for a ruling that seems to contradict common sense. Yesterday, MSNBC's Rachel Maddow -- PolitiFact's most vocal critic -- went to town on the group for claiming that an assertion made by Florida Senator Marco Rubio was "mostly true" when it was, in fact, false.

Rubio was probably correct, so if Byers intends to say that the statement was false then he needs to do better fact checking himself.  The construction of the sentence allows him to blame it on Maddow, I suppose.

Now the hit:

So, here's a thought. Get rid of the 'Truth-O-Meter.'

Okay, so I was peddling that idea back in 2008, but it's nice to see others picking up on the notion.

Byers scores another hit in supporting his suggestion:

I asked Adair today if PolitiFact would ever consider getting rid of its rulings and just present the facts on their own.

"The Truth-O-Meter is a key part of PolitiFact's work," he said. "We independently research political claims, analyze their overall accuracy and rate them from True to Pants on Fire. The rating allows readers our assessment to see the overall accuracy at a glance; they can read our analysis for more details."

Here is a less generous interpretation of that claim: The "Truth-O-Meter" allows PolitiFact to market its research -- which is painstaking and time-consuming -- to a political discourse that doesn't have time to read its analysis. The "Truth-O-Meter" is what enables pundits to put politicians on the spot by saying, "Ok, but PolitiFact found that that statement was "'mostly false.'" It is what enables political opposition to sound the siren whenever something is ruled "Pants on Fire." And without these convenient rulings, people might stop paying attention.

The "Truth-O-Meter" is a marketing gimmick.  And despite the fact that the meter's design by nature degrades PolitiFact's journalism, PolitiFact is so wedded to it that no divorce is possible.

There's a sense in which PolitiFact's marketing approach is a "savior" to print journalists.  That perception probably helped PolitiFact capture its 2008 Pulitzer Prize.  Hard news reporting was made popular while newspaper circulation numbers steadily declined.

But the false prophecies are getting more difficult to overlook.



Jeff adds: Since when is Rachel Maddow "PolitiFact's most vocal critic"? Perhaps the voices of James Taranto or Mark Hemingway aren't able to break through the echo echo chamber chamber?

Bryan adds:  Maybe she's the "most vocal" critic because her televised messages are audible while most others just write?

PFB Smackdown: Lawrence O'Donnell, Rachel Maddow and Tommy Christopher

Uh-oh!  Liberals are once again scandalized by a PolitiFact fact check!

MSNBC's Lawrence O'Donnell appeared in the following political ad:




PolitiFact looked into O'Donnell's claim about critics calling the GI Bill "welfare" and ruled it "Mostly False."   The fact check does have some problems.

PolitiFact went easy on O'Donnell

The fact check contains a huge error.  PolitiFact overlooks the fact that O'Donnell is making an equivocal argument.  O'Donnell stresses that the GI Bill was an education program.  But when PolitiFact pressed MSNBC to support O'Donnell's claim, the latter responded by providing criticisms that almost exclusively aimed at unemployment benefits that were part of the bill.  O'Donnell's argument is a bait-and-switch.

PolitiFact claims to take such abuse of context into account.  Some of the "Truth-O-Meter" grades, in fact, carry clear signs of the perils of making a claim with limited context.

Rachel Maddow

The Weekly Standard: "Liberal Pundits Shocked to Discover PolitiFact Not Always Factual"

Mark Hemingway of the Weekly Standard has earned himself the reputation as perhaps PolitiFact's top critic.  As evidence of that, Hemingway beat me to the "late to the party" theme by about a month after the progressive outrage over PolitiFact's "Lie of the Year" selection for 2011.

I'm sorry I missed his article before now.

Hemingway:

So the liberal punditry woke up today to find that PolitiFact has declared the "Lie of the Year" to be Democrats's claim that Paul Ryan's budget will "end Medicare" or "end Medicare as we know it." They're having quite the collective freakout—see Paul Krugman, Jonathan Chait, Matt Yglesias, Brian Beutler, Steve Benen, et al.

Hemingway concedes the "end Medicare" claim has some truth to it:

Accusing Republicans of trying to end Medicare as we know it is also a stupid criticism because the implementation of the Independent Payment Advisory Board (IPAB) in the Patient Protection and Affordable Care Act will also "end Medicare as we know it." And unlike Ryan's plan, Democrats already made IPAB the law of the land. Under IPAB, unelected federal bureaucrats chosen by the president will bypass Congress and set the Medicare budget, and this will likely have pretty dramatic consequences for the program, such as severely restricting doctor access and rationing. It might well prove unconstitutional to boot.

So why all the outrage if Medicare as we know it is already dead and gone?  Hemingway has a hypothesis:

Liberals are freaking out over this because they're so used to PoltiFact and other fact checkers breaking things their way.

Ouch!

But he's probably right.  And, as usual, it's well worth reading the whole article.



Correction 2/21/2012:  Fixed spelling of "Pundits" in the title.

Don Surber: "Hey PolitiFact, here's your death panel"

Thanks to the Charleston Daily Mail and columnist Don Surber, a little reminder that PolitiFact's "Lie of the Year selection for 2011 isn't the first to receive well-grounded criticism:

The liberal apologists at the Tampa Bay Times’ PolitiFact have denied for more than 2 years that Obamacare has death panels. How it could assure us in 2009 of just what was in a law that was not finalized until 2010 is a mystery that defies the laws of chronology; maybe in addition to having the power to divine the truth in politics, the personnel of PolitiFact have the power of prophecy.

Surber's just getting started, so scoot on over to the Daily Mail's website and read the whole thing.

Positives: PolitiFact adds corrections page

A bit of optional background, first.

• Keeping up appearances at PolitiFact

• PolitiFlub: Mike Pence/Medicare/How deep the memory hole? (Updated x3)

• PolitiFact's Ucci correction/update

• Probing PolitiFact's mostly useless corrections policy (Updated x3)

On Feb. 7 of this year, PolitiFact announced the addition of a corrections page.

It's about time, better late than never, etcetera.

As a corrections page the new feature is pretty skimpy.  Essentially it's just a list of stories that have corrections or updates and looks like any other list of stories in the politifact.com domain.  One interested in finding a list of stories that required a change in rating is out of luck, at least with the present version of the page.

Caveats aside, we applaud the modest improvement in transparency.  Now we just have to figure out why the total number of corrections on the page disagrees with the number we counted on our "(Annotated) Principles of PolitiFact" page.

Looks like their corrections page needs a few corrections.  Or at least a clarification to illuminate the fact that PolitiFact does not intend to admit corrections from more than three weeks prior to the publication of its "Principles of PolitiFact and the Truth-O-Meter" on Feb. 21, 2011.

Welcome to the wonderful world of PolitiFact fact checking.

PolitiFact's prophylactic CYA

Yesterday PolitiFact rolled out a CYA article in response to the blowback to the oft-floated claim that 98 percent of all Catholic women use contraception.  PolitiFact rated that claim from an Obama administration official on Feb. 6, finding it "Mostly True."  PolitiFact's treatment of the issue provided little evidence of earnest journalistic curiosity and left its readers with no real means of independently verifying the data.

Watch how PolitiFact deftly avoids taking any responsibility for failing to present a clear account of the issue:

For the past week, thoughtful readers have let us know that we were wrong to give a Mostly True to the claim from a White House official that "most women, including 98 percent of Catholic women, have used contraception."

They said we overlooked a chart in a study from the Guttmacher Institute that showed the percentage was far more limited. But there’s a good reason we didn’t rely on the chart — it wasn’t the right one.

PolitiFact doesn't tell you that the Feb 6 story doesn't refer at all to the relevant chart.  PolitiFact claims to provide its sources. The source list doesn't include the relevant chart.  Instead, it features the charts that drew so much attention in the published criticisms.

Guttmacher Institute, "Contraceptive Use Is The Norm Among Religious Women," April 13, 2011

Guttmacher Institute, "Countering Conventional Wisdom: New Evidence on Religion and Contraceptive Use," April 2011

Centers for Disease Control and Prevention, "National Survey of Family Growth," accessed Feb. 2, 2012

Centers for Disease Control and Prevention, "Key Statistics from the National Survey of Family Growth," accessed Feb. 6, 2012

PolitiFact's mission (bold emphasis added):

PolitiFact relies on on-the-record interviews and publishes a list of sources with every Truth-O-Meter item. When possible, the list includes links to sources that are freely available, although some sources rely on paid subscriptions. The goal is to help readers judge for themselves whether they agree with the ruling.

Um, yeah, whatever.

So did PolitiFact fact check the item without checking the facts or simply forget to link the relevant data in the source list? 

Don't look for a confession in a CYA:

To double-check, we reviewed the criticism, talked with the study’s lead researcher, and reviewed the report and an update from the institute. We’re confident in our original analysis.

We can take that statement for what it's worth, given that the original analysis never produced a baseline for determining the error of the 98 percent figure.  We're left to guess whether the CYA intends to assure us that the original item includes data sufficient to help readers judge for themselves whether to agree with the ruling.

PolitiFact is suggesting that the fact check was perfectly fine, and those of you who used their references to try to reach your own conclusions mishandled the facts.

PolitiFact:

The spate of blog posts and stories this week — some directly claiming to debunk our reporting — unfortunately rely on a flawed reading of a Guttmacher Institute study.

They were easy mistakes to make, confusing the group of women who have "ever used" contraceptives with those who are "currently using" contraceptives — and misapplying footnote information about those "currently using" to the 98 percent statistic.

The "flawed reading" results directly from the fact that neither the Guttmacher Institute nor PolitiFact provided access to the data that might have supported the key claim.  I'll quote from the PFB assessment:  "That's fact checking?"

If PolitiFact had checked the claim properly in the first place then PolitiFact could have answered the criticisms without the wholesale review.  In fact, the criticisms would be clearly wrong based on material included in or linked from the original fact check.

More from PolitiFact:

The critics of our reporting — bloggers for the Weekly Standard, CatholicVote.org and GetReligion.org — were relying on an analysis from Lydia McGrew in her blog, "What's Wrong With The World," which was also cited by the Washington Post's WonkBlog.

PFB highlighted McGrew's analysis, certainly.  But our criticisms expanded beyond McGrew's and recognized that the Guttmacher Institute report may have included data that PolitiFact neglected to explain to its readers.  One would think from PolitiFact's response above that no criticism of its reporting on this issue contains merit.

Focus on McGrew

PFB Smackdown: Rachel Maddow (Updated)

We agree with Rachel Maddow up through about the 55 second mark.  Yes, PolitiFact is bad, and PolitiFact is so bad at fact checking that it doesn't deserve frequent citations as a trustworthy source. 

After that, our level of agreement starts to drop.

Sen. Rubio (R-Fla.) stated that most Americans are conservative and went on to argue the point based on attitudes toward the labels "conservative" and "liberal."

Maddow ignores the context of Rubio's remarks and attacks it using survey data about the way Americans self-identify politically.

Maddow is supposed to be ultra smart.  So how come she can't figure out that Rubio's statement isn't properly measured against self-identification numbers?

It appears that Maddow uncritically followed PolitiFact's approach to judging Rubio's accuracy.  The self-identification numbers serve as interesting context, but it's perfectly possible for 100 percent of Americans to self-identify as "liberal" yet reasonably classify as majority conservative.  That's because people can have inaccurate perceptions of their location on the political spectrum.

So, was Rubio correct that the majority of Americans are conservative?  That depends on his argument.  Rubio didn't cite surveys about self-identification.  He used a method concerned with attitudes toward the respective labels.  One can argue with the method or the application of the method, but using an inappropriate benchmark doesn't cut it.

When you ask people which party they lean toward, the independents split up so that the country is almost evenly divided. For the year of 2011, Gallup reported that 45 percent of Americans identified as Republicans or leaned that way, while 45 percent identified as Democrats or leaned that way.

Is "Republican" the same label as "conservative"?  No, of course not.

PolitiFact came close to addressing Rubio's point by looking at the political leanings of moderates, but fell short by relying on the wrong label along with the self-identification standard.  Maddow's approach was even worse, as she took Rubio's comment out of context and apparently expected PolitiFact to do the same thing.

Meanwhile, PolitiFact defends itself with the usual banalities:

“Our goal at PolitiFact is to use the Truth-O-Meter to show the relative accuracy of a political claim,” Adair explained. “In this case, we rated it Mostly True because we felt that while the number was short of a majority, it was still a plurality. Forty percent of Americans consider themselves conservative, 35 percent moderate and 21 percent liberal. It wasn’t quite a majority, but was close.”

“We don’t expect our readers to agree with every ruling we make,” he continued.

Pretty weak, isn't it?


Update 2/19/2012:

With a hat tip to Kevin Drum of Mother Jones (liberal mag), we have survey data that help lend support to Marco Rubio (as well as to my argument in his defense):

(click image to enlarge)

1)  The survey, from Politico and George Washington University, is limited to likely voters.
2)  The poll essentially forces likely voters to choose between "liberal" and "conservative."
3)  A plurality of those surveyed (43 percent) lean Democrat or self-identify as Democrat.
4)  Despite the plurality of Democrats in the survey sample, 61 percent identify as conservative ("Very conservative" or "Somewhat conservative").