PolitiFact's prophylactic CYA

Yesterday PolitiFact rolled out a CYA article in response to the blowback to the oft-floated claim that 98 percent of all Catholic women use contraception.  PolitiFact rated that claim from an Obama administration official on Feb. 6, finding it "Mostly True."  PolitiFact's treatment of the issue provided little evidence of earnest journalistic curiosity and left its readers with no real means of independently verifying the data.

Watch how PolitiFact deftly avoids taking any responsibility for failing to present a clear account of the issue:

For the past week, thoughtful readers have let us know that we were wrong to give a Mostly True to the claim from a White House official that "most women, including 98 percent of Catholic women, have used contraception."

They said we overlooked a chart in a study from the Guttmacher Institute that showed the percentage was far more limited. But there’s a good reason we didn’t rely on the chart — it wasn’t the right one.

PolitiFact doesn't tell you that the Feb 6 story doesn't refer at all to the relevant chart.  PolitiFact claims to provide its sources. The source list doesn't include the relevant chart.  Instead, it features the charts that drew so much attention in the published criticisms.

Guttmacher Institute, "Contraceptive Use Is The Norm Among Religious Women," April 13, 2011

Guttmacher Institute, "Countering Conventional Wisdom: New Evidence on Religion and Contraceptive Use," April 2011

Centers for Disease Control and Prevention, "National Survey of Family Growth," accessed Feb. 2, 2012

Centers for Disease Control and Prevention, "Key Statistics from the National Survey of Family Growth," accessed Feb. 6, 2012

PolitiFact's mission (bold emphasis added):

PolitiFact relies on on-the-record interviews and publishes a list of sources with every Truth-O-Meter item. When possible, the list includes links to sources that are freely available, although some sources rely on paid subscriptions. The goal is to help readers judge for themselves whether they agree with the ruling.

Um, yeah, whatever.

So did PolitiFact fact check the item without checking the facts or simply forget to link the relevant data in the source list? 

Don't look for a confession in a CYA:

To double-check, we reviewed the criticism, talked with the study’s lead researcher, and reviewed the report and an update from the institute. We’re confident in our original analysis.

We can take that statement for what it's worth, given that the original analysis never produced a baseline for determining the error of the 98 percent figure.  We're left to guess whether the CYA intends to assure us that the original item includes data sufficient to help readers judge for themselves whether to agree with the ruling.

PolitiFact is suggesting that the fact check was perfectly fine, and those of you who used their references to try to reach your own conclusions mishandled the facts.

PolitiFact:

The spate of blog posts and stories this week — some directly claiming to debunk our reporting — unfortunately rely on a flawed reading of a Guttmacher Institute study.

They were easy mistakes to make, confusing the group of women who have "ever used" contraceptives with those who are "currently using" contraceptives — and misapplying footnote information about those "currently using" to the 98 percent statistic.

The "flawed reading" results directly from the fact that neither the Guttmacher Institute nor PolitiFact provided access to the data that might have supported the key claim.  I'll quote from the PFB assessment:  "That's fact checking?"

If PolitiFact had checked the claim properly in the first place then PolitiFact could have answered the criticisms without the wholesale review.  In fact, the criticisms would be clearly wrong based on material included in or linked from the original fact check.

More from PolitiFact:

The critics of our reporting — bloggers for the Weekly Standard, CatholicVote.org and GetReligion.org — were relying on an analysis from Lydia McGrew in her blog, "What's Wrong With The World," which was also cited by the Washington Post's WonkBlog.

PFB highlighted McGrew's analysis, certainly.  But our criticisms expanded beyond McGrew's and recognized that the Guttmacher Institute report may have included data that PolitiFact neglected to explain to its readers.  One would think from PolitiFact's response above that no criticism of its reporting on this issue contains merit.

Focus on McGrew

What's Wrong With the World: "How to Lie with Statistics, Example Umpteen"

Jeff and I hugely appreciate bloggers who delve into the more complicated PolitiFact-related issues.

Lydia McGrew of the "What's Wrong With the World" blog gives a proper dressing-down to the Obama administration, the Guttmacher Institute and our beloved PolitiFact over the supposedly "Mostly True" claim that 98 percent of Catholic women use birth control.

As is our wont, we'll focus primarily on PolitiFact's role in the mess.

McGrew:

(T)his Politifact evaluation of the meme gets it wrong again and again, and in both directions.

First, the Politifact discussion insists that the claim is only about women in this category who have ever used contraception. When I first heard that and hadn't looked at the study, I immediately thought of the fact that such a statistic would presumably include women who were not at the time of the study using contraception and had used it only once in the past. It was even pointed out to me that it would include adult converts whose use might easily have been prior to their becoming Catholic. However, that isn't correct, anyway. The study expressly was of current contraceptive use. That's, in a sense, "better" for the side that wants the numbers to be high.

McGrew pointed out earlier that the Guttmacher Institute study uses data for "women at risk for
unintended pregnancy, whom we define as those who had had sex in the three months prior to the survey and were not pregnant, postpartum or trying to get pregnant."  The women surveyed were additionally in the 15-44 age range.  Yet PolitiFact describes the findings like so:

We read the study, which was based on long-collected, frequently cited government survey data. It says essentially that — though the statistic refers specifically to women who have had sex, a distinction Muñoz didn’t make.

But that’s not a large clarification, since most women in the study, including 70 percent of unmarried Catholic women, were sexually experienced.

That's fact checking?

McGrew:

(O)n this point, too, the Politifact evaluation is completely wrong. Politifact implies that only the supplementary table on p. 8 excluded these groups and that Figure 3 on p. 6 included them! But this is wrong. The table on p. 8 is simply supplementary to Figure 3, and both are taken from the same survey using the same restrictions! This is made explicit again and again in the study.

McGrew's exactly right.  The same information accompanies the asterisk for each table (bold emphasis added):  "*Refers to sexually active women who are not pregnant, postpartum or trying to get pregnant."

It doesn't occur to PolitiFact that restricting the survey population like that throws a serious spanner in the works.

That kind of credulity goes by a different name:  gullibility.

Visit What's Wrong With the World and read all of McGrew's skillful fisking of the liberal trio.  It's well worth it.


Addendum:

The Guttmacher Institute drew its data ultimately from here.

It may be the case that the Guttmacher study is reliable.  Regardless of that, PolitiFact did virtually nothing to clarify the issue.  A recent Washington Post story does shed some light on things, however:

I called up Rachel Jones, the lead author of this study, to have her walk me through the research. She agrees that her study results do not speak to all Catholic women. Rather, they speak to a specific demographic: women between 15- and 44-years-old who have ever been sexually active.

Jeff Adds (2/15/2012): Over on PolitiFact's Facebook page, frequent PF critic Matthew Hoy offered up his usual spot on commentary:

I find [PolitiFact's] failure to note that the Alan Guttmacher Institute is closely allied with Planned Parenthood a troubling omission. It isn't some neutral observer and its studies shouldn't be taken at face value without some healthy skepticism.

This isn't the first time PolitiFact has ignored Guttmacher's relationship with Planned Parenthood. Regardless of the studies accuracy, the alliance deserves at least a cursory disclosure. It's also important to note that PolitiFact used a similar connection to justify the rating of Florida Governor Rick Scott's claim about high-speed rail projects:

Scott bases his claims on hypothetical cost overruns from a suspect study written by a libertarian think tank...We rate Scott's claim False.

We highlighted that rating here.



Correction 2/17/2012:  "Guttmacher" was misspelled in the next-to-last paragraph.