PolitiFact's prophylactic CYA

Yesterday PolitiFact rolled out a CYA article in response to the blowback to the oft-floated claim that 98 percent of all Catholic women use contraception.  PolitiFact rated that claim from an Obama administration official on Feb. 6, finding it "Mostly True."  PolitiFact's treatment of the issue provided little evidence of earnest journalistic curiosity and left its readers with no real means of independently verifying the data.

Watch how PolitiFact deftly avoids taking any responsibility for failing to present a clear account of the issue:

For the past week, thoughtful readers have let us know that we were wrong to give a Mostly True to the claim from a White House official that "most women, including 98 percent of Catholic women, have used contraception."

They said we overlooked a chart in a study from the Guttmacher Institute that showed the percentage was far more limited. But there’s a good reason we didn’t rely on the chart — it wasn’t the right one.

PolitiFact doesn't tell you that the Feb 6 story doesn't refer at all to the relevant chart.  PolitiFact claims to provide its sources. The source list doesn't include the relevant chart.  Instead, it features the charts that drew so much attention in the published criticisms.

Guttmacher Institute, "Contraceptive Use Is The Norm Among Religious Women," April 13, 2011

Guttmacher Institute, "Countering Conventional Wisdom: New Evidence on Religion and Contraceptive Use," April 2011

Centers for Disease Control and Prevention, "National Survey of Family Growth," accessed Feb. 2, 2012

Centers for Disease Control and Prevention, "Key Statistics from the National Survey of Family Growth," accessed Feb. 6, 2012

PolitiFact's mission (bold emphasis added):

PolitiFact relies on on-the-record interviews and publishes a list of sources with every Truth-O-Meter item. When possible, the list includes links to sources that are freely available, although some sources rely on paid subscriptions. The goal is to help readers judge for themselves whether they agree with the ruling.

Um, yeah, whatever.

So did PolitiFact fact check the item without checking the facts or simply forget to link the relevant data in the source list? 

Don't look for a confession in a CYA:

To double-check, we reviewed the criticism, talked with the study’s lead researcher, and reviewed the report and an update from the institute. We’re confident in our original analysis.

We can take that statement for what it's worth, given that the original analysis never produced a baseline for determining the error of the 98 percent figure.  We're left to guess whether the CYA intends to assure us that the original item includes data sufficient to help readers judge for themselves whether to agree with the ruling.

PolitiFact is suggesting that the fact check was perfectly fine, and those of you who used their references to try to reach your own conclusions mishandled the facts.

PolitiFact:

The spate of blog posts and stories this week — some directly claiming to debunk our reporting — unfortunately rely on a flawed reading of a Guttmacher Institute study.

They were easy mistakes to make, confusing the group of women who have "ever used" contraceptives with those who are "currently using" contraceptives — and misapplying footnote information about those "currently using" to the 98 percent statistic.

The "flawed reading" results directly from the fact that neither the Guttmacher Institute nor PolitiFact provided access to the data that might have supported the key claim.  I'll quote from the PFB assessment:  "That's fact checking?"

If PolitiFact had checked the claim properly in the first place then PolitiFact could have answered the criticisms without the wholesale review.  In fact, the criticisms would be clearly wrong based on material included in or linked from the original fact check.

More from PolitiFact:

The critics of our reporting — bloggers for the Weekly Standard, CatholicVote.org and GetReligion.org — were relying on an analysis from Lydia McGrew in her blog, "What's Wrong With The World," which was also cited by the Washington Post's WonkBlog.

PFB highlighted McGrew's analysis, certainly.  But our criticisms expanded beyond McGrew's and recognized that the Guttmacher Institute report may have included data that PolitiFact neglected to explain to its readers.  One would think from PolitiFact's response above that no criticism of its reporting on this issue contains merit.

Focus on McGrew

The rest of the CYA focuses on McGrew's criticism.  After mentioning McGrew's education and quoting from her self-description in a manner apparently intended to imply bias--an approach notably missing from PolitiFact's treatment of the Guttmacher Institute--PolitiFact gets to the criticism:

(McGrew) read the Guttmacher study that’s the source of the "98 percent of Catholic women" statistic.

The study doesn’t provide much explanation of the statistic, including it in the text on Page 4, but not displaying the related data in any charts. It says: "Among all women who have had sex, 99 percent have ever used a contraceptive method other than natural family planning. This figure is virtually the same, 98 percent, among sexually experienced Catholic women."

That summary pretty much accounts for the explanation of the 98 percent statistic.  And PolitiFact's original analysis added nothing to the explanation, resulting in a journalistic hole in the PolitiFact analysis.  PolitiFact stands behind that analysis, of course.

PolitiFact:

McGrew and others found far more exclusions in the Guttmacher study — but they weren’t the right ones. They focused on a chart in Figure 3 on Page 6 that was about a different group of women. Instead of being about "women who have had sex," it was about "sexually active women who are not pregnant, post-partum or trying to get pregnant."

They noted that in Figure 3, 11 percent of those sexually active women were using "no method" of contraception, arguing that also contradicted Munoz's claim and our rating.

Except that’s not the group the 98 percent statistic was based on.

No, it wasn't the group the 98 percent figure was based on--it was just the only group for which either the Guttmacher Institute or PolitiFact provided supporting evidence.  Again:  That's fact checking?

The Guttmacher Institute jumped into the discussion Wednesday with a clarification that indicated that, indeed, its study showed that 98 percent of Catholic women ages 15 to 44 who have had sex have used contraceptives:

More accurately, its report based on a government study claimed that 98 percent of Catholic women ages 15 to 44 who have had sex have used contraceptives despite not including any evidence in support of the claim.

Continued:

"Guttmacher’s analysis of data from the federal government’s National Survey of Family Growth found that the vast majority of American women of reproductive age (15–44) — including 99% of all sexually experienced women and 98% of those who identify themselves as Catholic — have used a method of contraception other than natural family planning at some point. Women may be classified as sexually experienced regardless of whether they are currently sexually active, using contraceptives, pregnant, trying to get pregnant or postpartum.


"By their early 20s, some 79% of never-married women — and 89% of never-married Catholic women — have had sex. (Presumably, all married women have done so.) In short, most American women (including Catholics) have had sex by their early 20s, and virtually all of them have used contraceptives other than natural family planning.


"The above statistics on women who have ever used contraceptives are not to be confused with data on women who are currently using contraceptives."

There's really no need to verify information from the Guttmacher Institute, is there?  One might as well stick a few paragraphs from Guttmacher into a fact check and present the information as authoritative.  And why bore their readers with an updated reference list including the government data on which the conclusions rely?

Aside from all of that, the 98 percent figure still excludes over 10 percent of women who have never had sex.  So the claim that received the "Mostly True" rating is off by no less than 10 percentage points and PolitiFact has never put a number on the error.  That's fact checking?

We’ve been in touch with McGrew and a few others since the Guttmacher update. The Christian Post published a clarification. McGrew said she planned to write a new post.

And if PolitiFact makes it seem like McGrew threw in the towel, think again.

If PolitiFact truly intends to inform its readers to the point where they can intelligently decide the issue for themselves then this ongoing item rates as a colossal failure.  This CYA attempt reads like a morning-after pill designed to do away with unwanted criticism.