What's Wrong With the World: "How to Lie with Statistics, Example Umpteen"

Jeff and I hugely appreciate bloggers who delve into the more complicated PolitiFact-related issues.

Lydia McGrew of the "What's Wrong With the World" blog gives a proper dressing-down to the Obama administration, the Guttmacher Institute and our beloved PolitiFact over the supposedly "Mostly True" claim that 98 percent of Catholic women use birth control.

As is our wont, we'll focus primarily on PolitiFact's role in the mess.

McGrew:

(T)his Politifact evaluation of the meme gets it wrong again and again, and in both directions.

First, the Politifact discussion insists that the claim is only about women in this category who have ever used contraception. When I first heard that and hadn't looked at the study, I immediately thought of the fact that such a statistic would presumably include women who were not at the time of the study using contraception and had used it only once in the past. It was even pointed out to me that it would include adult converts whose use might easily have been prior to their becoming Catholic. However, that isn't correct, anyway. The study expressly was of current contraceptive use. That's, in a sense, "better" for the side that wants the numbers to be high.

McGrew pointed out earlier that the Guttmacher Institute study uses data for "women at risk for
unintended pregnancy, whom we define as those who had had sex in the three months prior to the survey and were not pregnant, postpartum or trying to get pregnant."  The women surveyed were additionally in the 15-44 age range.  Yet PolitiFact describes the findings like so:

We read the study, which was based on long-collected, frequently cited government survey data. It says essentially that — though the statistic refers specifically to women who have had sex, a distinction Muñoz didn’t make.

But that’s not a large clarification, since most women in the study, including 70 percent of unmarried Catholic women, were sexually experienced.

That's fact checking?

McGrew:

(O)n this point, too, the Politifact evaluation is completely wrong. Politifact implies that only the supplementary table on p. 8 excluded these groups and that Figure 3 on p. 6 included them! But this is wrong. The table on p. 8 is simply supplementary to Figure 3, and both are taken from the same survey using the same restrictions! This is made explicit again and again in the study.

McGrew's exactly right.  The same information accompanies the asterisk for each table (bold emphasis added):  "*Refers to sexually active women who are not pregnant, postpartum or trying to get pregnant."

It doesn't occur to PolitiFact that restricting the survey population like that throws a serious spanner in the works.

That kind of credulity goes by a different name:  gullibility.

Visit What's Wrong With the World and read all of McGrew's skillful fisking of the liberal trio.  It's well worth it.


Addendum:

The Guttmacher Institute drew its data ultimately from here.

It may be the case that the Guttmacher study is reliable.  Regardless of that, PolitiFact did virtually nothing to clarify the issue.  A recent Washington Post story does shed some light on things, however:

I called up Rachel Jones, the lead author of this study, to have her walk me through the research. She agrees that her study results do not speak to all Catholic women. Rather, they speak to a specific demographic: women between 15- and 44-years-old who have ever been sexually active.

Jeff Adds (2/15/2012): Over on PolitiFact's Facebook page, frequent PF critic Matthew Hoy offered up his usual spot on commentary:

I find [PolitiFact's] failure to note that the Alan Guttmacher Institute is closely allied with Planned Parenthood a troubling omission. It isn't some neutral observer and its studies shouldn't be taken at face value without some healthy skepticism.

This isn't the first time PolitiFact has ignored Guttmacher's relationship with Planned Parenthood. Regardless of the studies accuracy, the alliance deserves at least a cursory disclosure. It's also important to note that PolitiFact used a similar connection to justify the rating of Florida Governor Rick Scott's claim about high-speed rail projects:

Scott bases his claims on hypothetical cost overruns from a suspect study written by a libertarian think tank...We rate Scott's claim False.

We highlighted that rating here.



Correction 2/17/2012:  "Guttmacher" was misspelled in the next-to-last paragraph.

Don Surber: "Obama proved it was government-run healthcare"

Don Surber notices the same connection between the actions of the Obama administration and PolitiFact's 2010 "Lie of the Year" that I pointed out last week.  We highlight it here at PolitiFact Bias because Surber makes the criticism of PolitiFact so effectively and directly:

On December 17, 2010, as part of its continuous support and defense of Obamacare, PolitiFact boldly declared as its Lie Of The Year “government takeover of health care.”

This week, President Obama’s own actions proved that PolitiFact’s editors were in error. By requiring that everyone’s health insurance (which will soon be mandatory) carry free birth control for women — no co-payments or no deductibles — not only does President Obama violate the 1st and 14th Amendments to the Constitution (religion and equal protection under the law) but President Obama lays bare the lie that this is not government-run health care.

Surber's observation serves as yet another indicator that liberals critical of the 2011 "Lie of the Year" selection were simply late to the party.  Not that a thorough progressivist indoctrination can't detect a full-on conservative bias at PolitiFact.

If PolitiFact has any chance of redemption on this issue, it comes from the doubt as to whether the executive branch has the authority it claims from the health care bill to require insurance companies to provide a service free of charge.

If no such authority exists, of course, it makes President Obama's position to the contrary a falsehood.

National Review: "Mark Hemingway on 'Fact-Checking'"

National Review's Reihan Salam amplifies (December 2011) the criticism of PolitiFact by Mark Hemingway and published in the Weekly Standard from the same month:

I’ve often noticed that the “fact-checkers” in question are often obtuse, misleading, or both, and Hemingway makes the case in systematic fashion.

Salam quotes Hemingway's article extensively and adds his hearty agreement to Hemingway's criticism, adding his own swipe at PolitiFact's lack of transparency with respect to ideological bias:

To the extent that the mission of PolitiFact is to offer richer context for the statements made by leading public officials, I’m all for it. That is part of what we try to do in this space. Yet there are at least two important distinctions: (a) our ideological perspective is clear; (b) we make an effort to make reference to and to provide links to contrary views, though perhaps not as much as we might if we presented ourselves as neutral observers on the political scene.

PolitiFact's use of the "non-partisan" label is a fig leaf.

Columbia Journalism Review: "What the Fact-Checkers Get Wrong"

If, in "What the Fact-Checkers Get Wrong" the Columbia Journalism Review only mentions the tip of the iceberg, we can at least take solace in the fact that this PolitiFact-related story is a step up from at least one previous effort.

Sure, the new item hints at liberal bias in that the piece appears to take for granted that only one of PolitiFact's last three "Lie of the Year" awards went to a statement that was defensibly true.

Regardless of that, it contains a solid criticism:

In fact, the sights of the broader fact-checking movement often seem to be set on something different than strict truth and falsehood. And by acknowledging that, the fact-checkers might grapple with some important questions about the project in which they’re engaged—and might see more clearly the box in which they’ve trapped themselves.

To get at those questions, it’s helpful to think about why “fact-checking” has emerged now. I’d argue that it’s a response to many journalists’ perception that they are ever more outgunned by the increasing volume and sophistication of professional political communication. The fact-check is a tool with which reporters can rescue themselves from oblivion. And the morally freighted language invoked by full-time fact-checkers—true and false, fact and lie—is a weapon, to be wielded by journalists with authority against other, presumably less trustworthy types who make political claims.

The CJR critique hits close (and the story acknowledges this explicitly) to one of the main points Mark Hemingway made in his controversial piece in the Weekly Standard in December:  Fact checkers are exalting their role in the epistemological chain, staking out for themselves a strategically valuable territory in the realm of public discourse.

Fact checking is, in a real sense, a power grab.

Don Surber: "PolitiFact fisked"

We're way overdue recognizing Don Surber's ongoing commentary regarding PolitiFact, but his "PolitiFact Fisked" is as good a start as any:

There is an old saying in the newspaper trade that if you are taking it from both sides, you must be doing something right. The reality is that you definitely are doing something wrong and in the case of PolitiFact, editor Bill Adair and company are doing plenty of things wrong.

Surber's always a great read, especially when he chooses PolitiFact as his subject.  The above is just a teaser for an excellent fisking of PolitiFact.  Read it all.

And could it be a coincidence that PolitiFact uses the fact of criticism from both sides to implicitly claim it's doing something right?

The Weekly Standard: "Pants on (three-alarm) Fire"

The Weekly Standard has some subscriber-only content criticizing PolitiFact Oregon.

Fortunately there's a preview:

PolitiFact Oregon—which works in partnership with the state’s most influential media outlet, the -Oregonian—has been trying and failing to play referee in the race by evaluating the candidates’ statements. First, PolitiFact gave Cornilles its “Pants on Fire” rating for an ad claiming that Bonamici, a state legislator, voted to raise taxes 60 times. Now depending on how broadly you define “tax,” the claim is defensible. But if you want to split hairs—and boy, does PolitiFact ever like to do that—then you would say not that Bonamici has raised taxes 60 times, but that she has raised taxes and fees 60 times.

Maybe PolitiFact Oregon has a Parse-O-Tron instead of a Truth-O-Meter.

There's more to preview at the link, but only subscribers get the whole story.

Relevant: "Measuring the Slant"

Power Line blog directs us to free online content from the Claremont Review of Books. 

CRB reviewer James Q. Wilson reviews Tim Groseclose's "Left Turn:  How Liberal Media Bias Distorts the American Mind" and its place in media bias studies.

We're following Power Line's lead by not including any excerpted material.  Go.  Read.

Hoystory: "Obama’s War on Religion and Conscience"

Matthew Hoy is back at it with his usual biting commentary on PolitiFact. This time he shares his thoughts on the current debate about the effect of PPACA mandates on institutions of the Roman Catholic Church.

Hoy deals broadly with the controversy, but we'll highlight his mention of PolitiFact. At issue is PolitiFact's treatment of Newt Gingrich's statement that the PPACA requires religious institutions to provide insurance coverage for contraceptives:

After honestly analyzing the rule and the law, Politifraud labels Gingrich’s charge “mostly false” as they engage in an amount of hand-waving that would enable human flight without the aid of wings, engines or the other commonly required tools.

Still, if you consider a Catholic church to be a "Catholic institution," or a synagogue to be a "Jewish institution," Gingrich isn’t correct that the recent federal rule on contraceptives applies. Those nonprofit religious employers could choose whether or not they covered contraceptive services.

It’s pretty clear that Gingrich chose his words carefully here and Politifraud is muddying the waters. When I hear the words “Catholic institution” I think of everything Catholic that isn’t the church. I think of hospitals, soup kitchens, homeless shelters, adoption services, the Knights of Columbus, etc. Maybe it’s just because I’m likely more familiar with religious terminology than the (snark on) godless heathens (snark off) who populate many newsrooms, that I interpret it this way. But if the difference between a “True” or “Mostly True” ruling and a “Mostly False” ruling is over whether the word “institution” includes the church or not, then there’s way too much parsing going on.

Parsing words is nothing new for PolitiFact. But that's not the biggest flub Hoy spots:

In the video Politifact links to of Gingrich’s statement (provided by none other than Think Progress), Gingrich makes it clear that he is talking about the rule issued “last week.” The rule issued last week was the one regarding religious employers covering contraceptives in their health plans. Politifraud dishonestly expands that specific criticism of that specific rule into states can set their own benchmarks. No, they can’t. Not when it comes to the rule that came down “last week.” That rule says they MUST cover contraceptives.

Once again Hoy is spot on, though as usual our brief review doesn't do his work justice. Head over to Hoystory and read the whole thing.