Friday, February 12, 2016

Dustin Siggins: "Fact-checking the fact-checkers: Did an abortion defunding bill try ‘to redefine rape’?"

Dustin Siggins continues to establish himself as one of the finest critics of the disastrously inept fact-checking outfit "PolitiFact."

Siggins' latest takedown of PolitiFact's hapless liberal bloggers looks at PolitiFact's fact check of the liberal interest group Emily's List, which claimed a Republican legislator had tried to redefine rape.

Newfangled PolitiFact Colorado looked at whether Emily's List was right that another Colorado Republican,  Mike Coffman, voted to redefine rape and found the Emily's List claim "Mostly True":
Emily’s List said that Coffman "co-sponsored a bill to redefine rape." The record shows Coffman did co-sponsor the bill to redefine a ban on federal funding for abortions to exempt "forcible rape." Yet he later voted on the floor for an amended version that had removed the "forcible" modifier from the bill.
How did it come to pass that it's "Mostly True" that redefining a ban on federal funding for abortions redefines rape?

Siggins, writing for LifeSiteNews, placed some focus on that inconsistency:
[PolitiFact Colorado] wrote, "Critics said the 'forcible rape' language could rule out other forms of sexual assault that are considered rape, including statutory rape, attacks where women are drugged or threatened, and date rapes."

Likewise, he quoted Coffman's Democratic opponent, Morgan Carroll, who said, "Rape is about the lack of consent – not the degree of force – and this definition takes us backwards."

However, which "definition" of rape are we talking about? Generally speaking, there are at least two broad definitions of rape – sexual intercourse between two people when one party is unwilling and sexual intercourse between a legal adult and a legal minor (which can be consensual).
Siggins went on to express doubt that having sex with an incapacitated person fails to count as a forcible rape.

We found support for Siggins' skepticism at
Forcible Rape—Rape by Force

Definition: The carnal knowledge of a female forcibly and against her will.


"Against her will" includes instances in which the victim is incapable of giving consent because of her temporary or permanent mental or physical incapacity (or because of her youth). The ability of the victim to give consent must be a professional determination by the law enforcement agency. The age of the victim, of course, plays a critical role in this determination. Individuals do not mature mentally at the same rate. For example, no 4-year-old is capable of consenting, whereas victims aged 10 or 12 may need to be assessed within the specific circumstances regarding the giving of their consent.
Put simply, the legal definition of "forcible rape" in effect during 2011, when Coffman's bill was debated, included lack of consent.

PolitiFact Colorado cited unnamed "critics" who argued otherwise:
Critics said the "forcible rape" language could rule out other forms of sexual assault that are considered rape, including statutory rape, attacks where women are drugged or threatened, and date rapes.
Does it matter to PolitiFact Colorado whether the "critics" were right? Is it okay to simply assume the anonymous critics raised a plausible and likely problem with the proposed law?
  • PolitiFact Colorado neglects to define "forcible rape"
  • PolitiFact Colorado offers no evidence it looked into the definition of "forcible rape"
  • PolitiFact Colorado cited anonymous "critics" as a key part of its analysis
We think at minimum a neutral fact-checker would take a more detailed look than PolitiFact Colorado did at the definition of "forcible rape."


We contacted the author and the editor of the PolitiFact Colorado fact check about problems we noted above. We will update this article if we receive any reply.

Correction Feb. 12, 2016: Corrected one instance in which the word "abortion" was mistakenly substituted for "rape".

Friday, February 5, 2016

Two Clinton fibs, two "Half True" ratings from PolitiFact

PolitiFact is too good to us.

Seriously. We don't have any formal agreement with PolitiFact binding them to produce horrible fact-checking in order to make sure we have stuff to write. They just do it anyway.

Case in point, co-editor Jeff D sent me an email about this item earlier today.

During the Democratic presidential debate on Feb. 4, 2016, Democratic presidential candidate Hillary Rodham Clinton said she waited until President Obama's Trans-Pacific Partnership agreement was finalized before passing judgment on it.

Try to wrap your head around PolitiFact's reasoning:
Did Clinton really withhold her support until the terms of the proposal had been finalized?


Speaking in Australia in 2012, Clinton hailed the deal as "setting the gold standard."

"This TPP sets the gold standard in trade agreements to open free, transparent, fair trade, the kind of environment that has the rule of law and a level playing field," Clinton said. "And when negotiated, this agreement will cover 40 percent of the world's total trade and build in strong protections for workers and the environment."
First, Clinton condemned the deal after it was finalized. It doesn't even make sense to ask whether Clinton withheld her support for the deal until after it was finalized. She never supported it after it was finalized. PolitiFact's headline makes the same error:

The statement supposedly from Clinton in PolitiFact's headline is flatly false. Clinton endorsed the deal before negotiations were finalized. The "Half True" doesn't belong within 10 miles of PolitiFact's headline.

If Clinton said she did not pass judgment on the deal before it was finalized that is likewise false.

The only way Clinton can escape with a shred of truth on this one is if she was saying she did not condemn the deal until after it was finalized. The problem? That statement does nothing to address the concern in the question posed by debate moderator Chuck Todd (transcript via The New York Times, bold emphasis added):
TODD: Secretary Clinton, let me turn to the issue of trade. In the ’90s you supported NAFTA. But you opposed it when you ran for the president in 2008. As secretary of state, you supported TPP, and then — which, of course, is that trade agreement with a lot of Asian countries, but you now oppose it as you make your second bid for president.

If elected, should Democrats expect that once you’re in office you will then become supportive of these trade agreements again?

CLINTON: You know, Chuck, I’ve only had responsibility for voting for trade agreements as a senator. And I voted a multinational trade agreement when I was senator, the CAFTA agreement, because I did not believe it was in the best interests of the workers of America, of our incomes, and I opposed it.

I did hope that the TPP, negotiated by this administration, would put to rest a lot of the concerns that many people have expressed about trade agreements. And I said that I was holding out that hope that it would be the kind of trade agreement that I was looking for.

I waited until it had actually been negotiated because I did want to give the benefit of the doubt to the administration. Once I saw what the outcome was, I opposed it.
It looks to us (see particularly the second paragraph of Clinton's response) like Clinton tried to downplay the support she offered the deal when she was Secretary of State. She seems to say that she didn't really support the deal back then. Apparently she was just being being a good soldier for President Obama, leaving her free to oppose the deal after she left the administration.

So ... no flip-flop because she was just doing Obama's bidding? And endorsing the deal before it was finalized is simply giving the administration--of which she was a part!--the benefit of the doubt?

Clinton's answer doesn't make much sense to us. She offers a thin excuse for her flip-flop.

PolitiFact's fact check doesn't make sense, either. The claim from PolitiFact's headline is false but receives a "Half True" rating. And it doesn't correctly capture what Clinton was saying in the first place.

What Clinton actually said might have been half true. She did not condemn the deal until after it was finalized. Though that claim carries a healthy dollop of misdirection downplaying her apparently insincere early endorsement of the TPP.

And did Clinton condemn the deal privately within the administration? Did President Obama hear from Clinton what it would take for her to support the deal? Ask the question, debate moderators.

The Second Fib

 Wait, didn't we say something about a second fib rated "Half True"?

Yes. Yes, we did.

The fact check we discuss above--containing the first fib--was from C. Eugene Emery, Jr., recently added to the staff at PolitiFact National after leading fact-check efforts for PolitiFact's Rhode Island franchise.

It looks like Emery relied on an earlier PolitiFact fact check for his analysis. That article contains the second fib, and probably helped nudge Emery toward his interpretation of Clinton's 2016 debate comments.

In that Oct. 13, 2015 fact check, it looks like Clinton did say she had reserved judgment on the TPP while Secretary of State. But with the contradictory evidence available and included in its story, PolitiFact gifted Clinton with a "Half True" rating on her claim that while serving as Secretary of State she merely "hoped" the TPP was the type of deal she could support.

 Try to figure out which half was true from PolitiFact's conclusion:
Clinton said when she was secretary of state, she was reserving judgment but "hoped (the Trans-Pacific Partnership) would be the gold standard."

She’s twisting her 2012 remarks a bit. Clinton said, "This TPP sets the gold standard in trade agreements," which is a more confident claim than if she had said she "hoped" it would meet that standard. This is in contrast to more recent comments where Clinton said she had concerns about the deal and that she ultimately opposes it.

The statement is distorting her previous comments. We rate it Half True.
Hooray for objective standards in fact-checking?

Michael F. Cannon: "Clean Up Your Act, PolitiFact"

Michael F. Cannon, the heath care expert who started a boycott of PolitiFact years ago over its penchant for calling people liars (without proving it), has an article at Forbes showing how PolitiFact flubbed a Jan. 29, 2016 rating of Republican presidential candidate Ted Cruz. Cruz called Obamacare the nation's No. 1 job killer and received a "Pants on Fire" rating from PolitiFact.

We can't do better than Cannon in summarizing his article:
In their rush to label Ted Cruz a liar, PolitiFact ignored economic theory, ignored economic consensus, ignored problems with the evidence they had amassed, ignored that some of the evidence they collected supports Cruz, ignored reams of anecdotal evidence, and dismissed Congressional Budget Office projections based on nothing more than a subjective and arbitrary distinction PolitiFact themselves invented.
But the summary's even better accompanied by the details, so please visit Forbes to read the whole thing.

Especially damning is PolitiFact's omission of Congressional Budget Office projections of labor loss as a result of the Affordable Care Act. PolitiFact's fact-check of Cruz does not mention the CBO but links to earlier PolitiFact fact checks dealing with the CBO's projections. Cannon succinctly explains the problem with PolitiFact's spin on the CBO's reports.

Our post just before this one talks about examining whether PolitiFact's corrections benefit Democrats more than Republicans. The other side of that coin whether PolitiFact acts more readily in response to strong criticisms from the left or from the right.

Here's betting PolitiFact will run no correction as a result of Cannon's criticisms. Indeed, we deem it very unlikely that PolitiFact will even publicly note Cannon's criticism.

That's just how PolitiFact rolls.


We can't resist attaching this howler from PolitiFact's fact check of Cruz.

Did you know the Affordable Care Act is actually a job creator? Read (bold emphasis added):
All of the job numbers have moved in a positive direction since April 2010, the first full month after President Barack Obama signed the bill into law. The unemployment rate has moved steadily from 9.9 percent to 5 percent. The economy has added about 10.7 million jobs. And the number of working people who have part-time work but would rather have full-time work has fallen by nearly 3 million. There was a brief rise in that number between April and September 2010, but the longer term trend is clearly one of decline.

That’s not to say the Affordable Care Act deserves all the credit. The fact is many factors drive the labor market and the overall recovery from the Great Recession is the dominant player in this regard.
Yes, ladies and gentlemen, that's PolitiFact arguing that correlation is evidence of causation despite the involvement of multiple causes. PolitiFact says that's a sin when conservative politicians do it.

 It's still sinful but not as much so when liberal politicians do it (research project is underway).

Thursday, February 4, 2016

As The "Wheel O' Meter" Turns

In the dog-bites-man department, PolitiFact continues to do fact-checking the wrong way this week.

It started with a fact check of Republican presidential candidate Ted Cruz. Cruz said blacks gained $5,000 in median income under Reagan. PolitiFact's crack fact-checking team, veteran writer Louis Jacobson and founding editor Bill Adair's heir, Angie Drobnic Holan performed their fact-checking magic and found Cruz's claim "Mostly True."

But a day passed, and PolitiFact unpublished the article and replaced it with a rewritten version giving Cruz a "Mostly False" rating. Why? Somebody complained:
CORRECTION, Feb. 4, 2016: After we published this fact-check, a reader wrote us to say that in our original article, we had used the wrong Census Bureau data table to analyze Cruz’s statement and rate it Mostly True. However, using the proper data table, Cruz is actually incorrect, so we have changed the rating to Mostly False. 
This case gives us a smorgasbord of items to criticize.
  • PolitiFact's A team committed a substantial flub
  • The archived "original version" hides Holan's role in editing the flawed version
  • The archived "original version" wipes (with a cloth?) the hotlinks leading to the wrong table 
  • Neither the new nor the old version bothered to look at how Cruz fared using nominal dollars
Having noted the main problems with PolitiFact's treatment of this issue, we'll address two questions. First, what specific mistake did PolitiFact make before publishing its less-than-totally-transparent correction? Second, was Cruz correct using nominal dollars?

The Wrong Chart

We found the PolitiFact's key figures from its original article on the same chart it used in its revised article. The chart gave figures for the United States as a whole followed by corresponding figures for different regions of the country. It appears PolitiFact used figures for the West region to test Cruz's claim.

Nominal Dollars

Using nominal dollars, Cruz's claim was correct. That's typically enough to eke a "Half True" from the PolitiFact gang's "Truth-O-Meter."

We reproduced the relevant part of the chart, putting the figures PolitiFact used in its new fact check in black boxes. The adjacent figures for nominal dollars ("Current dollars") we placed within red boxes. The difference over Reagan's two terms in office was $5,643.

We're not saying there's nothing wrong with giving increases in median income using nominal dollars. It's potentially every bit as misleading as when Democrats use the raw gender wage gap to emphasize the need for legislation punishing pay differences based solely on gender discrimination. What we are saying is Cruz made a true statement given a normal interpretation of his words and PolitiFact failed to acknowledge it either in the text of the story or in its (new) rating of Cruz.

PolitiFact tends to rate that bit of liberal misdirection no lower than "Half True," by the way.

What will it take for PolitiFact to regard its "Truth-O-Meter" rating system as a mistake?


Does PolitiFact's faulty first rating of Cruz show PolitiFact has a pro-conservative bias?

We don't think that's the best explanation. The more parsimonious explanation is that PolitiFact simply made a mistake. So this example shows PolitiFact's fallibility. This was the type of mistake bloggers make.

This example does point toward another potential evidence of bias at PolitiFact. Which political party receives the most benefit when PolitiFact runs a correction?

Tuesday, February 2, 2016

Left Jab: PolitiFact National vs. PF New Hampshire on per capita health care spending

I was hunting for some liberal criticism of PolitiFact associated with Democratic presidential candidate Bernie Sanders, one associated with polling numbers. I found a pretty good item aside from the one I sought.

Reddit commenter "wittenbunk" offered the following observation:
Yesterday Politifact published a rating of Bernie's often repeated claim that "We spend almost twice as much per capita on health care as do the people of any other country". They rated the claim false.

The issue is that on April 30th Politifact rated a nearly identical claim by Ben Carson Carson was quoted as "We spent twice as much per capita for health care in this country as the next closest nation". Politifact rated the claim Mostly False.

Despite the fact that Carson's wording allowed for much less interpretation, Politifact gave his quote a more truthful rating.
Wittenbunk went on to say that his example qualifies as a rare clear example of media bias. The post was solid up through that point. PolitiFact clearly used inconsistent standards in achieving the two different ratings for Sanders and Carson. But single cases of inconsistency make poor examples of media bias.

That's why we've always said the appropriate way to look for media bias at PolitiFact is to look for trends in unfair harm. Sen. Sanders was hit with unfair harm in this case. I've documented a separate case of unfair harm to Sanders at Zebra Fact Check. There may well be others.

This case does feature some special circumstances. It's a rating from a state operation, PolitiFact New Hampshire, conflicting with the rating from PolitiFact National. PolitiFact National published the rating that's probably harder to justify. We'll note that the story was written by intern Will Cabaniss, but since PolitiFact's "star chamber" of editors decides the rating we're not inclined to blame Cabaniss.

Note to liberal critics of PolitiFact: Open your eyes. This type of inconsistency is normal at PolitiFact.

Saturday, January 30, 2016

PolitiFact's second-most popular response to criticism

We all know PolitiFact's most popular response to criticism: Ignore it.

Today we reveal PolitiFact's second-most popular response to criticism, via example. The example comes from PolitiFact's Mailbag feature, Jan. 29, 2016:
One reader criticized our story, "The presidential scorecards so far," which listed the summaries of our fact-check ratings for the 2016 presidential field, candidate by candidate.

Such a comparison, the reader wrote, "is absolutely meaningless, because the statements selected for Truth-O-Meter ratings for each candidate were neither scientifically nor randomly selected. In previous responses to my emails, Truth-O-Meter personnel have stated that the criteria used for selection of statements is documented and thus is sufficient for journalistic integrity. I have no problem with what statements you select for fact-checking, or the process used in the selection. However, trying to lump all such statements together into a scorecard, and then comparing scorecards for different people, when different statements were rated, is scientifically meaningless, and, in my opinion, advocacy journalism rather than fact-based journalism."

And there you have it.

PolitiFact's second-most popular response to criticism involves acknowledging the existence of the criticism and offering no response to it.

Friday, January 29, 2016

Response to G, of the G'nO

Our post on the worst of PolitiFact's Reddit AMA drew a lengthy response from "G, of the G'nO." Interested readers can find that comment through the link in uninterrupted form. Here, we'll respond point by point in conversational fashion, breaking G's comment up into separate sections as we offer comment.
Wow. I was actually excited to see this site but, it looks so far like I might be the only one who has -- but, I'll step up and comment on this anyway. Overall, I think the questions you posed in this post are valid, on track, and worthwhile asking. Unfortunately, I also feel your interpretation of the answers to the questions posed completely nullifies them, takes them way off base, and makes it all a virtual waste of time.
We can't wait to read the specifics.
Your first two-part question:
Part 1: I would have to say that I believe (but, of course cannot state with certainty) that Sharockman most likely knows or, at least, has a large degree of insight into the ideological make up of his staff. My gut agrees with you in saying that he dodged that question.
Okay, on this one it sounds like we agree. Just to be clear, however, the question was not originally posed by us. It is "ours" in the sense that we used it as a heading in our post.
Part 2: From this point you state "it gets worse" but what actually happens is you just get a bit silly. Sharockman gave a reasonable answer -- you just didn't like it. That said, your idea that they could have "more balance" could certainly hold water. But then it just begs the question as to who wants to expend the energy schlepping that bucket of water around. After all, this portion of the two part question primarily seems to address the possible bias around the rating system itself. If I make the claim "that purple ball is blue" we would be able to read it a multitude of ways: false, mostly false, half true, mostly true. But, regardless of its ultimate rating, the fact still remains that the ball is purple and everyone except the colorblind should be able to get the point.
While we did not write anything about not liking Sharockman's answer, we did offer a substantive criticism of his answer that you do not appear to acknowledge. If Sharockman doesn't know the ideologies of his staff, he has no reason to have any sort of confidence that three or four people in PolitiFact's "star chamber" will produce judgments based on ideological balance. Without that assurance, Sharockman's reassuring words are empty.
Your follow-up question about the predominance of conservatives being fact checked -- I agree: great question.
Again, it wasn't our question. And we did not offer the opinion that it was a great question. We thought Sharockman could have offered a good answer but did not.
But, here again, Sharockman actually provided a good answer and, while it seems like you might be on the scent of making a good point, you never really make it -- and your response is fatally flawed from multiple angles:
** First, your statement "The only way the number of fact checks of Obama can carry is relevance is if that number is greater than the number of ratings of conservatives." That's just an absolutely silly assertion. Why would Obama (a single, individual liberal) need a greater number of ratings than all conservatives combined to carry relevance? In the vein of your flimflammery verbiage, I say hogwash!
Quite simply, touting the large number of ratings given to a special case (the only U.S. president PolitiFact has ever really bothered to rate, as well as a two-time presidential candidate) does nothing to address the imbalance charged by the reader. It's a meaningless metric in answer to the question, and any fact checker should know better. It's like saying more Raiders than Texans make the Pro Bowl and somebody objecting by saying "But the Texans' Joe Smith has been named to the Pro Bowl three years straight." It doesn't address the issue.

Combining the Obama rating with the ratings of other Democrats could mean something. But the Obama ratings by themselves mean nothing. It's a nutty case of cherry-picking.
** From there you say you found 68 ratings for Clinton since 2010. OK, good. You then make the point that a Clinton statement in 2008 is not relevant to 2016. Agreed -- but, the statement from 2008 lands outside of the range of those 68 ratings you found and doesn't really seem to apply to anything. However, since you made the point, that point can actually be used to poke holes in your next comment -- you found 86 ratings for Trump since 2011. So what? Sharockman's claim that, [of the 2016 candidates, Clinton was fact checked the most] may have actually been made within the context of relevance to the 2016 campaign -- which would pretty much make everything before 2015 moot. How many times has Clinton been rated since the beginning of the 2016 campaign? I don't know. I haven't checked. Like I said, you may be on the scent of making a good point -- but you have yet to make it.
Don't neglect our point, which is that Sharockman isn't making any sort of reasonable point. If you look at the ratings, both Trump and Clinton are arguably putting themselves forward (as candidates--ed.) at the beginning of the timeline we identified. We didn't go into detail because we're not fact-checking Sharockman. We're just poking holes in his argument. Compare Trump to Clinton however you wish as a 2016 candidate. Chances are exceedingly high you'll need to cheat to put Clinton in the lead on the number of ratings.

You don't appear to have made any sort of argument that rescues Sharockman from the charges we've made against him.
As to the difference between "false" and "pants on fire" -- I actually like this question but, really, who cares?
We do. And so should you. One of the main features of our site is a research section. The most developed research project we've published so far looks at PolitiFact's bias in applying its "Pants on Fire" ratings. The key premise of that research is the lack of any objective means of distinguishing between "False" ratings and "Pants on Fire" ratings. We're always amused when figures from PolitiFact address the issue in a way that supports the premise of our research.
There are obviously five general ratings which are all fairly defined with the criteria for each. False is false -- and, you know what -- pants on fire is also false. It just has a little flair attached for style and enjoyment of the readers -- it's childlike and funny. It points out things of a ridiculous nature -- like the post to which I write this reply. Is that subjective? Sure, I guess it is…
The other ratings are better defined than the giant blur that divides "False" from "Pants on Fire." But there's no good evidence PolitiFact pays particular attention to the definitions it gives for its ratings. A literally true statement can receive any rating. We pick on the "Pants on Fire" rating because the definition offers no real guidance at all in applying any objective criterion. And you appear to at least lean toward our view that the rating is essentially a subjective measure.

So, let's see what we've got:

  • You more-or-less agreed with us twice.
  • You said Sharockman's reasoning was good but didn't support your statement.
  • You said our view of the importance of the Obama ratings was hogwash but didn't say why.
  • You think maybe Clinton was rated more times than Trump (do maybes support Sharockman?).
  • You don't know why the line between "False" and "Pants on Fire" is important. Hopefully we cleared that up for you.

You really have to marvel over how much better we are than PolitiFact at responding to critics.