Saturday, May 31, 2014

PolitiFact Bias is biased

We get stuff like this all the time:
I read a bit of that site, and it happens to have its own biases.
Well, right.  The authors, Jeff D. and Bryan White, both admit to a conservative bias.

So what?

Again, we freely admit to our personal bias on our About page.  We're honest about our political biases.  That's something you won't get from PolitiFact.  PolitiFact's founding editor (now at Duke University and serving PolitiFact in an advisory role) Bill Adair could have a life-sized cutout of President Obama in his office and they'd never willingly tell you about it because it might make them seem less non-partisan. So that makes PolitiFact more objective than PolitiFact Bias.  Seriously?

There are two kinds of bias in journalism. One is bias that's nestled in the brain, and we're all stuck with it to one degree or another, every one of us.  The important bias is the one that comes out in writing and reporting.  When we make our case that PolitiFact is biased, our goal is to make that case without bias.  We're not perfect, but that's the goal.  And if we're not meeting that goal, we'd love to hear about it.  And you know what doesn't help one bit in helping us meet that goal?  It's comments like this:
I read a bit of that site, and it happens to have its own biases.
If there was some specific case mentioned then maybe we could use it to make an improvement.  But comments like the above typically serve as fallacious attacks on our credibility.  We're not wrong because we bring a conservative bias to our work.  If we're wrong on account of bias, it's because we let that bias introduce errors or fallacious thinking into our work.

We love to hear good criticism, whether constructive or not.  The world could do with less criticism that lacks content, focus or specifics.


Wednesday, May 28, 2014

When is a correction not a correction?

This correction is not a correction:
Correction: A Rand survey put the previously uninsured rate at about 36 percent of new marketplace enrollees. An earlier version of this story described the percentage as the "insured rate."
This correction is a correction:
Correction: About 60 percent of people living in Crimea identify themselves as Russian. An earlier version of this story described the statistic differently. This post was updated at 1 p.m. March 3, 2014.
And this is how we can tell the difference.

We pointed out the error in the Sununu fact check over a month ago. To its credit, PolitiFact appended a correction to the article after only a little external prodding.  But PolitiFact has never added the Sununu item to its list of corrections or updates.

The takeaway?  Don't view PolitiFact's corrections page as any kind of accurate measure of its mistakes.  PolitiFact doesn't even put all the mistakes it admits on the corrections page, let alone the mistakes it refuses to acknowledge.


PolitiFact, percent error, partisanship

In last Sunday's post about PolitiFact's math on uninsured Americans, we mentioned the worth of tracking PolitiFact's application of a basic math equation used to calculate error percentages.

To calculate the percent error, one takes the difference between the right figure and the wrong figure, and divides the difference by the right figure.

For example, if we guess Scarlett Johansson weighs 175 lbs and her actual weight is 100 lbs, we take the difference, 75, and divide it by the correct figure, 100.  That gives us 75 percent, so that's the percentage error.  It is incorrect to divide by the incorrect estimate, 175 lbs.  That procedure ends up giving us a figure of about 43 percent, which is the wrong figure for the percentage error even if the calculation is performed correctly.  It's the wrong formula.  Our estimate was off by 75 percent, not 43 percent.

We propose, given the difficulty journalists often display with math problems, that this calculation can serve as an indicator of ideological bias.  Where a journalist performs the wrong equation to the benefit of one political party over the other, we have an evidence of ideological bias.

The Experiment

We used the search string "off by" and the term "percentage" and performed a Google search limited to the politifact.com domain.  We then combed the results for the target results: instances of percentage error calculations.

We found only 14, which is a low number for a study.  Two of the fact checks had two (wrong) calculations in them, but we count each fact check as just one case.

Out of 14 equations, PolitiFact performed the wrong calculation an appalling nine times.  Over half.


Good Fortune for Democrats

Since there are two ways to perform the percentage error calculation, albeit one is the wrong way, we tracked both methods for their correlation with benefit or harm to political parties.  The reason is simple.  One of the calculations will normally minimize the error compared to the other calculation.  If Republicans get the wrong calculation every time the wrong calculation minimizes the error and the right calculation every time it likewise minimizes the error then just counting the number of times the wrong formula helped Republicans doesn't adequately tell the story.  The other half of the story is the uncanny ability to use the right equation when it helps Republicans.

Does anybody sincerely expect PolitiFact to help Republicans?  Do you work for Media Matters or what?


Wrong calculation helped Dem., harmed Rep.   6  
Wrong calculation helped Rep., harmed Dem.    3   



Right calculation helped Dem, harmed Rep.
3

Right calculation helped Rep., harmed Dem.
0


Each of the nine times PolitiFact used the percentage error concept for a fact check of a Democrat in our sample, the Democrat received the calculation that provided the greatest benefit.  That's not counting the two ambiguous cases. [incorrectly phrased; ignore--ed.]

Where the type of calculation made a noticeable difference in the degree of error, the liberal point of view got the benefit 75 percent of the time.

Three times PolitiFact aided Republicans with the wrong calculation.  One case occurred in a fact check by an intern.  Another case helped out Republican Allen West by a measly 2 percentage points.  The third case helped a Republican criticizing the influence of a special interest group on an election.  In other words, the truth is even worse than the table makes it look.


Correction 5/29/2014:  Allen West received the benefit of two measly percentage points, not Herman Cain as I originally wrote.  Cain occurs in our data as a case of harm from the wrong calculation.


Data notes

Sunday, May 25, 2014

PolitiMath on uninsured Americans

An pseudonymous tipster pointed out problems with an old PolitiFact rating from 2009.

PolitiFact rated President Obama "Mostly True" for his statement that nearly 46 million Americans lack health insurance.

PolitiFact examined Census Bureau data confirming the president's figure, but noted it included 9.7 million non-citizens.  Our tipster pointed out that the number also included an estimated 14 million already eligible for government assistance in getting health insurance. 
The 2004 Census Current Population Survey (CPS) identified 44.7 million non-elderly uninsured in 2003. Blue Cross and Blue Shield Association contracted with the Actuarial Research Corporation (ARC) to provide a detailed analysis of the uninsured identified by the Census Bureau, which found:
  • Nearly one-third were reachable through public programs, such as Medicaid and the SCHIP program for children
  • One-fifth earn $50,000 or more annually and may be able to afford coverage
  • Almost half may have difficulty affording coverage because they earn less than $50,000 per year. Many of these people work for small firms that do not offer health coverage
Given that Obama was using the number of uninsured to promote the need for government intervention, PolitiFact should have mentioned the number of uninsured already able to take advantage of government help.  We're seeing that this year as at least 380,000 of those the administration says are gaining Medicaid through the ACA were already eligible before the law was passed. The administration can claim some credit for getting eligible persons signed up, but it's misleading to say all those signing up for Medicaid are gaining their coverage thanks to the ACA, just as it was misleading to use 14 million assistance-eligible Americans to show the need to offer more of the same kind of assistance.  The need was exaggerated, and PolitiFact failed to properly notice the size of the exaggeration.

The PolitiMath angle

We use the term PolitiMath of the relationships between PolitiFact's math equations and its "Truth-O-Meter" ratings.  Many journalists have trouble properly calculation error percentage, and in this item we find PolitiFact's former chief editor (Bill Adair) and its present chief editor (Angie Drobnic Holan) making a common mistake:
Getting back to Obama's statement, he said, "Nearly 46 million Americans don't have health insurance coverage today." That is the most recent number for the U.S. Census available, but he messes it up in one way that would tend to overcount the uninsured and in another way that would tend to undercount them.

It's an overcount because it counts noncitizens. Take out the 9.7 million noncitizens and the actual number is closer to 36 million. 

... So Obama is sloppy by saying it is for "Americans" but not accounting for the noncitizens, which leaves him off by about 22 percent.
PolitiFact's likely equation:  (46-36)/46   _21.7 percent_

It's the wrong equation, and this is not controversial.  It's basic math.  To find the percentage error the accurate value belongs in the denominator.

The right equation:  (46-36)/36    _27.7 percent_

Marc Caputo of the Miami Herald, a PolitiFact partner paper, made the same mistake months ago and vigorously defended it on Twitter.  Caputo argued that it's okay to do the equation either way.  One can execute the equation accurately in either form, but executing the wrong equation gives the wrong final figure.  Journalists need to consider the ramifications of having two different options for calculating an error percentage.  If one chooses the method in a way that favors one party over another then a pattern of that behavior turns into evidence of political bias.

Caputo used the method more damaging to the Republican to whom he referred.

In Adair and Holan's case, guess which party received the benefit of the wrong equation?

It's a statistic worth following.

Monday, May 19, 2014

Is PolitiFact incapable of objectivity on climate change?

We noted PolitiFact's failure to report accurately on Marco Rubio's climate change statement from May 11, 2014.  Whatever's much bigger than doubling down on its disgraceful reporting on climate change, that's what PolitiFact's doing with its May 19 fact check of California governor Jerry Brown.

Brown said virtually no Republican in Washington D.C. accepts climate change science.

PolitiFact's fact check of Brown's claim is comical.

PolitiFact cites polls showing Republicans are more skeptical of mankind's role in causing global warming.  That isn't directly relevant to whether Republicans in D.C. reject the science of climate change.  PolitiFact doesn't bother telling its readers over 20 percent of Republicans in a 2013 Pew Research poll think humans are the primary cause of global warming.

PolitiFact touts many (mostly unnamed) examples of Republicans questioning climate change science "to some degree."  The degree is kind of important when we're talking about rejecting science, isn't it?

PolitiFact cites Marco Rubio as a recent example of a climate change denier.  We showed why PolitiFact's charge against Rubio is false.

PolitiFact cites Republicans John Boehner and Ted Cruz in a similar way.  The Boehner and Cruz examples share essentially the same flaws as the Rubio one.  The press takes statements out of context and draws its preferred conclusion.

PolitiFact cites the Organizing For Action's lengthy list of supposed "climate change deniers," assuring readers that OFA shows evidence for each one.  OFA was President Obama's campaign organization before it changed its name and purpose.  Therefore it's just as objective as press reports taken out of context.

PolitiFact cites an article about John McCain, saying it shows he's changed from his former acceptance of man-caused climate change.  We invite anyone to strain the article for that finding.

After that, we get the list of eight Republicans who supposedly accept climate change science.

And after that, PolitiFact admits that there may be more than eight.  PolitiFact doesn't tell you how many more there might be.  That would involve fact checking.

After all that, PolitiFact rates Gov. Brown "Mostly True":
Brown said that "virtually no Republican" in Washington accepts climate change science. When it comes to on-the-record comments of members of Congress, Brown’s characterization is about right.

We found at least eight Republicans in Congress who publicly voiced support for the scientific consensus and many more conservative legislators who deny either a human link to the changing climate, or the fact that the climate is changing altogether.

A reason for caution, however, is comments from someone like Yarnold — who suggest GOP members of Congress acknowledge climate change science behind closed doors but avoid the talk in public for political reasons.

We rate Brown’s claim Mostly True.
There are two major problems with PolitiFact's rating.

First, it's a mistake to use an all-or-nothing approach to acceptance of climate science.  That approach isn't used in establishing measurements of scientific consensus on the issue, so that measuring stick gives us an apples-to-oranges comparison.

Second, unless PolitiFact is accepting OFA's list at face value, PolitiFact simply assumes that over 200 Republicans are climate change deniers.  And even if PolitiFact accepts OFA's list at face value, PolitiFact is still assuming more than 100 Republicans are climate change deniers.  Those assumptions fly directly in the face of one of PolitiFact's principles, which look more and more like Pirates of the Caribbean "guidelines" with each passing day:
Burden of proof – People who make factual claims are accountable for their words and should be able to provide evidence to back them up. We will try to verify their statements, but we believe the burden of proof is on the person making the statement.
Don't worry, Gov. Brown. PolitiFact will pretend to have the proof you don't have.

Think about it.  If just half the 128 not accounted for from the OFA list plus McCain (Cruz and Rubio are on OFA's list, McCain isn't) and the elite eight, then the percentage of Republicans accepting the supposed science of climate change is 26 percent.  Even overlooking the mind-boggling sloppiness of the fact check, we're left with a range of 3-49 percent (counting McCain as a denier).

It's irresponsible journalism to use biased secondary sources like OFA as the basis for a fact-check finding.  It's incumbent on the journalist to verify the accuracy of such sources.  We see no indication of that from PolitiFact.

This is PolitiFact fact checking.  But there's another name for it.  Crap.


Addendum

Context, Context

Hot Air has a little item on Gov. Brown's statement revealing its original context.  Brown brought up climate change as a cause of California's current problem with wild fires.

It's settled science or something.  Wouldn't PolitiFact have questioned it otherwise?


Correction/Update 5/20/2014:
Fixed assorted grammatical problems and added a parenthetical "mostly unnamed."

Friday, May 16, 2014

More on PolitiFact's deceptive Rubio/climate correction

We've uncovered a bit more evidence of PolitiFact's dishonest correction of its climate-change hit piece on Sen. Marco Rubio (R-Fla.).

For review, here's the correction notice PolitiFact attached to its amended article:
CORRECTION: This story was updated on May 15 to clarify that 97.1 percent of the studies that took a position on global warming agreed that there's been a negative human impact on the atmosphere; more than half the studies did not take a position. Also, the story clarifies that the 2013 report looked at studies, not individual scientists.
The original article wasn't unclear about the 2013 report. It flatly said the report indicated 97.1 percent of scientists disagree with Rubio's supposed claim (PolitiFact blew and continues to blow the reporting on what Rubio said) that humans do not contribute to climate change.

Here's how PolitiFact was publicizing the Rubio fact check on its list of stories (red oval added to draw attention to the false reporting):


PolitiFact's clarification is not a clarification.  It's a gloss on a reporting error.

Here's how the Rubio blurb appears today:


PolitiFact's original article encouraged readers to conclude that 97 percent of scientists agree the Antarctic ice shelf is collapsing because of human-caused climate change.  That's a deception far worse than the Jeep ad from the Romney campaign that PolitiFact awarded its 2012 "Lie of the Year."  And the current version remains more misleading than that Romney ad.

PolitiFact continues climate change smear of Rubio

I noted over at Zebra Fact Check last year how PolitiFact has enlisted itself to aid in tarring various Republican politicians as "climate change deniers."  PolitiFact continued that effort this month:
Scientists have been issuing more new reports on the irreversible effects of climate change in recent weeks. Two groups reported on May 12, 2014, that the global sea level will rise at least 10 feet, accelerating to a dangerous pace after the next century.

Just a day before those reports were released, Sen. Marco Rubio, R-Fla., sat down with ABC’s Jonathan Karl on This Week. Talk turned to climate change, where the possible Republican presidential candidate denied a link between humans and the changing environment.
PolitiFact wasn't alone in its interpretation of Rubio's remark.  Unfortunately, PolitiFact didn't included enough context to make clear what Rubio was talking about.  Lucy McAlmont, writing at Patterico's Pontifications, took note of the cyclone of media spin and provided that context (bold emphasis carried over from McAlmont's transcript):
KARL: Miami, Tampa, are two of the cities that are most threatened by climate change. So putting aside your disagreement with what to do about it, do you agree with the science on this? I mean, how big a threat is climate change?

RUBIO: I don’t agree with the notion that some are putting out there, including scientists, that somehow, there are actions that we can take today that would actually have an impact on what’s happening in our climate. Our climate is always changing. And what they have chosen to do is take a handful of decades of research and say that this is now evidence of a longer term trend that’s directly and almost solely attributable to manmade activity. I do not agree with that.

KARL: You don’t buy that.

RUBIO: I don’t know of any era in history where climate has been stable. Climate is always evolving and natural disasters have always existed.

KARL: Let me get this straight. You do not think that human activity, the production of CO2 has caused warming to our planet?

RUBIO: I do not believe that human activity is causing these dramatic changes to our climate the way these scientists are portraying it. That’s what I do not — And I do not believe that the laws we pass will do anything about it. Except it will destroy our economy.
Reading Rubio's responses to Karl, it should be obvious that PolitiFact reports falsely when it says Rubio denied a link between humans and the changing environment. Rubio acknowledges people contribute to climate change but questions some of the more extreme claims.
 

PolitiFact provided a great example of an extreme claim.  We assume PolitiFact did this unknowingly.

The Ice Sheet


PolitiFact led the Rubio fact check with this:
Scientists have been issuing more new reports on the irreversible effects of climate change in recent weeks. Two groups reported on May 12, 2014, that the global sea level will rise at least 10 feet, accelerating to a dangerous pace after the next century.
That rise in sea level is a result of the irreversible effects of climate change?  And relevant to Rubio's skepticism regarding the size of mankind's role in climate change?

The hotlink embedded in PolitiFact's story leads to an article in The New York Times, where we find this:
Scientists said the ice sheet was not melting because of warmer air temperatures, but rather because relatively warm water that occurs naturally in the depths of the ocean was being pulled to the surface by an intensification, over the past several decades, of the powerful winds that encircle Antarctica.
The Times' story goes on to note that (unnamed) researchers think global warming may have contributed to the wind pattern affecting the ice sheets.

For PolitiFact, somehow the tenuous scientific link between global warming and the collapsing ice sheet makes a perfect intro to its Rubio fact check.

That's irony.

A 97.1 Percent Consensus and a PolitiFact Correction


The original version of the Rubio fact check claimed 97.1 percent of scientist disagreed with Rubio.  We looked forward to dissecting that blunder.  Now we find PolitiFact buried it with the following correction:
CORRECTION: This story was updated on May 15 to clarify that 97.1 percent of the studies that took a position on global warming agreed that there's been a negative human impact on the atmosphere; more than half the studies did not take a position. Also, the story clarifies that the 2013 report looked at studies, not individual scientists.
That's how to do a dishonest correction.  PolitiFact buried its inaccurate reporting by clarifying that the 97.1 figure was a select group of science papers, not scientists.  And simply eliminates the inaccurate reporting, pretending it never happened.

See Also


James Taylor of Media Trackers Florida also brought some attention to PolitiFact's smear of Rubio. We've picked out a section that includes some of PolitiFact's pre-correction malfeasance:
PolitiFact Florida is flat-out wrong regarding its scientific assertions. PolitiFact Florida justified its “False” ruling by claiming, “A May 2013 report analyzing all scientific papers that address the causes of climate change showed 97.1 percent of scientists agree that there’s been a negative human impact on the atmosphere.”
Taylor's quotation makes clear the magnitude of some the false reporting PolitiFact covers up with its correction spin.


Also See

Zebra Fact Check has an article that helps sort out what various studies show about the consensus on climate change.

PolitiFact could have benefited by consulting it before publishing its Rubio smear.

Wednesday, May 7, 2014

Nothing To See Here: Koch brothers drive climate change (Updated)

Senate Majority Leader Harry Reid (D-Nev.) says the Koch brothers cause much of global climate change.

Via Politico:
“While the Koch brothers admit to not being experts on the matter, these billionaire oil tycoons are certainly experts at contributing to climate change. That’s what they do very well. They are one of the main causes of this. Not a cause, one of the main causes,” Reid said.
 That's not really something that needs fact checking, is it?  It's only the Senate majority leader.

Nothing to see here.  Move along.


Update 5/8/2014

The Washington Post Fact Checker hit this topic today, giving Reid three "Pinocchios."  PolitiFact still hasn't weighed in.


Jeff Adds (5/8/2014):

Andrew Stiles of the Washington Free Beacon helpfully performs the fact check that Angie Holan is apparently too busy to do:
The multizillionaire Koch brothers

False. “Zillion” is not a real number.

The two richest people in the world

Nope. According to Forbes, the two richest people in the world are Democratic donor Bill Gates, and New York Times investor Carlos Slim.

Tuesday, May 6, 2014

Media Trackers: "PolitiFact Florida Scorecard Reveals Pro-Crist Bias in Governor’s Race"

Media Trackers' Florida affiliate takes a healthy whack at PolitiFact Florida for its Crist-leaning coverage of the gubernatorial election.  James Taylor makes a number of solid points, including this one:
In its third ruling, PolitiFact Florida fact-checked the Republican Party of Florida’s assertion that Crist supported “cuts to the Medicare Advantage program.” PolitiFact Florida assigned a flat-out “False” ruling to this claim. It is a legislative fact that Obamacare cuts funding for the Medicare Advantage program. Since becoming a Democrat, Crist has been an ardent supporter of Obamacare. Accordingly, Crist ardently supports a bill that necessitates “cuts to the Medicare Advantage program.” PolitiFact Florida nevertheless justified its “False” ruling by claiming Crist said he personally does not like this particular provision of Obamacare.
Taylor cinches the point by emphasizing the degree to which Obamacare relies on Medicare Advantage cuts to shrink its own price tag.

It's fine for Democratic candidate Charlie Crist to oppose Medicare Advantage cuts.  But they're an integral part of Obamacare, so opposing them is tantamount to opposing at least a part of Obamacare.  And if Crist doesn't oppose the cuts enough to oppose Obamacare over them then he effectively supports the cuts.

It's almost like PolitiFact is playing the "No True Scotsman" game.

True Obamacare doesn't cut Medicare Advantage.  Or something.