Tuesday, January 26, 2016

PolitiFact's partisan correlation correlation

The co-founder and co-editor of PolitiFact Bias, Jeff D, noted this outstanding example of PolitiFact's inconsistency. Jeff's a bit too busy to write up the example and so granted me that honor.

The problem Jeff noted has to do with PolitiFact's treatment of factual correlations. A correlation occurs when two things happen near the same time. When the correlation occurs regularly, it is often taken as a sign of causation. We infer that one of the things causes the other in some way.

Correlation, however, is not a proof of causation. PolitiFact recognizes that fact, as we can see from the explanation offered in a fact check of Lieutenant Governor Dan Patrick (R-Texas):
Lott’s study shows a 25 percent decrease in murder and violent crime across the country from 2007 to 2014, as well as a 178 percent rise in the number of concealed-carry permits. Those two trends may be correlated, but experts say there’s no evidence showing causation. Further, gun laws may have little to nothing to do with rates of falling crime.
PolitiFact ruled Patrick's statement "Mostly False," perhaps partly because Patrick emphasized open carry while Lott's research dealt with concealed-carry.

PolitiFact also noted that correlation does not equal causation while evaluating a claim from Democrat presidential nominee Hillary Clinton. Clinton said recessions happen much more often under Republican presidents:
The numbers back up Clinton’s claim since World War II: Of the 49 quarters in recession since 1947, eight occurred under Democrats, while 41 occurred under Republicans.

It’s important to note, however, that many factors contribute to general well-being of the economy, so one shouldn’t treat Clinton’s implication -- that Democratic presidencies are better for the economy -- with irrational exuberance.
Okay, maybe PolitiFact was a little stronger with its warning that correlation does not necessarily indicate causation while dealing with the Republican. But that doesn't necessarily mean that PolitiFact gave Clinton a better rating than Patrick.

Clinton's claim received a "Mostly True" rating, by the way.

Was Patrick's potentially faulty emphasis on open carry the reason he fared worse with his rating? We can't rule it out as a contributing factor, though PolitiFact wasn't quite crystal clear in communicating how it justified the rating. On the other hand, Clinton left out details from the research supporting her claim, such as the fact that the claim applied to the period since 1947. We see no evidence PolitiFact counted that against her.

Perhaps this comparison is best explained via biased coin tosses.


Post-publication note: We'll be looking at PolitiFact's stories on causation narratives to see if there's a partisan pattern in their ratings.

 

No comments:

Post a Comment