Thursday, March 29, 2012

PolitiFact rolls out new feature, continues to ignore selection bias

Crossposted from Sublime Bloviations


I've belabored the point that PolitiFact pushes its candidate "report cards" without owning up to the selection bias that is overpoweringly likely to skew the grades.

PolitiFact editor Bill Adair has been asked about selection bias.

Bill Adair probably (based on IP address trails) reads commentary here and at PolitiFact Bias [more appropriately "here and at Sublime Bloviations" after crossposting] regarding PolitiFact's problem with selection bias.

Yet minutes ago we get this:
It's spring, which means it's report card time. So we're unveiling a new feature that allows you to compare the PolitiFact report cards for individuals and groups we check.

Our report cards have always been a popular feature and often generate interesting commentary. Now, you'll be able to compare the report cards more easily.
Adair still won't inform his readers that the process leading to the report card grades is rife with selection bias problems.  Yes, Adair at least linked to a New York Times blog ("interesting") that provided the minimum type of disclaimer that PolitiFact should offer.  The other link ("commentary") was the sort of pointless statistical exercise that simply elaborates on the results of PolitiFact's fundamentally flawed process (the former link I gave a positive review, the latter author I've given a less-than-positive review).

Bottom line, PolitiFact continues to publish candidate "report cards" that appear minus critical context.  PolitiFact (allegedly) rules statements missing critical context "Half True."

PolitiFact apparently knows about its selection bias problem and is deliberately downplaying it.  Selection bias is not merely "interesting."  It is critical to an accurate understanding of the meaning of PolitiFact's "report cards."


Updated 3/29/2012:  Added editor's note in third paragraph

No comments:

Post a Comment