Sunday, August 12, 2012

PFB Smackdown: Mediaite's Tommy Christopher (Updated/Corrected)

We use the PFB Smackdown feature to critique the worst of the left's best critiques of PolitiFact.

I get another excuse to say the political Left's critiques of PolitiFact are generally poor, thanks to Mediaite's political editor and White House correspondent Tommy Christopher.

Take it away, Christopher:
Political ads have been an especially hot topic this week, with surrogates from both presidential campaigns alternately citing, and arguing with, vaunted fact-checking outfits like the Pulitzer Prize-winning Politifact. Although controversial rulings have eroded the magic of such efforts, it is worth noting that, by Politifact’s numbers, Republican presidential candidate Mitt Romney is 58% more likely to lie than President Obama.
What magic?

PolitiFact's ratings have always drawn well-deserved criticism, from Bill Adair's brain-dead analysis of Joe Biden's hyperbole through last week's continuation of PolitiFact's series of misdirections about effective tax rates.  Why is it worth noting PolitiFact's comparison of Romney to Obama after we add the problem of selection bias to PolitiFact's inability to apply consistent standards or even achieve a reasonable minimum standard of quality?

Other than the fact that it might serve Christopher's politics, that is?  It's hard not to notice that both of Christopher's examples of supposed "controversial rulings" allegedly caused unfair harm to the Left.

It doesn't take many blown calls to produce an 58 percent difference between two individuals' "Truth-O-Meter" report cards, nor does it take much selection bias to produce that type of difference.

In short, contrary to Christopher's suggestion, aggregating PolitiFact's ratings is a useless exercise for purposes other than evaluating PolitiFact.

We have at least two examples of the latter so far:

"Selection Bias? PolitiFact Rates Republican Statements as False at 3 Times the Rate of Democrats"
"Bias in PolitiFact’s ratings: Pants on Fire vs. False"

The utility of its "report cards" stands as one of PolitiFact's most spectacular lies.  Don't buy it.

Update Aug. 14, 2012:  

Tommy Christopher tweets in response:
@PolitiFactBias Before you "smack" anyone "down," you ought to learn some math. It was a 17-point difference, not 8. Romney 46% Obama 29%
Christopher has a point in that the numbers I used were incorrect.  The passage was intended from the first to read "a 58 percent difference," and with this update that reading shows above.  My primary mistake was in failing to see a typographical error instead of a math error when I did yesterday's correction.

The change in percentage does not significantly affect the thrust of the criticism of Christopher's claim.  Without a control on selection bias,--and there is no good evidence of any such control--using PolitiFact's ratings other than to find out things about PolitiFact just doesn't make sense.

Will Christopher address that point or allow the typographical error to serve as a red herring covering for his mushy thinking?

Correction Aug. 13, 2012:  Math mistake:  Was:  "an 8 percent difference."  Is:  "an 8 percentage point difference."  Apologies for the error.
Correction correction Aug. 14, 2012:  See update above. 
Corrected update Au. 14, 2012:   Thanks to Jeff Dyberg for pointing out that I had incorrectly posted the original intent as "a 58 percentage point difference.  Rather, the intended figure was as Christopher expressed it, as a 58 percent difference.


No comments:

Post a Comment

Thanks to commenters who refuse to honor various requests from the blog administrators, all comments are now moderated. Pseudonymous commenters who do not choose distinctive pseudonyms will not be published, period. No "Anonymous." No "Unknown." Etc.