PFB Smackdown: Kos clueless on Ostermeier study

The reality-based community hasn't taken kindly to Eric Ostermeier's attempts to rescue reality from its liberal bias.

Steve Singiser for the Daily Kos is the latest to take a swipe at the study, writing under the title "Documented proof that Republicans are the biggest liars in politics":

(T)he study (which can be found here) focuses on Politifact itself, charging the nonpartisan analysis done by the St. Petersburg Times fact-checking unit with a systemic bias against Republicans.

Singiser misstates the charge made by Ostermeier.  Ostermeier argues that PolitiFact's presentation suggests Republicans lie more than Democrats, but in scientific terms that conclusion only follows if the conclusions from the data account for selection bias.  But the evidence we have of PolitiFact's selection process offers no assurance against ideological bias in the results.  The stark contrast in the results, Ostermeier says, places a burden of proof on PolitiFact to assure readers that the implicit conclusion is not the result of selection bias.  In other words, make the selection process transparent.  Ostermeier is not charging PolitiFact with "a systemic bias against Republicans."  But he's pointing out that it may well be the case.

After constructing a straw man position for Ostermeier, Singiser proceeds to attack it:

(T)here are other explanations that are equally, if not more, plausible than charging Politifact with grading in bad faith.

True, among them an unconscious bias (good faith, bad methods) among PolitiFact journalists that results in the type of selection bias Ostermeier hypothesizes may account for the disparity in PolitiFact's cumulative grades.  And the latter is an extraordinarily plausible (and parsimonious) explanation.  Singiser would be hard pressed to match it, and that's why Ostermeier argues that PolitiFact should do something to provide a sure foundation for the conclusions it implicitly encourages.

Singiser continues:

For one thing, the study was conducted during a time when the GOP was out of power. The party out of power, it could reasonably be assumed, is going to take more chances with their rhetoric, in an effort to turn the electorate against the party in office.

That's all well and good, but it does nothing to mitigate Ostermeier's real argument.  Remember, PolitiFact grades approximately the same number of statements from figures in both parties.  If PolitiFact used a policy of simply grading chancy rhetoric then one would expect PolitiFact to rate far more statements from one party than the other if the conditions Singiser suggests were to prevail.  And if PolitiFact ends up padding the Democrats' numbers with statements chosen according to a different criterion then we have selection bias by definition.

In short, Singiser's suggested explanation doesn't help.

He doesn't get it.



Feb 17, 2011:  Corrected a recurrent misspelling of Markos Moulitsas' last name (no "z" as it turns out).  Sincere apologies to Mr. Moulitsas.
Jan. 24, 2012:  Double apologies to Markos Moulitsas.  While going back to reference the Kos blog entry I found the link broken.  Shortly after I found an identical version of the post clearly credited to Steve Singiser.