Saturday, February 12, 2011

PFB Smackdown: Why grant PolitiFact the presumption of neutrality?

The smackdown is actually courtesy of Eric Ostermeier, publisher of the study highlighted recently here at PolitiFact Bias and originally published at the University of Minnesota's Smart Politics blog.

Ostermeier sniffed out a criticism of his work at mnpACT! by progressive blogger Dave Mindeman.  After noting two of the central facts in Ostermeier's study, that PolitiFact rates about the same number of statements from Democrats as for Republicans and that Republicans get the worst ratings of the two, Mindeman offers his two cents:
Occam's razor. The GOP get the worst ratings because they make the worst statements.

Ostermeier concentrates on making Politifact defend their selection process, but overlooks the facts about the statements themselves. Could it be possible that the Republicans make more outrageous and indefensible assertions?

Politifact is certainly going to be drawn to statements that get the most attention and the more outrageous the statement, the more attention it gets.
Ostermeier promptly addressed the second paragraph by posting in the commentary section:
FYI: this very possibility was in fact addressed in my report:

"One could theoretically argue that one political party has made a disproportionately higher number of false claims than the other, and that this is subsequently reflected in the distribution of ratings on the PolitiFact site."
Mindeman (as DaveM) answered back:
Then why ask title your post with "selection bias"? It would seem that most of the data says there is none.
And Ostermeier responded again:
What data is that? You can't use the data I published from coding PolitiFact's stories itself (noting the site attributes more false statements to the GOP) as proof that the GOP lies more and thus there is de facto no selection bias. That's circular reasoning. Note: My report did not definitively prove there is such bias, but the data published shifts the burden, I would argue, to PolitiFact. And greater transparency in their selection methodology would shine a light on this very question.
Ostermeier is exactly right.  Mindeman apparently wants to entitle PolitiFact to the presumption of neutrality on the issue of selection bias.  But there simply isn't any basis for that presumption.  The presumption would follow if PolitiFact chose its stories at random.  Lacking that, the reader has no good reason to take PolitiFact as a neutral party minus the transparent methodology Ostermeier mentions at the conclusion of his second comment.

Mindeman has the last word on Ostermeier for the moment:
I fail to understand why Politifact has to "prove" anything. They examined political statements that interested them. Most reporters choose their own stories. If they do a good job of reporting the story, do they still have to prove that they have no inherent bias? Maybe I should assume you have an inherent conservative bias because your analysis deals with GOP favorable data?? But I don't, because I think you use the data in a broad enough sense that it tells something regardless of the outcome.
By this time Mindeman is offering some clues that he doesn't understand selection bias.  But the biggest problem in his analysis is actually his attempt to employ Occam's razor (aka the principle of parsimony) to disfavor Ostermeier's hypothesis that PolitiFact displays a selection bias favorable to Democrats.

Occam's razor favors simple explanations, and Mindeman seems to understand it that far.  But it isn't at all clear why he regards a pattern of lying among a large set of entities as a simpler explanation than political bias from a much smaller group of journalists.  Rather than using Occam's razor to legitimately favor a simpler explanation, Mindeman wields it more like a magic wand that produces a supernatural sphere of protection around the ideas he favors.

No comments:

Post a Comment