Sunday, September 23, 2012

Relevant: The AllSides project

A new venture called "AllSides" tries to meet democracy's need for trustworthy information by using ideological transparency, crowdsourcing and some technical inspiration from Tim Groseclose and Jeff Milyo.

We've previously emphasized the relevance of Groseclose's book "Left Turn" to our ongoing critique of PolitiFact's bias problem.

AllSides appears to represent an entirely new attempt to address the problem of getting quality information to voters in a constitutional republic.  We're not big fans of crowdsourcing, but it seems like a potentially reasonable approach to grading sources for their degree of bias.

And speaking of the degree of bias, the quotation that largely accounts for our interest in the AllSides project (bold emphasis added):
During the Democratic National Convention (DNC), Bill Clinton asserted that over the last 52 years, America had experienced more job growth under past Democratic presidents (42 million) than under Republican presidents (24 million).

In covering this assertion, both PolitiFact and The Washington Post's Fact Checker determined that Bill Clinton's job numbers were essentially correct. PolitiFact (AllSides Bias Rating "Left") gave it a "True" rating and went on to make the case that the numbers were even stronger than they appear.
PolitiFact's "Left" rating for bias especially interested me because I've often wondered how PolitiFact would stack up against other fact checkers in terms of sustaining reader trust.

Unless AllSides confused PolitiFact with Politico (which looks somewhat likely), it looks like PolitiFact wears a broad reputation for liberal bias.

We'll look forward to more from AllSides.


Jeff adds:

Consider me one of the skeptics. Crowdsourcing has undeniable value. Whether it's Yelp reviews or an Ebay member's rating, the opinions of large amounts of random people can mean something. But the authority of the masses doesn't hold the same weight with regard to objective reality. An overwhelming number of otherwise rational people vouched for the Macarena's awesomeness. Large groups of people can be wrong. Reality is unencumbered by the burdens of popularity. The fact that most people think PolitiFact is biased to the left doesn't make it so. That finding is better explored through critical study with verifiable and reproducible evidence.

I took AllSides' 'test' and it's no surprise I ended up on the 'Right' side of their scale. The whole process seemed a bit push-poll-ey to me. If someone considers flag burning immoral, but supports flag burning as a constitutional right, which box do they choose on AllSides test? Do you feel extremely strong that abortion is a right [left box] while rejecting the notion that it's the government's responsibility to pay for it [right box]?

Self-assessment is inherently flawed. And it's even less reliable when dealing with convoluted subjects like political philosophy and moral convictions.

Regardless, I say kudos to AllSides for assembling a respectable team of experts. There's no doubt about their sincerity and they deserve the benefit of the doubt. I commend them on their sincere efforts and it's possible they will provide interested readers with valuable, if only anecdotal, information as their project progresses.

I'm not convinced, but I'm looking forward to them proving me wrong. At the very least they deserve credit for producing a much more honest and transparent project than PolitiFact ever has.

[Note: A draft version of this Jeff adds portion was inadvertently published simultaneously with the original post and then immediately removed.]


2 comments:

  1. Checked them out; they have CNN, NPR, and ABC listed as being unbiased, so either theirs or their user's personal biases are skewing the results away from reality.

    As their "Bias" page says, "If you've got a pulse, you've got a bias. But hidden bias misleads and divides us." Too often, people don't recognize their own biases. I've had to explain to people why their positions on certain topics and their values put them into a category other than what they claim (usually far left 20-somethings that consider themselves "Independent").

    I'm a conservative, myself, and have no trouble at all admitting that colors my perception.

    ReplyDelete
  2. In a way you're making a good point, but the scale AllSides uses has only five positions to indicate relative bias. It seems to me a false assumption that "centrist" reporting means "unbiased."

    And yes of course the bias of the people doing the ratings affects the outcome.

    We think the AllSides emphasis on disclosure/exposure is healthy. But we don't endorse every little detail.

    It's worth noting that the Groseclose/Milyo studies did put NPR pretty close to centrist for its reporting. Note that their measurements involved the pattern of citations a news sources used.

    ReplyDelete