Sunday, January 20, 2013

PolitiFact Georgia publishes inaccurate stats for 2012?

These are definitely the people we want doing our fact checks.

PolitiFact Georgia published a statistical breakdown of its fact checks for 2012.

There's just one problem.  The statistics don't appear to have any solid relationship to reality.

Let's see how PolitiFact Georgia editor Jim Tharpe tells it:

Most ratings, as in 2011, fell between the extreme ratings of True and Pants On Fire.

Ratings for the GOP/conservative fact checks broke down like this: 26 True, 22 Mostly True, 32 Half True, 14 Mostly False, 16 False and eight Pants On Fire.

Ratings for the Democratic/liberal fact checks broke down like this: 10 True, 14 Mostly True, 23 Half True, 10 Mostly False, 15 False and four Pants On Fire.

For fact checks for groups and individuals we labeled as "other, " the ratings broke this way: seven True, 14 Mostly True, eight Half True, eight Mostly False, eight False and three Pants On Fire.

It's gotta be the glasses.

Going by the numbers appearing on its website, PolitiFact Georgia did less than 160 fact checks in 2012.  Tharpe gives the number 242.  Looking at all the "Truth-O-Meter" ratings, the site gives 20 per page in reverse chronological order.  On the eighth page of results we reach 2011.  Toss in 36 promises from Gov. Nathan Deal and we get to 196, not that PolitiFact Georgia has actually done that many ratings of Deal's promises.

Where do Tharpe's numbers come from?  Is there a secret list of unpublished "Truth-O-Meter" ratings from PolitiFact Georgia?  Does PolitiFact Georgia count PolitiFact National articles published in the Atlanta Journal-Constitution?  Should PolitiFact Georgia take credit for those stories if that's how Tharpe arrived at his total?

Tharpe isn't from outer space simply with respect to the total number of stories.  None of it adds up, so far as I can tell.  Add Tharpe's total stories for Democrats and Republicans and we end with a total of 194 stories--just two short of the inflated total we get by rounding up to 160 and adding in Deal's promises.  And that doesn't count an additional 48 stories from Tharpe's "other" category.

If there's one saving grace, at least the total of all those stories does agree with Tharpe's overall total of 242 stories.  It's just hard to tell from which planet those stories originated.

Here's the tale of the numbers directly from the PolitiFact Georgia website for 2012:

PolitiFact Georgia published 148 "Truth-O-Meter" fact check stories.  Of that total, 30 were "True" ratings, 33 were "Mostly True" ratings, 35 were "Half True" ratings, 20 were "Mostly False" ratings, 22 were "False" ratings and 8 "Pants on Fire" ratings.

I emailed Tharpe asking about the discrepancy.  He answered that PolitiFact Georgia keeps "a very accurate count" and confirmed that PF Georgia counted stories from PolitiFact national as well as those from the "Deal-O-Meter."  We've already touched on the problems with those methods.

If any "Deal-O-Meter" ratings count in the totals then the breakdown I show shouldn't add up.  If we don't have any "Deal-O-Meter" ratings in the total then it takes 94 stories from PF national to get to Tharpe's total of 242 stories.

On Jan. 3 I wrote again to Tharpe:
When I added together your three breakdowns of stories in the classes "Republican," "Democrat" and "other" the totals agreed with your overall total.  But all three classes are broken down into "Truth-O-Meter" ratings like "True" and "Mostly False."  There are no ratings like "Promise Kept" in those groups. Could you explain how the "Deal-O-Meter" claims fit with the total of 242 fact checks?

You affirm that your totals include some stories from PolitiFact's national operation.  At the same time, your article reports "a year of more than 200 fact checks by your local team of truth-seekers, collectively known as PolitiFact Georgia."  How does that claim jibe with using stories from PolitiFact's national operation in the totals?

When Tharpe first replied he said I could phone him if I needed more help with the numbers. Perhaps it is because I contacted him again by email instead of by phone that the message remains unanswered.

No comments:

Post a Comment