One classic example of that genre comes from PolitiFact's treatment of illustrated numbers comparisons. It makes sense to PolitiFact to compare the frequency of voter fraud to shark attack. But comparing the frequency of gun crimes by concealed-carry permit holders to the frequency of alligator attacks doesn't make sense because gun crimes and alligator attacks are so different from each other.
Predictably, PolitiFact's treatment of the former case proved a benefit to liberalism. PolitiFact's treatment of the latter case unfairly harmed conservatism.
On July 19, 2016, PolitiFact California approved the comparison of the number of licensed gun dealers to the number of McDonald's restaurants.
Of course the fact check contains no discussion at all whether McDonald's restaurants are similar enough to gun dealers to justify the comparison. And we're okay with that, aside from the inconsistency it shows from PolitiFact, because sometimes a numbers comparison is just that. One may compare the number of ants at a picnic to the number of stars in the galaxy without needing to show another similarity between ants and stars.
PolitiFact California went the extra mile for liberalism on this fact check, however.
PolitiFact Georgia rated a similar claim in 2013 as Mostly True. A civil rights activist said: "There are twice as many gun shops as McDonald’s in the United States." Our California claim deals with licenses in this state only, and not necessarily brick-and-mortar gun shops.Above, PolitiFact California starts to make a mockery of PolitiFact's principled assurance that "words matter."
The claim was that licensed gun dealers more than double the number of McDonald's restaurants. Counting those who have licenses to deal guns but who do not sell guns is like adding closed or re-purposed McDonald's locations to the total number of McDonald's restaurants. And the distinction matters in this case:
The ATF keeps a monthly log on its website of how many dealers and pawnbrokers are licensed to sell guns in each state.PolitiFact's fact-finding found the claim from the gun control group "Safety For All" was not quite true using those numbers. So Safety for All pointed out that adding in the number of licenses not necessarily connected to a gun seller would provide the numbers needed to make the claim true.
This month’s tally shows 2,315 dealers and pawnbrokers licensed to sell guns in California. That’s a shade less, 15 in fact, than twice as many McDonald’s.
PolitiFact California was fine with that approach and rated the claim "True."
The problem? That approach fails to jibe with the spirit of the comparison. Safety For All was trying to emphasize that licensed vendors from whom one might buy a gun more than double the number of McDonald's restaurants. Would the comparison work if half the licensed gun dealers did not sell guns? Of course not. Having a license to sell a gun does not necessarily make one a "gun seller" in the sense implied by the Safety For All claim.
It is not appropriate to shoehorn in persons licensed to sell guns who do not ordinarily act as gun sellers.
One could go further and nitpick the comparison to death (example here). We will stick with the point that a competent and neutral fact-checker does not illegitimately maximize a number to the benefit a political point of view. We expect that type of behavior from biased fact checkers.
PolitiFact California went the extra-extra mile by working up a meme-worthy graphic to promote Safety For All's not-quite-true factoid:
Update July 22, 2016: Added link to PolitiFact California fact check