So hold my beer, Nelly.
We say that PolitiFact's graphs and charts, including its PunditFact collections of ratings for news networks, routinely mislead readers. But PolitiFact Editor Angie Drobnic Holan says there isn't much reader confusion.
Quite a few folks apparently have no clue at all that PolitiFact's charts and graphs lack anything like a scientific basis. Others know that something isn't right about the charts and graphs but against all reason find some value in them anyway.
PolitiFact itself would fall in the latter camp, based on the way it uses its charts and graphs.
So would Luciano Gonzalez, writing at Patheos. Gonzalez listened to Speaker of the House Paul Ryan's speech announcing his impending retirement and started wondering about Ryan's record of honesty.
PolitiFact's charts and graphs don't tell people about the honesty of politicians because of many flaky layers of selection bias, but people can't seem to help themselves (bold emphasis added):
I decided after hearing his speech at his press conference to independently check if this House Speaker has made more honest claims than his predecessors. To did this I went to Politifact and read the records of Nancy Pelosi (House Speaker from January 4th 2007-January 3rd 2011), John Boehner (House Speaker from January 5th 2011-October 29th, 2015), and of course of the current House Speaker Paul Ryan (October 29th 2015 until January 2019). I admit that there are flaws in this, such as the fact that not every political claim a politician makes is examined (or even capable of being examined) by Politifact and of course the inherent problems in giving political claims “true”, “mostly”, “half-true”, “mostly false”, “false”, & “pants on fire” ratings but it’s better than not examining political claims and a candidate’s level of honesty or awareness of reality at all.If we can't have science, Gonzalez appears to say, pseudoscience is better than nothing at all.
Gonzalez proceeds to crunch the meaningless numbers, which "support" the premise of his column that Ryan isn't really so honest.
That accounts for the great bulk of Gonzalez's column.
Let's be clear: PolitiFact encourages this type of irresponsible behavior by publishing its nonsense graphs without the disclaimers that spell out for people that the graphs cannot be reasonably used to gauge people's honesty.
PolitiFact encourages exactly the type of behavior that fact checkers ought to discourage.