Monday, January 13, 2025

PolitiFact's "Pants on Fire" bias in 2024

For years, PolitiFact Bias has tracked the proportion of false ("False" plus "Pants on Fire") statements PolitiFact rated "Pants on Fire." As PolitiFact has never established an objective distinction between the two ratings, we infer that the difference between the two is substantially or wholly subjective. That makes this dividing line perhaps the best means of using PolitiFact's own ratings to measure its political bias.

As we are looking at proportions and not raw numbers for the bias measurement, the results cannot be dismissed on the basis that Republicans supposedly lie more.

The Tale of the Tape in 2024

Graphs-a-plenty this year!

We'll start with the dual graph of the PoF Bias number along with the story selection proportion number. The PoF Bias number could be expressed either of two ways. As the numbers pretty consistently have show an anti-Republican/pro-Democrat bias, this number shows that anti-Republican bias when the number is greater than 1. Using this option a PoF Bias Number less than 1 shows the PoF bias harmed Democrats. Our chart shows that occurring for four different years (2007, 2011, 2013, 2015). But it's important to point out that the state franchises accounted for the apparent relative evenhandedness for the latter three years. We tracked PolitiFact National separately, and only 2007 and 2017 showed the anti-Democrat bias. The year 2007 counts as a statistical anomaly, we would say. PolitiFact treated the "Pants on Fire" rating as a joke at first.


The chart shows that after 2007 Republicans consistently had more false ratings than Democrats. In 2024 that preference for GOP falsehoods fell just short of the record for 2020. For both years, PolitiFact gave the GOP more than five times the number of false ratings it gave Democrats.

Because Republicans lie more?

Not so fast! Here's where the PoF Bias number shows its value. The PoF Bias number compares the percentages of false statements rated "Pants on Fire" for each party. PolitiFact has never offered an objective means of separating ridiculously false statements from those that are merely false. As the number represents a proportion, it is immune from influence by the sheer number of false ratings. Put another way, it's entirely independent of the Selection Proportion number.

In 2024, PolitiFact was over six times more likely to (subjectively rate a Republican false claim "Pants on Fire" than a false claim from a Democrat. That figure easily eclipsed the old record of 4.58 times more likely set in 2020.

Democrats Lie Less?

Interestingly, the recent year spikes in the PoF Bias Number are not driven by increases in "Pants on Fire" ratings given to Republicans. Those have actually moderated. The driver of the higher bias number stems from PolitiFact having increasing difficulty bringing itself to rate a Democrat "Pants on Fire." In 2010, PolitiFact meted out 31 "Pants on Fire" ratings to Democrats. That number has shrunken pretty steadily over time, with the Democrats setting a new record for PoF avoidance. Only one false Democrat claim received a "Pants on Fire" rating, just 4 percent of the total false ratings.




PolitiFact encountered a similar distaste for giving Democrats false ratings of any kind. "False" and "Pants on Fire" combined fell from a peak of 135 in 2012 to 25 in 2024. That figure was the lowest for any presidential election year over PolitiFact's entire history.

Republicans Lie More and Less?

Probably thanks to social media dollars drawing fact checkers away from politicians and toward fake news and social media hijinx, PolitiFact is finding fewer false claims from Republicans. No! I'm not kidding.



Check the presidential election year peaks.

2008 : 55 (PolitiFact's infancy)
2012: 247 (Good ol' Romney)
2016: 237 (Dawn of the Trump)
2020: 253 (Day of the Trump)
2024: 132 (Return of the Trump)

PolitiFact barely gave the GOP half the false ratings it did in 2020. When former PolitiFact editor Bill Adair runs around in support of his new book decrying an explosion in political falsehoods, what is he talking about? PolitiFact has apparently cut Democrat falsehoods down to almost nothing and cut Republican falsehoods nearly in half.

No, of course we don't believe that. A fool would believe that. We say the "Truth-O-Meter" numbers give us information about PolitiFact, not about the figures on whom they stand in judgment.