The "Pants on Fire" bias study updated through 2023

 We have updated our "Pants on Fire" bias study with data from 2023.

What is it? We use a spreadsheet to track all "False" and "Pants on Fire" ratings given to partisan Republicans or Democrats whether candidate, officeholder or appointed administration official plus party officials or organizations. We then calculate the percentage of false ("False" plus "Pants on Fire") ratings given the "Pants on Fire" rating.

Why do we do it? Because PolitiFact has never offered an objective means of distinguishing its "False" rating from its "Pants on Fire" rating, we infer that the difference is either substantially or wholly subjective. Assuming the substantial subjectivity of the ratings, we expect that differences in the percentages will help identify PolitiFact's partisan bias, if any.

Here's the updated chart:

What have we learned so far?

We've learned that national PolitiFact after 2007 shows a consistent bias for Democrats/against Republicans. That trend shows poorly on the graph above because this graph includes ratings from PolitiFact's various state operations. Before PolitiFact changed its website making it far less clear which franchise was responsible for what, we kept track of each part of the organization separately. The years from 2010 through 2015 show a moderation of bias thanks to state operations that sometimes were legitimately tough on Democrats. PolitiFact Wisconsin was notably tough on Democrats during that period, for example.

By looking at the total number of various ratings given to the political parties, we've also noted that Republicans (after 2007) receive far more of PolitiFact's bottom two ratings. That effect may stem from Republicans lying more or simply because of bias in story selection and ratings. We've documented enough of the latter two factors to reasonably prefer the second option. That's where the evidence leads.

If, as the available evidence suggests, PolitiFact's "Pants on Fire" rating has no objective basis, "Republicans lie more" carries no objective explanatory value respecting the percentages on our graph.

We've also learned that harsh ratings for both parties are on the decline, in terms of raw numbers. The most obvious explanation for that trend stems from PolitiFact's social media partnerships. If PolitiFact fact checks a politician, revenue consists of donations, grants and ad revenue. But if PolitiFact fact checks something for its social media partners, there's a payday for that. PolitiFact discloses that more than 5 percent of its revenue comes from the social media company Meta. The Chinese social media company TikTok likewise accounts for over 5 percent of PolitiFact's revenue.  

Why doesn't PolitiFact offer more transparency than that regarding its income? Good question, but we don't have an answer free of conjecture.

As for our study of PolitiFact's numbers in 2023, the Republican average fell well below its historic norm, establishing an all-time low for the GOP. PolitiFact's ratings of Democrats pulled their historic average down for the eighth straight year.

A potential weird Trump effect?

The percentages for Republicans haven't really changed much over the years, defying the existence of any Trump effect in terms of increasing Republican dishonesty (in PolitiFact's data, anyway). But the percentages for Democrats have declined noticeably since around 2016 as Trump ascended politically.

Could Trump help explain an increase in Democratic Party honesty?

More likely those changes happen because the makeup of PolitiFact's franchises has shifted over time. State franchises no longer take the edge off the pro-Democrat bias of national PolitiFact.