Wednesday, December 24, 2014

PolitiFact editor explains the difference between "False" and "Pants on Fire"

During an interview for a  "DeCodeDC" podcast, PolitiFact editor Angie Drobnic Holan explained to listeners the difference between the Truth-O-Meter ratings "False" and "Pants on Fire":

Our transcript of the relevant portion of the podcast follows, picking up with the host asking why President Barack Obama's denial of a change of position on immigration wasn't rated more harshly (bold emphasis added):
Why wouldn't that be "Pants on Fire," for example?

You know, that's an interesting question.

We have definitions for all of our ratings. The definition for "False" is the statement is not accurate. The definition for "Pants on Fire" is the statement is not accurate and makes a ridiculous claim. So, we have a vote by the editors and the line between "False" and "Pants on Fire" is just, you know, sometimes we decide one way and sometimes decide the other. And we totally understand when readers might disagree and say "You rated that 'Pants on Fire.' It should only be 'False.'" Or "You rated that 'False." Why isn't it 'Pants on Fire'?" Those are the kinds of discussions we have every day ...
One branch of our research examines how PolitiFact differentially applies its "Pants on Fire" definition to false statements by the ideology of the subject. Holan's description accords with other statements from PolitiFact regarding the criteria used to distinguish between "False" and "Pants on Fire."

Taking PolitiFact at its word, we concluded that the line of demarcation between the two ratings is essentially subjective. Our data show that PolitiFact National is over 70 percent more likely to give a Republican's false statement a "Pants on Fire" rating than a Democrat's false statement.

We don't necessarily agree with PolitiFact's determinations of what is true or false, of course. What's important to our research is that the PolitiFact editors doing the voting believe it.

Holan's statement helps further confirm our hypothesis regarding the subjective line of demarcation between "False" and "Pants on Fire."

We'll soon publish an update of our research, covering 2014 and updating cumulative totals.


  1. Now I understand the difference between false and pants on fire.

    But this opens for me a new level:
    There should be the ultimate level for repeated outrageous lies which have a serious impact on the people.

    And to give this level a lot of weight, there should be maybe every 3 month a tv show in which the politicians get confronted with this repeated outrageous lie. In this show get the speaker of this person called and asked what he say to it.

    It would be interesting to see if he still tell this lie after he got direct confronted with it.

  2. Frank Meier wrote:

    **Now I understand the difference between false and pants on fire.**

    You do? Could you express it in your own words?

    **There should be the ultimate level for repeated outrageous lies which have a serious impact on the people.**

    Frank, adding 494 more categories to the "Truth-O-Meter" scale will not make it any less subjective *unless PolitiFact connects the ratings to objective benchmarks of some sort.* And then actually uses the benchmarks consistently.

    I hope you're not missing that point.

  3. False is just a wrong information in a statement.
    Pants on Fire is deliberate telling a lie. It is a much higher level of false information in statements to the people.

    Is this correct?

    But as I said, Pants on Fire does not tell that he is doing this continuously.

    1. Frank, PolitiFact defines "False" as a statement that is not true. PolitiFact defines "Pants on Fire" as a statement that is not true and is also ridiculous. PolitiFact denies that it says statements are lies (deliberate deception).

      Google "principles of punditfact" (using "punditfact" instead of "politifact" will take you to PolitiFact's most recent statement of principle.

      If you don't know of any way to distinguish objectively between a false statement and a ridiculously false statement, neither do we. We believe that the line of demarcation is subjective, and this is supported by the statements of PolitiFact editors.

  4. Bryan:

    You say "subjective"?

    I think that many true or false statements can be OBJECTIVE valued.

    If somebody says "I am a woman" but the person is a man ... and this person gets a Pants on Fire for this ice cold lie, this classification is not subjective.

    The areas between this easy TRUTH or LIE statements are subjective.

    And I still think that is interesting to get the information that the lying person use a "Pants-On-Fire" statement not only one time, but as a weapon and even after he got caught that it is a lie.

    I hope that you understand what I mean

    Your comment sounds like that you downgrade this "Truth-O-Meter".

    1. Frank Meier wrote:

      **I think that many true or false statements can be OBJECTIVE valued.**

      I think so, too. But PolitiFact's "Truth-O-Meter" is of little use for that purpose. The definitions are too vague, and in any case PolitiFact appears to pay them little mind. PolitiFact's founding editor Bill Adair recently said the "Truth-O-Meter" ratings are subjective. Is he a liar?

      **I hope that you understand what I mean**

      I think I do. But you have to understand it's shocking to me that somebody would propose something like a useful feature of the "Truth-O-Meter" and actually expect PolitiFact to use it according to consistent principle. ;-)

      **Your comment sounds like that you downgrade this "Truth-O-Meter".**

      Yes, I think PolitiFact's "Truth-O-Meter" is rubbish. It was poorly designed in the first place (vague definitions) and PolitiFact journalists don't seem to know the first thing about applying principles consistently. Indeed, sometimes they appear to not care, as when they unblushingly admit the ratings are subjective and then proceed to often publish "candidate report cards" to guide voters in their support of a candidate. It seems dishonest or at least unethical to me.


Thanks to commenters who refuse to honor various requests from the blog administrators, all comments are now moderated. Pseudonymous commenters who do not choose distinctive pseudonyms will not be published, period. No "Anonymous." No "Unknown." Etc.