Why does PolitiFact claim it allows license for hyperbole?

It makes sense for a fact checker to account for figures of speech such as hyperbole. And PolitiFact plainly states in its description of its principles that it allows license for hyperbole (bold emphasis added):

In deciding which statements to check, we consider these questions:

• Is the statement rooted in a fact that is verifiable? We don’t check opinions, and we recognize that in the world of speechmaking and political rhetoric, there is license for hyperbole.

For us, the mystery is why PolitiFact makes this claim and then blatantly fails to honor it.

Today we add another example to the collection.

The $38 Burrito?

Turning Point USA commentator Jordan Rachel tweeted about the $15 minimum wage:

Rachel's tweet fits perfectly the form of hyperbole. She chose an extremely high price for the future burrito and took care to offer no clear sign the statement was intended to be taken literally. One would encourage literal interpretation by plainly stating something like "Increasing the minimum wage to $15 will result in $38 burritos."

PolitiFact's graphic presentation of its fact check misleadingly frames Rachel's statement as exactly the latter type of claim:

PolitiFact unfairly manipulates Rachel's claim. Her claim is an if/then statement emphasizing the truth that the $15 minimum wage increase counts as an inflationary policy.

Rachel gets no credit for that point from PolitiFact. We don't know precisely why she gets no credit for that point, but we can at least say with certainty that PolitiFact claim that it allows license for hyperbole rings false.

Hyperbole is not mere exaggeration

As a figure of speech, hyperbole is not the same thing as mere exaggeration. If it was, then fact checkers shouldn't need to take it into account. They could simply show in factual terms how far off the exaggeration fell from the truth and leave it at that.

Hyperbole counts as common figure of speech. Perhaps the modern journalist simply understands it poorly. In the interest of public education, we recommend to our readers the account of hyperbole at YourDictionary.com:

Hyperbole, from a Greek word meaning “excess,” is a figure of speech that uses extreme exaggeration to make a point or show emphasis. It is the opposite of understatement.

You can find examples of hyperbole in literature and everyday speech. You wouldn’t want to use it in nonfiction works, like reports or research papers, but it’s perfect for creative writing and communication, especially when you want to add color to a character or humor to a story.

We encourage readers to click the link and read through it all.

No Sign PolitiFact Considered the Possibility of Hyperbole

As for the fact checkers at PolitiFact, we see no sign at all they considered interpreting Rachel's statement as hyperbole.

The summary section of the fact check fairly displays the rush to judgment:

Our ruling

Rachel said a $15-an-hour minimum wage would raise the price of a Taco Bell burrito to $38.

This claim is countered by available evidence, including the current burrito prices at Taco Bell locations in cities and counties where a $15 minimum wage is in effect. Four economists characterized the claim as a far-off estimate at odds with economic theory.

We rate Rachel’s statement False.

 Misleading and disgraceful.

We expect a conscientious fact-checking organization to make a plausible attempt to achieve consistency between policy and practice. If the policy is A and the practice is B, change one of them to match the other.

In claiming to allow license for hyperbole and failing to allow license for hyperbole, PolitiFact deceives its readers.

Remember back when it was False to say Nixon was impeached?

I remember reading a story years back about a tire company that enterprisingly tried to drum up business by sending out a team to spread roofing nails on the local roads.

Turns out there's a version of that technique in PolitiFact's fact-checking tool box.

Nixon was Never Impeached

Back on June 13th, 2019 PolitiFact's PunditFact declared it "False" that Nixon was impeached. PunditFact said "Nixon was never officially impeached." We're not sure what would count as "unofficially impeached." We're pretty sure it's the same as saying Nixon was not impeached.

But that was way back in June. Over three months have passed. And it's now sufficiently true that Nixon was impeached so that PolitiFact can spread the idea on Twitter and write an impeachment PolitiSplainer that refers multiple times to the Nixon impeachment.

Nixon was Impeached

Twitter

Johnson, Nixon, Clinton, Trump: We answer nine questions about impeachment. https://t.co/fzgdYA0iqE pic.twitter.com/MLh6cODpuN

— PolitiFact (@PolitiFact) October 12, 2019

Edit: (if embed isn't working use hotlink above)

Is Nixon a good example to include with Johnson and Clinton (let alone Trump) if Nixon wasn't impeached?

More than anything, the procedural details are derived from historical precedent, from the impeachment of President Andrew Johnson in the 1860s to that of President Richard Nixon in the 1970s and President Bill Clinton in the 1990s.

Got it? The impeachment of President Nixon. Because Nixon was impeached, right?

Experts pointed to a variety of differences between the Trump impeachment process and those that went before.

The differences begin with the substance of the charges. All prior presidential impeachments have concerned domestic issues — the aftermath of the Civil War in Johnson’s case, the Watergate burglary and coverup under Nixon, and the Monica Lewinsky affair for Clinton.

Got it? Nixon was impeached over the Watergate burglary. Because Nixon was impeached, right?

The impeachments of both Nixon and Clinton did tend to curb legislative action by soaking up all the attention in Washington, historians say.

Obviously a fact-checker will not refer to "the impeachments of both Nixon and Clinton" if Nixon was not impeached. Therefore, Nixon was impeached. Right?

Some congressional Republicans have openly supported Trump’s assertion that the allegations against Trump are dubious. This contrasts with the Nixon impeachment, when "on both sides there was a pretty universal acknowledgement that the charges being investigated were very important and that it was necessary to get to the bottom of what happened," said Frank O. Bowman III, a University of Missouri law professor and author of the book, "High Crimes and Misdemeanors: A History of Impeachment for the Age of Trump."

Obviously a fact-checker will only draw a parallel to the Nixon impeachment if Nixon was impeached. Therefore Nixon was impeached. Right?

Trump is facing possible impeachment about a year before running for reelection. By contrast, both Nixon and Clinton had already won second terms when they were impeached. (Johnson was such an outcast within his own party that he would have been an extreme longshot to win renomination, historians say.)

Got it? Nixon and Clinton had already won second terms when they were impeached. Because Nixon was impeached, right?

On the eve of impeachment for both Nixon and Clinton, popular support for impeachment was weak — 38% for Nixon and 29% for Clinton, according to a recent Axios analysis. (There was no public opinion polling when Johnson was president.)

Got it? "On the eve of impeachment for both Nixon and Clinton," because a fact checker doesn't refer to the eve of the Nixon impeachment if there was no Nixon impeachment.

Is there a Christmas Eve if there's no Christmas?

That's six times PolitiFact referred to the Nixon impeachment in just one PolitiSplainer article. And about three months after PolitiFact's PunditFact said Nixon was not impeached.

Want a seventh? We've got a seventh:

During Nixon’s impeachment, "people counted on the media to serve as arbiters of truth," he said. "Obviously, we don’t have that now."

 "During Nixon's impeachment" directly implies Nixon was impeached. Seven.

We've been going in order, too.

(Nixon Wasn't Impeached)

But behold! Context at last!

The uncertainty about Senate process stems from the rarity of the process. Nixon resigned before the House could vote to send articles to the Senate, leaving just one precedent -- Clinton’s trial — in the past century and a half.

Admittedly, that's not PolitiFact saying "Nixon was not impeached." On the other hand, it's PolitiFact directly implying Nixon was not impeached. Blink and you might miss it amidst all the talk about the Nixon impeachment.

Can we get to eight after that bothersome bit of context?

Nixon was Impeached, Continued 

We can:

The impeachments of both Nixon and Clinton did tend to curb legislative action by soaking up all the attention in Washington, historians say.

We're curious which historians PolitiFact talked to who explicitly referred to the impeachment of Nixon. There are no quotations in the text of the PolitiSplainer that would support this claim about what historians say.

PolitiFact flirted with nine in the next paragraph. We're capping the count at eight.

In summary, we'll just say this: If there's a sense of "impeachment" that doesn't mean literally getting impeached by Congress and standing trial in the Senate, then Jimmy Kimmel is entitled to that understanding when he says Nixon was the last president to be impeached.

Contrary to PolitiFact's framing, Kimmel was wrong not because Nixon was not impeached. Kimmel was wrong because President William J. Clinton was the last president to be impeached. There was never any need for PunditFact to focus on the fact Nixon wasn't impeached, unless it was to avoid emphasis on Clinton.

This all works out very well for PolitiFact. PolitiFact does what it can to spread the misperception Nixon was impeached. And then it can draw clicks to its PunditFact fact check showing that claim false.

Just like dropping roofing nails on the road.

PolitiFact misses obvious evidence in Broward recount fact check

On Nov. 13, 2018 PolitiFact's "PunditFact" brand issued a "Pants on Fire" rating to conservative Ken Blackwell for claiming Democrats and their allies were manufacturing voters in the Florida election recount.

The problem?

PolitiFact somehow overlooked obvious evidence reported in the mainstream media. The Tampa Bay Times, former owner of PolitiFact before it was transferred to the nonprofit Poynter Institute, published a version of the story:

Broward's elections supervisor accidentally mixed more than a dozen rejected ballots with nearly 200 valid ones, a circumstance that is unlikely to help Brenda Snipes push back against Republican allegations of incompetence.

The mistake — for which no one had a solution Friday night — was discovered after Snipes agreed to present 205 provisional ballots to the Broward County canvassing board for inspection. She had initially intended to handle the ballots administratively, but agreed to present them to the canvassing board after Republican attorneys objected.

The Times story says counting the 205 provisional ballots resulted in at least 20 illegal votes ending up in Broward County's vote totals.

The Times published its story on Nov. 10, 2018.

PolitiFact/PunditFact published its fact check on Nov. 13, 2018 (2:24 p.m. time stamp). The fact check contains no mention at all that Broward County included invalid votes in its vote totals.

Instead, PolitiFact reporter John Kruzel gives us the breezy assurance that neither he nor the state found evidence supporting Blackwell's charge.

Our ruling

Blackwell said, "Democrats and their allies (...) are manufacturing voters."

We found no evidence, nor has the state, to support this claim. Blackwell provided no evidence to support his statement.

We rate this Pants on Fire.

Inconceivable, you say?



via GIPHY

Another partisan rating from bipartisan PolitiFact

"We call out both sides."

That is the assurance that PolitiFact gives its readers to communicate to them that it rates statements impartially.

We've pointed out before, and we will doubtless repeat it in the future, that rating both sides serves as no guarantee of impartiality if the grades skew left whether rating a Republican or a Democrat.

On December 1, 2017, PolitiFact New York looked at Albany Mayor Kathy M. Sheehan's claim that simply living in the United States without documentation is not a crime. PolitiFact rated the statement "Mostly True."

PolitiFact explained that while living illegally in the United States carries civil penalties, it does not count as a criminal act. So, "Mostly True."

Something about this case reminded us of one from earlier in 2017.

On May 31, 2017, PolitiFact's PunditFact looked at Fox News host Gregg Jarrett's claim that collusion is not a crime. PolitiFact rated the statement "False."

These cases prove very similar, not counting the ratings, upon examination.

Sheehan defended Albany's sanctuary designation by suggesting that law enforcement need not look at immigration status because illegal presence in the United States is not a crime.

And though PolitiFact apparently didn't notice, Jarrett made the point that Special Counsel Mueller was put in charge of investigating non-criminal activity (collusion). Special Counsels are typically appointed to investigate crimes, not to investigate to find out if a crime was committed.

On the one hand, Albany police might ask a driver for proof of immigration status. The lack of documentation might lead to the discovery of criminal acts such as entering the country illegally or falsifying government documents.

On the other hand, the Mueller investigation might investigate the relationship (collusion) between the Trump campaign and Russian operatives and find a conspiracy to commit a crime. Conspiring to commit a crime counts as a criminal act.

Sheehan and Jarrett were making essentially the same point, though collusion by itself doesn't even carry a civil penalty like undocumented immigrant status does.

So there's PolitiFact calling out both sides. Sheehan and Jarrett make almost the same point. Sheehan gets a "Mostly True" rating. Jarrett gets a "False."

That's the kind of non-partisanship you get when liberal bloggers do fact-checking.



Afters

Just to hammer home the point that Jarrett was right, we will review the damning testimony of the  three impartial experts who helped PunditFact reach the conclusion that Jarrett was wrong.

Nathaniel Persily at Stanford University Law School said one relevant statute is the Bipartisan Campaign Reform Act of 2002.

"A foreign national spending money to influence a federal election can be a crime," Persily said. "And if a U.S. citizen coordinates, conspires or assists in that spending, then it could be a crime."

The conspiracy to commit the crime, not the mere collusion, counts as the crime.

Next:

Another election law specialist, John Coates at Harvard University Law School, said if Russians aimed to shape the outcome of the presidential election, that would meet the definition of an expenditure.

"The related funds could also be viewed as an illegal contribution to any candidate who coordinates (colludes) with the foreign speaker," Coates said.

Conspiring to collect illegal contributions, not mere collusion, would count as the crime. Coats also offered the example of conspiring to commit fraud.

Josh Douglas at the University of Kentucky Law School offered two other possible relevant statutes.

"Collusion in a federal election with a foreign entity could potentially fall under other crimes, such as against public corruption," Douglas said. "There's also a general anti-coercion federal election law."

The corruption, not the mere collusion, would count as the crime.

How PolitiFact missed Jarrett's point after linking the article he wrote explaining what he meant is far beyond us.

Hewitt v. PolitiFact: Two facts clearly favor Hewitt

Over the past week, conservative radio host Hugh Hewitt claimed on Sunday the health insurance industry has entered a "death spiral," PolitiFact rated Hewitt's claim "False" and Hewitt had PolitiFact Executive Director Aaron Sharockman on his radio show for an hour long interview (transcript here).

Aside from the central dispute over the "death spiral," where PolitiFact's work arguably commits a bifurcation fallacy, we have identified two areas where Hewitt has the right of the argument. PolitiFact has published (and superficially defended) false statements in both areas.

The New York Times article

PunditFact (PolitiFact):

Hewitt referred to a New York Times article that quotes the president of Aetna saying that in many places people will lose health care insurance.

We couldn’t find that article, but a simple remark on how premiums are rising and insurers are leaving the marketplace is not enough evidence to meet the actuarial definition of a death spiral.

We found the article in just a few minutes (it was dead easy; see hit No. 5). Two quotations from it will show it matches the content, other than the term "president," that Hewitt described on his Sunday television appearance.

One:

Aetna CEO Mark Bertolini has pronounced the ACA's health insurance markets in a "death spiral."

Two:

___

This story has been corrected to show that consumers have reduced options, not that some consumers have no health care options.

So we have the "death spiral" comment from the head of AETNA that Hewitt described, as well as the dire statement that some people have no options, though that part was reported in error in the AP story that appeared in the Times.

During his radio interview, Sharockman tried to pin on Hewitt PolitiFact's failure to find the described Times story and flatly said the article was not in the Times:

AS: You said the president of Aetna. It’s the chairman and CEO, and it was not in the New York Times, as you also know. It was originally probably in the Wall Street Journal.

The article was in The New York Times, and we have informed PolitiFact writer Allison Graves (via Twitter) and Sharockman (via email).

We expect ethical journalists to make appropriate corrections.

Where does the CBO stand on the "death spiral"?

The mainstream media widely interpreted the CBO report addressing President Trump's health care proposal as a judgment the ACA has not entered a "death spiral."

PolitiFact did likewise in its fact check of Hewitt:

CBO, independent analysis: No death spiral
Others have also concluded that the Affordable Care Act is not in a death spiral. The nonpartisan Congressional Budget Office, as part of its recent analysis of the GOP legislation, described the Affordable Care Act as stable.

Though PolitiFact did not link to the CBO report in its fact check (contrary to PolitiFact's statement of standards), we believe the claim traces to this CBO report, which contains this assessment (bold emphasis added):

Stability of the Health Insurance Market

Decisions about offering and purchasing health insurance depend on the stability of the health insurance market—that is, on having insurers participating in most areas of the country and on the likelihood of premiums’ not rising in an unsustainable spiral. The market for insurance purchased individually (that is, nongroup coverage) would be unstable, for example, if the people who wanted to buy coverage at any offered price would have average health care expenditures so high that offering the insurance would be unprofitable. In CBO and JCT’s assessment, however, the nongroup market would probably be stable in most areas under either current law or the legislation.

Note the CBO report does not call the insurance market "stable" under the ACA. Instead it projects that insurance markets will probably remain stable in most areas. Assuming PolitiFact has no better support from the CBO than the portion we have quoted, we find PolitiFact's version a perversion of the original. The CBO statement leaves open the possibility of a death spiral.

Sharockman stood behind the fact check's apparent spin during his appearance on the Hugh Hewitt Show:

AS: Hugh, you’re misleading the listeners.

HH: …is that we have gone from 7…

AS: You’re misleading the listeners. The same CBO report that you’re quoting said that the markets are stable whether it’s the AHCA…

Again, unless Sharockman has some version of a CBO report different from what we have found, we judge that Sharockman and PolitiFact are misleading people about the content of the report.

We used email to point out the discrepancy to Sharockman and asked him to provide support for his and PolitiFact's interpretation of the CBO report.

We will update this article if we receive a response from Sharockman that includes such evidence.

NTSH: 95 percent of Clinton's claims "Mostly True" or better?

We tip our hats to Power Line blog for making it easy to add a "Nothing To See Here" item.

With "Nothing To See Here" we take note of political statements deserving of a fact check. But we tend to doubt one will occur. Power Line blog noted a problem with a Nicholas Kristoff column in The New York Times. Kristoff, a liberal columnist, wrote a column highlighting Clinton's position head-and-shoulders above the competition when it comes to PolitiFact report cards. But there was a problem: Kristoff got the key numbers wrong.

Power Line's Steven Hayward compared the original version of Kristoff's column with the Times' later correction of the article.

At the bottom of the column is this short correction:

Correction: April 23, 2016: An earlier version of this column misstated some of the percentages of true statements as judged by PolitiFact.

So how did the original version of Kristof’s column read? Here:

PolitiFact, the Pulitzer Prize winning fact checking site, calculates that of the Clinton statements it has examined, 95 percent are either true or mostly true.

That’s more than twice as high as the percentages for any of the other candidates, with 46 percent for Bernie Sanders’s, 12 percent for Trump’s, 23 percent for Ted Cruz’s and 33 percent for John Kasich’s. Here we have a rare metric of integrity among candidates, and it suggest that contrary to popular impressions, Clinton is far more honest and trustworthy than her peers.

So we go from 95 percent true to 50 percent true and switch out “far more honest and trustworthy than her peers” for “relatively honest by politician standards,” with the blink of a mere correction.

We've repeatedly noted PolitiFact's weak-to-nonexistent efforts to police the misuse of its "report card" data. If PunditFact and PolitiFact let Kristoff slide on this one, what else are they willing to overlook?

The Unbeliebable Hilarity of Abortion

If you're the kind of prude that can't joke about abortion, then PunditFact isn't the nonpartisan fact-checker for you.

Recently PunditFacter C. Eugene Emery Jr. wondered if Sarah Palin made a true claim about the abortion stances of Donald Trump and Justin Bieber. Palin claimed that both Trump and Bieber held "pro-choice" positions in the past but now disavow those positions. The post originally caught my attention because I couldn't figure out what exactly Emery Jr. was fact checking. The closest we get to an explanation is this:

I'm not sure if the term "conversion" is intentional snark directed at religious believers or if it's just an inapt choice of words, but in any event the sentence is vague. Is PundiFact simply trying to verify Bieber was formerly pro-choice and is now opposed to abortion? Are they determining if Bieber came to his new position via an affirmation of faith or if his new public stance is a matter of political convenience? Without knowing specifically what Emery Jr. is checking we can't determine if the evidence supports his ruling or if it's even a verifiable fact to check. [Note: I used a screengrab for PunditFact's sentence instead of quoting the text for reasons discussed below.]

If ambiguity was the only problem with the piece I would have probably ignored it. Unfortunately for his readers, Emery Jr. chose to use the serious topic of abortion to display his comedic talents. Note the opening sentence of the post:

What do Donald Trump and Justin Bieber have in common?
Humongous hair?
Big bucks?
How about a case of abortion contortion?

Ha! Get it? Abortion contortion! ZING!

And that's just the beginning of the yuk-yuk tone throughout the article. Emery Jr's entire post is peppered with a repeat gag wherein he uses a specific word that is hyperlinked to a Justin Bieber video. For example, when Emery Jr. uses the word "sorry," the text links to Justin Bieber's music video for the song "Sorry." Emery Jr. seems to think this spoof is so hilarious he repeats it with the word "believe," and the phrases "never say never," "No Pressure," and "What do you mean."

Whatever one's views on abortion, it is not a funny topic. At the very least it involves a complicated decision and often an invasive surgery for a woman. For those who subscribe to the notion that life begins somewhere between conception and birth, abortion can quite sincerely be viewed as the death of a viable human being.

It's beyond the scope and purpose of this website to weigh in on the moral, scientific, and philosophical debate surrounding abortion. But anyone holding conventionally accepted sensibilities and decency recognizes that abortion is not a subject ripe for clever barbs and smartassery. 

Eugene Emery Jr. and his editor, Katie Sanders, betray a woeful lack of both seriousness and good judgement when they yuk it up over such a solemn issue. Further, their humor displays a comfort level with mockery that we think a right-leaning voice in PunditFact's newsroom would disapprove of.

It seems that for PunditFact, abortion jokes aren't just acceptable watercooler chat, they're worthy of publishing.

PunditFact's poor choices don't end there. In the screengrab above readers will note that the word "overboard" is highlighted indicating a hyperlink. The link leads to a YouTube version of Bieber's song "Overboard."  We chose not to link to it ourselves because unlike all the other instances where PunditFact links to the official Justin Bieber Vevo account, Emery Jr. links to a pirated version of the "Overboard" song, presumably unauthorized by Bieber or his label.

In case readers had any doubt about PunditFact's ethics, note that making a joke in an abortion article was so important to them they chose to promote music piracy to make the gag work when they couldn't find an official version offered by the artist. (Perhaps this was accidental, but that would only highlight their shoddy editing standards.)

PunditFact's journalistic clowns had another embarrassing trick up their sleeve that my co-editor Bryan pointed out. Here's how PunditFact laid out some of its evidence:

The Chicago Sun-Times said in a Jan. 21, 2013, story that it had spoken with an unnamed "longtime Bieber associate" who claimed, "I don't believe he agrees with his mom on this issue."

Some might interpret that to mean that Bieber is now pro-abortion — which would signal a new position on abortion that, sorry, does not fit Palin’s point. However, PolitiFact has a policy of not relying on unnamed or secondhand sources to speak for what others believe.

In case you missed the subtlety, PunditFact includes a source that they immediately disavow since it goes against PolitiFact's policy. If there's a policy against using unnamed sources then why include an unnamed source in your piece? This is just another example in a long list of cases where the PolitiFact enterprise ignores its own policies.

Finally, we get to the rating. Unsurprisingly, Palin gets a "False" rating but it's the way they arrived at that conclusion that is so offensive to those of us that care about the truth:

At PolitiFact, we believe it's the responsibility of the person making a claim to provide the evidence to back it up...Palin’s claim is not substantiated.

In PunditFact's world, absence of evidence is evidence of absence. Of course, PolitiFact often shirks its own burden of proof.

Perhaps it's a stretch to infer any ideological leanings from these PunditFacters behaving so daftly while writing about abortion. But beyond whatever bias it may or may not expose, Emery Jr's piece showcases a jaw-dropping level of poor taste and substandard journalism.

C. Eugene Emery Jr. is funny, but he's only funny by mistake.

The 2015 "Pants on Fire" bias for PunditFact and the PolitiFact states

Earlier this week we published our 2015 update to our study of PolitiFact's bias in applying its "Pants on Fire" rating.

The premise of the research, briefly, is that no objective criterion distinguishes between a "False" rating and a "Pants on Fire" rating. If the ratings are subjective then a "Pants on Fire" rating provides a measure of opinion and nothing more.

In 2015 the states provided comparatively little data. State franchises, with a few exceptions, seem to have a tough time giving false ratings. The state PolitiFact operations also tend to vary widely in the measurement of the "Pants on Fire" bias. PolitiFact Wisconsin's "Pants on Fire" ratings proportionally treat Democrats more harshly than Republicans, for example.

PolitiFact Florida: PolitiFact Florida's data roughly matched those from PolitiFact National and PunditFact. Those three franchises are the most closely associated with one another since all are based in Florida and tend to share writing and editorial staff. In 2015 the PoF bias number was within a range of five hundredths for each. That's so close that it's suspicious on its face. All three gave Republicans more false ratings than Democrats (4.83, 3.50, 2.43).

PolitiFact Georgia: Though PolitiFact Georgia has operated for a good number of years and has in the past provided us with useful data, that wasn't the case in 2015. PolitiFact Georgia's false ratings went to apparently non-partisan claims.

PolitiFact New Hampshire: PolitiFact New Hampshire historically provides virtually nothing helpful in terms of the PoF bias number. But false ratings for Democrats outnumbered false ratings for Republicans (blue numerals indicate that bias).

PolitiFact Rhode Island: PF Rhode Island rated two statements from Democrats "False."

PolitiFact Texas: PF Texas gave Republicans false ratings an astonishing 8 times more than Democrats. But at the same time, PF Texas produced a PoF bias number harming Democrats. The key to both figures? PF Texas only doled out two false ratings to partisan Democrats. Both were "Pants on Fire" ratings. An entire year with no "False" ratings for Democrats? Texas' previous low for "False" ratings was four (twice).

PolitiFact Virginia: PF Virginia achieved perfect neutrality in terms of our PoF bias number. That's the meaning of a 1.00 score. The "Pants on Fire" claims as a percentage of all false claims was equal for Republicans and Democrats.

PolitiFact Wisconsin: PF Wisconsin continued its trend of giving Democrats the short end of the PoF bias measure. That's despite giving Republicans a bigger share than usual of the total false ratings. The 5.00 selection bias number was easily the all-time high for PF Wisconsin, besting the old mark of 1.57 back in 2011.

PunditFact: PunditFact, we should note, produces data we class in our "Group B." PunditFact tends not to rate partisan candidates, officeholders, partisan appointees or party organizations. It focuses more on pundits, as the name implies. We consider group B data less reliable as a measure of partisan bias than the group A data. But we do find it interesting that PunditFact's data profile lines up pretty closely with the most closely associated PolitiFact entities, as noted above. That finding proves consistent with the idea that PolitiFact ratings say something about the viewpoint of the ones giving the ratings.

PunditFact's editorial page

Oops--are we being redundant? Sorry! Here's a portion of PunditFact's main page from Dec. 28, 2015.

Partial screen capture from PolitiFact.com/PunditFact/ Dec. 18, 2015

That, ladies and gentlemen, is a pictorial editorial.

It's not an unaltered photograph of Donald Trump. It's not a photo at all. It is an artist's rendering of Donald Trump, created to communicate an editorial message to PunditFact's readers. Note how Trump's fingers are crossed as he speaks, a traditional gesture of those who know they are speaking falsely.

Material like this makes PunditFact itself a pundit of sorts.

We've long noted the fact that PolitiFact and its various franchises blur the traditional line between straight news reporting--what some might expect from journalism billed as "fact-checking"--and editorializing. That's why we call PolitiFact "the Fox News of fact-checking" and PolitiFact staffers "liberal bloggers."

Their journalism is not objective. When PolitiFact creator Bill Adair calls fact checking a "new form of journalism" perhaps he has in mind that deliberate blurring of the lines.

Or not. Either way, we don't appreciate it.

Lauren Carroll cannot contain herself

PolitiFact/PunditFact writer Lauren Carroll couldn't resist pushing back against criticism she received on her story looking at the containment of ISIS.

Carroll suggested on Twitter that Breitbart.com's John Nolte had not read her fact check. The evidence?

1) I am the only byline on the story 2) I fact-checked Ben Rhodes, not Obama. @NolteNC— Lauren Carroll (@LaurenFCarroll) November 16, 2015

The fact is that PunditFact gave Carroll's story more than one presentation.

In one of the presentations, a version of Carroll's story was combined with another story from a Sunday morning news show. That second version of the story has Linda Qiu listed on the byline. So Carroll's claim she's the only one on the byline rates a "Half True" on the Hack-O-Meter. Combined with her whinge about fact-checking Obama proxy (deputy national security advisor) Ben Rhodes instead of President Obama, Carroll provides an astonishingly thin defense of her work.

The critiques from Breitbart.com and the Washington Examiner both made the point that Obama was answering a question about ISIS' strength, not the range of its geographical control. Carroll completely accepted Rhodes' spin and ignored the point of the question Obama was asked.

Where's Carroll's explanation of her central error? It's certainly not in her clumsy jabs at John Nolte.

ISIS "contained"?

When President Obama called ISIS ("ISIL") "contained" in a televised interview on Nov. 12, 2015, other politicians, including at least one Democrat, gave him some grief over the statement.

Mainstream fact checker PunditFact came to the president's defense. PunditFact said Obama was just talking about territorial expansion, so what he said was correct.

Conservative media objected.

John Nolte from Breitbart.com:

PolitiFact’s transparent sleight-of-hand comes from basing its “True” rating — not on the question Obama is asked — but how the President chose to answer it.

Stephanopoulos asks, “But ISIS is gaining strength aren’t they?”

T. Becket Adams from the Washington Examiner:

PunditFact has rated the Obama administration's claim that the Islamic State has been "contained" as "true," even after a recent series of ISIS-sponsored events around the world have claimed the lives of hundreds of civilians.

For the fact-checker, the White House's doesn't believe ISIS is no longer a global threat, as fatal attacks last week in Beirut and Paris would show. The president and his team merely believe that the insurgent terrorist group controls a smaller portion of the Middle East today than it did a few months ago.

We think PunditFact has a bit of a point when it claims the president's remarks are taken out of context. But as Nolte and Adams point out, the specific context of the Obama interview was the strength of ISIS, not its territorial expansion.

If the president was saying that containing ISIL's geographic control equates with containing its strength, then PunditFact ends up taking the president out of context to justify claiming the president was taken out of context.

There's something not quite right about that.


Clarification Dec. 10, 2015: Changed "wasn't" to "was" in the next-to-last paragraph

PunditFact's PolitiMath on the GDP of 29 countries

We do "PolitiFact" stories to examine how PolitiFact's ratings correlate to percentage error. Claims where the ratings seem based purely or mainly on the degree of error serve as the best case studies. PunditFact gives us a great study example with its article on the claim that a boxing match would generate more revenue than the GDP of 29 different countries.

PunditFact ruled that claim "Pants on Fire," finding only six countries with a GDP lower than that predicted for the fight: $400 million.

Jim Lampley's figure of 29 exaggerates PunditFact's total by 383 percent. That substantial error, we suppose, justifies the "Pants on Fire" rating.

On the other hand, PunditFact gave Cokie Roberts a "Half True" rating for a claim she exaggerated by over 9,000 percent. PunditFact gave Roberts credit for her underlying point, that the risk of getting murdered in Honduras is greater than for New York City.

Apparently Lampley has no valid underlying point that the Mayweather-Pacquiao fight would generate a great deal of revenue.

You be the judge.


Update May 3, 2015

While researching and wondering how Lampley ended up with 29 countries producing a GDP under $400 million, we noticed a perhaps-coincidental statistic: The World Bank's 2013 GDP rankings have 29 countries with a GDP above $400 billion.

Lampley's claim may have started with this statistic. After mixing up millions with billions and mistaking the top of the list for the bottom, Lampley's claim makes perfect sense, in a way.


Correction May 4, 2015: Fixed spelling of "Pacquiao."

PunditFact amends pundit's claim about amendments

We've pointed out before how PolitiFact will fault statements made on Twitter for lacking context despite the 140-character limit Twitter imposes.

This week PunditFact played that game with the following tweet from conservative pundit Phil Kerpen:

Amazing fact: Senate has already voted on more amendments in 2015 than Reid allowed ALL YEAR last year.
— Phil Kerpen (@kerpen) January 23, 2015

PunditFact found that the new Republican-controlled Senate has already voted on more amendments in 2015 than Reid allowed in the Democrat-controlled Senate for all of 2014: "On the numbers, that is right."

But PunditFact went on to find fault with Kerpen for leaving out needed context:

On the numbers, that is right. But experts cautioned us that the claim falls more in the interesting factoid category than a sign of a different or more cooperative Senate leadership.

The statement is accurate but needs clarification and additional information. That meets our definition of Mostly True.

We'll spell out the obvious problem with PunditFact's rating: Kerpen's tweet doesn't say anything about different or more cooperative Senate leadership. If Kerpen's not making that argument (we found no evidence he was), then it makes no sense at all to charge him with leaving out information. In effect, PunditFact is amending Kerpen's tweet, giving it context that doesn't exist in the original. Kerpen's statement doesn't need clarification or additional information to qualify as simply "True."

PunditFact's rating offers us a perfect opportunity to point out that if Kerpen's statement isn't simply "True" then there's probably no political claim anywhere that's immune to the type of objection PunditFact used to justify its "Mostly True" rating of Kerpen. A politician could claim the sky is blue and the fact checker could reply that yes, the sky is blue but no thanks to the policies of that politician's party! There are endless ways to rationalize withholding a "True" rating.

This rating convinces us that it would be productive to look at the breakdown between "True" and "Mostly True" ratings to look for a partisan bias. Since there's always context missing from political claims, drawing that line between "True" and "Mostly True" may prove no more objective than the line between "False" and "Pants on Fire."

PunditFact officially less accurate than PolitiFact

Careful research over the last tens of hours has convinced us that PunditFact reports less accurately than PolitiFact.

On Jan 27, 2015, PunditFact posted an article about an update for PunditFact's networks scorecards. PunditFact falsely reported that its scorecards check "all claims made by pundits on air." We noted that announcement with a post that same day, linking to an archived version of the story. But the mistake remains as of this writing.

On Jan. 29, 2015, PolitiFact posted an article covering the same subject matter, coming to identical conclusions but without the claim that the scorecards check all claims made by pundits on air.

We conclude that between PunditFact and PolitiFact, PolitiFact reports more accurately. We've made the judgment easy for our readers with a side-by-side comparison:

Of course we're just having fun with this at PolitiFact's expense. PunditFact is simply a part of PolitiFact, the part that focuses on rating statements from pundits. PunditFact posted the faulty story, then PolitiFact posted the same story two days later with the false statement amended. I guess that way there's no need to publish a correction notice or fix the version of the story containing the error. It's enough to have one version of the story published without the mistake.

Or something.

Bless your heart, PolitiFact.


(Moments after publishing fixed side-by-side image captions, changing two cases of "2014" to "2015")

PunditFact: Our scorecards measure all claims made by pundits on air

Bless your heart, PunditFact.

Some people think PolitiFact's network scorecards can't be taken as a reliable litmus test of network truthfulness. After all, these skeptics say, PunditFact doesn't check every claim by every pundit on the networks.

But today PunditFact broke the news that it does check every claim on the networks (bold emphasis added):

MSNBC and CNN have improved ever so slightly on our TV network scorecards, while Fox News has moved a touch in the opposite direction.

We last looked at our network scorecards, which examine all the claims made by pundits on air, in September. The scorecards measure statements made by a pundit or a host or paid contributor on a particular network. They do not include statements made by elected leaders, declared candidates or party officials.

So that lends a great deal of credibility to PunditFact's scorecards. After all, if the scorecards are examining all the claims then there's no issue with selection bias. Right?

Or not. PunditFact added this later on in the article:

As we have said in the past, be cautious about using the scorecards to draw broad conclusions. We use our news judgment to pick the facts we’re going to check, so we certainly don’t fact-check everything. And we don’t fact-check the five network groups evenly.

Bless your heart, PunditFact.

PunditFact's "Pants on Fire" bias, 2014

We pledged to apply our "Pants on Fire" bias research methods to PolitiFact's "PunditFact" project.

PunditFact is the branch of PolitiFact that looks at and rates statements from pundits. PunditFact uses the same cheesy "Truth-O-Meter" system to which PolitiFact has wedded itself.

Our "Pants on Fire" research project looks at how PolitiFact disproportionately applies its "Pants on Fire" rating.

It's important to note that we don't simply look at the higher numbers of "Pants on Fire" ratings PolitiFact gives to Republicans and conservatives. Nor do we focus on which party receives more "False" ratings. We note that PolitiFact has never provided anything resembling an objective criterion for distinguishing between "False" and "Pants on Fire" ratings. We conclude from the evidence that PolitiFact very probably distinguishes between the two ratings subjectively. From that, we conclude that the proportion of "Pants on Fire" ratings in relation to the total number of false ("False" plus "Pants on Fire") ratings tells us something about the ideology of the people applying the ratings.

PunditFact shows a remarkable skew to the left.

PunditFact's PoF Bias number for 2014 came in at 2.19. That simply means PunditFact was 119 percent more likely to rate a conservative's false statement "Pants on Fire" than a liberal's.

Adding the small amount of data from the end of 2013, when PunditFact was first starting out, we obtain a cumulative figure of 2.57 for the PoF Bias number. So over PunditFact's entire lifespan, conservatives were 157 percent more likely than liberals to have a false statement rated "Pants on Fire."

We also have a hint in our data that PunditFact shows much more interest in statements coming from conservatives.

Of course we're only looking at two categories of statements, so we place no great importance on that aspect of the chart. We note, however, that PolitiFact's oft-stated criterion for choosing its subject matter ("Is that true?") fits just as well (if not better) for the hypothesis of liberal media bias than for the idea that conservatives simply lie more.

It's more natural to question statements that do not fit with what one accepts as true, after all.



Correction Jan 19, 2015: Revised 7th graf to correct description of cumulative totals.

PolitiFact's coin flips

We've often highlighted the apparent non-objective standards PolitiFact uses to justify its "Truth-O-Meter" ratings. John Kroll, a former staffer at the Cleveland Plain Dealer, PolitiFact's former partner with PolitiFact Ohio, said the choice between one rating and another was often difficult and said the decisions amounted to "coin flips" much of the time.

Heads the liberal wins, tails the Republican loses, at least in the following comparison of PolitiFact's ratings of Stephen Carter (liberal) and Ted Cruz (Republican).

I'll simply reproduce the email PolitiFact Bias editor Jeff D. sent me, reformatted to our standard PFB presentation:

Read the last three paragraphs of each one (emphasis mine):

Carter said that more than 70 percent of American adults have committed a crime that could lead to imprisonment. Based on a strictly technical reading of existing laws, the consensus among the legal experts we reached is that the number is reasonable. Way more than a majority of Americans have done something in their lives that runs afoul of some law that includes jail or prison time as a potential punishment.

That said, experts acknowledged that the likelihood of arrest, prosecution or imprisonment is exceedingly low for many of Americans’ "crimes." 

As such, we rate the claim Mostly True.

Cruz said that "Lorne Michaels could be put in jail under this amendment for making fun of any politician."

Most experts we talked to agreed that the proposed amendment’s language left open the door to that possibility. But many of those same experts emphasized that prosecuting, much less imprisoning, a comedian for purely political speech would run counter to centuries of American tradition, and would face many obstacles at a variety of government levels and run headlong into popular sentiment.

In the big picture, Cruz makes a persuasive case that it’s not a good idea to mess with the First Amendment. Still, his SNL scenario is far-fetched. The claim is partially accurate but leaves out important details, so we rate it Half True.

One wonders if PolitiFact sought the consensus of experts while considering whether blacks were convicted at a higher rate than whites in a recent fact check. Rudy Giuliani received a "False" rating since PolitiFact could locate no official statistics backing his claim. Looks like official statistics aren't really needed if experts think a claim seems reasonable.
 

Jeff Adds: 

Though former Cleveland Plain Dealer (PolitiFact Ohio) editor John Kroll admits PolitiFact's ratings often amount to coin flips, their other journalistic standards are applied with the same consistency. Take for instance their Dec. 2 dodge of the claim Obama's executive order on immigration would create a $3000 incentive to hire undocumented workers:

The claim isn’t so much inaccurate as it is speculative. For that reason, we won’t put this on our Truth-O-Meter.

Was there an unannounced policy change at PolitiFact? Aaron Sharockman was editor on both the Cruz and Carter checks. An unnamed editor signed off on the Incentive claim, adding flip flops to coin flips.

Here's a timeline:

• On Sept. 11, 2014, there was enough established, tangible evidence for something that may or may not happen in the future to say Ted Cruz' prediction was half wrong

• On Dec. 2, 2014, PolitiFact suddenly has a policy against checking speculative claims, but felt compelled enough to spend an entire article Voxsplaining their work to readers.

• On Dec. 8th, 2014, PolitiFact is back in the future-checking business and found enough proof of something that hasn't actually happened yet to definitively determine a liberal's claim is Mostly True.

Remember also that Mitt Romney won the Lie of the Year award for a TV ad that claimed implied Chrysler would be moving Jeep production to China. So in 2012, PolitiFact's most notable falsehood of the year was a campaign ad implying something would happen in the future.

But does Obama's executive order offer a certain economic incentive, as in the Dec. 2 article? Sorry, PolitiFact says it doesn't rate speculative claims.

Different strokes for different folks

Folk A: President Barack Obama

Obama claimed border crossings are at the lowest level since the 1970s.

We cannot directly check Obama's literal claim -- which would include the number of people who failed and succeeeded to cross the border -- because those statistics are not maintained by the federal government.

Truth-O-Meter rating: "Half True"

Folk B: Rudy Giuliani

Giuliani said blacks and whites are acquitted of murder at about the same rate.

We couldn't find any statistical evidence to support Giuliani’s claim, and experts said they weren't aware of any, either. We found some related data, but that data only serves to highlight some of the racial disproportion in the justice system.

We found "related data" PolitiFact apparently couldn't find:

Blacks charged with murder, rape and other major crimes are more likely to be acquitted by juries or freed because of a dismissal than white defendants, according to an analysis of Justice Department statistics.

Truth-O-Meter rating: "False"

Different strokes for different folks.

PunditFact PolitiFail on Ben Shapiro, with PolitiMath

On Nov. 6, 2014 PunditFact provided yet another example why the various iterations of PolitiFact do not deserve serious consideration as fact checkers (we'll refer to PolitiFact writers as bloggers and the "fact check" stories as blogs from here on out as a considered display of disrespect).

PunditFact reviewed a claim by Truth Revolt's Ben Shapiro that a majority of Muslims are radical. PunditFact ruled Shapiro's claim "False" based on the idea that Shapiro's definition of "radical" and the numbers used to justify his claim were, according to PunditFact, "almost meaningless."

Lost on PunditFact was the inherent difficulty of ruling "False" something that's almost meaningless. Definite meanings lend themselves to verification or falsification. Fuzzy meanings defy those tests.

PunditFact's blog was literally filled with laughable errors, but we'll just focus on three for the sake of brevity.

First, PunditFact faults Shapiro for his broad definition of "radical," but Shapiro explains very clearly what he's up to in the video where he made the claim. There's no attempt to mislead the viewer and no excuse to misinterpret Shapiro's purpose.



Second, PunditFact engages in its own misdirection of its readers. In PunditFact's blog, it reports how Muslims "favor sharia." Pew Research explains clearly what that means: Favoring sharia means favoring sharia as official state law. PunditFact never mentions what Pew Research means by "favor sharia."

Do liberals think marrying church and state is radical? You betcha. Was PunditFact deliberately trying to downplay that angle? Or was the reporting just that bad? Either way, PunditFact provides a disservice to its readers.

Third, PunditFact fails to note that Shapiro could easily have increased the number of radicalized Muslims in his count. He drew his totals from a limited set of nations for which Pew Research had collected data. Shapiro points this out near the end of the video, but it PunditFact either didn't notice or else determined its readers did not need to know.

PolitiMath

PunditFact used what it calls a "reasonable" method of counting radical Muslims to supposedly show how Shapiro engaged in cherry-picking. We've pointed out at least two ways PunditFact erred in its methods, but for the sake of PolitiMath we'll assume PunditFact created an apt comparison between its "reasonable" method and Shapiro's alleged cherry-picking.

Shapiro counted 680 million radical Muslims. PunditFact counted 181.8 million. We rounded both numbers off slightly.

Taking PunditFact's 181.8 million as the baseline, Shapiro exaggerated the number of radical Muslims by 274 percent. That may seem like a big enough exaggeration to warrant a "False" rating. But it's easy to forget that the bloggers at PunditFact gave Cokie Roberts a "Half True" for a claim exaggerated by about 9,000 percent. PunditFact detected a valid underlying argument from Roberts. Apparently Ben Shapiro has no valid underlying argument that there are plenty of Muslims around who hold religious views that meet a broad definition of "radical."

Why?

Liberal bias is as likely an explanation as any.


Addendum:

Shapiro makes some of the same points we make with his own response to PunditFact.

Larry Elder: 'PunditFact Lies Again'

Conservative radio show host Larry Elder has an Oct. 30 criticism of PunditFact posted at Townhall.com. It's definitely worth a read, and here's one of our favorite bits:

Since PunditFact kicks me for not using purchasing power parity, surely PunditFact's parent, Tampa Times, follows its own advice when writing about the size of a country's economy? Wrong.

A Tampa Times' 2012 story headlined "With Slow Growth, China Can't Prop Up the World Economy" called China "the world's second-largest economy," with not one word about per capita GDP or purchasing power parity. It also reprinted articles from other papers that discuss a country's gross GDP with no reference to purchasing power parity or per capita income.

Elder does a nice job of highlighting PolitiFact's consistency problem. PolitiFact often abandons normal standards of interpretation in its fact check work. Such fact checks amount to pedantry rather than journalistic research.

A liberal may trot out a misleading statistic and it will get a "Half True" or higher. A figure like Sarah Palin uses CIA Factbook ratings of military spending and receives a "Mostly False" rating.

Of course Elder makes the point in a fresh way by looking at the way PolitiFact's parent paper, the Tampa Bay Times, handles its own reporting. And the same principle applies to fact checks coming from PolitiFact. The fact checkers don't follow the standard for accuracy they apply to others.