Friday, September 8, 2017

PolitiFact's hypocrisy

PolitiFact manifests many examples of hypocrisy. This post will focus on just one.

On August 21, 2017 Speaker of the House Paul Ryan (R-Wis.) said American has dozens of counties with zero insurers. Ryan was talking about insurers committed to serving the exchanges that serve individual market customers.

On August 24, 2017, PolitiFact published a fact check rating Ryan's claim "Pants on Fire." PolitiFact noted that Ryan had relied on outdated information to back his claim. PolitiFact said only one county was expected to risk having no insurer, and Ryan should have been aware of it:
Now technically, that report wasn’t published until two days after Ryan spoke. But the government had the information, and a day before Ryan spoke, Politico reported that just one county remained without a potential insurance carrier in 2018. The Kaiser Family Foundation published the same information the day of Ryan’s CNN town hall.

And a week earlier, the government said there were only two counties at risk of having no participating insurer. Ryan was way off no matter what.
Fast forward to Sept. 7, 2017. PolitiFact elects to republicize its fact check of Ryan, reinforcing its message that only one county remains at risk no not having any insurance provider available through the exchange. PolitiFact publicized it on Twitter:
And PolitiFact publicized it on Facebook as well.

The problem? On Sept. 6, 2017, the Kaiser Family Foundation updated its information to show 63 counties at risk of having no insurer on the exchange. The information in the story PolitiFact shared was outdated.

Paul Ryan got a "Pants on Fire" for peddling outdated information.

What does PolitiFact get for doing the same thing?

Another Pulitzer Prize?

Thursday, September 7, 2017

"Not a lot of reader confusion" V

When will PolitiFact give up its absurd notion that its graphs and tables do not mislead large numbers of people?

Joy Behar of ABC's "The View" recently challenged White House spokesperson Sarah Sanders on the basis that PolitiFact says 95 percent of President Donald Trump's statements are untrue:
Joy Behar asked Sanders about a PolitiFact report that found 95 percent of the president's statements were less than completely true.

"The problem with that, Joy, is that you are doing exactly what we're talking about," Sanders responded. "Pushing a false narrative."
Apparently Sanders was the only person on the set who challenged the false narrative Behar was peddling.

For those PolitiFact continues to mislead, we repeat that if PolitiFact fails to use a representative sample of statements when it publishes its graphs and charts then the percentages tell you the opinions of PolitiFact editors for a select set of statements, not the percentage chance that a typical Trump statement is untrue.

(fingers crossed that the ABC embed works)

Not a lot of reader confusion? Seriously?

Give us a break, PolitiFact.

Wednesday, September 6, 2017

PolitiFact & Roy Moore: A smorgasbord of problems

When PolitiFact unpublished its Sept. 1, 2017 fact check of a claim attacking Alabama Republican Roy Moore, we had our red flag to look into the story. Taking down a published story itself runs against the current of journalistic ethics, so we decided to keep an eye on things to see what else might come of it.

We were rewarded with a smorgasbord of questionable actions by PolitiFact.

Publication and Unpublication

PolitiFact's Sept. 1, 2017 fact check found it "Mostly False" that Republican Roy Moore had taken $1 million from a charity he ran to supplement his pay as as Chief Justice in the Supreme Court of Alabama.

We have yet to read the original fact check, but we know the summary thanks to PolitiFact's Twitter confession issued later on Sept. 1, 2017:

We tweeted criticism of PolitiFact for not making an archived version of the fact check immediately available and for not providing an explanation for those who ended up looking for the story only to find a 404-page-not found-error.  We think readers should not have to rely on Twitter to know what is going on with the PolitiFact website.

John Kruzel takes tens of thousands of dollars from PolitiFact

(a brief lesson in misleading communications)

The way editors word a story's title, or even a subheading like the one above, makes a difference.

What business does John Kruzel have "taking" tens of thousands of dollars from PolitiFact? The answer is easy: Kruzel is an employee of PolitiFact, and PolitiFact pays Kruzel for his work. But we can make that perfectly ordinary and non-controversial relationship look suspicious with a subheading like the one above.

We have a parallel in the fact check of Roy Moore. Moore worked for the charity he ran and was paid for it. Note the title PolitiFact chose for its fact check:

Did Alabama Senate candidate Roy Moore take $1 million from a charity he ran?

 "Mostly True." Hmmm.

Kruzel wrote the fact check we're discussing. He did not necessarily compose the title.

We think it's a bad idea for fact-checkers to engage in the same misleading modes of communication they ought to criticize and hold to account.

Semi-transparent Transparency

For an organization that advocates transparency, PolitiFact sure relishes its semi-transparency. On Sept. 5, 2017, PolitiFact published an explanation of its correction but rationed specifics (bold emphasis added in the second instance):
Correction: When we originally reported this fact-check on Sept. 1, we were unable to determine how the Senate Leadership Fund arrived at its figure of "over $1 million," and the group didn’t respond to our query. The evidence seemed to show a total of under $1 million for salary and other benefits. After publication, a spokesman for the group provided additional evidence showing Moore received compensation as a consultant and through an amended filing, bringing the total to more than $1 million. We have corrected our report, and we have changed the rating from Mostly False to Mostly True.
PolitiFact included a table in its fact check showing relevant information gleaned from tax documents. Two of the entries were marked as for consulting and as an amended filing, which we highlighted for our readers:

Combining the two totals gives us $177,500. Subtracting that figure from the total PolitiFact used in its corrected fact check, we end up with $853,375.

The Senate Leadership Fund PAC (Republican) was off by a measly 14.7 percent and got a "Mostly False" in PolitiFact's original fact check? PolitiFact often barely blinks over much larger errors than that.

Take a claim by Sen. Brad Schneider (D-Ill.) from April 2017, for example. The fact check was published under the "PolitiFact Illinois" banner, but PolitiFact veterans Louis Jacobson and Angie Drobnic Holan did the writing and editing, respectively.

Schneider said that the solar industry accounts for 3 times the jobs from the entire coal mining industry. PolitiFact said the best data resulted in a solar having a 2.3 to 1 job advantage over coal, terming 2.3 "just short of three-to-one" and rating Schneider's claim "Mostly True."

Schneider's claim was off by over 7 percent even if we credit 2.5 as 3 by rounding up.

How could an error of under 15 percent have dropped the rating for the Senate Leadership Fund's claim all the way down to "Mostly False"?

We examine that issue next.

Compound Claim, Or Not?

PolitiFact recognizes in its statement of principles that sometimes claims have more than one part:
We sometimes rate compound statements that contain two or more factual assertions. In these cases, we rate the overall accuracy after looking at the individual pieces.
We note that if PolitiFact does not weight the individual pieces equally, we have yet another area where subjective judgment might color "Truth-O-Meter" ratings.

Perhaps this case qualifies as one of those subjectively skewed cases.

The ad attacking Moore looks like a clear compound claim. As PolitiFact puts it (bold emphasis added), "In addition to his compensation as a judge, "Roy Moore and his wife (paid themselves) over $1 million from a charity they ran."

PolitiFact found the first part of the claim flatly false (bold emphasis added):
He began to draw a salary from the foundation in 2005, two years after his dismissal from the bench, according to the foundation’s IRS filings. So the suggestion he drew the two salaries concurrently is wrong.
Without the damning double dipping, the attack ad is a classic deluxe nothingburger with nothingfries and a super-sized nothingsoda.

Moore was ousted as Chief Justice in the Alabama Supreme Court, where he could have expected a raise up to $196,183 per year by 2008. After that ouster Moore was paid a little over $1 million over a nine-year period, counting his wife's salary for one year, getting well under $150,000 per year on average. On what planet is that not a pay cut? With the facts exposed, the attack ad loses all coherence. Where is the "more" that serves as the theme of the ad?

We think the fact checkers lost track of the point of the ad somewhere along the line. If the ad was just about what Moore was paid for running his charity while not doing a different job at the same time, it's more neutral biography than attack ad. The main point of the attack ad was Moore supplementing his generous salary with money from running a charitable (not-for-profit) organization. Without that main point, virtually nothing remains.

PolitiFact covers itself with shame by failing to see the obvious. The original "Mostly False" rating fit the ad pretty well regardless of whether the ad correctly reported the amount of money Moore was paid for working at a not-for-profit organization.

Assuming PolitiFact did not confuse itself?

If PolitiFact denies making a mistake by losing track of the point of the ad, we have another case that helps amplify the point we made with our post on Sept. 1, 2017. In that post, we noted that PolitiFact graded one of Trump's claims as "False" based on not giving Trump credit for his underlying point.

PolitiFact does not address the "underlying point" of claims in a consistent manner.

In our current example, the attack ad on Roy Moore gets PolitiFact's seal of "Mostly" approval only by ignoring its underlying point. The ad actually misled in two ways, first by saying Moore was supplementing his income as judge with income from his charity when the two source of income were not concurrent, and secondly by reporting the charity income while downplaying the period of time over which that income was spread. Despite the dual deceit, PolitiFact graded the claim "Mostly True."

"The decision about a Truth-O-Meter rating is entirely subjective"

Cases like this support our argument that PolitiFact tends to base its ratings on subjective judgments. This case also highlights a systemic failure of transparency at PolitiFact.

We will update this item if PolitiFact surprises us by running a second correction.


On top of problems we described above, PolitiFact neglected to tag its revised/republished story with the "Corrections and Updates" tag its says it uses for all corrected or updated stories.

PolitiFact has a poor record of following this part of its corrections policy.

We note, however, that after we pointed out the problem via Twitter and email PolitiFact fixed it without a long delay.

Friday, September 1, 2017

PolitiFact disallows Trump's underlying point?

PolitiFact's defenders sometimes opine that PolitiFact always justifies its rulings.

We accept that PolitiFact typically includes words in its fact checks intended to justify its rulings. But we detect bias in PolitiFact's inconsistent application of principles when it tries to justify its ratings.

Our example this time comes from an Aug. 31, 2017 fact check of President Donald Trump's claim that illegal border crossings have slowed by 78 percent.

Zebra Fact Check on Aug. 30, 2017 published criticisms of the way PolitiFact and the Washington Post Fact Checker handled this claim. PolitiFact's latest version corrects none of the specified problems, including the failure to attempt a reasonable fact check of how much of a drop in illegal Southwest border crossings Trump can claim.

As with its earlier fact check, PolitiFact offers examples of various cherry-picked statistics, implicitly demonstrating that cherry-picking leads to a divergent set of outcomes:
Here’s how the number of apprehensions have changed:
• From July 2016 to July 2017, down 46 percent;
• From June 2017 to July 2017, up 13 percent;
• From November 2016 to July 2017, down 61 percent.
As I explained over at Zebra Fact Check, a serious attempt to measure a drop in border crossings explains the use of a proxy measure (border apprehensions) and then picks a representative baseline against which to measure the change.

Zebra Fact Check calculated a 56 percent change. calculated 58 percent.

PolitiFact has yet to make a reasonable attempt to establish a representative baseline. The best attempt in the cherry-picked set we quoted was the comparison of July 2016 to July 2017, showing a 46 percent change. That comparison suffers from offering a narrow picture, made up of individual months separated by a year in time. Also, 2016 was not a typical year for border apprehensions under the Obama administration. But at least it compared Trump to Obama in an apples-to-apples sense.

PolitiFact's 46 percent figure ends up in the ballpark with the 58 percent figure produced.

That's where the problem comes in.

Where is Trump's underlying point?

Back in 2008, PolitiFact Editor Bill Adair published an article trying to explain how PolitiFact treats numbers claims.
To assess the truth for a numbers claim, the biggest factor is the underlying message.
Adair used as one of his examples a claim by then-presidential candidate Barack Obama that his mixed-marriage birth was illegal in 12 states when he has born. PolitiFact found it was illegal in 22 states but rated the claim "Mostly True." That is the potential power of the underlying argument. If one makes Obama's claim into "My mixed marriage birth was illegal in many states when I was born" then it's essentially accurate, but you can drop Obama down to just "Mostly True" since he underestimated the number of states by 10.

Does Trump have an underlying point that illegal Southwest border crossings have decreased under his watch?

It appears that he does, and it appears that PolitiFact offers him no credit for it. PolitiFact's fact check shows no interest at all in Trump's underlying point.


Correction Sept. 1, 2017: Swapped out "Did" for "Does" in the third-to-last paragraph.

Wednesday, August 30, 2017

PolitiFact: "There are no sharks swimming in the streets of Houston or anywhere else"

We were amused when we noticed PolitiFact inquiring about the faked image of a shark swimming on a Houston freeway thanks to Hurricane Harvey.

"Is it true PolitiFact wonders if that's true?" we wondered.

Our amusement multiplied when we saw the headline over PolitiFact's story, albeit not a scoop, exposing the fakery:

There are no sharks swimming in the streets of Houston or anywhere else

No, seriously. That is how PolitiFact titled its story.

No sharks swimming in the Mediterranean?

No sharks swimming in the Indian Ocean?

No sharks swimming in the Atlantic Ocean?

No sharks swimming in the Pacific Ocean?

Are these questions silly? Of course, until we consider that PolitiFact is the fact checker that fact checks something if it can be construed to mean something, basing the fact check on the ability of some to construe creatively.

If we have "the streets of Houston" and "anywhere else," we don't see why we can't construe that to mean the Pacific Ocean, or even the shark exhibit at Sea World.

Still, we believe in charitable interpretation. What if PolitiFact was just trying to say that there were no sharks swimming "in the streets" in Houston or anywhere else?

Well, immediately we put that together with PolitiFact's "Half True" ruling on President Obama's claim that fish swim in the streets of Miami at high tide.

If fish can swim in the streets of Miami at high tide, then what about a little 'ol bonnethead shark? Couldn't a Miami-area bonnethead put the lie to PolitiFact's claim that no sharks swim in any streets anywhere? And what about submerged cities such as Port Royal? What keeps the sharks away from those streets?

PolitiFact lives in a glass house, throwing stones.

Wednesday, August 16, 2017

Speaking hearsay to power: Joy Reid & PolitiFact

Sometimes PolitiFact publishes fact-checking so irresponsible that we find it hard to believe that unconscious bias serves as an adequate explanation.

On the Aug. 13, 2017 edition of NBC's "Meet the Press," pundit Joy-Ann Reid directly implied that the Trump White House contains white nationalists. On Aug. 15, 2017, PolitiFact published a fact-check style article without a "Truth-O-Meter" rating but with a "Share the Facts"/Google label conclusion judging her words "a bit too strong."

A reasonable person might translate "a bit too strong" into "Mostly True" or "Half True," but probably not "Mostly False," "False" or "Pants on Fire."

Hold on--Something's not quite alt-right

If the evidence supported something akin to a "Half True" or "Mostly True" rating, then we would not have much to complain about. But the ruling-not-ruling flies in the face of the evidence PolitiFact collected.

PolitiFact went to liberal experts (?) like the Southern Poverty Law Center and could not get a single one of them to declare evidence that one or more white nationalists populate the White House. The article was filled with things like this:
When we asked this question of several independent experts, they all agreed that none of the four were white nationalists themselves. However, several said that they had placed themselves uncomfortably close to white nationalists.
Are we to infer from PolitiFact's "a bit too strong" rating that guilt-by-association is fair game in fact-checking?

More to the point, is it okay to publicly accuse others of racism using guilt-by-association? That is what Reid did, and PolitiFact gave her the equivalent of a "Mostly True" rating.

PolitiFact even tried to downplay its own implicit interpretation ("Are there white nationalists in the White House?") of Reid's claim.

Hey! Let's fact check something Reid supposedly did not say!

PolitiFact flip-flops on whether Reid said there were white nationalists in the White House. PolitiFact's introductory paragraphs paint Reid as having "crystallized" the issue of White Nationalists in the White House:
The "Unite the Right" march in Charlottesville has brought the issue of white nationalism to the top of the nation’s agenda -- specifically, whether white nationalists are part of the White House staff.

Remarks by liberal commentator Joy-Ann Reid on the Aug. 13 edition of NBC’s Meet the Press crystallized these questions.

Just a few paragraphs later, Reid's crystal has turned to ash (bold emphasis added):
It’s important to note that Reid did not explicitly accuse any of the four individuals she named of being white nationalists or alt-right members per se. But she suggested that the four were sympathetic to people who do fall into that category.

PolitiFact contradicts its own quotation of Reid (bold emphasis added):
"Who's writing the talking points that he was looking down and reading from? He has people like Stephen Miller, claimed as a mentee by Richard Spencer, who is an avowed open white nationalist. He has Steve Bannon, who's been sort of allowed to … meld into … the normalcy of a governmental employee, but who ran, which I reread today, the post that's still on their website, where they self-describe as the home of the alt-right.

What is the alt-right? It is a dressed-up term for white nationalism. They call themselves white identitarianism. They say that the tribalism that's sort of inherent in the human spirit ought to be also applied to white people.

That is who is in his government. Sebastian Gorka, who wore the medal of Vitézi Rend, a Nazi organization, being paid by the taxpayer, in the government of Donald Trump. The former Publius Decius blogger Michael Anton in the government.

He is surrounded by these people. It isn't both sides. He's in the White House -- they're in the White House with him."

We can't even imagine the level of expertise in mental gymnastics needed to deny the fact that Reid is saying the alt-right is a white nationalist group and is represented in the White House by the people she named. Nothing occurs in the context to diminish Reid's clear implication.

Shame on you, Joy Reid. Shame on you, PolitiFact.

Editor's note: It appears we published while attempting to preview this post. We're not aware of any significant change, other than adding an embedded URL, to the content since the original publication. Most or all the changes only affect HTML formatting.

Update Aug. 23, 2017: Fixed formatting to make clear the "crystallized" line was a quotation from PolitiFact

Friday, August 11, 2017

National Review: "PolitiFact, Wrong Again on Health Care"

We've noted with interest Avik Roy's articles noting that the CBO's assessments of insurance loss from GOP health care reform bills place much of the responsibility on repeal of the individual mandate.

We anticipated this research would impact PolitiFact's fact-checking of GOP reform efforts, and National Review's Ramesh Ponnuru delivers the expected assessment in "PolitiFact, Wrong Again on Health Care."

When House Speaker Paul Ryan (R-Wis.) said most of those losing insurance under a GOP proposal were choosing not to buy something they did not want instead of having something taken away, PolitiFact rated his statement "Mostly False."

Ponnuru explains:
The root problem is that (PolitiFact's Jon) Greenberg assumed that the fines on people without insurance—Obamacare’s “individual mandate”—operate only in the market for individually purchased health insurance and that getting rid of them has no effect on Medicaid enrollment. So he thinks that all of the decline in Medicaid enrollment that CBO projects are the result of reforms to Medicaid that would have kept people who want it from getting it, and Ryan is exaggerating the effect of the fines.
Here's how Greenberg explained it in PolitiFact's fact check (bold emphasis added):
The biggest single chunk of savings under the Senate bill comes out of Medicaid. The CBO said that compared with the laws in place today, 15 million fewer people who need insurance would be able to get it through Medicaid or anywhere else.

Ryan’s answer flipped the CBO presentation. According to the CBO, the Senate bill’s impact on people who would get coverage through Medicaid is double that of people who buy on the insurance market. That’s where people make the kind of choices Ryan was talking about.
It looks like Ponnuru has Greenberg dead to rights.

We made the same assumption as Greenberg, though not published in a fact check, which led us to puzzle how to reconcile the high impact of the individual mandate for the CBO's prediction for insurance loss in 2018 with the apparently shrinking impact of the individual mandate in 2025.

Ponnuru's article helps explain the discrepancy, and his explanation exposes one of PolitiFact's claims as false: "The CBO said that compared with the laws in place today, 15 million fewer people who need insurance would be able to get it through Medicaid or anywhere else."

A decent slice of that 15 million, about 7 million by Ponnuru's estimate, will still maintain Medicaid eligibility. They simply won't sign up if not threatened with a fine.  But they can sign up after they fall ill and obtain retroactive coverage for up to three months. If that segment of the population needs Medicaid insurance, it can get Medicaid insurance, contrary to what PolitiFact claimed.

Yes, PolitiFact was wrong again.


Considering PolitiFact's penchant for declining to change its stories even after critics point out flaws, we wonder if PolitiFact will update its stories affected by the truths Ponnuru mentions.