'Just the facts' from PolitiFact? Think again

We've pointed out before (including recently) that subjectivity affects the interpretation PolitiFact brings to the statements it rates on its trademarked "Truth-O-Meter." A May 31, 2016 fact check of Republican presidential candidate Donald Trump offers a great example of PolitiFact applying spin to a politician's statement.

We'll start with the version of the statement PolitiFact cited in its fact check, from the Washington Post:

Wind is very expensive. I mean, wind, without subsidy, wind doesn't work. You need massive subsidies for wind. There are places maybe for wind. But if you go to various places in California, wind is killing all of the eagles.

You know if you shoot an eagle, kill an eagle, they want to put you in jail for five years. Yet the windmills are killing hundreds and hundreds of eagles. One of the most beautiful, one of the most treasured birds — and they're killing them by the hundreds and nothing happens. So wind is, you know, it is a problem.

A fact checker needs to answer a basic question before proceeding with a fact check of the number of eagles killed by windmills: Which windmills is Trump talking about? Is it just windmills in "various places in California"? Windmills in the entire state of California? Or was California just an example of a place where windmills are killing eagles and the "hundreds and hundreds" mentioned later may come from locations across the United States?

We do not detect any strong argument for limiting Trump's claim about the number of eagles killed just to California. If Trump had said "Yet the windmills in California are killing hundreds and hundreds of eagles" then, sure, go for it. He's talking about California. But Trump is speaking in South Dakota about national energy policy. He has reason not to limit his comments to the wind energy industry's effects to California.

PolitiFact, surprising us not in the slightest, offers no good reason for concluding Trump was saying California windmills kill hundreds of eagles.

PolitiFact (bold emphasis added):

Setting aside Trump’s exaggeration about killing all of the eagles, we wondered, are wind turbines in California killing "hundreds and hundreds," as he said?

The fact check assumes without justification Trump was talking just about California. That assumption leads directly to the "Mostly False" rating PolitiFact pins on Trump's claim.

But what if Trump was talking about the whole of the United States? The words Trump used fit that interpretation at least as well as the interpretation PolitiFact used.

Adjusting the math

PolitiFact accepted an estimate of about 100 eagles killed per year by California wind turbines. PolitiFact accepted that wind turbines rarely kill Bald Eagles. Assuming Golden Eagles, the variety making up the majority killed in California, count as the type most vulnerable to wind turbine deaths, we can very roughly estimate eagle deaths by looking at the Golden Eagle's U.S. range.

Golden Eagles occur year-round in six of the top 15 states for generating wind energy. They occur in all 15 states for at least part of the year. California comes in at No. 2, after Texas. Golden Eagles range into the part of Texas known for generating wind energy.

It seems reasonable to estimate that wind turbines kill over 200 eagles per year in the United States.

That interpretation of Trump's comment makes his claim true, though based on an estimate, and supportive of his underlying point that wind turbines kill many eagles otherwise protected by federal law.

PolitiFact's interpretation lacks clear justification in the context of Trump's remarks, but fits PolitiFact's narrative about Trump.

A politician's lack of clarity does not give fact checkers justification for interpreting statements as they wish. The neutral fact checker notes for readers the lack of clarity and then examines the possible interpretations that are at the same time plausible. The neutral fact checker applies the same standard of charitable interpretation to all, regardless of popular public narratives.

The point? PolitiFact's rating of Trump was not based simply on the facts. It was based on an unjustified (and unjustifiable?) interpretation of what Trump said.


Jeff Adds (June 2, 2016): 
I'll give Greenberg a hat tip for applying PolitiFact's on-again, off-again policy on rating hyperbole for ignoring Trump's "wind is killing all of the eagles" blurb, but my praise ends there.

Bryan is right that Greenberg offers no reason to limit the death count to those killed in California. Likewise, Greenberg offers no justification for limiting Trump's claim to eagle deaths per year. Greenberg's own expert acknowledged "probably about 2,000" eagles had been killed in California alone since the early 80's. It is generally understood that thousands and thousands are more than hundreds and hundreds.

Greenberg's article is also an example of PolitiFact disregarding an underlying point in favor of the raw numbers. We think raw facts are the only objective thing "fact-checkers" can check, and evaluating underlying points is best counted as analysis or editorial. But PolitiFact has applied its methods so inconsistently we've likened it to playing the game of Plinko.

When PolitiFact fails to employ a consistent method for rating claims, it can't be taken seriously as  neutral or objective journalism.