Great Moments in the Annals of Subjectivity (Updated)

Did Republican Donald Trump win the electoral college in a landslide?

We typically think of a "landslide" as an overwhelming victory, and there's certainly doubt whether Trump's margin of victory in the electoral college unequivocally counts as overwhelming.

"Overwhelming" itself is hard to pin down in objective terms.

So that's why we have PolitiFact, the group of liberal bloggers that puts "fact" in its name and then proceeds to publish "fact check journalism" based on subjective "Truth-O-Meter" judgments.

When RNC Chairman Reince Priebus (and Trump's pick for his chief of staff) called Trump's electoral college victory a "landslide," PolitiFact Wisconsin's liberal bloggers sprang into action to do their thing (bold emphasis added):

Landslide, of course, is not technically defined. When we asked for information to back Priebus’ claim, the Republican National Committee merely recited the electoral figures and repeated that it was a landslide.

If "landslide" is not technically defined then what fact is PolitiFact Wisconsin checking? Is "landslide" non-technically defined to the point one can judge it true or false?

PolitiFact Wisconsin follows typical PolitiFact procedure in collecting expert opinions about whether Priebus' use of "landslide" matches its non-technical definition. One of the 10 experts PolitiFact consulted said Trump's margin was "close" to a landslide. PolitiFact said the other nine said it fell short, so PolitiFact ruled Priebus' claim "False."

Priebus said Trump’s win was "an electoral landslide."

But aside from the fact Trump lost the popular vote, his margin in the Electoral College isn’t all that high, either. None of the 10 experts we contacted said Trump’s win crosses that threshold.

We rate Priebus’ claim False.

One has to marvel at expertise sufficient to say whether the use of a term meets a non-technical definition.

One has to marvel all the more at fact checkers who concede that a term has a mushy definition ("not technically defined") and then declare that some use of the term fails to cross "that threshold."

What threshold?

One of the election experts said if Trump won by a landslide then Obama won by an even greater landslide.

RollCall, 2015:

In 2006, Democrats won back the House; two years later, President Barack Obama won by a landslide.

LA Times, 2012:

Obama officially wins in electoral vote landslide.

NPR, 2015:

President Obama won in a landslide.

NYU Journalism, 2008:

Obama Wins Landslide Victory, Charts New Course for United States.

Since Obama did not win by a landslide, therefore one cannot claim Trump won by a landslide? Is that it?

It is folly for fact checkers to try to judge the truth of ambiguous claims. PolitiFact often pursues that folly, of course, and in the end simply underscores what it occasionally admits: The ratings are subjective.

Finding experts willing to participate in the folly does not reduce the magnitude of the folly. This would have been a good subject for PolitiFact to use in continuing its Voxification trend. PolitiFact might have produced an "In context" article to talk about electoral landslides and how experts view the matter. But trying to corral the use of a term that is traditionally hard to tame simply makes a mockery of fact-checking.

Jeff Adds (Dec. 1, 2016):

Add this to a long list of opinions that PolitiFact treats as verifiable facts, including these two gems:

- Radio host John DePetro opined that the Boston Marathon bomber was buried "not far" from President John Kennedy. PolitiFact used their magical powers of objective divinity to determine the unarguable demarcation of "not far."

- Rush Limbaugh claimed "some of the wealthiest American's are African-Americans now." Using the divine wizardry of the nonpartisan Truth-O-Meter, PolitiFact's highly trained social scientists were able to conjure up a determinant definition of what "wealthiest" means, and specifically which people were included in the list.

Reasonable people may discount Trump's claim of a "landslide" victory assuming the conventional use of the term, but it's not a verifiable fact that can be confirmed or dismissed with evidence. It's an opinion.

The reality is that the charlatans at PolitiFact masquerade as truthsayers when they do little more than contribute to the supposed fake news epidemic by shilling their own opinions as unarguable fact. They're dangerous frauds whose declaration of objectivity doesn't withstand the slightest scrutiny.

PolitiFact finds true Rush Limbaugh claim "False"

Unbelievable.  That's PolitiFact's "PunditFact."

Rush Limbaugh said African Americans are now some of the wealthiest people in America.  PunditFact sprang into action:

The key to PunditFact's "False" rating for Limbaugh was simplicity itself.  PunditFact defined "wealthiest" to mean that a person appeared on the Forbes list of wealthiest people.

No, really.  That's what PunditFact did.

"You've got a black president. You've got a black attorney general. You've got the wealthiest TV performer in American history is a African-American woman. That would be The Oprah," Limbaugh said. "Some of the wealthiest Americans are African-American now."

That last line is quite incorrect.

The go-to source to learn about the wealthiest Americans is Forbes, which tracks the fortunes of the world’s elite. It publishes an annual list of the 400 wealthiest Americans and a list of the world’s billionaires.

PunditFact reasoned that since Oprah Winfrey was the only African American to appear on the list of the 400 wealthiest Americans, therefore what Limbaugh said was false.

No, really.  That's what PunditFact did.

Did Limbaugh say something in context to justify PunditFact narrowing the definition of "wealthiest" to the Forbes top 400?  Not from what we can tell.  Certainly the fact check makes no mention of it.  PunditFact's decision seems entirely arbitrary, especially given how commonly media outlets like Vanity Fair, CNBC, and the Cleveland Plain Dealer use the term for the top 1 percent of earners.

Are those media outlets lying to us?  PolitiFact's analysis suggests they are.  But it gets worse.

 

Hypocrites

PolitiFact is also among the media outlets comfortable with using "wealthiest Americans" to mean something other than the Forbes top 400.

PolitiFact New Jersey did it in a fact check of (Democrat) Steve Rothman.

PolitiFact Florida did it in a fact check of Debbie Wasserman Schultz, where defining "wealthiest Americans" as the top 400 would have reversed the ruling (DWS would have received a "True" rating, implying that CNN's Wolf Blitzer was wrong for claiming the wealthiest Americans foot most of the U.S. tax bill).

National PolitiFact did it in a Obameter item in 2013, rating President Obama's promise that he would raise taxes on those making over $200,000 per year--a figure PolitiFact paraphrased as the earning floor for the "wealthiest Americans."

PolitiFact Texas did it in a fact check of (Democrat) Lloyd Doggett, equating the top 1 percent of earners with the "wealthiest Americans" in a paraphrase.

Oh, and PunditFact did it in a fact check of MSNBC's Joe Scarborough earlier this year.

We could provide many more examples from PolitiFact, but you get the idea.  This is hypocrisy of the highest order.  PolitiFact has no right to decide where Limbaugh draws the line on what constitutes the "wealthiest Americans."  Limbaugh can draw that line anywhere he likes.  And if he happens to draw it at the top 1 percent of earners, like PolitiFact often does, then his statement is quite simply true.

The competition for worst fact check of the year is pretty intense in 2014.  And it's still early.

PunditFact's PolitiMath

Here at PolitiFact Bias, we keep some tabs on what we call "PolitiMath"--the mathematical indicators that correspond (or not) to various positions on its "Truth-O-Meter" scale.

This week offers us another potentially informative case, as PunditFact looks at whether Michael Eric Dyson was accurate in claiming that Sunday morning political talk shows "usually" feature conservative white men.

In terms of math, this case is fairly simple.  PunditFact counted 25 percent of the Sunday show guests as conservative white males--a little short of a plurality.

PunditFact also shared parallel figures compiled by the left-wing Media Matters organization.  Media Matters put the figure for conservative white men at about 29 percent, which did count as a plurality.

"Usually" means more than half the time, so using PunditFact's count Dyson was off by 50 percent.  Using the Media Matters count, Dyson was off by about 42 percent.

PunditFact rated Dyson's claim "Mostly False":

Dyson described the Sunday shows as having been "given over" to conservative white males. While that phrase isn't exact, it does suggest a dominant presence. The numbers don’t back that up. Conservatives outman the liberals but by the time you drill down to white, male, conservatives, they lose much of the edge.

Dyson pushed too far on his adjectives. We rate the claim Mostly False.

We don't understand PunditFact focusing on "given over."  It is the extended phrase "mostly given over" that provides the basis for the fact check.  "Mostly" communicates a "given over" figure exceeding 50 percent.

Going by the "Principles of PolitiFact, PunditFact and the Truth-O-Meter," a "Mostly False" statement contains an element of truth.  We are unable to identify what PunditFact thinks is the element of truth in Dyson's statement.

This case featuring Dyson compares very naturally with PolitiFact Florida's rating of Sen. Marco Rubio's statement claiming Americans are mostly conservative.  PolitiFact Florida gave Rubio a "Half True" for that one, though only one of three polls had conservatives self-identifying in majority numbers.

We certainly think there's much to criticize in the way PolitiFact's fact checkers went about rating both of these claims, but in terms of PolitiMath we can set those concerns aside and simply look at how PunditFact's numbers correlated to the "Truth-O-Meter" rating.

By the two measurements PunditFact offered, Dyson was at least 42 percent in error.  That resulted in a "Mostly False" rating.  By three measurements, Rubio was correct on one and off by a maximum of 36 percent by the two polls that showed conservatives as the plurality.

It's easy to see how Rubio's statement could count as at least partly true.  One poll unambiguously supported him.  But Dyson?  Not so much.

That's how PolitiMath works, for what it's worth.


Edit 11/16/2014: Added Link to PunditFact article in 6th graph - Jeff

PolitiFact is "Not Far" from "Large-Scale" Inconsistencies

Just over a week ago we highlighted PolitiFact's dubious rating of radio host John DePetro. The problem with that rating was simple: DePetro said the Boston Bomber was buried "not far" from John F. Kennedy. PolitiFact did a search on Google Maps to find the linear distance between the two graves, and said DePetro's claim was "mildly ridiculous." Somehow, PolitiFact Rhode Island was able to determine the specific distance of "not far."

PolitiFact explained their scientific conclusion:

Saying that Kennedy is buried "not far" from Tsarnaev is like saying Newport is not far from the eastern tip of Cape Cod, that Rhode Island's State House is not far from Derry, N.H., or that the site of the Boston Marathon bombing is "not far" from the the southernmost tip of Narragansett, R.I.

And to say that such a distance should somehow spark offense strikes us as mildly ridiculous, so we rate his statement Pants on Fire!

Last Thursday, after hearing President Obama claim "there have been no large scale attacks on the United States" during his presidency, I fired off an email to my co-editor Bryan White: "What are the odds PF gives Obama leeway because 'large scale' is too vague?"

As it turns out, the Predictability Gods were listening to my prayers:

Indeed, the definitions of "large-scale" are sufficiently vague that there’s a lot of room for Obama... 

That's right. PolitiFact can determine the linear distance of "not far," but the definition of "large scale" is beyond their ability to comprehend.

Their final ruling is so pathetically protective of Obama it's actually insulting (emphasis added)

Obama said that since he has taken office, "there have been no large-scale attacks on the United States."

Two attacks on Obama’s watch that might qualify as "large scale" -- the Fort Hood shootings and the Boston Marathon bombing. They caused substantially fewer deaths than the biggest terrorist attacks of recent years, and they are believed to have been carried out by "lone wolf" attackers with limited connections to large-scale terrorist networks. But where to draw the line between small, medium and large attacks is open to interpretation. Obama's formulation is plausible, but not the only one. We rate it Half True.

There's a lot wrong with this paragraph. What qualities are necessary for an attack to be deemed "biggest"? Loss of life? Property damage? Number of terrorists involved? How far back do "recent" years go? Notice that "small," "medium" and "large" are "open to interpretation," but "not far" can be quantified by a Google search.

The reality is that reasonable people are entitled to their own opinions about what constitutes a "large scale" attack, or how close is too close to bury a terrorist to a fallen president. In either case, though, it amounts to an opinion, and is rightly beyond the bounds of the clinical world of fact checking. 

PolitiFact does decent work as an editorial page. And they tend to provide valid arguments in favor of their opinions. But it's dishonest to label themselves "fact checkers," let alone pretend to be unbiased. This latest Obama rating is not a legitimate uncovering of facts. It's a defense of a spin of a denial. PolitiFact may as well have their own podium next to Jay Carney.


Bryan adds:

The way fact checkers rule on ambiguous claims over time reveals much about their ideology.  If one side gets a statistically significant advantage over the other then we have a strong indication of ideological bias.

A Fact Too Far

PolitiFact Rhode Island published an article yesterday that highlights the distance between fact checking and what PolitiFact actually does.

Image from PolitiFact

At issue is radio host John DePetro's comments regarding the current resting place of the deceased Boston bomber: "You know, in a way, think of who else is there. That is, President Kennedy is buried not far from there, in Virginia,"

PolitiFact's findings?

We used the Google Maps Distance Calculator to find the actual span between Kennedy's grave at Arlington and the Al-Barzakh Cemetery on Sadie Lane in Doswell, Virginia.

Driving distance: 87 miles.
Bee-line distance: 74 miles.
That's about 55,817 casket lengths.

When we informed DePetro of the distance and asked if he was still bothered, he wrote in an e-mail, "Yes. Insult to bury him so close to JFK. Johnston landfill was my choice or out to sea."

Notice anything missing?  PolitiFact failed to provide a standardized measurement for the linear distance of "not far." Probably because no such definition exists. It's an opinion, and one DePetro articulated quite effectively.

Any guess on PolitiFact's rating?

[T]o say that such a distance should somehow spark offense strikes us as mildly ridiculous, so we rate his statement Pants on Fire!

This is a wholly inappropriate sentence to include in a supposed fact check. We've long argued that there is simply no objective definition of what makes a claim "ridiculous." It's a subjective term determined only by the personal inclinations of PolitiFact's editors. Compounding that subjectivity, PolitiFact finds DePetro's claim only mildly ridiculous in this case. So now not only is the Pants on Fire rating based on an opinion, it's also subject to a sliding scale, the standards for which have yet to be published. Is it a Truth-O-Meter or a mood ring?

It would be interesting to learn when PolitiFact acquired the magical gift of objectively defining what should or should not cause offense. The fact that PolitiFact Rhode Island isn't offended does not make something inoffensive. That's a personal judgement that has no place in a dispassionate determination of fact.

This article is yet another example of how PolitiFact operates as an editorial site sheathed in a false blanket of objectivity. There is simply no way for them to measure the accuracy of DePetro's opinion, and even less possible for them to place a factual determination on what is or isn't offensive.

This is an opinion piece. It's not a fact check. It is dishonest for PolitiFact to suggest otherwise.