Monday, November 21, 2016

Great Moments in the Annals of Subjectivity (Updated)

Did Republican Donald Trump win the electoral college in a landslide?

We typically think of a "landslide" as an overwhelming victory, and there's certainly doubt whether Trump's margin of victory in the electoral college unequivocally counts as overwhelming.

"Overwhelming" itself is hard to pin down in objective terms.

So that's why we have PolitiFact, the group of liberal bloggers that puts "fact" in its name and then proceeds to publish "fact check journalism" based on subjective "Truth-O-Meter" judgments.

When RNC Chairman Reince Priebus (and Trump's pick for his chief of staff) called Trump's electoral college victory a "landslide," PolitiFact Wisconsin's liberal bloggers sprang into action to do their thing (bold emphasis added):
Landslide, of course, is not technically defined. When we asked for information to back Priebus’ claim, the Republican National Committee merely recited the electoral figures and repeated that it was a landslide.
If "landslide" is not technically defined then what fact is PolitiFact Wisconsin checking? Is "landslide" non-technically defined to the point one can judge it true or false?

PolitiFact Wisconsin follows typical PolitiFact procedure in collecting expert opinions about whether Priebus' use of "landslide" matches its non-technical definition. One of the 10 experts PolitiFact consulted said Trump's margin was "close" to a landslide. PolitiFact said the other nine said it fell short, so PolitiFact ruled Priebus' claim "False."
Priebus said Trump’s win was "an electoral landslide."

But aside from the fact Trump lost the popular vote, his margin in the Electoral College isn’t all that high, either. None of the 10 experts we contacted said Trump’s win crosses that threshold.

We rate Priebus’ claim False.
One has to marvel at expertise sufficient to say whether the use of a term meets a non-technical definition.

One has to marvel all the more at fact checkers who concede that a term has a mushy definition ("not technically defined") and then declare that some use of the term fails to cross "that threshold."

What threshold?

One of the election experts said if Trump won by a landslide then Obama won by an even greater landslide.

RollCall, 2015:
In 2006, Democrats won back the House; two years later, President Barack Obama won by a landslide.
LA Times, 2012:
Obama officially wins in electoral vote landslide.
NPR, 2015:
President Obama won in a landslide.
NYU Journalism, 2008:
Obama Wins Landslide Victory, Charts New Course for United States.
Since Obama did not win by a landslide, therefore one cannot claim Trump won by a landslide? Is that it?

It is folly for fact checkers to try to judge the truth of ambiguous claims. PolitiFact often pursues that folly, of course, and in the end simply underscores what it occasionally admits: The ratings are subjective.

Finding experts willing to participate in the folly does not reduce the magnitude of the folly. This would have been a good subject for PolitiFact to use in continuing its Voxification trend. PolitiFact might have produced an "In context" article to talk about electoral landslides and how experts view the matter. But trying to corral the use of a term that is traditionally hard to tame simply makes a mockery of fact-checking.

Jeff Adds (Dec. 1, 2016):

Add this to a long list of opinions that PolitiFact treats as verifiable facts, including these two gems:

- Radio host John DePetro opined that the Boston Marathon bomber was buried "not far" from President John Kennedy. PolitiFact used their magical powers of objective divinity to determine the unarguable demarcation of "not far."

- Rush Limbaugh claimed "some of the wealthiest American's are African-Americans now." Using the divine wizardry of the nonpartisan Truth-O-Meter, PolitiFact's highly trained social scientists were able to conjure up a determinant definition of what "wealthiest" means, and specifically which people were included in the list.

Reasonable people may discount Trump's claim of a "landslide" victory assuming the conventional use of the term, but it's not a verifiable fact that can be confirmed or dismissed with evidence. It's an opinion.

The reality is that the charlatans at PolitiFact masquerade as truthsayers when they do little more than contribute to the supposed fake news epidemic by shilling their own opinions as unarguable fact. They're dangerous frauds whose declaration of objectivity doesn't withstand the slightest scrutiny.

Sunday, November 13, 2016

PolitiFact's "many" problems

On Nov. 3, 2016 we brought some focus to the "Mostly False" rating PolitiFact gave Donald Trump for saying many Americans were paying more for health care than for their mortgage or rent.

PFB co-editor Jeff D. today reminded me about an "Afters" section I added to a post from Sept. 1, 2016:
PolitiFact exaggerated the survey evidence supposedly supporting Clinton by claiming "many" teachers blamed Trump for increasing bullying and harassment:
Many of these teachers, unsolicited, cited Trump’s campaign rhetoric and the accompanying discourse as the likely reason for this behavior.
The Zebra Fact Check investigation suggests PolitiFact was misled about the number of teachers saying Trump was responsible for increasing bullying or harassment. Out of almost 2,000 teachers participating in the survey, 849 answered the question about bullying or biased language and of those 123 mentioned Trump. A fraction of those placed any kind of blame on Trump for anything. We would generously estimate that 25 teachers blamed Trump for something (not necessarily bullying or harassment) in answering that question. This implies that, to PolitiFact, "many" can be less than 1.25 percent of 2,000.
That's right. The hypocritical liberal bloggers at PolitiFact said "many" teachers cited Trump's rhetoric as the likely reason for bullying and/or harassing behavior. PolitiFact shoveled that to its readers as a fact, though the data showed a small fraction of the surveyed teachers offered that opinion. Then. about a month later, PolitiFact said Trump's statement about health care costs was "Mostly False."

Could PolitiFact be right in both cases?

That seems like a stretch.

Wrong in both cases?

That's more likely.

Wednesday, November 9, 2016

Another day, another deceptive PolitiFact chart

On election day, PolitiFact helpfully trotted out a set of its misleading "report card" graphs, including an updated version of its comparison between Democrat Hillary Clinton and Republican Donald Trump.

What is the point of publishing such graphs?

The graphs make an implicit argument to prefer the Democratic Party nominee in the general election. See how much more honest she is! Or, alternatively, see how the Republican tells many falsehoods!

The problem? This is the same PolitiFact deception we have pointed out for years.

The chart amounts to a political ad, making the claim Clinton is more truthful than Trump. But to properly support that conclusion, the underlying data should fairly represent typical political claims from Clinton and Trump--the sort of thing scientific studies capture by randomizing the capture of data.

In the same vein, a scientific study would allow for verification of its ratings. A scientific study would permit this process by using a carefully defined set of ratings. One might then duplicate the results by independently repeating the fact check and reaching the same results.

Yet none of that is possible with these collected "Truth-O-Meter" ratings.

Randomly selected stories aren't likely to grip readers. So editors select the fact-checks to maximize reader interest and/or serve some notion of the public good.

So much for a random sample.

And trying to duplicate the ratings through following objective scientific procedure counts as futile. PolitiFact editor Bill Adair recently confirmed this yet again with the frank admission that "the decision about a Truth-O-Meter rating is entirely subjective."

So much for objectively verifying the results.

PolitiFact passes off graphs of its opinions as though they represent hard data about candidate truthfulness.

This practice ought to offend any conscientious journalist, and that should go double for any conscientious fact-checking journalist.

We have called for PolitiFact to include some type of disclaimer each time it publishes this type of item. Such disclaimers happen only on occasion. The example embedded in this post contains no hint of a disclaimer.

Wonder why Republicans and Trump voters do not trust mainstream media fact-checking?

Take a look in the mirror, PolitiFact.

Saturday, November 5, 2016

PolitiFact founder Bill Adair: "Lord knows the decision about a Truth-O-Meter rating is entirely subjective"

 What have we said for years? PolitiFact's "Truth-O-Meter" rating system is hopelessly subjective.

And now, thanks to an interview by freelance journalist Michael Schulson, PolitiFact's founding editor has made perhaps his clearest statement yet confirming that key charge against PolitiFact (bold emphasis added):
[Michael Schulson]
But there is some subjectivity baked into the process, in terms of which claims you check, and where you draw the line between statements of opinion and statements of fact. Objective journalists are still making subjective choices.

[Bill Adair]
Oh, absolutely. But they always have!

I think that transparency is key. You need to have your own guidelines on how you select what you fact-check.

But yeah, we’re human. We’re making subjective decisions. Lord knows the decision about a Truth-O-Meter rating is entirely subjective. As Angie Holan, the editor of PolitiFact, often says, the Truth-O-Meter is not a scientific instrument.
How often does PolitiFact's Angie Drobnic Holan say the "Truth-O-Meter" is not a scientific instrument? Not nearly enough for our tastes. PolitiFact announces new candidate report card updates and comparisons by the week. But a Google search on Nov. 5, 2016 for "Truth-O-Meter" and "scientific instrument" drew only seven pages of hits. And a good number of those were not directly related to PolitiFact. Many of the rest were duplicates of this page (38 hits).

Would it alter PolitiFact's impact if every "report card" or report card story it published wore the disclaimer that the "Truth-O-Meter" ratings are subjective?

We think it would. Plus, doing so would represent an important step toward full transparency for PolitiFact.

So why don't they do it?

Note: Those who read Schulson's interview of Adair may wish to also read my response.

Friday, November 4, 2016

The Daily Caller: "PolitiFact Used Doctored Clinton Foundation Memo On Its HIV/AIDS Program"

The conservative website The Daily Caller has wound up in a bit of a feud with PolitiFact. The Caller ran an article criticizing the Clinton Foundation. PolitiFact did a fact check of the Caller in response. And the Caller has responded blow for blow.

This one'll leave a mark:
High-ranking officers with the Clinton Foundation gave PolitiFact a doctored version of a 2008 memo lauding its HIV/AIDS program presumably to defend against congressional charges that the charity distributed ‘watered down’ drugs to poor patients on the African continent, according to new information acquired by The Daily Caller News Foundation Investigative Group.

The altered memo went to Politifact Sept. 21, 2016, three days after TheDCNF published a story entitled, “Clinton Foundation AIDS Program Distributed ‘Watered-Down’ Drugs To Third World Countries.” (RELATED: EXCLUSIVE: Clinton Foundation AIDS Program Distributed ‘Watered-Down’ Drugs To Third World Countries)
The Caller also achieved a minor miracle by getting PolitiFact's Aaron Sharockman to respond.
Steps were reportedly taken to verify the authenticity of the Clinton Foundation document, Aaron Sharockman, executive director of PolitiFact, claimed in a statement to TheDCNF.

He told TheDCNF that the memo “was provided to us by the Clinton Foundation in response to our questions,” adding that PolitiFact “verified its authenticity through emails sent and received at that time.”
Read the whole thing here.

Correction/clarification Nov. 6, 2016: "The Daily Caller has wound up a bit of a feud"=>"The Daily Caller has wound up in a bit of a feud"

Thursday, November 3, 2016

PolitiFact: "Mostly False" that many are paying more for health care than for mortgage or rent

Sometimes reading a PolitiFact fact check is like being whisked off to Wonderland for a conversation with the Mad Hatter.

Case in point: Donald Trump says that many Americans are paying more for health care than to pay their rent or mortgage for the first time in history. PolitiFact finds it "Mostly False."

This rating drew our attention right away because of the ambiguity of Trump's claim. How does a fact-check go about making a truth distinction about "many instances"? How many is "many"? And if pinning down "many" poses a challenge, how does one go from that challenge to finding out whether it's happening for the first time in history?

After getting into the text of the fact check, it was a matter of trying to control laughter over the way PolitiFact approached the problem.

Wednesday, November 2, 2016

Fact, motivation, and PolitiFact's inconsistency

One of the oldest legislative tricks involves introducing a bill that will not pass so that one party can slam the member of the opposing party for not supporting one part of the bill.

We've seen the Democrats use that technique to terrific effect with the "Violence Against Women Act." And Republicans do the same type of thing to Democrats.

A Democrat or a Republican might have motivations behind their opposition that undercut the message their opponents try to use against them.

But does PolitiFact treat these same types of campaign ads the same way for both parties?

It sure doesn't look like it.

PolitiFact Missouri today graded a Republican claim in this category "Half True."

Note how PolitiFact Missouri justifies its conclusion (bold emphasis added):
Greitens says Koster voted against a 2007 bill requiring the state to pay for rape victims’ medical exams.In reality, the bill did more than that.

Koster says he objected to wording that made it possible for convicted murderers to be granted parole by claiming they were victims of domestic abuse. Koster said the language made it possible for murderers to manufacture evidence to be released before the completion of their sentence.

Greitens is cherry-picking one part of the legislation to paint his opponent as soft on domestic abuse. We rate his claim Half True.
As we noted back in August, PolitiFact Florida gave a "True" rating to Democrat Patrick Murphy when he made a parallel claim about his Republican opponent:

And note how PolitiFact Florida justifies its conclusion:
Murphy said Rubio "voted against the bipartisan Violence Against Women Act."

Rubio voiced support for the original law, but he and some Republicans in both the Senate and House opposed certain provisions added to the bill pertaining to spending and federal oversight. Rubio voted against the bill in 2012 and 2013, but it passed with bipartisan support the second time.

Even though he had clearly stated his reasons why, Rubio still voted nay. We rate Murphy’s statement True.
Both cases feature the same type of deception, and PolitiFact's fact checkers take note of the deception in both cases. But the Republican gets a "Half True" rating while the Democrat gets a "True" rating.

This type of example isn't atypical. It's just another day at the office for PolitiFact's left-leaning fact checkers.


It's worth pointing out that our previous post shows PolitiFact Wisconsin using essentially this same illicit ad technique against Senator Ron Johnson (R-Wisc.).
The next year, Johnson voted against a Senate amendment to affirm that human activity significantly contributes to climate change.

While all but one senator supported an earlier amendment affirming the existence of climate change, only five Republicans this time voted to acknowledge there is a human impact. The amendment, seen as a symbolic effort by the Democrats to force GOP senators to take a position, failed 50 to 49 (it required a 3/5 majority).
PolitiFact Wisconsin saw nothing wrong with using Johnson's opposition to the amendment as a solid evidence that Johnson thinks humans have no role in climate change even though the amendment did not narrowly address that issue.

Q: What's the difference between PolitiFact and the Democratic Party?
A: The Democratic Party doesn't claim to be nonpartisan.