Monday, November 21, 2016

Great Moments in the Annals of Subjectivity (Updated)

Did Republican Donald Trump win the electoral college in a landslide?

We typically think of a "landslide" as an overwhelming victory, and there's certainly doubt whether Trump's margin of victory in the electoral college unequivocally counts as overwhelming.

"Overwhelming" itself is hard to pin down in objective terms.

So that's why we have PolitiFact, the group of liberal bloggers that puts "fact" in its name and then proceeds to publish "fact check journalism" based on subjective "Truth-O-Meter" judgments.

When RNC Chairman Reince Priebus (and Trump's pick for his chief of staff) called Trump's electoral college victory a "landslide," PolitiFact Wisconsin's liberal bloggers sprang into action to do their thing (bold emphasis added):
Landslide, of course, is not technically defined. When we asked for information to back Priebus’ claim, the Republican National Committee merely recited the electoral figures and repeated that it was a landslide.
If "landslide" is not technically defined then what fact is PolitiFact Wisconsin checking? Is "landslide" non-technically defined to the point one can judge it true or false?

PolitiFact Wisconsin follows typical PolitiFact procedure in collecting expert opinions about whether Priebus' use of "landslide" matches its non-technical definition. One of the 10 experts PolitiFact consulted said Trump's margin was "close" to a landslide. PolitiFact said the other nine said it fell short, so PolitiFact ruled Priebus' claim "False."
Priebus said Trump’s win was "an electoral landslide."

But aside from the fact Trump lost the popular vote, his margin in the Electoral College isn’t all that high, either. None of the 10 experts we contacted said Trump’s win crosses that threshold.

We rate Priebus’ claim False.
One has to marvel at expertise sufficient to say whether the use of a term meets a non-technical definition.

One has to marvel all the more at fact checkers who concede that a term has a mushy definition ("not technically defined") and then declare that some use of the term fails to cross "that threshold."

What threshold?

One of the election experts said if Trump won by a landslide then Obama won by an even greater landslide.

RollCall, 2015:
In 2006, Democrats won back the House; two years later, President Barack Obama won by a landslide.
LA Times, 2012:
Obama officially wins in electoral vote landslide.
NPR, 2015:
President Obama won in a landslide.
NYU Journalism, 2008:
Obama Wins Landslide Victory, Charts New Course for United States.
Since Obama did not win by a landslide, therefore one cannot claim Trump won by a landslide? Is that it?

It is folly for fact checkers to try to judge the truth of ambiguous claims. PolitiFact often pursues that folly, of course, and in the end simply underscores what it occasionally admits: The ratings are subjective.

Finding experts willing to participate in the folly does not reduce the magnitude of the folly. This would have been a good subject for PolitiFact to use in continuing its Voxification trend. PolitiFact might have produced an "In context" article to talk about electoral landslides and how experts view the matter. But trying to corral the use of a term that is traditionally hard to tame simply makes a mockery of fact-checking.


Jeff Adds (Dec. 1, 2016):

Add this to a long list of opinions that PolitiFact treats as verifiable facts, including these two gems:

- Radio host John DePetro opined that the Boston Marathon bomber was buried "not far" from President John Kennedy. PolitiFact used their magical powers of objective divinity to determine the unarguable demarcation of "not far."

- Rush Limbaugh claimed "some of the wealthiest American's are African-Americans now." Using the divine wizardry of the nonpartisan Truth-O-Meter, PolitiFact's highly trained social scientists were able to conjure up a determinant definition of what "wealthiest" means, and specifically which people were included in the list.

Reasonable people may discount Trump's claim of a "landslide" victory assuming the conventional use of the term, but it's not a verifiable fact that can be confirmed or dismissed with evidence. It's an opinion.

The reality is that the charlatans at PolitiFact masquerade as truthsayers when they do little more than contribute to the supposed fake news epidemic by shilling their own opinions as unarguable fact. They're dangerous frauds whose declaration of objectivity doesn't withstand the slightest scrutiny.

Sunday, November 13, 2016

PolitiFact's "many" problems

On Nov. 3, 2016 we brought some focus to the "Mostly False" rating PolitiFact gave Donald Trump for saying many Americans were paying more for health care than for their mortgage or rent.

PFB co-editor Jeff D. today reminded me about an "Afters" section I added to a post from Sept. 1, 2016:
PolitiFact exaggerated the survey evidence supposedly supporting Clinton by claiming "many" teachers blamed Trump for increasing bullying and harassment:
Many of these teachers, unsolicited, cited Trump’s campaign rhetoric and the accompanying discourse as the likely reason for this behavior.
The Zebra Fact Check investigation suggests PolitiFact was misled about the number of teachers saying Trump was responsible for increasing bullying or harassment. Out of almost 2,000 teachers participating in the survey, 849 answered the question about bullying or biased language and of those 123 mentioned Trump. A fraction of those placed any kind of blame on Trump for anything. We would generously estimate that 25 teachers blamed Trump for something (not necessarily bullying or harassment) in answering that question. This implies that, to PolitiFact, "many" can be less than 1.25 percent of 2,000.
That's right. The hypocritical liberal bloggers at PolitiFact said "many" teachers cited Trump's rhetoric as the likely reason for bullying and/or harassing behavior. PolitiFact shoveled that to its readers as a fact, though the data showed a small fraction of the surveyed teachers offered that opinion. Then. about a month later, PolitiFact said Trump's statement about health care costs was "Mostly False."

Could PolitiFact be right in both cases?

That seems like a stretch.

Wrong in both cases?

That's more likely.

Wednesday, November 9, 2016

Another day, another deceptive PolitiFact chart

On election day, PolitiFact helpfully trotted out a set of its misleading "report card" graphs, including an updated version of its comparison between Democrat Hillary Clinton and Republican Donald Trump.

What is the point of publishing such graphs?

The graphs make an implicit argument to prefer the Democratic Party nominee in the general election. See how much more honest she is! Or, alternatively, see how the Republican tells many falsehoods!

The problem? This is the same PolitiFact deception we have pointed out for years.

The chart amounts to a political ad, making the claim Clinton is more truthful than Trump. But to properly support that conclusion, the underlying data should fairly represent typical political claims from Clinton and Trump--the sort of thing scientific studies capture by randomizing the capture of data.

In the same vein, a scientific study would allow for verification of its ratings. A scientific study would permit this process by using a carefully defined set of ratings. One might then duplicate the results by independently repeating the fact check and reaching the same results.

Yet none of that is possible with these collected "Truth-O-Meter" ratings.

Randomly selected stories aren't likely to grip readers. So editors select the fact-checks to maximize reader interest and/or serve some notion of the public good.

So much for a random sample.

And trying to duplicate the ratings through following objective scientific procedure counts as futile. PolitiFact editor Bill Adair recently confirmed this yet again with the frank admission that "the decision about a Truth-O-Meter rating is entirely subjective."

So much for objectively verifying the results.

PolitiFact passes off graphs of its opinions as though they represent hard data about candidate truthfulness.

This practice ought to offend any conscientious journalist, and that should go double for any conscientious fact-checking journalist.

We have called for PolitiFact to include some type of disclaimer each time it publishes this type of item. Such disclaimers happen only on occasion. The example embedded in this post contains no hint of a disclaimer.

Wonder why Republicans and Trump voters do not trust mainstream media fact-checking?

Take a look in the mirror, PolitiFact.

Saturday, November 5, 2016

PolitiFact founder Bill Adair: "Lord knows the decision about a Truth-O-Meter rating is entirely subjective"

 What have we said for years? PolitiFact's "Truth-O-Meter" rating system is hopelessly subjective.

And now, thanks to an interview by freelance journalist Michael Schulson, PolitiFact's founding editor has made perhaps his clearest statement yet confirming that key charge against PolitiFact (bold emphasis added):
[Michael Schulson]
But there is some subjectivity baked into the process, in terms of which claims you check, and where you draw the line between statements of opinion and statements of fact. Objective journalists are still making subjective choices.

[Bill Adair]
Oh, absolutely. But they always have!

I think that transparency is key. You need to have your own guidelines on how you select what you fact-check.

But yeah, we’re human. We’re making subjective decisions. Lord knows the decision about a Truth-O-Meter rating is entirely subjective. As Angie Holan, the editor of PolitiFact, often says, the Truth-O-Meter is not a scientific instrument.
How often does PolitiFact's Angie Drobnic Holan say the "Truth-O-Meter" is not a scientific instrument? Not nearly enough for our tastes. PolitiFact announces new candidate report card updates and comparisons by the week. But a Google search on Nov. 5, 2016 for "Truth-O-Meter" and "scientific instrument" drew only seven pages of hits. And a good number of those were not directly related to PolitiFact. Many of the rest were duplicates of this page (38 hits).

Would it alter PolitiFact's impact if every "report card" or report card story it published wore the disclaimer that the "Truth-O-Meter" ratings are subjective?

We think it would. Plus, doing so would represent an important step toward full transparency for PolitiFact.

So why don't they do it?

Note: Those who read Schulson's interview of Adair may wish to also read my response.

Friday, November 4, 2016

The Daily Caller: "PolitiFact Used Doctored Clinton Foundation Memo On Its HIV/AIDS Program"

The conservative website The Daily Caller has wound up in a bit of a feud with PolitiFact. The Caller ran an article criticizing the Clinton Foundation. PolitiFact did a fact check of the Caller in response. And the Caller has responded blow for blow.

This one'll leave a mark:
High-ranking officers with the Clinton Foundation gave PolitiFact a doctored version of a 2008 memo lauding its HIV/AIDS program presumably to defend against congressional charges that the charity distributed ‘watered down’ drugs to poor patients on the African continent, according to new information acquired by The Daily Caller News Foundation Investigative Group.

The altered memo went to Politifact Sept. 21, 2016, three days after TheDCNF published a story entitled, “Clinton Foundation AIDS Program Distributed ‘Watered-Down’ Drugs To Third World Countries.” (RELATED: EXCLUSIVE: Clinton Foundation AIDS Program Distributed ‘Watered-Down’ Drugs To Third World Countries)
The Caller also achieved a minor miracle by getting PolitiFact's Aaron Sharockman to respond.
Steps were reportedly taken to verify the authenticity of the Clinton Foundation document, Aaron Sharockman, executive director of PolitiFact, claimed in a statement to TheDCNF.

He told TheDCNF that the memo “was provided to us by the Clinton Foundation in response to our questions,” adding that PolitiFact “verified its authenticity through emails sent and received at that time.”
Read the whole thing here.



Correction/clarification Nov. 6, 2016: "The Daily Caller has wound up a bit of a feud"=>"The Daily Caller has wound up in a bit of a feud"

Thursday, November 3, 2016

PolitiFact: "Mostly False" that many are paying more for health care than for mortgage or rent

Sometimes reading a PolitiFact fact check is like being whisked off to Wonderland for a conversation with the Mad Hatter.

Case in point: Donald Trump says that many Americans are paying more for health care than to pay their rent or mortgage for the first time in history. PolitiFact finds it "Mostly False."


This rating drew our attention right away because of the ambiguity of Trump's claim. How does a fact-check go about making a truth distinction about "many instances"? How many is "many"? And if pinning down "many" poses a challenge, how does one go from that challenge to finding out whether it's happening for the first time in history?

After getting into the text of the fact check, it was a matter of trying to control laughter over the way PolitiFact approached the problem.

Wednesday, November 2, 2016

Fact, motivation, and PolitiFact's inconsistency

One of the oldest legislative tricks involves introducing a bill that will not pass so that one party can slam the member of the opposing party for not supporting one part of the bill.

We've seen the Democrats use that technique to terrific effect with the "Violence Against Women Act." And Republicans do the same type of thing to Democrats.

A Democrat or a Republican might have motivations behind their opposition that undercut the message their opponents try to use against them.

But does PolitiFact treat these same types of campaign ads the same way for both parties?

It sure doesn't look like it.

PolitiFact Missouri today graded a Republican claim in this category "Half True."


Note how PolitiFact Missouri justifies its conclusion (bold emphasis added):
Greitens says Koster voted against a 2007 bill requiring the state to pay for rape victims’ medical exams.In reality, the bill did more than that.

Koster says he objected to wording that made it possible for convicted murderers to be granted parole by claiming they were victims of domestic abuse. Koster said the language made it possible for murderers to manufacture evidence to be released before the completion of their sentence.

Greitens is cherry-picking one part of the legislation to paint his opponent as soft on domestic abuse. We rate his claim Half True.
As we noted back in August, PolitiFact Florida gave a "True" rating to Democrat Patrick Murphy when he made a parallel claim about his Republican opponent:


And note how PolitiFact Florida justifies its conclusion:
Murphy said Rubio "voted against the bipartisan Violence Against Women Act."

Rubio voiced support for the original law, but he and some Republicans in both the Senate and House opposed certain provisions added to the bill pertaining to spending and federal oversight. Rubio voted against the bill in 2012 and 2013, but it passed with bipartisan support the second time.

Even though he had clearly stated his reasons why, Rubio still voted nay. We rate Murphy’s statement True.
Both cases feature the same type of deception, and PolitiFact's fact checkers take note of the deception in both cases. But the Republican gets a "Half True" rating while the Democrat gets a "True" rating.

This type of example isn't atypical. It's just another day at the office for PolitiFact's left-leaning fact checkers.


Afters

It's worth pointing out that our previous post shows PolitiFact Wisconsin using essentially this same illicit ad technique against Senator Ron Johnson (R-Wisc.).
The next year, Johnson voted against a Senate amendment to affirm that human activity significantly contributes to climate change.

While all but one senator supported an earlier amendment affirming the existence of climate change, only five Republicans this time voted to acknowledge there is a human impact. The amendment, seen as a symbolic effort by the Democrats to force GOP senators to take a position, failed 50 to 49 (it required a 3/5 majority).
PolitiFact Wisconsin saw nothing wrong with using Johnson's opposition to the amendment as a solid evidence that Johnson thinks humans have no role in climate change even though the amendment did not narrowly address that issue.

Q: What's the difference between PolitiFact and the Democratic Party?
A: The Democratic Party doesn't claim to be nonpartisan.

Tuesday, November 1, 2016

More PolitiFact climate change shenanigans, featuring PolitiFact Wisconsin

One of PolitiFact's more reliable bends to the left occurs on the issue of climate change. The arbiters of truth, for example, class Republicans as climate change deniers if they do not go on record affirming man-made climate change. So much for PolitiFact's burden of proof criterion, right?

Hypocrites.

This related example comes from PolitiFact Wisconsin, checking on a claim from Senate candidate Russ Feingold that his Republican opponent does not believe humans contribute to climate change.


PolitiFact Wisconsin's approach to the fact check resembles the incompetent methods used by other iterations of the PolitiFact family. A Zebra Fact Check critique of PolitiFact's past fact check foreshadows the problems with PolitiFact Wisconsin's fact check of Feingold:
First, interpret an unclear statement according to a more clear statement by the same source.  Second, in judging what a person thinks in the present place greater weight on more recent statements.
PolitiFact Wisconsin does not apply these commonsense principles.

PolitiFact Wisconsin's evidences, in chronological order

2010
Johnson: "I absolutely do not believe in the science of man-caused climate change. It’s not proven by any stretch of the imagination."

2014
"There are other forces that cause climate to change," Johnson told Here and Now’s Robin Young. "So climate does change and I don’t deny that man has some effect on that. It certainly has a great deal of effect on spoiling our environment in many different ways."

But Johnson softened his view as soon as the next sentence: "I’ve got a very open mind, but I don’t have the arrogance that man can really do much to affect climate."
2015
Johnson votes against a proposed amendment to a bill touching the Keystone Pipeline. The amendment would have described the sense of the Senate on the issue of anthropogenic climate change, including the ideas that humans "significantly" affect the climate and that climate change increases the severity of extreme weather events (such as hurricanes).

2015
"Man-made global warming remains unsettled science. World-renowned climate experts have raised serious objections to the theories behind these claims. I believe it is a bad idea to impose a policy that will raise taxes on every American, will balloon energy prices and will hurt our economic competiveness (sic) – especially on such uncertain predictions."
2016
"Listen, man can affect the environment; no doubt about it," he said. "The climate has always changed, it always will. … The question is, how much does man cause changes in our environment, changes in our climate, and what we could possibly even do about it?"

Assessing PolitiFact Wisconsin's evidences

Following the principles mentioned above, Johnson's clearest statements on humans having some role in climate change comes from the 2014 and 2016 quotations. In 2014, Johnson said he does not deny humans have a role in climate change. In 2016, Johnson said humans "clearly" have a role in changing the climate. Johnson's clearest statements on the subject directly contradict Feingold's claim.

Our principles also guide us toward giving a preference to more recent statements. Therefore, we consider the 2015 climate change amendment for some sign that Johnson denied a human role in climate change.

Is there a worse proof of a legislator's specific views on a topic than their willingness to vote in favor of a "sense of the Senate" amendment? Particularly when that amendment does not feature language narrowly tailored to suit the question?

Would Johnson have voted in favor of the amendment if he believed there was good evidence that undefined "climate change" causes an increase in severe weather events? Who knows? We don't. But if you're PolitiFact Wisconsin you can simply assume the answer is "no" and call it fact-checking.

PolitiFact concludes:
Johnson did not support a Senate amendment to acknowledge a man-made role in climate change and expressed skepticism each of the few times he acknowledged humans might contribute. He has acknowledged at times that humans can play a role but downplayed how significant that role might be.

For a statement that is accurate but needs additional clarification, our rating is Mostly True.

PolitiFact's conclusion consists of spin.

The Senate amendment was not simply about "a man-made role in climate change." It stipulated a significant role as well as a worsening effect on severe weather.

When Johnson said humans play a role in climate change he did not express skepticism about whether humans play a role. He expressed skepticism about the extent of that role. They're not the same thing, and skepticism about the latter does not contradict Johnson's recognition that humans play some role in affecting the climate. PolitiFact says Johnson says humans "can" play a role. But that's just more spin. Johnson did not simply say humans "can" play a role. He said humans do play a role, and he said he does not deny humans play a role.

If Johnson says humans play some role in causing climate change, that statement cannot support Feingold's claim that Johnson does not believe humans play any role in climate change.

Johnson's statement cannot reasonably justify the "Mostly True" rating with which PolitiFact Wisconsin gifted Feingold. The statements could reasonably justify "False" or "Mostly False" ratings if PolitiFact's definitions for its ratings meant something.


PolitiFact's continued inability to apply simple logic in the course of its fact checks continues to boggle our minds. At the same time, we're not surprised. This is the type of error that results when left-leaning journalists rate the truth of political statements on a subjective scale.