Monday, December 5, 2016

Handicapping PolitiFact's "Lie of the Year" for 2016

A full plate of stuff to write about has left me a little behind in getting to PolitiFact's list of finalists for its "Lie of the Year" award--the award that makes it even more obvious that PolitiFact does opinion journalism, since judging the importance of a "lie" obviously requires subjective judgments.

I clipped an image of the most important part of the electronic ballot:

One thing jumped out right away. Typically the list of finalists includes about 10 specific fact checks from a number of sources. This year's menu includes only four specific fact checks, two each from Hillary Clinton and Donald Trump. PolitiFact rounds out the menu with two category choices, the whole 2016 election and the "fake news" phenomenon that, without much hint of irony, has galvanized the mainstream press to make even greater efforts to recapture its (legendary?) role as the the gatekeeper of what people ought to know and accept.

Given PolitiFact's recent tendency to select a "Lie of the Year" made up of multiple finalists, these changes make a great deal of sense. We've already pointed out one of the advantages PolitiFact gains from this approach. Having a multi-headed hydra as the winner allows PolitiFact to dodge criticisms of its choice. Oh, that hydra head got lopped off? No worries. These others continue to writhe and gnash their teeth.

Without further ado, I'll rate the chances of the six listed finalists. Doubtless my co-editor Jeff D. will weigh in at some point with his own comments and predictions.

Clinton "never received nor sent any email that was marked as classified" on her private email server while acting as Secretary of State

Of the specific claims listed, this one probably had the biggest impact on the election. Clinton made this claim a key part of her ongoing defense of her use of the private email server. When FBI Director James Comey contradicted this part of Clinton's story, it cinched one of Clinton's key negatives heading into the 2016 election. This one would serve as a pretty solid choice for "Lie of the Year." The main drawback of this selection stems from liberal denial of Clinton's weakness as a presidential candidate. This choice might generate some lasting resentment  from a significant segment of PolitiFact's liberal fanbase, some of whom will insist Clinton was telling the truth.

Clinton says she received Comey's seal of approval regarding her truthfulness about the email server

This item gave us a notable case where a major political figure made a pretty much indefensible and clear statement that was quickly publicized as such. Was this one politically significant? I think journalists were a bit shocked that Clinton made this unforced error. But I doubt voters regarded this case as anything other than a footnote to Clinton's earlier dissembling about her email server.

Trump claims "large-scale voter fraud"

Talk about awkward!

Trump was pilloried by the mainstream press along with pundits and politicians aplenty for his statements calling the presidential election results into doubt. But the political importance of this one gets complicated by liberal challenges to the election results in states where Trump's margin of victory was not particularly narrow (Michigan, Pennsylvania, and Wisconsin). Why challenge the results if they were not skewed by some form of large-scale fraud? This selection also suffers from the nature of the evidence. Trump received the rating not because it is known that no large-scale voter fraud has taken place in 2016, but because of a lack of evidence supporting the claim.

Donald Trump said he was against the war in Iraq

This one counts as the weakest of the specific fact checks on the list. PolitiFact and its fact-checking brethren built a very weak case that Trump had supported the Iraq War. Making this one by itself the "Lie of the Year" will result in some very good challenges in the mainstream and conservative press.

"The entire 2016 election, when falsehoods overran the facts"

Now things get interesting! Could PolitiFact opt for a "Lie of the Year" awarded to a candidate even more generalized than "campaign statements by Donald Trump," which won in 2016? And does PolitiFact have the ability to objectively quantify this election's overrunning of the facts compared to elections in the past? And could PolitiFact admit that falsehoods overran the facts despite proclamations that fact-checking enjoyed a banner year? If falsehoods overrun the facts while fact checkers enjoy a banner year then what will journalists prescribe to remedy the situation? More of what hasn't worked?

This choice will likely have good traction with PolitiFact's editors if they see a way toward picking this one while avoiding the appearance of admitting failure.

The fake news phenomenon(?)

Fake news has been around a good while, but it's the new hotness in journalistic circles. If mainstream journalism can conquer fake news, then maybe the mainstream press can again take its rightful place as society's gatekeepers of information! That idea excites mainstream journalists.

This surprise nominee has everything going for it. Fake news is fake by definition, so who can criticize the choice? It's total journalistic hotness, as noted. And the choice represents a call to action, opposing fake news, in symphony with a call that is already reverberating in fact-checking circles.

Is it a lame choice? Yes, it's as lame as all get out. I'd doubt journalists even have a clue about the impact of fake news, not to even mention the role fact checkers play in supporting false news memes that liberals favor.


Clinton's claim she never sent or received material marked as classified on her private server is the favorite according to the early established norms of the "Lie of the Year" award. But the fake news choice serves as the clear favorite in terms of sympathy with its Democratic-leaning readership and promoting its own sense of mission. I expect the latter favorite to prevail.

Jeff Adds:

I don't see much to disagree with Bryan. You can dismiss any claim relating to Trump right off the bat. Giving the award to Trump would neither shock people that hate him nor would it upset people that love him (who presumably already have low regard for liberal fact checkers.) It would be a yawner of a pick that would fail to generate buzz.

The Clinton pick would be a favorite in any other year. Because Clinton has already lost the election and her status on the left has been diminished, handing her the award wouldn't do any harm to her, but it would provide PolitiFact with a bogus token of neutrality ("See! We call Democrats liars too!") Likewise, the resulting outrage of PolitiFact's devoted liberal fanbase would generate plenty of clicks, and typically that's what the Lie of the Year has been about. It's true they would temporarily upset the faithful, but we've seen this exact scenario play out before with little consequence. Historically, PolitiFact seems motivated by clicks (and even angry liberal clicks will do, not to mention they keep the "we upset both sides" charade going.)

But the Fake News pick is the obvious favorite here. It's the hottest of hot topics in journalist circles, and PolitiFact sees themselves on the front lines in the war against opposing viewpoints unfacts. They're already trying to rally the troops and want to be seen as a beacon of truthiness in a sea of deceit.

It's been my view that while PolitiFact formerly cared primarily about generating buzz, since Holan's ascension [Angie Drobnic Holan replacing Bill Adair as chief editor--bww] they've behaved more and more like political activists. The Clinton choice would get more clicks, but I'd bet on Fake News being this year's rally cry for PolitiFact's army of Truth Hustlers.

Viva la Factismo!

Monday, November 21, 2016

Great Moments in the Annals of Subjectivity (Updated)

Did Republican Donald Trump win the electoral college in a landslide?

We typically think of a "landslide" as an overwhelming victory, and there's certainly doubt whether Trump's margin of victory in the electoral college unequivocally counts as overwhelming.

"Overwhelming" itself is hard to pin down in objective terms.

So that's why we have PolitiFact, the group of liberal bloggers that puts "fact" in its name and then proceeds to publish "fact check journalism" based on subjective "Truth-O-Meter" judgments.

When RNC Chairman Reince Priebus (and Trump's pick for his chief of staff) called Trump's electoral college victory a "landslide," PolitiFact Wisconsin's liberal bloggers sprang into action to do their thing (bold emphasis added):
Landslide, of course, is not technically defined. When we asked for information to back Priebus’ claim, the Republican National Committee merely recited the electoral figures and repeated that it was a landslide.
If "landslide" is not technically defined then what fact is PolitiFact Wisconsin checking? Is "landslide" non-technically defined to the point one can judge it true or false?

PolitiFact Wisconsin follows typical PolitiFact procedure in collecting expert opinions about whether Priebus' use of "landslide" matches its non-technical definition. One of the 10 experts PolitiFact consulted said Trump's margin was "close" to a landslide. PolitiFact said the other nine said it fell short, so PolitiFact ruled Priebus' claim "False."
Priebus said Trump’s win was "an electoral landslide."

But aside from the fact Trump lost the popular vote, his margin in the Electoral College isn’t all that high, either. None of the 10 experts we contacted said Trump’s win crosses that threshold.

We rate Priebus’ claim False.
One has to marvel at expertise sufficient to say whether the use of a term meets a non-technical definition.

One has to marvel all the more at fact checkers who concede that a term has a mushy definition ("not technically defined") and then declare that some use of the term fails to cross "that threshold."

What threshold?

One of the election experts said if Trump won by a landslide then Obama won by an even greater landslide.

RollCall, 2015:
In 2006, Democrats won back the House; two years later, President Barack Obama won by a landslide.
LA Times, 2012:
Obama officially wins in electoral vote landslide.
NPR, 2015:
President Obama won in a landslide.
NYU Journalism, 2008:
Obama Wins Landslide Victory, Charts New Course for United States.
Since Obama did not win by a landslide, therefore one cannot claim Trump won by a landslide? Is that it?

It is folly for fact checkers to try to judge the truth of ambiguous claims. PolitiFact often pursues that folly, of course, and in the end simply underscores what it occasionally admits: The ratings are subjective.

Finding experts willing to participate in the folly does not reduce the magnitude of the folly. This would have been a good subject for PolitiFact to use in continuing its Voxification trend. PolitiFact might have produced an "In context" article to talk about electoral landslides and how experts view the matter. But trying to corral the use of a term that is traditionally hard to tame simply makes a mockery of fact-checking.

Jeff Adds (Dec. 1, 2016):

Add this to a long list of opinions that PolitiFact treats as verifiable facts, including these two gems:

- Radio host John DePetro opined that the Boston Marathon bomber was buried "not far" from President John Kennedy. PolitiFact used their magical powers of objective divinity to determine the unarguable demarcation of "not far."

- Rush Limbaugh claimed "some of the wealthiest American's are African-Americans now." Using the divine wizardry of the nonpartisan Truth-O-Meter, PolitiFact's highly trained social scientists were able to conjure up a determinant definition of what "wealthiest" means, and specifically which people were included in the list.

Reasonable people may discount Trump's claim of a "landslide" victory assuming the conventional use of the term, but it's not a verifiable fact that can be confirmed or dismissed with evidence. It's an opinion.

The reality is that the charlatans at PolitiFact masquerade as truthsayers when they do little more than contribute to the supposed fake news epidemic by shilling their own opinions as unarguable fact. They're dangerous frauds whose declaration of objectivity doesn't withstand the slightest scrutiny.

Sunday, November 13, 2016

PolitiFact's "many" problems

On Nov. 3, 2016 we brought some focus to the "Mostly False" rating PolitiFact gave Donald Trump for saying many Americans were paying more for health care than for their mortgage or rent.

PFB co-editor Jeff D. today reminded me about an "Afters" section I added to a post from Sept. 1, 2016:
PolitiFact exaggerated the survey evidence supposedly supporting Clinton by claiming "many" teachers blamed Trump for increasing bullying and harassment:
Many of these teachers, unsolicited, cited Trump’s campaign rhetoric and the accompanying discourse as the likely reason for this behavior.
The Zebra Fact Check investigation suggests PolitiFact was misled about the number of teachers saying Trump was responsible for increasing bullying or harassment. Out of almost 2,000 teachers participating in the survey, 849 answered the question about bullying or biased language and of those 123 mentioned Trump. A fraction of those placed any kind of blame on Trump for anything. We would generously estimate that 25 teachers blamed Trump for something (not necessarily bullying or harassment) in answering that question. This implies that, to PolitiFact, "many" can be less than 1.25 percent of 2,000.
That's right. The hypocritical liberal bloggers at PolitiFact said "many" teachers cited Trump's rhetoric as the likely reason for bullying and/or harassing behavior. PolitiFact shoveled that to its readers as a fact, though the data showed a small fraction of the surveyed teachers offered that opinion. Then. about a month later, PolitiFact said Trump's statement about health care costs was "Mostly False."

Could PolitiFact be right in both cases?

That seems like a stretch.

Wrong in both cases?

That's more likely.

Wednesday, November 9, 2016

Another day, another deceptive PolitiFact chart

On election day, PolitiFact helpfully trotted out a set of its misleading "report card" graphs, including an updated version of its comparison between Democrat Hillary Clinton and Republican Donald Trump.

What is the point of publishing such graphs?

The graphs make an implicit argument to prefer the Democratic Party nominee in the general election. See how much more honest she is! Or, alternatively, see how the Republican tells many falsehoods!

The problem? This is the same PolitiFact deception we have pointed out for years.

The chart amounts to a political ad, making the claim Clinton is more truthful than Trump. But to properly support that conclusion, the underlying data should fairly represent typical political claims from Clinton and Trump--the sort of thing scientific studies capture by randomizing the capture of data.

In the same vein, a scientific study would allow for verification of its ratings. A scientific study would permit this process by using a carefully defined set of ratings. One might then duplicate the results by independently repeating the fact check and reaching the same results.

Yet none of that is possible with these collected "Truth-O-Meter" ratings.

Randomly selected stories aren't likely to grip readers. So editors select the fact-checks to maximize reader interest and/or serve some notion of the public good.

So much for a random sample.

And trying to duplicate the ratings through following objective scientific procedure counts as futile. PolitiFact editor Bill Adair recently confirmed this yet again with the frank admission that "the decision about a Truth-O-Meter rating is entirely subjective."

So much for objectively verifying the results.

PolitiFact passes off graphs of its opinions as though they represent hard data about candidate truthfulness.

This practice ought to offend any conscientious journalist, and that should go double for any conscientious fact-checking journalist.

We have called for PolitiFact to include some type of disclaimer each time it publishes this type of item. Such disclaimers happen only on occasion. The example embedded in this post contains no hint of a disclaimer.

Wonder why Republicans and Trump voters do not trust mainstream media fact-checking?

Take a look in the mirror, PolitiFact.

Saturday, November 5, 2016

PolitiFact founder Bill Adair: "Lord knows the decision about a Truth-O-Meter rating is entirely subjective"

 What have we said for years? PolitiFact's "Truth-O-Meter" rating system is hopelessly subjective.

And now, thanks to an interview by freelance journalist Michael Schulson, PolitiFact's founding editor has made perhaps his clearest statement yet confirming that key charge against PolitiFact (bold emphasis added):
[Michael Schulson]
But there is some subjectivity baked into the process, in terms of which claims you check, and where you draw the line between statements of opinion and statements of fact. Objective journalists are still making subjective choices.

[Bill Adair]
Oh, absolutely. But they always have!

I think that transparency is key. You need to have your own guidelines on how you select what you fact-check.

But yeah, we’re human. We’re making subjective decisions. Lord knows the decision about a Truth-O-Meter rating is entirely subjective. As Angie Holan, the editor of PolitiFact, often says, the Truth-O-Meter is not a scientific instrument.
How often does PolitiFact's Angie Drobnic Holan say the "Truth-O-Meter" is not a scientific instrument? Not nearly enough for our tastes. PolitiFact announces new candidate report card updates and comparisons by the week. But a Google search on Nov. 5, 2016 for "Truth-O-Meter" and "scientific instrument" drew only seven pages of hits. And a good number of those were not directly related to PolitiFact. Many of the rest were duplicates of this page (38 hits).

Would it alter PolitiFact's impact if every "report card" or report card story it published wore the disclaimer that the "Truth-O-Meter" ratings are subjective?

We think it would. Plus, doing so would represent an important step toward full transparency for PolitiFact.

So why don't they do it?

Note: Those who read Schulson's interview of Adair may wish to also read my response.

Friday, November 4, 2016

The Daily Caller: "PolitiFact Used Doctored Clinton Foundation Memo On Its HIV/AIDS Program"

The conservative website The Daily Caller has wound up in a bit of a feud with PolitiFact. The Caller ran an article criticizing the Clinton Foundation. PolitiFact did a fact check of the Caller in response. And the Caller has responded blow for blow.

This one'll leave a mark:
High-ranking officers with the Clinton Foundation gave PolitiFact a doctored version of a 2008 memo lauding its HIV/AIDS program presumably to defend against congressional charges that the charity distributed ‘watered down’ drugs to poor patients on the African continent, according to new information acquired by The Daily Caller News Foundation Investigative Group.

The altered memo went to Politifact Sept. 21, 2016, three days after TheDCNF published a story entitled, “Clinton Foundation AIDS Program Distributed ‘Watered-Down’ Drugs To Third World Countries.” (RELATED: EXCLUSIVE: Clinton Foundation AIDS Program Distributed ‘Watered-Down’ Drugs To Third World Countries)
The Caller also achieved a minor miracle by getting PolitiFact's Aaron Sharockman to respond.
Steps were reportedly taken to verify the authenticity of the Clinton Foundation document, Aaron Sharockman, executive director of PolitiFact, claimed in a statement to TheDCNF.

He told TheDCNF that the memo “was provided to us by the Clinton Foundation in response to our questions,” adding that PolitiFact “verified its authenticity through emails sent and received at that time.”
Read the whole thing here.

Correction/clarification Nov. 6, 2016: "The Daily Caller has wound up a bit of a feud"=>"The Daily Caller has wound up in a bit of a feud"

Thursday, November 3, 2016

PolitiFact: "Mostly False" that many are paying more for health care than for mortgage or rent

Sometimes reading a PolitiFact fact check is like being whisked off to Wonderland for a conversation with the Mad Hatter.

Case in point: Donald Trump says that many Americans are paying more for health care than to pay their rent or mortgage for the first time in history. PolitiFact finds it "Mostly False."

This rating drew our attention right away because of the ambiguity of Trump's claim. How does a fact-check go about making a truth distinction about "many instances"? How many is "many"? And if pinning down "many" poses a challenge, how does one go from that challenge to finding out whether it's happening for the first time in history?

After getting into the text of the fact check, it was a matter of trying to control laughter over the way PolitiFact approached the problem.