PolitiFact's Jolly problem

PolitiFact hired David Jolly at some point in February, with the hire date depending on whether his job title was "reader representative" or "Republican guest columnist."

PolitiFact said the hire was intended to build trust in PolitiFact across party lines. We've viewed the experiment with justified skepticism. And Jolly's work so far as the "Republican guest columnist" only solidifies our skepticism.

Jolly's first guest column was published March 2, 2018. We noted that Jolly used that column to address a subject tied to his own advocacy of "common sense gun control." We doubted many PolitiSkeptical conservatives would hear their voices in that column. We judged that Jolly was using his position at PolitiFact to essentially write an op-ed about one of his pet political issues.

Jolly's March 7, 2018 column followed that pattern.

Instead of critiquing PolitiFact, Jolly used his column to attack the target of a PolitiFact fact check. The target of that fact check? President Donald Trump.

Jolly (bold emphasis added):

As the nation continues to debate which gun policies might provide for the safety of our schools and communities, PolitiFact demonstrated in a single column the critical importance fact-checkers serve in both informing the American public as well as holding politicians and advocates on both sides of the debate accountable for their assertions.

Jolly played his gun control theme song in the background again and again we ask: How does Jolly's approach to his columns build trust among conservatives skeptical of PolitiFact? Does he think that just having a Republican say something like "PolitiFact is right" will budge the needle of partisan mistrust?

We'll go out on a limb and predict that approach has a snowball's chance in hell of working.

Conservative mistrust in PolitiFact stems primarily from two factors:

• Conservatives see PolitiFact turning a blind eye to conservative arguments
• PolitiFact commonly makes errors of fact and logic damaging to conservatives

Jolly's column serves as an example of both problems, despite his willingness to identify as a Republican.

Jolly reinforces PolitiFact's left-leaning bias

Jolly lauded PolitiFact for rating "False" the claim that an armed civilian might have stopped the "Pulse" nightclub shooting in Orlando (bold emphasis added):

"You take Pulse nightclub," Trump said. "If you had one person in that room that could carry a gun and knew how to use it, it wouldn’t have happened, or certainly to the extent that it did."

The problem for both the president and his theory is that an armed officer and 15-year veteran of the Orlando Police Department, Adam Gruler, was actually working security at Pulse that fateful night, and indeed engaged the shooter directly with gunfire. Forty-nine people still lost their lives.

PolitiFact rightly rated as False the president's statement that an armed security guard could have saved those 49 victims.

The conservative would have read the PolitiFact fact check looking for evidence of fair treatment of the mainstream conservative point of view. That view is absent, and when Jolly fails to notice its absence and celebrates PolitiFact's fact check in spite of that, the conservative cannot take Jolly's columns any more seriously than the fact check in the first place.

The problem with the fact check

What's the problem with PolitiFact's fact check?

The fact check pretends that the armed guard working security in the Pulse parking lot counts as "one person in that room that could carry a gun and knew how to use it."

Is the Pulse parking lot inside the Pulse nightclub?

Shouldn't that make a difference for non-partisan fact checkers and GOP columnists alike?

Note PolitiFact's description of armed guard Anthony Gruler's involvement in the Pulse incident (bold emphasis added):

The Justice Department in 2017 released a nearly 200-page report detailing the Orlando police response to the shooting. Here’s the report’s account of Gruler’s initial confrontation with Mateen:

"Outside, in the Pulse parking lot, (Gruler), who was working extra duty at the club — to provide outside security and to provide assistance to security personnel inside the club if needed — heard the shots that were being fired; at 2:02:17 a.m., he broadcast over the radio, 'Shots fired, shots fired, shots fired,' and requested additional officers to respond.

"The detective told the assessment team that he immediately recognized that his Sig Sauer P226 9mm handgun was no match for the .223 caliber rifle being fired inside the club and moved to a position that afforded him more cover in the parking lot. Two patrons attempted to flee through an emergency exit on the south side of the club. When the detective saw the suspect shoot them, he fired at the suspect."

According to an Orlando Police Department report, additional officers arrived on the scene about a minute after Gruler’s call for backup was broadcast. A second backup officer arrived about a minute after that.

 PolitiFact's conclusion, sadly, serves as an adequate summary of its argument:

Talking about the Pulse nightclub shooting, Trump said, "If you had one person in that room that could carry a gun and knew how to use it, it wouldn’t have happened, or certainly to the extent that it did."

An armed, off-duty police officer in uniform was at the club during the shooting, and exchanged gunfire with the shooter, who managed to kill 49 people.

We rate this False.

Notice that PolitiFact says Gruler was "at the club," not "outside the club."

The "False" rating gets its premise from the fiction that Gruler was in the same room with Mateen while the latter was murdering club patrons. Jolly, PolitiFact's voice of the GOP, signs on with that falsehood.

In this case the deception comes from PolitiFact and Jolly. Not from Trump.

PolitiFact demonstrated nothing false in Trump's statement, yet pinned a "False" rating on his statement.

Jolly cheered PolitiFact's work, missing the key discrepancy in its fact check.

That's all kinds of wrong.

The first critique from PolitiFact's Republican "guest columnist" David Jolly (Updated)

In early February, PolitiFact tabbed two former Washington D.C. politicians as Democrat and Republican "guest columnists." PolitiFact said the hires were intended to raise its credibility across partisan lines.

We looked the the first column from the Democrat back on Feb. 12 and judged it pointless crap.

Republican David Jolly published his first critique on March 2, 2018. And though we were aware of charges Jolly counts as an "MSNBC Republican," we were still a bit surprised at first blush when Jolly used his column to apparently suggest PolitiFact had rendered a too-harsh judgment on a Democrat's claim about gun violence.

It's worth noting that Jolly has been busy lately promoting "common sense" gun control legislation, so maybe it was just too tempting to resist tying that political cause to his role at PolitiFact.

PolitiFact gave a gun violence statistic from Rep. Ted Deutch (D-Fla) a rating of "Mostly False." Jolly appeared to think Deutch deserved a better fate (bold emphasis added)

Rep. Deutch is said to have relied on statistics provided by The Century Foundation, a left-leaning think tank, which plainly assessed there had been "an increase of over 200%" in mass shootings since the expiration of the assault weapons ban. PolitiFact, however, rated Deutch’s statement Mostly False, largely on the basis that the commentary upon which the congressman relied was substantively challenged by other experts in the field. Presumably, had Deutch said in the town hall, "According to The Century Foundation, mass shootings went up 200% in the decade after the assault weapons ban expired," PolitiFact would have found the Congressman’s statement True on its face, while ruling the findings of The Century Foundation Mostly False.

Moreover, had PolitiFact evaluated Deutch’s statement simply on the numbers, there is ample evidence in the PolitiFact article to support a ruling of Mostly True.

Jolly's critique hits at PolitiFact's standard operating procedure. Yes, of course PolitiFact's ratings oversimplify complexities. Will writing a column pointing that out for the umpteenth time change anything?

Is that the extent of Jolly's point?  That's what his conclusion suggests:

In this case, a congressman’s statement seems to have been ruled Mostly False on two primary factors — his citing a credible think tank’s commentary on gun violence statistics, and a drawn inference by fact-checkers that may or may not have been intended in the congressman’s statement. Neither makes PolitiFact’s ruling right or wrong, but it reflects the enormous challenge faced by politicians, fact-checkers and ultimately voters in today’s political environment.

Newsflash for David Jolly: The fact checkers do not see the subjectivity of their ratings as a problem. The problem, in their eyes, is that readers do not place enough trust in fact checkers. And they recruited their "guest columnists" to help build bipartisan trust in their subjective judgments.

Jolly's critique is about as deep as Barbie's "Math is hard" critique of high school and as such works better as a soft-sell version of his gun control op-ed that we linked above. But we like it for the fact that its premise arguably strikes against the fundamental subjectivity of the PolitiFact approach.

Expect a short run for this PolitiFact experiment if Jolly's future work similarly attacks the PolitiFact premise.


Afters

We learned about Jolly's column through PolitiFact's Facebook page, where it featured a link to his column.

PolitiFact's main page at PolitiFact.com does a pretty awesome job of burying these columns. The reader would have to scroll near the bottom of the page and see the "Inside the Meters" section in the right sidebar.

It's literally the last element on the right sidebar, and there's no way to directly reach the "Inside the Meters" posts using the navigation buttons at the top of the PolitiFact website.

Afters 2

Unsurprisingly, PolitiFact uses an "even the Republican thinks we were too tough on the Democrat" frame in its Facebook announcement of his column:

Note:

See why he said "there is ample evidence ... to support a ruling of "Mostly True."

PolitiFact left out enough context of Jolly's column to earn its own "Half True" (bold emphasis added):

Moreover, had PolitiFact evaluated Deutch’s statement simply on the numbers, there is ample evidence in the PolitiFact article to support a ruling of Mostly True.

Sometimes PolitiFact judges simply on the numbers, sometimes it doesn't. No objective, nonpartisan principle appears to guide the decision.

We suggest PolitiFact's manipulation of Jolly's statement misleads its audience about Jolly's point. Jolly's overall point, albeit delivered weakly, was the difficulty of making the ratings (thanks to subjectivity). PolitiFact spins that into the Republican saying PolitiFact was too hard on the Democrat.

PolitiFact's spin helps stuff Jolly's column into a frame that assists PolitiFact's purpose of fluffing up its own credibility: They're so nonpartisan! They gave a Democrat a "Mostly False" when a Republican said he should have had a "Mostly True!!"

Afters 3

Exit question for David Jolly:

PolitiFact sees your role as a guest columnist as a way for it to help build its own credibility. It has said as much in public statements. Describe the difference between the way you see your role as a guest columnist and the way PolitiFact sees it.



Update: Jeff adds

Cherry-picking the source of a claim

One might assume PolitiFact's newest critic would know something about PolitiFact, until you read PolitiFact's newest critic's newest PolitiFact critique.

Jolly's gripe, and his solution to it, betray a gross ignorance of the way his employer works.

"PolitiFact would have found the Congressman’s statement True on its face, while ruling the findings of The Century Foundation Mostly False."

Jolly finds fault with PolitiFact giving the False rating to Rep. Deutch, when Deutch was simply repeating a claim made by a (Jolly's words) "credible think tank." He argues the rating should have gone to the think tank.

That's nice, but it's also a problem spelled out on these pages for nearly a decade. One of the ways  PolitiFact shows its liberal lean comes from its choice of the source for a claim. We've called that "attribution bias."

In 2011, when The New York Times, ABC, and NPR all repeated a bogus number published in an official Inspector General report regarding $16 muffins, PolitiFact added a "False" rating to conservative Bill O'Reilly's "report card." 

In 2012, when Time, CNN, and the New York Post published a faulty stat from a dubious research firm, it was conservative media group American Crossroads that earned the demerit on its PolitiFact scorecard.

More recently in 2017 PolitiFact ignored a widely publicized liberal talking point espoused by numerous talking heads in the national media for months, and then quickly jumped in to deem it "False" within a day of Donald Trump repeating it.

Though in Deutch's case PolitiFact gave the rating to a Democrat instead of the think tank responsible for the stat, it still exemplifies PolitiFact's subjective, cherry-picking process for assigning its ratings. That Jolly seems to think his observation is novel only highlights his own naivety about PolitiFact.

The critique overall?

 Jolly's overall critique comes off as compelling as warm beer. It's almost as if he quickly cobbled together the column immediately after being criticized for not producing a critique during the first month of PolitiFact's new "readers advocate" feature.

We're not sure what value Jolly adds to the operation here, unless being duped into being a real life version of PolitiFact's lame "we get it from both sides!" defense is his goal. He didn't provide any insight or commentary that hasn't been offered in a better form on this blog for the last several years. We think having an in-house, right-leaning critic at PolitiFact is a good idea and think it could improve its work immensely. So far it appears Jolly is not the vehicle for the improvement.

In any event, congratulations to Jolly for finding a way to get his first critique of PolitiFact from the right to dovetail so nicely with his conservative efforts to support ... gun control.

Top five reasons why David Jolly did not critique PolitiFact in February (Updated)

Update March 2, 2018

We've been curious, especially in light of the gun debate's current popularity, what has happened to PolitiFact's experiment involving "reader advocates." Particularly we noted the absence of Republican David Jolly. We published a post yesterday  [March 1--bww] addressing this curiosity and included a healthy dose of snark. 

Mr. Jolly has since responded via Twitter that his silence was the result of personal matters, including a death in the family and threats he's been receiving.

Our sarcasm was intended to mock the same problems we typically do with PolitiFact. While there's nothing offensive in anything we wrote, we've decided to remove the post in deference to Mr. Jolly's personal circumstances.

We'll continue to use snark to highlight PolitiFact's dubious operation, but in this instance we don't see the value in unnecessary jokes in light of Mr. Jolly's response.
(above by Jeff D.)
Update continued:

Instead of deleting the post or preserving it offsite via the Internet Archive, we're putting it below a page break and changing the text to white (on a white background) to make it mostly invisible to all except the insatiably curious.

Some blue highlights from embedded URLs will remain visible.

The bird's-eye lowdown on PolitiFact's partisan reader advocates (Updated)

Today PolitiFact announced it will publish content from two reader advocates at its PolitiFact.com website.

The announcement didn't go so well. Shortly after making the announcement PolitiFact nixed Democrat Alan Grayson's planned involvement.

The Hill reports:

Fact-checking website PolitiFact on Thursday announced that it had hired former Florida Reps. David Jolly (R) and Alan Grayson (D) as “reader advocates” before hours later nixing Grayson's hire after fierce backlash.

The dumping of Grayson aside, does this represent a sincere effort from PolitiFact to help improve its product?

Probably not, despite the we're-so-humble sales job from PolitiFact Executive Director Aaron Sharockman.

Huh. Well, we intended to link directly to PolitiFact's announcement about its new reader representatives, but it looks like PolitiFact unpublished it. We'll go with the reporting from the Hill instead, which has coincidentally also been altered since its publication: (an earlier version carried a hyperlink to PolitiFact's announcement)

The two former lawmakers had been set to critique the website’s fact-checks and provide additional insight on political issues as part of a pilot program that will run through the end of April, according to a Thursday PolitiFact post announcing the hires. The post has since been deleted.

“David and Alan are both particularly qualified, we think, to critique the work of PolitiFact, because they’ve been subject to our fact-checks as members of Congress,” PolitiFact Executive Director Aaron Sharockman wrote an his initial statement.

Well-qualified to critique the work of PolitiFact! That implies that the critiques may prove valid, right?

PolitiFact said it would learn from Jolly and Grayson, though it's hard for us to prove that now that PolitiFact is disposing of the evidence. Like a good fact-checker should, I guess.

We think PolitiFact's flirtation with reader advocates fits well with its old narrative about how it receives criticism from both sides. Getting criticized from both sides, using mumbo-jumbo logic, shows the reliability of the entity getting criticized. That's a superb narrative for PolitiFact to promote. It's certainly far better than the alternative narrative, that getting criticized by both sides means you're probably doing something wrong.

PolitiFact's founding editor, Bill Adair, did research suggesting that PolitiFact receives most of its substantive criticism from the right. Will PolitiFact allow such research to impact its choice of which narrative to promote? We doubt it.

We captured images from Sharockman's Twitter feed--ones he had so far elected not to delete. The one at the top pretty much shows PolitiFact's thinking behind this experiment. It's not about improving the product. It's about encouraging the public to trust in the existing product.

Note that PolitiFact's experiment was only slated to run through April 2018. Three months. If PolitiFact detects signs that people are trusting it less during the experiment, it will terminate the experiment.

Does that sound like a sincere effort to improve the product? Or more like a cynical ploy intended to trick people into trusting PolitiFact?

How many times do we have to say it? One gains trust by proving trustworthy. One proves one's trustworthiness through accuracy and transparency.

Making published works entirely disappear is not transparency.



Jeff adds:


We wonder what it says about PolitiFact that they considered Grayson representative of a conventional Democrat voice.

We'd also like to congratulate the fact checkers on selecting conservative powerhouse David ... uh ... *checks notes* ... Jolly.

We're not surprised that PolitiFact's ill-advised attempt to gain credibility resulted in more of the same deception we've come to expect. For example, un-publishing articles is somehow a sign of trustworthiness?

For years we've said PolitiFact would benefit from an inside critic of their work and suggested most of their obvious blunders would have been prevented with a heterodox voice on staff. On its face the notion of "reader advocates" seems like a step in that direction, until you realize it's just another click-seeking gimmick (Is Grayson really the top pick for any serious endeavor?).

If PolitiFact were actually sincere about gaining reader trust there's more effective ways than adding sideshow acts performed by clowns and cranks. For instance, they could unequivocally disavow their longtime use of stealth edits. That might help bring them into compliance with the International Fact Checking Network code of principles that they currently violate.

But the most obvious thing they could do to improve their image is to credibly rebut the volumes of legitimate, earnest criticism of their work. So far, PolitiFact's response to charges of bias has been to call critics "mental" or to ignore them altogether. For some reason PolitiFact does not view an honest defense of their work as a viable remedy for reader distrust.

PolitiFact's incompetent and biased editorializing has never earned credibility. Adding more clowns to the car won't change that. This "readers advocate" stunt only shows how unserious PolitiFact is about providing readers the truth.






Edit 0835PST 2/2/2018: Added word "PolitiFact" in penultimate graph -Jeff