Friday, June 22, 2018

PolitiFact Corrects, We Evaluate the Correction

PolitiFact corrected an error in one of its fact checks this past week, most likely in response to an email we sent on June 20, 2018.
Dear PolitiFact,

A recent PolitiFact fact check contains the following paragraph (bold emphasis added):
Soon after, in February 2017, Nehlen wrote on Twitter that Islam was not a religion of peace and posted a photo of a plane striking the World Trade Center with the caption, "9/11 would’ve been a Wonderful #DayWithoutImmigrants." In the following months, Nehlen also tweeted that "Islam is not your friend," implied that Muslim communities should be bombed and retweeted posts saying Bill and Hillary Clinton were murdering associates.

The hotlink ("implied") leads to an archived Twitter page. Unless I'm missing somelthing [sic], the following represents the best candidate as a supporting evidence:


Unless "Muslim no-go zones" represent typical Muslim communities, PolitiFact's summary of Nehlen's tweet distorts the truth. If a politician similarly omitted context in this fashion, would PolitiFact not mete out a "Half True" rating or worse?

If PolitiFact excuses itself from telling the truth where people accused of bigotry are involved, that principle ought to appear in its statement of principles.

Otherwise, a correction or clarification is in order. Thanks.
We were surprised to see that PolitiFact updated the story with a clarification within two days. And PolitiFact did most things right with the fix, which it labeled a "clarification."

Here's a checklist:
  1. Paid attention to the criticism
  2. Updated the article with a clarification
  3. Attached a clarification notice at the bottom of the fact check
  4. Added the "Corrections and Updates" tag to the article, ensuring it would appear on PolitiFact's "Corrections and Updates" page
Still, we think PolitiFact can do better.

Specifically, we fault PolitiFact for its lack of transparency regarding the specifics of the mistake.

Note what Craig Silverman, long associated with PolitiFact's owner, the Poynter Institute, said in an American Press Institute interview about letting readers know what changed:

News organizations aren’t the only ones on the internet who are practicing some form of journalism. There are a number of sites or blogs or individual bloggers who may not have the same standards for corrections. Is there any way journalists or anyone else can contribute to a culture of corrections? Where does it start?

SILVERMAN: Bloggers actually ended up doing a little bit of correction innovation. In the relatively early blogging days, you’d often see <strike>strikethrough</strike> used to cross out a typo or error. This was a lovely use of the medium, as it showed what was incorrect and also included the correct information after. In that respect, bloggers modelled good behavior, and showed how digital corrections can work. We can learn from that.

It all starts with a broad commitment to acknowledge and even publicize mistakes. That is the core of the culture, the ethic of correction.
We think Silverman has it right. Transparency in corrections involves letting the reader know what the story got wrong. In this case, PolitiFact reported that a tweet implied that somebody wanted to bomb Muslim communities. The tweet referred, in fact, to a small subset of Muslim communities (so small PolitiFact says they do not exist hey that one failed its fact check and I forgot to remove it from the first published draft) referred to as "no-go zones"--areas where non-Muslims allegedly face unusual danger to their person and property.

PolitiFact explained its error like this:
This fact-check has been updated to more precisely refer to a previous Nehlen tweet
That notice is transparent about the fact the text of the fact check was changed and transparent about the part of the fact check that was changed (information about a Nehlen tweet). But it mostly lacked transparency about what the fact check got wrong and the misleading impression it created.

We think journalists, including PolitiFact, stand to gain public trust by full transparency regarding errors. Though that boost to public trust assumes that errors aren't so ridiculous and rampant that transparency instead destroys the organization's credibility.

Is that what PolitiFact fears when it issues these vague descriptions of its inaccuracies?

Still, we're encouraged that PolitiFact performed a clarification and mostly followed its corrections policy. Ignoring needed corrections is worse than falling short of best practices with the corrections.

Monday, June 18, 2018

PolitiFact Wisconsin: The Future is Now!

A May 2, 2018 fact check from PolitiFact Wisconsin uses projected numbers from the 2018-2019 budget year to assess a claim that Wisconsinites are now paying twice as much for debt service on road work as they were paying in 2010-2011 before Republican Scott Walker took over as Wisconsin's governor.


Democratic candidate for governor Kelda Helen Roys and her interviewer used a 22-23 percent figure to represent current spending on road work debt service in Wisconsin.

PolitiFact Wisconsin gave both a pass on their fudging of the facts, but lowered Roys' rating from "True" down to "Mostly True" because the numbers used were mere estimates:
The figure is projected to reach 20.9 percent during the second year of the current two-year state budget Walker signed, which is nearly doubling.

With the caveat that the figure for the current budget is an estimate, we rate Roys’ statement Mostly True.
We think that reasoning would work better as a fact check of Roys' claim if the estimated number represented what Wisconsin is paying now for debt service on its road work. Unless PolitiFact Wisconsin is saying the future is now, the estimate for budget year 2017-2018 would better fit the bill.

PolitiFact Wisconsin reported the 2017-2018 estimate as 20 percent but used the higher figure for the following budget year to judge Roys' accuracy.

And that was just one of three ways PolitiFact Wisconsin massaged the Democrat's statement into a closer semblance of the truth.

What is "Just Basic Road Repair and Maintenance"?

Roys' claimed the debt service was "for just basic road repair and maintenance," which would apparently exclude new construction. PolitiFact tested her claim using the numbers for the transportation-related share of the budget (bold emphasis added):
In analyzing 2017-’19 two-year state budget enacted by Walker and the GOP-controlled Legislature, the bureau provided figures on the total of all transportation debt service as a percentage of gross transportation fund revenue -- in other words, what portion of transportation revenue for road work would be going to paying off debt.
PolitiFact's other truth-massage credited Roys with making clear that the debt service increase she spoke of was the debt service amount as a percentage of total spending on roads. Aside from the fact Roys talked about "just basic road repair and maintenance," she offered listeners no clue that she used the same measure PolitiFact Wisconsin used to fact check her claim.

The clue that likely drove PolitiFact to check the debt service as a percentage of road work expenses came from WisconsinEye senior producer Steve Walters, who conducted the interview of Roys. Walter referred no less than twice to a "22 to 23 percent" figure for debt service during the interview.

Since that number came from Walters, PolitiFact Wisconsin apparently felt no need to fact check its accuracy.

Does Some Road Construction Go Beyond 'Basic'?

We think the phrase "basic road repair and maintenance" may leave some members of the audience with the impression that more involved road work such as replacing bridges would balloon the cost of debt service even higher than described.

We found a page run by the Wisconsin Department of Transportation describing its road projects. Here's the description of one costing $9.6 million:
Description of work: The project consists of a full reconstruction of WIS 55 (Delanglade Street) from I-41 to Lawe Street in the city of Kaukauna. Improvements will include roundabouts at the intersections of I-41 ramps, Maloney/Gertrude, and County OO. New traffic signals will be installed at County J/WIS 55/WIS 96, and bike/pedestrian accommodations will be added throughout the project limits along WIS 55. Other work includes storm sewer, sanitary sewer, water main, sidewalks, retaining walls, street lighting, and incidentals.
It appears to us that PolitiFact Wisconsin simply assumed that all the described work rightly fits under Roys' description.

We're skeptical that such assumptions hold a rightful place among the best practices for fact checkers.

Summary


If we assume that Roys was talking about all expenses attached to road work, and also assume she was talking about the increase in the estimated dollar amount of debt service in raw dollars, her estimate is off by only about 7 percent. In that case, PolitiFact Wisconsin did not really need to use future estimates to justify Roys' statement about how much Wisconsin is spending now. It could have just used the measure Roys' described and rated that against the estimate for this year's spending.

But a fact checker could easily have justified asking Roys to define what she meant by "basic road repair and maintenance" and then using that definition to grade her accuracy. A better fact check would likely result.

We wonder if Roys would need to join the Republican Party to make that happen.

Thursday, June 14, 2018

Different Strokes for Different Quotes: What does "voted for tax cuts" really mean?

"What I find is it's hard for me to take critics seriously when they never say we do anything right. Sometimes we can do things right, and you'll never see it on that site."

-PolitiFact Editor Angie Drobnic Holan



Sometimes PolitiFact can do things right.

PolitiFact New York did something right recently that deserves mention because it's the correct way to journomalist:





PolitiFact added the Trump camp "did not get back to us with information supporting his claim, so we can't say for sure what he was talking about in his endorsement."

PolitiFact noted that Trump tweeted about the Tax Cuts and Jobs Act "four other times in May" but acknowledged Trump did not reference that law in the tweet it fact checked.

In our view this is the correct approach.

We think a persuasive argument could be made that Trump inaccurately implied Donovan was a Tax Cut and Jobs Act supporter, but that argument belongs on the editorial page, not in a fact check. PolitiFact examined the claim Trump made without inventing assumptions about what he meant or what he was implying. In this case PolitiFact stuck to the facts.

Notwithstanding our longtime opposition to rating facts on a sliding scale, we think PolitiFact did this one right and we're happy to point it out.


The Other Guy

Readers may wonder "How could a fact checker screw this one up?" Donovan had a documented history of voting for tax cuts, and Trump's claim was not only unambiguous but also easy to check.

How could a serious fact checker get this wrong?







When the Washington Post's unabashed Trump basher/unbiased truthsayer tweeted that "fact checkers sometimes disagree" we were curious. PolitiFact rated Trump's tweet as accurate, while Kessler deemed the exact same tweet false. How can that be?

As it turns out the two fact checkers aren't disagreeing at all.

PolitiFact correctly identified the claim Trump made and ruled based on his actual words. Kessler invented a claim and then gave Trump a false rating for his own fantasy. The fact checkers aren't disagreeing because they're not checking the same claim.

Kessler says Trump's claim that Donovan "voted for tax cuts" is false because "Donovan voted against Trump's tax cut three times." For those of you that aren't experts in journalism or logic, voting against the Tax Cut and Jobs Act does not negate the fact that Donovan has previously voted for other tax cuts.

As far as we can tell, Kessler offered no justification for calling Trump's claim false other than Donovan's opposition to the 2017 tax bill.

Kessler's reasoning here is flatly wrong. And if one wanted to treat Kessler with the same painful pedantry as he applies to Trump in his chart, one could note there's no such thing as "Trump's tax cuts" because only Congress can pass tax bills.

Petty word games aside, this "disagreement" among fact checkers affirms that our fact-divining betters are neither scientific agents of truth nor objective determiners of evidence. When a fact checker can substitute a person's actual words for their own interpretation of what that person meant it counts as commentary, not an adjudication of facts.

Kudos to PolitiFact New York for taking the correct approach. Sometimes PolitiFact can do things right.


Monday, June 11, 2018

PolitiFact 2016: Despite No Evidence Supporting Our Conclusion, It's "Half True" Donald Trump Doesn't Believe in Equal Pay for Equal Work


Our part-time effort to hold PolitiFact accountable allows many problems to slip through the cracks. Sometimes our various research projects bring a problematic fact check to our attention.

Case in point:


PolitiFact's Nov. 2, 2016 "fact check" found Democratic presidential nominee Hillary Clinton's claim "Half True" despite finding no evidence supporting it other than a fired campaign organizer's complaint of gender discrimination.

We must be exaggerating, right?

We challenge anybody to find concrete evidence of Trump's disbelief in equal pay in PolitiFact's "fact check" apart from the allegation we just described.


In fact, PolitiFact's fact check makes it look like the fact checkers have difficulty distinguishing between the raw pay gap and differences in pay stemming from discrimination. Trump comes across looking like he makes that distinction. PolitiFact comes across looking like it interprets Trump's insistence on that distinction as support for Clinton's claim.

Take for example this unsupportive piece of supporting evidence PolitiFact received from the Clinton campaign:
Clinton’s campaign pointed to another August 2015 interview, in which CNN’s Chris Cuomo asked Trump if he would pass equal pay legislation.

Trump said he was looking into it "very strongly."

"One of the problems you have is you get to have an economy where it's no longer free enterprise economy," Trump said.

Trump said he favored the concept, but that it’s "very complicated."

"I feel strongly -- the concept of it, I love," Trump said. "I just don't want it to be a negative, where everybody ends up making the same pay. That's not our system. You know, the world, everybody comes in to get a job, they make -- people aren't the same."
Trump says he favors the concept (PolitiFact's paraphrase) and in Trump's own words he "loves" the concept. Trump cautions that everybody might end up making the same pay. Does that sound like equal pay for equal work? Is all work equal?

If anything, Clinton's evidence against Trump helped undermine its own case.

Making this fact check even more bizarre, PolitiFact's summary fails to reference its best evidence of Trump's disbelief in equal pay--the went-nowhere gender discrimination case:
Our ruling

Clinton said Trump "doesn't believe in equal pay."

Trump’s campaign website does not have a stipulated stance on equal pay for men and women, but his campaign says he supports "equal pay for equal work." Trump has said men and women doing the same job should get the same pay, but it’s hard to determine what’s "the same job," and that if everybody gets equal pay, "you get away from capitalism in a sense."

Trump has also said pay should be based on performance, not gender -- so he does appear to favor uniform payment if performance is alike.

Clinton’s statement is partially accurate but leaves out important details or takes things out of context. We rate it Half True.
Put bluntly, PolitiFact put nothing in its summary in support of Clinton's claim.

Noting that "Trump's campaign website does not have a stipulated stance on equal pay for men and women" counts as an argument from silence. Making matters worse for Clinton, the campaign breaks its silence to endorse the concept of equal work for equal pay.

PolitiFact claims it places the burden of proof on the one making the claim, in this case Hillary Clinton. The evidence suggests PolitiFact instead placed the burden of proof on the Trump campaign.

PolitiFact makes a snippet mosaic out of Trump's statements that appear to show that he doesn't believe men and women should make equal pay regardless of whether they do equal work. Is that supposed to serve as evidence Trump does not believe in equal pay for equal work?

In the end, PolitiFact gave Clinton a "Half True" rating despite finding no real evidence in support of her claim and an abundance of evidence contradicting it.


Afters I

The 2016 discrimination complaint from Elizabeth Mae Davidson was never litigated and was dropped earlier this year.

Afters II

PolitiFact's description of the gender wage gap in the Clinton fact check puts it somewhat at odds with some of its own past fact checks. Note this from the Clinton fact check:
We’ve detailed key issues about the gender wage gap in our PolitiFact Sheet, but a consistent argument is that women earn 77 cents on the dollar that men earn.

(...)

The Institute for Policy Women’s Research says discrimination is a big factor for why the gender wage gap still persists. Experts consider "occupational segregation" another reason for the wage gap, which means women more often than men work in jobs that pay low and minimum wages.
If you've got a "big reason" and "another reason" which reason should you expect to have the greatest effect? The "big reason," right? Some of PolitiFact's past fact checks have correctly cast serious doubt on that proposition. PolitiFact's summary article on the gender wage gap  (the same "PolitiFact Sheet" referenced above in the Clinton fact check) features a fine example (bold emphasis added):
THE BIG PICTURE

Just before Obama took office in 2009, the Department of Labor released a study because, as a deputy assistant secretary explained it, "The raw wage gap continues to be used in misleading ways to advance public policy agendas without fully explaining the reasons behind the gap." The study by CONSAD Research Corp. took into account women being more likely to work part-time for lower pay, leave the labor force for children or elder care, and choose work that is "family friendly" with fuller benefit packages over higher pay. The study found that, when factoring in those variables, the gap narrows to between 93 cents and 95 cents on the dollar.
We would remind readers that the CONSAD study is not saying that gender discrimination accounts for 5 to 7 percent of the raw gender wage gap. It estimates that 5 to 7 percent of the gender wage gap is not explained by a combination of women's occupational and family choices. Those aren't the only factors influencing the raw wage gap.

So about two-thirds of the raw wage gap is explained by the job choices women make, and 7 percent remains unexplained with part of that 7 percent perhaps explained by gender discrimination.

Did PolitiFact not make that clear?

Afters III

In the search from some charitable explanation for PolitiFact's fact-checking, I had to consider the possibility that Clinton's claim is literally correct: Trump does not believe in equal pay if "equal pay" means paying everyone equally regardless of the job or the quality of the work.

Using that interpretation of "equal pay" would make Clinton's claim literally true but at the same time a whopper of deceit. PolitiFact appeared to take Clinton to mean "equal pay for equal work" except possibly when it used Trump's statements in support of Clinton's claim.

If it was PolitiFact's position that Clinton was saying Trump did not believe men and women should earn the same regardless of the job or the work performed then it should have stated so clearly.

Either way, PolitiFact's fact check looks incoherent.