Wednesday, March 13, 2019

Gender Pay Gap Shenanigans from PolitiFact Virginia

PolitiFact has a long history of botching gender wage gap stories, often horrifically.

PolitiFact Virginia's March 13, 2019 treatment of the subject does nothing improve the reliability of PolitiFact's reporting on the topic.


It's not true that women earn 80 percent of the pay men earn doing the same job, though Democrats proclaim otherwise from time to time. And that's probably what Scott did, using "similar" in its role as a synonym for "same."

PolitiFact was apparently very eager to use the technique of charitable interpretation--most likely because Scott is a Democrat. Republicans rarely receive the benefit of that feature of competent fact-checking from PolitiFact.

We're partial to the using charitable interpretation when appropriate, but PoltiFact Virginia ends up running data through the confirmation bias filter in its effort to bail out Scott.

We'd judge Scott's use of the term "similar" as an ambiguity. PolitiFact Virginia calls it "nuance":
Scott’s statement, however, is nuanced. He says women get 80 percent pay for doing "similar" jobs as white men, which is different than saying the "same" job as men.
PolitiFact Virginia apparently skipped the step of checking the thesaurus to see if the terms "similar" and "same" may be used interchangeably. They can. The two terms have overlapping meanings, in fact.

We find it notable that PolitiFact Virginia set aside the usual PolitiFact practice of relying on explanations from spokespeople representing the figure being fact-checked.

Scott's staff said he got his numbers from sources relying on Census Bureau data.

PolitiFact Virginia:
Stephanie Lalle, Scott’s deputy communications director, told us the congressman got the statistic from separate reports published in late 2018 by the Institute for Women’s Policy Research, and the National Partnership for Women and Families.

Both reports said the statistic comes from the U.S. Census Bureau. The latest gender-gap statistics from the Bureau show in 2017 women earned 80.5 percent of what men made - the same percentage as in 2016.
PolitiFact Virginia's determination to defend Scott's statement leads it to spout statistical mumbo-jumbo. Based on apparently nothing more the Scott's "nuanced" use of the term "similar," PolitiFact Virginia tried to reverse engineer an explanation of his statistic to replace the explanation offered by Scott's staff.

What if "similar" meant broad classes of jobs, and data from the Bureau of Labor Statistics showed white men making more than women in those classes of jobs?

PolitiFact Virginia thought it was worth a shot:
Women out-earned men in three occupations: wholesale and retail buying; frontline supervisor of construction trades and extraction workers; and, as we mentioned, dining room and cafeteria attendants and bartender helpers.

Fact-checking Scott, however, requires a deeper dive. The percentages we just discussed compare the full-time weekly earnings of all women to all men in these occupations. Scott, in his statement, compared the earnings of all women to white men in similar jobs.

The BLS’s data set that compares gender pay by specific jobs does not sort men and women by race. It does, however, categorize the jobs into 29 broad fields of work and, in each of those fields, breaks down women and men by sex.

Overall in 2018, women earned 78.7 percent less than white men in the same areas of work. The comparison of women’s pay to white men’s produces a bigger gender gap than the comparison to all men. That’s because white males tend to earn more than black males.

White men out-earned women in all 29 fields of work.
Note that PolitiFact Virginia isn't really showing its work. And what it does show contains appalling mistakes.

Let's break it down piece by piece.

Piece by Piece, Step by Step


"The percentages we just discussed compare the full-time weekly earnings of all women to all men in these occupations."

Do the percentages compare the full-time weekly earnings of all women to all men in those occupations? It's hard to tell from PolitiFact's linked source document. If author Warren Fiske was talking about Table 18, as we believe, then the fact check should refer to Table 18 by name.

Looking at Table 18, it seems Fiske reasoned improperly. The table mentions 121 groups of occupations but most of occupations nested under the list headers have no estimate of a gender wage gap, entering a dash instead of a number. In the notes at the bottom of the table, BLS warns that a dash means "no data or data that do not meet publication criteria." That makes it improper to extrapolate the listed results into a nationally representative number. Nor should a fact-checker assume that the subject of the fact check had such creative reasoning in mind.

In short, using numbers from Table 18 to support Scott represents unjustifiable cherry-picking.

"Scott, in his statement, compared the earnings of all women to white men in similar jobs."

We cannot find any citation in PolitiFact Virginia's fact check that offers data addressing the racial aspect of Scott's claim. Without that data, how can the fact checker reach a reasonable conclusion about the claim?

"The BLS’s data set that compares gender pay by specific jobs does not sort men and women by race."

That's bad news for this fact check. As noted above, without the data on race there's no checking the claim.

"It does, however, categorize the jobs into 29 broad fields of work and, in each of those fields, breaks down women and men by sex."

Breaking down men and women by sex is not the same as breaking them down by race, so this "however" doesn't point the way to a solution to the problem.

"Overall in 2018, women earned 78.7 percent less than white men in the same areas of work."

The fact checkers probably meant 78.7 cents to the dollar compared to men, not 78.7 percent less (about 22 cents to the dollar). But we still do not know the source of the race-based claim. We'd love to see a clear clue from PolitiFact Virginia regarding the specific source of this claim.

"The comparison of women’s pay to white men’s produces a bigger gender gap than the comparison to all men. That’s because white males tend to earn more than black males."

This part we follow. But it doesn't help us understand how PolitiFact can claim how white men out-earned women in all 29 fields of work in the BLS data.

"White men out-earned women in all 29 fields of work."

Based on what? We just went through PolitiFact's argument step by step. There's no reasoning in the fact check to justify it, and we can't find any citation that appears to lead to a justifying document.

Summary

PolitiFact failed to offer information justifying its key data point ("White men out-earned women in all 29 fields of work). It failed to show how cherry-picking that information, even if legitimately sourced, would justify Scott's statement in the context of "fair pay" legislation. And it simply blundered with the claim that women earned 78.7 percent less than white men (in same areas of work or otherwise).

Making the bad news worse, we could write another article about the problems in PolitiFact Virginia's wage gap story without repeating the same points.


Update March 14, 2019

After we contacted PolitiFact Virginia on March 13, 2019 it corrected the "78.7 percent less" mistake.

PolitiFact Virginia attached no editor's note to the story indicating either a correction or clarification.

Note this from PolitiFact's policy on corrections:
Errors of fact – Errors of fact that do not impact the rating or do not change the general outlook of the fact-check receive a mark of correction at the bottom of the fact-check.

The text of the fact-check is updated with the new information. The correction states the correct information that has been added to the report. If necessary for clarity, it repeats the incorrect information. Corrected fact-checks receive a tag of "Corrections and updates."

Typos, grammatical errors, misspellings – We correct typos, grammatical errors, misspellings, transpositions and other small errors without a mark of correction or tag and as soon as they are brought to our attention.
So we're supposed to believe writing "earned 78.7 percent less" instead of "78.7 percent as much" counts as one of the following:
  • typo
  • grammatical error
  • misspelling
  • transposition
  • other small error
Who buys it?

No comments:

Post a Comment