Thursday, March 23, 2017

Rorschach context

It seems as though the liberal bloggers (aka "mainstream fact checkers") at PolitiFact treat context like a sort of Rorschach inkblot, to interpret as they see fit.

What evidence prompts these unkind words? The evidence runs throughout PolitiFact's history, but two recent fact-checks inspired the imagery.

The PolitiFact Florida Lens

In our previous post, we pointed out the preposterous "Mostly True" rating PolitiFact Florida gifted on a Florida Democrat who equated the raw gender wage gap with the gender wage gap caused by sex discrimination. The fact checkers did not interpret words uttered in context, "simply because she isn't a man," as an argument that the raw wage gap was entirely the result of gender discrimination. Perhaps it wasn't specific enough, like saying the difference in pay occurred despite doing the same work ("Mostly False")?

Whatever the case, PolitiFact opted not to accept a crystal clear clue that it was checking a claim that mirrored one it had previously rated "Mostly False" and rated the similar claim "Mostly True."

The PolitiFact California Lens

A recent fact check from PolitiFact California makes for a jarring contrast with the PolitiFact Florida item.

California Lt. Governor Gavin Newsom tweeted that Republican Jason Chaffetz had treated the cost of an iPhone to the cost of health care "as if the 2 are the same." Newsom was making the point that health care costs more than an iPhone, so saying the two are the same misses the mark by a California mile.

But did Chaffetz say the costs are the same?

First let's look at how the PolitiFact California lens processed the evidence, then we'll put that evidence together with some surrounding context.

PolitiFact California:
We also examined Newsom’s final claim that Chaffetz had compared the iPhone and health care costs "as if they are the same."

Chaffetz’ comments, particularly his phrase "Americans have choices. And they’ve got to make a choice," leave the impression that obtaining health care is as simple as sacrificing the purchase of a smartphone.
It's worth noting at the outset that PolitiFact California's key evidence doesn't mention the iPhone and does not even imply any type of cost comparison. The only way to adduce Chaffetz's quotation as evidence of a price comparison would have to come from the context of Chaffetz's remarks. And a fact-checker ought to explain to readers how that works, unless the fact checker can count on his audience sharing his ideological bias.

Chaffetz (as quoted at length in the PolitiFact California fact check; bold emphasis added):
"Well we're getting rid of the individual mandate. We're getting rid of those things that people said they don't want. And you know what? Americans have choices. And they've got to make a choice. And so, maybe rather than getting that new iPhone that they just love and they want to go spend hundreds of dollars on that, maybe they should invest it in their own health care. They've got to make those decisions for themselves."
Chaffetz in no way offers anything approaching a clear suggestion that the cost of an iPhone equals the cost of health care or health insurance. His words about people having choices come right after he says the health care bill would eliminate the individual mandate. After that comes the mention of an iPhone costing "hundreds of dollars" that one might instead invest in health care. In context, the statement is just one example of a great number of choices one might make about paying for health care.

The PolitiFact California lens (like magic!) turns Chaffetz's words conveniently into what is needed to say the Democrat said something "Mostly True."

It's the bias, stupid.

We have PolitiFact Florida ignoring clear context to give a Democrat a more favorable rating than she deserves. We have PolitiFact California finding clear evidence from the context where none exists to give a Democrat a more favorable rating than he deserves.

Point out the absurdity to PolitiFact (as we did for the PolitiFact Florida flub) and somebody from the Tampa Bay Times will read the critique and no changes to the article will result.
How are they able to repeatedly overlook problems like these?

The simplest explanation? Because they're biased. Biased to the left. Biased to trust their own work (despite the incongruity with other PolitiFact fact checks!). And Dunning-Kruger out the wazoo.

Tuesday, March 14, 2017

There You Go Again: PolitiFact Florida makes a hash of another gender wage gap ruling

Though PolitiFact is an unreliable fact-checker, at least one can bank on the mainstream fact-checker's ability to flub gender wage gap claims.

We hit PolitiFact on this issue often, but this latest one from PolitiFact Florida is a doozy, rivaling PolitiFact Oregon's remarkable turd from 2014.

Drum roll: PolitiFact Florida, March 14, 2017:

We're presenting a big hunk of the fact check as it appears at PolitiFact Florida to show how PolitiFact Florida effectively contradicts its own reasoning.

In the next-to-last paragraph of its summary, PolitiFact Florida explains that "differences in pay can be affected by the careers women and men choose and taking time off to care for children." Those aren't the only factors affecting the raw wage gap, by the way.

Yet in the ironically named "Share The Facts" version, the "Mostly True" rating blares its message aside Democrat Patricia Farley's claim the disparities occur purely based on gender ("simply because she isn't a man"). In other words, the cause is gender discrimination, not different job choices and the like--directly contradicting PolitiFact Florida's caveat. Farley didn't just leave out context. She explicitly denied the key bit of context.

Anyone who knows the difference between the raw gender wage gap and the wage gap based solely on gender discrimination but uses the large former gap in the context of arguing for legislation to reduce gender discrimination is deceiving people. The raw gender wage gap is not a realistic representation of gender discrimination in wages because of other factors, such as men and women tending to choose careers that pay differently.

So, yes, we're saying that unless Patricia Farley is ignorant about the difference between the gender wage gap and the wage gap caused by pay discrimination, she is lying, as in deliberately deceiving her audience. And PolitiFact Florida is calling her falsehood and potentially intentional deception "Mostly True."

The PolitiFact Florida wage gap fact check is below average for PolitiFact--and that's like failing to leap over a match box.


Correction March 15, 2017: Posted the intended URL for the PolitiFact Florida fact check. We had mistakenly used the URL to a related fact check concerning Donald Trump.

Monday, February 27, 2017

Daily Caller: "Politifact Says Trump Is Right, But Rates His Remark ‘Mostly False'"

The Daily Caller notes an item from PolitiFact where President Trump tweeted something PolitiFact found true, after which the fact checkers proceeded to rate the claim "Mostly False."

The Daily Caller's Alex Pfeiffer has the skinny:
The tweet from Trump came after Gateway Pundit reported on the change in the national debt under the two respective presidents and after former Godfather Pizza CEO Herman Cain brought up the figures on Fox News.

Politifact wrote: “The numbers check out. And in fact, the total public debt has dropped another $22 billion since the Gateway Pundit article published, according to data from the U.S. Department of Treasury.”

Despite this, Politifact still gave Trump a rating of “mostly false” and titled its article, “Why Donald Trump’s tweet about national debt decrease in his first month is highly misleading.”
We saw this item and considered writing it up. It seemed to us the type of thing that liberal (or even moderate) readers might excuse, judging that PolitiFact did enough to justify the "Mostly False" rating it gave to Trump's tweet.

The case needs additional information to show that it does not represent a fair fact check.

The definition of "Mostly False"

Did PolitiFact show that Trump's tweet met its definition of "Mostly False"? Here is the definition:
MOSTLY FALSE – The statement contains an element of truth but ignores critical facts that would give a different impression.
Trump's tweet did not simply contain "an element of truth." It was true (and misleading). PolitiFact's "Truth-O-Meter" definitions mean little. PolitiFact does not used objective criteria to decide the rating. If objective criteria decided the rating, then PolitiFact's creator would not declare that "Truth-O-Meter" ratings are "entirely subjective."

Sauce for the gander?


If PolitiFact applied its judgments consistently, then the Daily Caller and sites like ours would have little to complain about. But vague definitions that ultimately fail to guide the final rating make it virtually impossible even for well-meaning left-leaning journalists to keep the scales balanced.

Consider an example from the PolitiFact Oregon franchise. PolitiFact Oregon rated Democrat Brad Avakian "Mostly True" for a false statement:
Avakian, citing Census data and echoing claims by Obama and others, said women in Oregon "earn an average of 79 cents for every dollar that men earn for doing the same job." The report he relied on noted that the 79-cent figure applies to full-time, year-round work, although Avakian didn’t include those stipulations.

For starters, the commissioner loses points for cherry-picking the 79-cent figure. Other means of measuring pay gaps between men and women put it considerably less.

The same can be said of the "for doing the same job" piece. As PolitiFact has found previously, the existence of a pay gap doesn’t necessarily mean that all of the gap is caused by individual employer-level discrimination, as Avakian’s claim implies. Some of the gap is at least partially explained by the predominance of women in lower-paying fields, rather than women necessarily being paid less for the same job than men are.

Finally, Avakian used the term "average" when the report he relied on said "median." He could have avoided that by simply saying women "make 79 cents for every dollar a man earns," but since the information he cited contains only median incomes, we find the difference to be inconsequential.

Those caveats aside, he still is well inside the ballpark and the ratio he cited is a credible figure from a credible agency. We rate the claim Mostly True.
That's an inexcusably tilted playing field. If Avakian had described the raw pay gap without saying it compared men and women doing the same job, then his claim would have paralleled Trump's: a true but misleading statement. But Avakian's statement was not true and misleading. It was false and misleading at the same time.

Yet it received a "Mostly True" rating compared to Trump's "Mostly False" rating.

Doesn't fact-checking need better standards than that?



Jeff Adds (1922PST 2/27/17):
We'd love to see PolitiFact reconcile their Mostly False rating of Trump's claim with the rationale behind this gem:



Was there anything misleading about Clinton's statement?
Clinton’s figures check out, and they also mirror the broader results we came up with two years ago. Partisans are free to interpret these findings as they wish, but on the numbers, Clinton’s right. We rate his claim True.
Ha! Silly factseekers. When Trump makes an accurate claim PolitiFact conjures their magical powers of objectivity to decide what is misleading. When lovable ol' Bill makes a claim, heck, PolitiFact is just checkin' the numbers and all you partisans can figure out what it means.

Note that PolitiFact gave Bill Clinton a True rating, which they define as "The statement is accurate and there’s nothing significant missing." Must be nice to be in the club.

We've pointed out how PolitiFact's application of standards is akin to the game of Plinko. With ratings like this it's difficult to view PolitiFact as serious journalists instead of carnival barkers.

Tuesday, February 21, 2017

Another nugget from the Hollyfield interview

In an earlier post we pointed out how managing editor Amy Hollyfield of PolitiFact described its "Truth-O-Meter" in terms hard to reconcile with those used by PolitiFact's creator, Bill Adair.

The Hollyfield interview published at The Politic (Yale University) contains other amusing nuggets, such as this howler (bold emphasis added):
We take accuracy very seriously. Transparency is one of the key things we focus on, which is why we publish all the sources for our fact checks. We flag every correction and have a subject tag called “correction,” so you can see every fact check we’ve put a correction on.
We find Hollyfield's assertion offensive, especially as it occurs in response to a question about this website, PolitiFact Bias.

PolitiFact does a poor job of consistently adding the subject tags to corrected articles.

We pointed out an example in December 2016. PolitiFact California changed the rating of a fact check from "True" to "Half True," publishing a new version of its fact check from months earlier. Weeks later, PolitiFact California has yet to add a tag to the article that would make it appear on PolitiFact's "Corrections and Updates" page.

Maybe PolitiFact California does not regard rewriting an article as a correction or update?

How about PolitiFact Pennsylvania from January 2017? Lawyers pointed out that the Pennsylvania PolitiFact franchise incorrectly described the standard of evidence courts use for criminal cases. PolitiFact Pennsylvania ran a correction (the correction made the fact check incoherent, but that's another story), but added no tag to the story.


So, contrary to what Hollyfield claims, the corrected story is not transparently presented on its "Corrections and Updates" page.

PolitiFact's spotty compliance with its statement of principles is not new. We even complained about the problem to Paul Tash, the president of the Tampa Bay Times (Nov. 18, 2016). But we've noticed no improvement.

PolitiFact does not have a page that transparently informs readers of all of its corrections.

Will you believe Amy Hollyfield or your own lyin' eyes?

Monday, February 20, 2017

PolitiFact's "Truth-O-Meter": Floor wax, or dessert topping?

The different messages coming from PolitiFact founder Bill Adair and current PolitiFact managing editor Amy Hollyfield in recent interviews reminded me of a classic Saturday Night Live sketch.

In one interview (Pacific Standard), Adair said deciding PolitiFact's "Truth-O-Meter" ratings was "entirely subjective."

In the other interview (The Politic), Hollyfield gave a different impression:
There are six gradations on our [Truth-O-Meter] scale, and I think someone who’s not familiar with it might think it’s hard to sort out, but for people who’ve been at it for so long, we’ve done over 13,000 fact checks. To have participated in thousands of those, we all have a pretty good understanding of what the lines are between “true” and “mostly true,” or “false” and “pants on fire.”
If PolitiFact's "star chamber" of editors has a good understand of the lines of demarcation between each of the ratings, that suggests objectivity, right?

Reconciling these statements about the "Truth-O-Meter" seems about as easy as reconciling New Shimmer's dual purposes as a floor wax and a dessert topping. Subjective and objective are polar opposites, perhaps even more so than floor wax and dessert topping.

If, as Hollyfield appears to claim, PolitiFact editors have objective criteria to rely on in deciding on "Truth-O-Meter" ratings, then what business does Adair have claiming the ratings are subjective?

Can both Adair and Hollyfield be right? Does New Shimmer's exclusive formula prevent yellowing and taste great on pumpkin pie?

Sorry, we're not buying it. We consider PolitiFact's messaging about its rating system another example of PolitiFact's flimflammery.

We think Adair must be right that the Truth-O-Meter is primarily subjective. The line between "False" and "Pants on Fire" as described by Hollyfield appears to support Adair's position:
“False” is simply inaccurate—it’s not true. The difference between that and “pants on fire” is that “pants on fire” is something that is utterly, ridiculously false. So it’s not just wrong, but almost like it’s egregiously wrong. It’s purposely wrong. Sometimes people just make mistakes, but sometimes they’re just off the deep end. That’s sort of where we are with “pants on fire.”
Got it? It's "almost like" and "sort of where we are" with the rating. Or, as another PolitiFact editor from the "star chamber" (Angie Drobnic Holan) memorably put it: "Sometimes we decide one way and sometimes decide the other."


Afters

Though PolitiFact has over the years routinely denied that it accuses people of lying, Hollyfield appears to have wandered off the reservation with her statement that "Pants on Fire" falsehoods on the "Truth-O-Meter" are "purposely wrong." A purposely wrong falsehood would count as a lie in its strong traditional sense: A falsehood intended to deceive the audience. But if that truly is part of the line of demarcation between "False" and "Pants on Fire," then why has it never appeared that way in PolitiFact's statement of principles?

Perhaps that criterion exists only (subjectively) in Hollyfield's mind?


Update Feb. 20, 2017: Removed an unneeded "the" from the second paragraph

Sunday, February 19, 2017

Power Line: "Trump 4, PolitiFact 1"

John Hinderaker, writing for the Power Line blog, does a quick rundown of five PolitiFact fact checks of President Donald Trump. Hinderaker scores the series 4-1 for Trump.

Read it through for the specifics.

Our favorite part occurs at the end:
We could go through this exercise multiple times every day. Correcting the Democratic Party “fact checkers” would be a full-time job that I don’t plan to undertake. Suffice it to say that Trump is more often right than are the press’s purported fact checkers who pretend to correct him.
We continue to marvel at PolitiFact's supernatural ability to ignore substantive criticism. How often does it answer charges that it has done its job poorly?

If PolitiFact is an honest and transparent attempt at objective fact-checking, then we think PolitiFact should aggressively defend itself against such charges, or else change its articles accordingly.

On the other hand, if PolitiFact is a sham attempt at objective fact-checking, maybe it's smart to ignore criticism, trusting that its readers will conclude the criticisms did not deserve an answer.

Maybe there's an explanation that splits the difference?

Friday, February 17, 2017

PolitiFact: That was then, this is now

Now (2017)

PolitiFact is independent! That means nobody chooses for PolitiFact what stories PolitiFact will cover. PolitiFact made that clear with its recent appeal for financial support though its "Truth Squad" members--persons who contribute financially to PolitiFact (bold emphasis added):
Our independence is incredibly valuable to us, and we don't let anyone — not politicians, not grant-making groups, not anyone — tell us what to fact-check or what our Truth-O-Meter rulings should be. At PolitiFact, those decisions are made solely by journalists. With your help, they always will be.
Got it? Story selection is done solely by PolitiFact journalists. That's independence.

Then (2015)

In early 2015, PolitiFact started its exploration of public funding with a Kickstarter program geared toward funding its live fact checks of the 2015 State of the Union address.

Supporters donating $100 or more got to choose what PolitiFact would fact check. Seriously. That's what PolitiFact offered:

Pledge $100 or more

Pick the fact-check. We’ll send you a list of four fact-checks we’re thinking of working on. You decide which one we do. Plus the coffee mug, the shout out and the mail.
We at PolitiFact Bias saw this scam for what it was back then: It was either a breach of journalistic ethics in selling its editorial discretion, or else a misleading offer making donors believe they were choosing the fact check when in reality the editorial discretion was kept in-house by the PolitiFact editors.

Either way, PolitiFact acted unethically. And if Angie Drobnic Holan is telling the truth that PolitiFact always has its editorial decisions made by journalists, then we can rest assured that PolitiFact brazenly misled people in advertising its 2015 Kickstarter campaign.


Clarification Feb. 18, 2017: Belatedly added the promised bold emphasis in the first quotation of PolitiFact.