Friday, September 28, 2012

PolitiFact Never Rates Hyperbole Sometimes

"We don’t check opinions, and we recognize that in the world of speechmaking and political rhetoric, there is license for hyperbole."



PolitiFact doesn't rate hyperbole.

In fact, in 2007 they "decided on a policy against it."

Just don't tell that to Republican George Allen, who recently criticized Democrat Tim Kaine for his position on tax hikes:

Image from PolitiFact.com (arrow added)

Poor George Allen. No license for hyperbole for you!

Some longtime PolitiFact readers may remember back in the olden days of three months ago that Harry Reid got a pass from the gimmicky graphic:
We recognize Reid was using hyperbole, so we won't put his claim to the Truth-O-Meter.
That line sparked the following exchange on PolitiFact's Facebook page:
Mark FitzSimmons: What? Wasn't the first pants on fire Biden referring to Bush as brain-dead? How is that not recognized as hyperbole?

PolitiFact: Mark,you have a very good memory! It was after that check (and partly because of that check) that we decided on a policy against it.
As we pointed out in a previous post, on three separate occasions since January of this year, PolitiFact has given a Pants on Fire rating to statements (all by Republicans) it described as hyperbolic. Since the Biden claim that was the impetus for the anti-hyperbole policy came out, they've rated roughly 20 statements described as hyperbolic. As far as we can tell, Reid is the only one who has escaped the Truth-O-Meter due to the policy.

But PolitiFact doesn't rate hyperbole.

They have a policy against it.

Take comfort, George Allen.


Bryan adds:

The evidence suggests that Republicans are much more likely to use hyperbole without a license.





Edit: 9/28/12: Changed the word "graph" to "graphic"-Jeff

Wednesday, September 26, 2012

Media Trackers: "At PolitiFact Ohio, Democrats Outnumber Republicans 4 to 1"

Media critics continue to identify bias in the political affiliation of fact checkers at PolitiFact.

Jason Hart of Media Trackers (Ohio) published a report today indicating a solid majority of the PolitiFact Ohio staff shows as registered Democrats according to election records.

Hart:
Records indicate the following twelve Plain Dealer employees assigned to PolitiFact Ohio are registered Democrats:

  • Robert Higgs, PolitiFact Ohio editor
  • Jane (Murphy) Kahoun, Plain Dealer deputy metro editor
  • Tom Feran, Plain Dealer reporter
  • Henry J. Gomez, Plain Dealer reporter
  • Aaron Marshall, Plain Dealer statehouse reporter
  • Reginald Fields, Plain Dealer statehouse bureau chief
  • Jo Ellen Corrigan, Plain Dealer librarian
  • James Ewinger, Plain Dealer reporter
  • Laura Johnston, Plain Dealer reporter
  • Peter Krouse, Plain Dealer reporter
  • James McCarty, Plain Dealer reporter
  • Robert Schoenberger, Plain Dealer reporter

We at PolitiFact Bias are on record defending the potential ability for a Democrat (or Republican) to report news or even perform news analysis fairly, so we take reports like this one with a grain of salt as supposed proof of a reporting bias.

Clearly, though, Hart's report exposes a failure of disclosure.  PolitiFact presents itself as non-partisan.  Part of PolitiFact's strategy for conveying an image of neutrality is to hide the political affiliations and leanings of its staff members.

In practice, the high proportion of Democrats on the PolitiFact Ohio staff can easily exert an ideological influence on its reporting.  It is a team of three editors who vote on the "Truth-O-Meter" ratings.  Hart identifies no Republican editors at PolitiFact Ohio.  So a Democrat is likely to do the reporting and a majority of Democrats will vote on the "Truth-O-Meter" rating.   Yet people will still criticize PolitiFact Bias compared to PolitiFact because Jeff and I admit we're personally biased against liberalism.

We're the ones exhibiting honesty, inviting readers to take our bias into account when they consider what we write.

If there's one secret the mainstream press is unwilling to divulge to its audience under any circumstances, personal ideology is it.  But it's dishonest to pretend to an objectivity of viewpoint that doesn't exist in reality.

Sunday, September 23, 2012

Relevant: The AllSides project

A new venture called "AllSides" tries to meet democracy's need for trustworthy information by using ideological transparency, crowdsourcing and some technical inspiration from Tim Groseclose and Jeff Milyo.

We've previously emphasized the relevance of Groseclose's book "Left Turn" to our ongoing critique of PolitiFact's bias problem.

AllSides appears to represent an entirely new attempt to address the problem of getting quality information to voters in a constitutional republic.  We're not big fans of crowdsourcing, but it seems like a potentially reasonable approach to grading sources for their degree of bias.

And speaking of the degree of bias, the quotation that largely accounts for our interest in the AllSides project (bold emphasis added):
During the Democratic National Convention (DNC), Bill Clinton asserted that over the last 52 years, America had experienced more job growth under past Democratic presidents (42 million) than under Republican presidents (24 million).

In covering this assertion, both PolitiFact and The Washington Post's Fact Checker determined that Bill Clinton's job numbers were essentially correct. PolitiFact (AllSides Bias Rating "Left") gave it a "True" rating and went on to make the case that the numbers were even stronger than they appear.
PolitiFact's "Left" rating for bias especially interested me because I've often wondered how PolitiFact would stack up against other fact checkers in terms of sustaining reader trust.

Unless AllSides confused PolitiFact with Politico (which looks somewhat likely), it looks like PolitiFact wears a broad reputation for liberal bias.

We'll look forward to more from AllSides.


Jeff adds:

Consider me one of the skeptics. Crowdsourcing has undeniable value. Whether it's Yelp reviews or an Ebay member's rating, the opinions of large amounts of random people can mean something. But the authority of the masses doesn't hold the same weight with regard to objective reality. An overwhelming number of otherwise rational people vouched for the Macarena's awesomeness. Large groups of people can be wrong. Reality is unencumbered by the burdens of popularity. The fact that most people think PolitiFact is biased to the left doesn't make it so. That finding is better explored through critical study with verifiable and reproducible evidence.

I took AllSides' 'test' and it's no surprise I ended up on the 'Right' side of their scale. The whole process seemed a bit push-poll-ey to me. If someone considers flag burning immoral, but supports flag burning as a constitutional right, which box do they choose on AllSides test? Do you feel extremely strong that abortion is a right [left box] while rejecting the notion that it's the government's responsibility to pay for it [right box]?

Self-assessment is inherently flawed. And it's even less reliable when dealing with convoluted subjects like political philosophy and moral convictions.

Regardless, I say kudos to AllSides for assembling a respectable team of experts. There's no doubt about their sincerity and they deserve the benefit of the doubt. I commend them on their sincere efforts and it's possible they will provide interested readers with valuable, if only anecdotal, information as their project progresses.

I'm not convinced, but I'm looking forward to them proving me wrong. At the very least they deserve credit for producing a much more honest and transparent project than PolitiFact ever has.

[Note: A draft version of this Jeff adds portion was inadvertently published simultaneously with the original post and then immediately removed.]


Sunday, September 16, 2012

Cathy Young: "But who fact-checks the fact-checkers?"

I was surprised yet pleased to see a PolitiFact-related criticism in the Star Tribune, traditionally a liberal newspaper.

Cathy Young's "But who fact-checks the fact-checkers?" was written for Newsday and treads on some familiar ground, namely fact checks of Paul Ryan regarding the closing of the GM plant in Janesville and allegations that the Obama administration "gutted" the back-to-work provisions in the Welfare reform bill signed by President Clinton in the 1990s.

Young doesn't expand much beyond past criticisms of the corresponding fact checks but very aptly describes one of PolitiFact's foundational problems:
Especially on complex policy issues, facts are rarely just facts. Is Obama's health care law a "government takeover" of health care, or merely an expansion of government's role? Would Ryan's Medicare reform plan represent the "end" of Medicare, or merely an overhaul? Is rhetorical exaggeration a lie? Is an out-of-context statement false?
Young's questions lead the reader toward a point we make repeatedly at PolitiFact Bias:  PolitiFact often reaches well past the bounds of objective fact checking to judging the limits of rhetoric.  The same impulse that leads PolitiFact to grade clearly hyperbolic statements as "False" or worse leads PolitiFact to make judgments as to whether politicians provide sufficient context with otherwise true statements.

Are such judgments within the purview of journalistic fact checking?  Arguably so, yet not without a label to ensure that readers know they're reading news analysis or even opinion.
— Distinguish between advocacy and news reporting. Analysis and commentary should be labeled and not misrepresent fact or context.
Credit to Young for a solid article and credit to Newsday and the Star Tribune for publishing it.  Young's story is especially good for introducing readers to the Janesville and Welfare reform fact checks.  Those unfamiliar with those stories will get a concise and accurate treatment by reading the whole of Young's story.

Friday, September 14, 2012

Ben Shapiro: "Politifact Cites Three Liberal 'Apology Experts' to Condemn Romney"

Ben Shapiro, writing for Breitbart.com's Big Peace, pre-emptively steals my thunder on PolitiFact's ridiculous story on Mitt Romney and the statement from the American embassy in Libya.

Shapiro:

Just when you think Politifact can’t make any more of a mockery of itself than it already has – over and over and over and over again – they wade into the breach today on foreign policy. More specifically, they took issue with Mitt Romney’s statement today that “I think it’s a terrible course for America to stand in apology for our values.”
PolitiFact has a history of denying that things Mitt Romney says are apologies are, in fact, apologies.  Shapiro has fun with PolitiFact's method of undercutting Romney in this case:
So, what did Politifact have to say? They interviewed three “apology experts.” Seriously. First, they interviewed Professor John Murphy, of the University of Illinois at Urbana-Champaign, who said it wasn’t an apology because “the statement does not use the word ‘apology’ or ‘apologize’ and does not use any synonym for that word.” Second, they interviewed Lauren Bloom, “an attorney and business consultant who wrote The Art of the Apology.” What did she say? Romney’s “once again allowing his emotional allergy to apology to interfere with his judgment.” Finally, they interviewed Professor Rhoda E. Howard-Hassman, who said the statement was “not an apology.”
But is that PolitiFact's fault?  PolitiFact tried to contact a fourth expert who did not respond.  By looking at the earlier fact checks we can confirm that the expert was conservative foreign policy analyst Nile Gardiner of the Heritage Foundation.

What did Gardiner have to say in PolitiFact's original story?  Here it is:
Nile Gardiner, a foreign policy analyst with the the conservative Heritage Foundation, said Obama is definitely apologizing, and it's not good. He co-wrote the Heritage analysis, "Barack Obama's Top 10 Apologies: How the President Has Humiliated a Superpower."

"Apologizing for your own country projects an image of weakness before both allies and enemies," Gardiner said. "It sends a very clear signal that the U.S. is to blame for some major developments on the world stage. This can be used to the advanage of those who wish to undermine American global leadership."

He noted that Obama tends to be most apologetic about how the U.S. has fought terrorism and its approach to the Iraq war. "There is a very strong partisan element to his apologies, but the biggest driving factor is Obama's personal belief that the U.S. is not an exceptional, uniquely great nation," he said.
As I noted in an earlier analysis, PolitiFact completely discounted Gardiner's statement in ruling Romney "Pants on Fire" for saying Mr. Obama went on an apology tour.  PolitiFact did not explain its reasons for discounting Gardiner's expertise.  If partisanship was a problem then we should expect PolitiFact to find an entirely new set of experts.  Choosing the expert opinion of three liberals over one conservative looks simply like an expression of partisan bias by the fact checker when unaccompanied by a solid rationale.

In the latest apology for Obama, PolitiFact's three experts make a show of distinguishing between condemnation and apology.  But that approach obscures a potential relationship between condemnation and apology.

One cannot condemn an entity and apologize for that same entity at the same time with the same statement.  Those aims work against each other.  But very clearly, one can easily work a condemnation into an apology:  "My son was bad, bad, bad, bad, bad--a thousand times bad for breaking your window, Mrs. Jones."

In the above example we have an apology and a condemnation in the same sentence.  It works because the apology is directed at one entity (Mrs. Jones) while the condemnation is directed at a third party (the son).  By throwing a natural ally under the bus for breaking the window, the condemner sends a clear implicit message of regret to the offended party, Mrs. Smith.

It's important to emphasize the role of an apology in both personal and international relations:  An apology is an attempt to smooth things over with the offended party.  Condemning the breaking of the window sends a message to Mrs. Jones that something will be done to the window breaker to help balance the scales of justice.  Absent that implication, condemning the window-breaker isn't likely to sooth Mrs. Jones' ire.
 
In the case of the Libyan embassy, embassy officials clearly released the statement with the aim of defusing anger at the United States.  One can claim that it was a condemnation rather than an apology, but that's obfuscation.

It was a classic apology, delivered by implicit means.


***
Shapiro's sharp, pithy and to the point.  Visit Big Peace and read the whole of his take.

Tuesday, September 11, 2012

PolitiFact and the gutting of Welfare reform (Updated)

Many of us on the political right heard about the Obama administration's proposed waivers for Welfare work requirements cast in a frame suggesting Obama had rolled back a significant aspect of the Republican-led Welfare reform plan Clinton signed into law in the 1990s.

The Romney campaign didn't take long to produce ads criticizing the change.

Almost as quickly, mainstream fact checkers found fault with such ads.

PolitiFact:
The ad’s claim is not accurate, and it inflames old resentments about able-bodied adults sitting around collecting public assistance. Pants on Fire!
 We were suspicious.  But to call the fact checkers on a mistake takes research.

Enter Mickey Kaus, piggybacking on The New York Times:
Here’s how the Times describes what Nevada wants to do:
[Nevada] asked to discuss flexibility in imposing those requirements. Perhaps, the state asked, those families hardest to employ could be exempted from the work requirements for six months while officials worked with them to stabilize their households. [E.A.]
“Exempted from the work requirements for six months.” That’s not just “weakening” work requirements–the safe, milder charge I chose to make a couple of days ago. It’s explicitly tossing them out the window for an extended period–“to allow time for their barriers to be addressed and their household circumstances stabilized”, in Nevada’s words.
Nevada, recall, was one of the states the Obama administration cited as requesting waivers from the Welfare reform work requirements.

This piece of evidence alone doesn't make the Romney ad accurate.  But it does render the "Pants on Fire" verdict very questionable.

And then there's Robert Rector, writing for the National Review:
(I)t appears the administration intends to do away with standards of the reform law that require 30 to 40 percent of the work-eligible TANF caseload to engage in clearly defined activities for 20 to 30 hours per week. It will replace those standards with a new standard urging that the work-eligible caseload engage in vaguely defined activities for as little as one hour per week. This sounds a lot like “gutting” to most reasonable people.
Read all of Rector's argument and look for additional information before making a final judgment.  Don't trust in mainstream fact checkers like PolitiFact.

We'll keep following this issue as it develops.


Update 9/11/2012:

Our apologies for neglecting information pointed out in August by Senior Attorney Hans Bader of the Competitive Enterprise Institute:
The Obama Administration’s move to gut welfare reform puzzled Kaus, who voted for Obama in 2008. But perhaps it shouldn’t have baffled him, since it reflects Obama’s longstanding antipathy to reforms of the welfare system aimed at reducing dependency on welfare and requiring welfare recipients to work. “For example, in the years immediately after passage of the [1996 reform] law, Barack Obama himself pledged to do all he could to undo it,” noted the Washington Examiner. As the Examiner's Chief Political Correspondent, Byron York, noted, on July 12, the Obama administration ‘released an official policy directive rewriting the welfare reform law of 1996’” to allow the “Department of Health and Human Services to waive the work requirement at the heart of welfare reform.”

Obama also gutted welfare reform in other ways, such as supporting and signing into law a stimulus package that rewarded states for promoting welfare dependency, giving state governors an incentive to try to water down any work requirements for welfare recipients to keep federal welfare money flowing.
In a subsequent essay, Bader delivers appropriate words for PolitiFact:
In arguing that waivers won’t lead to the gutting of the 1996 welfare reform law, since the Obama administration now says it won’t approve waivers unless it makes welfare reform more successful, left-leaning “independent” fact-checkers like PolitiFact and Factcheck.org chose to rely on political spin from the Obama administration in response to the furor over its action, and self-serving, unsubstantiated, and non-binding statements about its intentions, rather than on what the Obama administration actually did in claiming for itself the broad authority to waive the work requirements at the heart of the welfare-reform law (and what it actually said in its July 12 HHS memo claiming that authority, which discussed “the sort of waivers they want to grant,” which do indeed “weaken work requirements,” and did so in response to a waiver request by Nevada, which expressly sought to weaken work requirements, as Mickey Kaus has noted at The Daily Caller).
We echo Bader's question:  Why did PolitiFact remain effectively silent regarding contrary expert opinion?

This is the game we end up with so often from PolitiFact.  PolitiFact ends "he said/she said" journalism by arbitrarily picking the clear winner between two plausible opposing viewpoints.  The result gives us "PolitiFact says" journalism.


Jeff adds:

I looked through PolitiFact's source list but wasn't able to see a poll they cited that determined Romney's ad "inflames old resentments about able-bodied adults sitting around collecting public assistance." How did PolitiFact determine the reaction millions of viewers had to the ad? Unless they offer some evidence of the audience reaction, PolitiFact is simply editorializing.

PolitiFact takes another cue from the opinion pages in its summary:
Romney’s ad says, "Under Obama’s plan (for welfare), you wouldn’t have to work and wouldn’t have to train for a job. They just send you your welfare check."

That's a drastic distortion of the planned changes to Temporary Assistance to Needy Families.
How is that a drastic distortion? If Romney had used Nevada's language, and said "You could be exempt from work requirements and you'll still get a welfare check" would PolitiFact have awarded Romney with a shiny True? PolitiFact's hyperbolic description is raw opinion that shouldn't be confused with verifiable evidence of a falsehood.

And check out this gem describing the same policy that allows states to provide a six month exemption from work requirements:
"The requirement was for more work, not less."
That's a portion of a Bill Clinton statement that PolitiFact rated True.

I'm also struggling to imagine the fantasy world where PolitiFact would ever write this paragraph:
That's a drastic distortion of the planned changes to the Affordable Care Act. By granting waivers to states, the Romney administration is seeking to make health insurance mandates more successful, not end them. What’s more, the waivers would apply to individually evaluated pilot programs -- HHS is not proposing a blanket, national change to ObamaCare.
In PolitiFact's world of facts, illegally granting waivers and exemptions to laws is a way to strengthen policy. Keep that in mind should the GOP control the White House and Congress next year.
 
Romney's claim is entirely accurate. Partisans can argue about whether or not it's misleading, or if it's a reasonable summation of Obama's policy. But Romney's statement that a person could still receive a welfare check without working or training is unarguably based on fact. Attempts to claim otherwise are pure editorial spin.

This is how the non-non-partisans at PolitiFact have always operated, though in the past they did better at keeping their campaign chaff out of their research wheat. They must be getting antsy, or perhaps they're getting cocky. Either way, their unabashed defense of liberal policies is less and less camouflaged as the election nears.


Thursday, September 6, 2012

Bearing Drift: "Who Watches The Watchers?"

There's been a lot of good criticism on PolitiFact lately, and Brian Schoeneman over at Bearing Drift provides a great example. While his post discusses fact checking and fact checkers in general, he hits the mark when panning three recent PolitiFact ratings that went against Paul Ryan:
What Ryan said in each statement he made was factually accurate.  Politifact claimed what he said was misleading, which is their opinion, not a fact.  Politicians using facts to persuade and not giving the entire context is to be expected, and there’s nothing wrong with it.  We can’t expect a politician to act like a journalist or a lobbyist, presenting both sides of every story.  That’s not what they do.  As Gabriel Mallor noted in the New York Daily News, “the bottom line is that the fact checker criticisms of Ryan’s speech come in only one form: ‘Yes it’s true, but here’s some context that Democrats want to talk about.’ That’s not fact checking; that’s advocacy. And it’s not persuasive, it’s absurd.”

And that’s the point – Ryan’s statements weren’t false, so calling them false is dumb. We should expect more from folks who want to sit in judgment over “the truth.”

When fact checkers stop actually checking facts and begin checking opinion or try to place facts in “context,” they enter into murky water where bias is inevitable.
Schoeneman goes on to offer advice for fact checkers to help them avoid the pitfalls of opinion journalism, and serves up plenty of thoughtful criticism while he's at it. I won't say that Schoeneman writes anything particularly unique or anything that we haven't been saying here for years, but his observations are spot on, well written and carry a message well worth repeating. His post isn't a specific critique of PolitiFact, but he hits so many solid points in such a short post we consider it recommended reading. Check out the entire article here.

Wednesday, September 5, 2012

Hans Bader: "'Fact Checker' Repeals The Laws Of Supply And Demand: The Bias Of PolitiFact"

Hans Bader of the Competitive Enterprise Institute brings his expertise to the art of criticizing PolitiFact this week:
The left-leaning, self-proclaimed “fact checker” PolitiFact ignored the most basic economic law, the law of supply and demand, in claiming that cap-and-trade legislation, which is designed to limit energy consumption and increase the price of energy from non-renewable sources, could actually result in “an average lower cost for consumers.” Even the supporters of such legislation, such as President Obama, have admitted that such legislation increases energy costs to consumers. In a January 17, 2008, interview with the San Francisco Chronicle, Obama said thatelectricity rates would necessarily skyrocket” under his cap-and-trade plan to fight global warming. Similarly, a CBS analyst pointed out that a Treasury Department analysis estimated the cost of the Obama administration’s cap-and-trade plan at $1,761 per year for the average American household.
After faulting PolitiFact for its understanding of economics, Bader gives something of a roll call for PolitiFact's most notable critics on the right, including this blog.

Our heartfelt thanks for Bader for the plug.  Please visit openmarket.org to read the rest of Bader's piece and visit some of the many worthy PolitiFact critiques Bader links.

PFB Smackdown: Dylan Otto Krider vs. Jon Cassidy (Updated)

Dylan Otto Krider's orbit around truth-hustler Chris Mooney helped bring him into direct conflict with Ohio Watchdog writer Jon Cassidy this week.  We've featured some of Cassidy's PolitiFact critiques here at PolitiFact Bias, and our review of Cassidy's longer article for Human Events is pending.

Krider, like Mooney, believes that statistics from PolitiFact indicating that Republicans receive harsher ratings than Democrats help show that Republicans simply have a more cavalier attitude toward telling the truth.  Cassidy's Human Events story challenged that interpretation and prompted a story in reply from Krider.

Krider's central point carries partial merit.  He challenges Cassidy's headline with his own:  "Does PolitiFact say Republicans lie nine times more? Really?"

That answer to that question is "no," but Krider used specious reasoning to reach the conclusion.  Examples follow.

Tuesday, September 4, 2012

Another dimension to the Janesville GM plant fact check

Hot Air's Ed Morrissey performs an important service by emphasizing that Paul Ryan's RNC speech mention of GM's Janesville plant was entirely accurate.

Morrissey:
Clearly, the job of “fact checker” in the mainstream media must not involve research skills.  Nor does it take much in comprehension, because these supposed fact checks started with a misrepresentation of what Ryan actually said.  Here are his actual words, emphasis mine:
President Barack Obama came to office during an economic crisis, as he has reminded us a time or two. Those were very tough days, and any fair measure of his record has to take that into account. My home state voted for President Obama. When he talked about change, many people liked the sound of it, especially in Janesville, where we were about to lose a major factory.

A lot of guys I went to high school with worked at that GM plant. Right there at that plant, candidate Obama said: “I believe that if our government is there to support you … this plant will be here for another hundred years.” That’s what he said in 2008.

Well, as it turned out, that plant didn’t last another year. It is locked up and empty to this day. And that’s how it is in so many towns today, where the recovery that was promised is nowhere in sight.

Morrissey points out in his post that a number of mainstream media fact checkers published stories claiming Ryan was inaccurate in talking about the Janesville plant.

Here's how PolitiFact continues to play it:
Here is a look at the accuracy of various statements Ryan made Wednesday night, based on past PolitiFact rulings and other sources as noted:

Closed GM plant: PolitiFact Wisconsin evaluated Ryan’s statement — made both before the convention and in his speech — that Obama broke his promise to keep the Janesville auto factory from closing.

The claim was rated False due to the lack of evidence Obama explicitly made such a promise and the fact the Janesville plant shut down before he took office.
During his convention speech, Ryan referred to Obama's promise of economic recovery, not a promise to keep the Janesville plant open.  Ryan quoted Obama accurately about the Janesville plant, in fact.  Given that Ryan said nothing about an Obama promise to keep the plant open, it is irrelevant when the plant closed.  And as to the timing of the closure, Ryan provides what seems like adequate context.  He says the plant was at risk when Obama made his speech.  He points out the plant didn't last another year.

Ryan's convention speech was too accurate for PolitiFact.  So PolitiFact did not fact check Ryan's convention speech regarding the Janesville plant.  It fact checked a different statement Ryan made earlier in 2012 and claimed it was fact checking Ryan's convention speech.
Ryan stirred memories of the factory on Aug. 16, 2012, attacking President Barack Obama during a campaign speech in Ohio.

"I remember President Obama visiting it when he was first running, saying he'll keep that plant open. One more broken promise," Ryan said.

He made the same point Aug. 29 during his speech to the Republican National Convention in Tampa.
The ensuing fact check emphasizes things only specifically found in the Aug. 16 speech.

These types of errors appear to go beyond mere incompetence.  They look like designed misdirection.  When a fact check claims to check something said at a convention it should stick with what was said at the convention.  That's something no competent reporter should bungle.

Are we to believe the team at PolitiFact responsible for this fact check was collectively that incompetent?


Afters: 

Morrissey updated his story with a new post including video and additional comments.  And it's worth reminding readers that even though PolitiFact checked a statement other than the one Ryan made at the convention it still blew the call.

Sunday, September 2, 2012

The Weekly Standard: "PolitiFact's Credulous Romney-Ryan Health Care Attacks"

The Weekly Standard's Mark Hemingway once again exemplifies what I'm talking about when I say the criticisms of PolitiFact from the right sustain a higher standard than those from the left.  Hemingway methodically dismantles PolitiFact's facade of misstatements surrounding health care claims by Mitt Romney and Paul Ryan.

Behold as Hemingway warms to his topic:
Perhaps if we all ignore PolitiFact, they'll go away. But for the time being, the supposedly independent organization continues to crank out skewed and partisan work. There's no better example of this than the the current jihad the "fact checking" organization is waging against the Romney-Ryan health care plan.
Hemingway goes on to point out PolitiFact's failure to acknowledge the power of the Independent Payment Advisory Board to implement policies that reduce services for Medicare beneficiaries by decreasing the supply of providers.

He then highlights the mendacity of the Obama campaign and its fact-checking lackey in promoting the claim that Medicare beneficiaries may bear an increased cost of $6,400 per year for Medicare insurance.  The Obama campaign attacked an obsolete version of a Ryan reform,  and PolitiFact evidently granted the Democrats the benefit of the doubt that attacking the old plan was not an attempt to mislead the audience.  The claim in the ad, says PolitiFact, is "Half True."

Hemingway:
PolitiFact presents no evidence that the current Romney-Ryan Medicare plan will costs [sic] seniors anywhere close to $6,000. So how the heck, in the total absen(c)e of evidence, does that statement rate even "half true"?
May I suggest to Hemingway that employing inconsistent standards for judgment can easily assist the opinion journalists at PolitiFact in reaching their apparently partisan conclusions?

Hemingway makes the complex easy to digest, so make the time to read the whole thing.

Disconnect at PolitiFact Ohio

Nothing's better than getting PolitiFact editors on the record about PolitiFact.  Their statements probably do more than anything else to show that the PolitiFact system encourages bias and the people who created it either don't realize it or couldn't care less.

Statements from editors at PolitiFact's associated newspapers come in a close second.

Ted Diadiun, reader representative for the Cleveland Plain Dealer (host of PolitiFact Ohio), answering a reader's question:
In July you printed a chart with two years of PolitiFact Ohio results. It showed Democrats with 42 ratings of Mostly False, False or Pants on Fire, while the Republicans had a total of 88 in those categories. Doesn't that prove you guys are biased?

Well, it doesn't necessarily prove that. It might prove instead that in the statements our PolitiFact team chose to check out, Republicans tended to be more reckless with the truth than Democrats.
Diadiun apparently doesn't realize that if PolitiFact Ohio chooses more Republican statements to treat harshly then it is a likely sign of institutional selection bias unless PolitiFact Ohio either randomizes its story selection (highly unlikely) or coincidentally chose a representative sample.  How would we ever know that the sample is representative unless somebody runs a controlled study?  Great question.  It's such a good question that it is reasonable to presume that a disparity in the treatment of the respective parties by PolitiFact results from an ideologically influenced selection bias.  That was the point of Eric Ostermeier's study of PolitiFact's 2011 "Truth-O-Meter" results.

Diadiun, continuing his answer to the same question:
Or, it might prove only that there are a lot more Republicans who have been elected to the major offices that provide most of the fodder for fact-checking.
It would prove nothing of the kind.  PolitiFact has one state operation in a state that is firmly controlled by Democrats:  PolitiFact Oregon.  PolitiFact Oregon, despite a state political climate dominated by Democrats, rates the parties about evenly in its bottom three "Truth-O-Meter" categories (Republicans fare slightly worse).

Diadiun:
It is also a fact that Republicans had a few more statements rated True than the Democrats did, but the Truth-O-Meter was indeed a bit tougher overall on Republicans. You can find the report here

Does that show bias? I've said it before, and I'll say it again here: The PolitiFact Truth-O-Meter is an arbitrary rating that has the often impossible job of summing up an arduously reported, complicated and nuanced issue in one or two words.
Diadiun goes on to tell his readers to "ignore" the Truth-O-Meter.

That's quite the recommendation for PolitiFact's signature gimmick.

He's partly right.  PolitiFact ratings cram complicated issues into narrow and ill-defined categories.  The ratings almost inevitably distort whatever truth ends up in the reporting.  So shouldn't we ask why a fact checker steadfastly continues to use a device that distorts the truth?

The answer is pretty plain:  The "Truth-O-Meter" gimmick is about money.  PolitiFact's creators think it helps market the fact checking service.  And doubtless they're right about that.

There is a drawback to selling out accuracy for 30 pieces of silver:  Contrary to Diadiun's half-hearted reassurances, the "Truth-O-Meter" numbers do tell a story of selection and ideological bias.  Readers should not ignore that story. 


Jeff adds:

The hubris on display from Diadiun could fill gallon-sized buckets. Notice that he completely absolves PolitiFact from the role they play in selecting which statements to rate, and immediately implies the "Republicans tended to be more reckless with the truth than Democrats." Incompetence or deceit are the only reasonable explanations for such an empty claim.

For the benefit of our new readers, I'd like to provide an exaggerated example of selection bias: Let's say I'm going to call myself an unbiased fact checker. Let's say I'm going to check 4 statements that interest me (as opposed to a random sample of claims). I'll check Obama's claim that he would close Guantanamo Bay, and his claim that he "didn't raise taxes once." I find he's telling falsehoods on both accounts.

Next, I'll check Rush Limbaugh's statement that he's happy to be back in the studio after a long weekend. I'll also check his claim that he's one of the most popular radio shows in the nation. Of course, these claims are true.

What can we learn from this? According to PolitiFact's metrics, Rush Limbaugh is a bastion of honesty while Barack Obama is a serial liar. I'll even put out "report cards" that "reveal patterns and trends about their truth-telling." I'll admit the "tallies are not scientific, but they provide interesting insights into a candidate's overall record for accuracy." It's unbiased because, as the popular defense goes, I checked both sides. The reality that I checked statements that interested me supposedly has no influence on the credibility of the overall ratings. If you don't like the results, it's because you're biased!

See how that trick works?

It's something to keep in mind the next time you see Obama earning a Promise Kept for getting his daughters a puppy or a True for his claim that the White Sox can still make the playoffs. A cumulative total of PolitiFact's ratings serves the purpose of telling readers about the bias of PolitiFact's editors and what claims are interesting to them. It's a worthless measure of anything else.

Saturday, September 1, 2012

PolitiFlub: PolitiFact grades Callista Gingrich by the wrong measure

Crossposted from Sublime Bloviations


Words matter -- We pay close attention to the specific wording of a claim. Is it a precise statement? Does it contain mitigating words or phrases?
--Principles of PolitiFact and the "Truth-O-Meter"

It's a testament to PolitiFact's warped self-image that it continues churning out journalistic offal even while enduring a wave of substantive criticism.

Our latest example comes again from the Republican National Convention, where Callista Gingrich claimed that the Obama administration's foreign policy has led to decreased respect for the United States.

A legitimate fact checking enterprise immediately suspects that Gingrich referred to respect from foreign governments in terms of recognizing the U.S. as a power to which deferral yields the most beneficial results.  In other words, other nations fear the United States depending on the degree to which they operate contrary to our policy designs.  Based on that premise, the legitimate fact checker asks Gingrich to clarify the intent and tries to find a verifiable statistic that measures her accuracy.

That's not PolitiFact:
While surveys are currently being undertaken in 20 nations, only 14 of those have been done for long enough to shed light on Callista Gingrich’s claim.

The question asked is, "Please tell me if you have a very favorable, somewhat favorable, somewhat unfavorable or very unfavorable opinion of ... the United States." While favorability isn’t exactly identical to respect, we think it’s very close and a good approximation.
Seriously?

No doubt PolitiFact used the opinions of foreign policy experts to determine that the Pew data were an appropriate measure.

Or maybe not:


Seriously?  No expert sources?  Not one?

That's not a responsible fact check.  The global standing of the United States does not depend on the popular view among the world's peoples.  It comes directly from the way the world's leaders view the United States and whether they believe they can flaunt their power contrary to U.S. interests.

PolitiFact chose the wrong measure.

Why does anyone take PolitiFact seriously?



Jeff adds (9-2-12): 

If there's any doubt that PolitiFact is peddling editorial pieces as objective reporting, check out this Bret Stephens op-ed in the Wall Street Journal last week discussing the same topic and using the same sources:
In June, the Pew Research Center released one of its periodic surveys of global opinion. It found that since 2009, favorable attitudes toward the U.S. had slipped nearly everywhere in the world except Russia and, go figure, Japan. George W. Bush was more popular in Egypt in the last year of his presidency than Mr. Obama is today.

It's true that these surveys need to be taken with a grain of salt: efficacy, not popularity, is the right measure by which to judge an administration's foreign policy. But that makes it more noteworthy that this administration should fail so conspicuously on its own terms. Mr. Obama has become the Ruben Studdard of the world stage: the American Idol who never quite made it in the real world.
Is PolitiFact accusing Mr. Stephens of lying? Inaccuracy? Or is the reality that the world's opinion of America is beyond the scope of objective, measurable standards? How could two reputable outfits come up with such contradictory interpretations of the same facts? What is the measuring stick that makes Louis Jacobson and the Truth-O-Meter the final arbiter of truth on one end and Bret Stephens a dishonest, partisan dolt on the other?

Callista Gingrich made a perfectly reasonable, if not politically rhetorical, statement about Obama's influence on the world's impression of our country. She offered an opinion that has solid, if not conclusive, support. PolitiFact's biggest lie is their claim that they can fit opinions onto a ratings scale and objectively disprove them with opinions of their own.

The reality is PolitiFact often publishes opinion pieces instead of fact checks. And if it expects to maintain whatever shred of credibility it has left, it should take a lesson from Mr. Stephens' employer, and publish its articles on the editorial page.


(Earlier today I explained even more problems with PolitiFact's treatment of Gingrich's claim in the comments section below, so I won't repeat them here.)

A word to the army of Davids

It was always our intention at PolitiFact Bias to encourage and highlight the work of an army of Davids in confronting PolitiFact's distortions of the truth.

We're happy to say that the army of Davids is growing and producing an abundance of accurate and effective criticism.

The volume of material from that army, unfortunately, presents a great challenge in terms of providing recognition.  We keep a bulletin board of sorts with a list of stories for review.  That list reached its all-time greatest length this week.

We'd love to issue a review of every one of the items on the list, but that's increasingly impractical.  Our "Contra PolitiFact" section in the sidebar features some consistently good sources of material, so please continue using those links to find good material we haven't yet highlighted.

To all those producing quality work critical of PolitiFact, we extend our thanks.  Special mention goes to Matthew Hoy of Hoystory, Jon Cassidy of watchdog.org  (also recently published in Human Events), and pseudonymous blogger "counterirritant."


Jeff adds:

Back in 2008 when I first discovered PolitiFact, it was very difficult for me to believe I was the only one that saw through the subtle twists in reasoning and the inconsistency in PolitiFact's application of standards that overwhelmingly benefited the left. It wasn't long before Bryan and I noticed each other making similar observations and comments on PolitiFact's Facebook page. Back then, we certainly felt very alone in a forest of praise for PolitiFact, and we eagerly shared whatever obscure and rare articles we could find that echoed or supplemented our concerns.

Gradually criticism mounted, and in January of 2011 we created this site with the goal of collecting and promoting the best critiques we could find. Our intention was to establish a collection point for people to research and discover the flaws and bias in PolitiFact's system. 

Almost two years later, that rumble of criticism has grown to a loud roar. An article highlighting a PolitiFact flub was once a rare find, and now the rebukes are so numerous we simply can't keep up.

We'd like to think we contributed to that, but we couldn't have done it without the many readers who email, Tweet, or share our site. A hearty thank you to the many of you who do. Your help in spreading the word has helped pull the perception of PolitiFact's credibility closer to where it belongs.