Wednesday, April 18, 2012

Nothing To See Here: Hilary Rosen "in context"

PolitiFact has published a second "In Context" item featuring a claim that piques the curiosity of the right-leaning fact checker.  PolitiFact published the comments from Democratic Party strategist Hilary Rosen that prompted the flurry of words surrounding Ann Romney's day job (stay-at-home mom).

This one kind of stuck out:
HILARY ROSEN, POLITICAL COMMENTATOR:  Well, first, can we just get rid of this word "war" on women? The Obama campaign does not use it. President Obama does not use it. This is something that the Republicans are accusing people of using, but they're actually the one spreading it.
How many campaign surrogates will eventually join Hilary Rosen under the bus after that one?

Thursday, April 12, 2012

PolitiFact Florida editor Angie Drobnic Holan on WMNF

Angie Drobnic Holan, recently named editor at PolitiFact Florida, appeared for an interview last month on WMNF, a Tampa Bay area public radio station.

The interview overall was relatively mundane, serving mostly as a forum for liberal callers to recycle their most popular recent complaints about PolitiFact ("Lie of the Year" for 2011, Rubio saying the U.S. is majority conservative, etc.). 

Drobnic Holan did offer up a self-assessment of the fact check operation that we might want to refer to later on down the line, however:
We've published almost 5,000 fact checks since we got started.  We do not get it right every single single (sic) time.  Mitch, you're a journalist [crosstalk]

Let me tell you what we do do.  We correct errors.  There were no, um, errors of fact in this particular report, but sometimes there have been, and we correct those quickly, we note them.  Every now and then, a handful of claims, we say "You know, we got the Truth-O-Meter rating on that one wrong." 

So, our typical procedure is, we go back, we report it again, uh, we have a procedure where a reporter researches and writes the report and three editors sign off on the ruling.  So when we look at rulings again, uh, we have all the editors look at it again.  Um, so, you know, that's a handful of rulings.  The majority--the vast majority--of rulings are, uh, you know, not second-guessed [If only I had the time!--ed.] in any way.

And I would also add that when you go to PolitiFact, when you read our reports, uh, we do something that you don't often see in journalism.  We have a source list where we list all of our sources., we hyperlink to all of the data, list everybody we interviewed, and in our story we carefully explain our logic.  So, what's interesting to me is oftentimes when readers, um, disagree with our work and very passionately disagree with us, they disagree with us using evidence that we gave them.  It's not like they're going out and researching these things and uncovering these facts, I mean it's, it's, it's very much of a, um, of a, of a--they know things now they didn't know before they read the report.
I find it at least as interesting the many times that others go beyond PolitiFact's research (finding additional facts or additional context; examples of both are legion), inform PolitiFact of the additional information and then PolitiFact does absolutely nothing discernible in response. 

Not that a critic needs to find new information to offer a legitimate criticism, of course.

But I guess her response serves Drobnic Holan's PR purposes better in the context of a radio interview.


Jeff adds: Drobnic Holan highlights PolitiFact's supposed transparency by listing "everybody we interviewed." What she failed to mention was that they don't include the transcripts of those interviews. More than one of those interviewees has publicly complained that PolitiFact took their responses out of context. One even pointed out the questions appeared biased from the start:
What struck me about Jacobson’s message was it asked if Romney’s statement was “technically true” and “what context does this ignore,” which carried the clear implication – as I warned Jacobson in my reply – that he’d already decided what he was going to write.
Given PolitiFact's propensity for statement distortion, simply naming people they emailed hardly inspires confidence about their integrity. Until PolitiFact provides the full context of their interviews, the list is little more than window dressing.  

Nothing To See Here: HBO series "The Newsroom"

Do facts matter in a fictionalized setting?

Here's betting that the fictionalized setting is used to try to communicate "truthiness" about the real world in the new HBO series about a cable newsroom.

The apparent lead character, who seems to represent "the most trusted name in news" (does that sound eerily familiar or what?), gets put on the spot regarding his political views.  He shocks what looks like an audience of college students with the following:
 We're not the greatest country in the world
Apparently he's speaking about some fictionalized version of Liberty College or some other right-wing diploma mill.  Don't almost all American universities inform students that the United States is not the greatest country in the world?

He continues:
We're seventh in literacy. We're second in science.  We're 29th in life expectancy.  One hundred seventy-eighth in infant mortality.  Third in median household income.  No. 4 in labor force and No. 4 in exports.
I wouldn't expect a fact checker to look into the logic, which seems like either a smart-ass answer or else a non sequitur.

Imagine an undefeated basketball team wondering if they're really No. 1: "We're seventh in free throw shooting.  We're second in overall defense.  We're 29th in points off the bench.   One hundred seventy-eighth in home attendance.  Third in offensive rebounds.  No. 4 in graduation rate and No. 4 in the number of players reaching the NBA." 

There's no way that team could be No. 1, is there?

On the other hand, the speech does contain a number of factoids that might interest mainstream media fact checkers.  After all, PolitiFact fact checked a "Saturday Night Live" sketch after a fashion.

Does Bill Adair understand selection bias?

My heart fairly skipped a beat minutes ago when I ran across an interview of PolitiFact editor and founder Bill Adair where he was directly asked how PolitiFact avoids selection bias.

National Press Foundation:
How do you avoid selection bias?

There are many things that go into deciding what we are going to choose. We try to be timely, we try to stay on top of the news and we try to have balance so we check people from both parties. That can be challenging though, because if you have eight voices speaking up in a Republican primary and only one Democratic incumbent – naturally you have eight times the number of statements being made on the Republican side than on the Democratic side. We try to check roughly the same number of claims by Democrats as we do for Republicans, but we have to go where the claims are and lately there have been more made by Republicans. In terms of avoiding selection bias, I think the key is to be guided by what serves the reader. Once you get past claims selections, our fact-check process is entirely driven by journalistic and independent assessment.
Shorter Bill Adair:  "We don't avoid selection bias."

The guy just doesn't get it.

Selection bias is simply the non-random selection of subject matter.  Adair has described two different methods of story selection.  First, PolitiFact tries to sniff out suspicious-sounding claims.  Second, PolitiFact tries to pick out roughly equal numbers of claims by Democrats and Republicans.

That's a recipe for producing selection bias, not a method for minimizing it.

 

Jeff adds: Adair's misdirection may placate PolitiFact's sympathetic followers, but his excuse that "naturally you have eight times the number of statements being made on the Republican side than on the Democratic side" because of the current campaigning doesn't pass the laugh test. That may be a plausible reason if PolitiFact limited it's fact checking only to current presidential contenders, but that's not the case.

PolitiFact checks statements by Congressmen, aging pop singers, television personalities, Senate conferences, and chain emails. Are right leaning Congressmen making more statements because of the upcoming election? Are conservative pop stars issuing statements at an 8-1 ratio to their liberal counterparts? No.

And as we've explained before, simply choosing an equal number of statements by opposing parties is an insufficient method to avoid selection bias. Adair knows this, and his evasion on the issue would be embarrassing if it wasn't so insulting.

Thursday, April 5, 2012

Nothing To See Here: DOJ denies President Obama misspoke

President Obama made news this week with his declaration that a Supreme Court finding against the health care reform bill would represent "an unprecedented, extraordinary step of overturning a law that was passed by a strong majority of a democratically elected Congress."
A panel of the Fifth Circuit Court of Appeals learned of the president's remarks and pointedly ask the government council for an assurance of their client's (Obama's) belief in the Court's power of judicial review.

Attorney General Eric Holder provided that assurance today, concluding with the following:
The President’s remarks were fully consistent with the principles described herein.
While I thought from the outset that Obama's statement was of likely interest to fact checkers, I think this one has a much better chance to stay under the radar. 

I'm counting on you to save the day, Glenn Kessler!

Tuesday, April 3, 2012

James Wigderson: "Looking for the 'Pants on Fire' rating"

James Wigderson of the Wigderson Library & Pub offered a critique of PolitiFact late last year that we failed to highlight (we'll excuse ourselves based on the huge amount of PolitiFact-related material published in December and January). 

Visit the Library & Pub for the details of the relatively gentle takedown; our takeaway was the final paragraph:
We also give the Milwaukee Journal Sentinel’s rating system a “pants on fire” rating for failing to have any standards for the public to use to judge whether the Journal Sentinel’s ratings have any meaning.
A most palpable hit.

One may pretend that some of the definitions PolitiFact gives for its ratings allow for objective categorization, but PolitiFact applies its ratings in a manner that seems to defy systematization.  Though some of PolitiFact's writers wisely make an attempt to correlate the fact check's findings to the appropriate definition, those attempts often (if not always) seem subjectively tinged.  The PFB team excepted, who even noticed when PolitiFact changed its definition of "Half True"?

By now so many have leveled the criticism that its novelty has faded well into the past, but it bears repeating:  PolitiFact's rating system is a flop.  It's not worthy of association with fact checking.  It is way too subjective for that.

Monday, April 2, 2012

Nothing To See Here: the President and "unprecedented" (Updated x2)

President Obama's remarks following a meeting with Canadian and Mexican leaders seem likely to draw some attention from fact checkers.
“Ultimately, I am confident that the Supreme Court will not take what would be an unprecedented, extraordinary step of overturning a law that was passed by a strong majority of a democratically elected Congress,” Obama said at a news conference with the leaders of Canada and Mexico.
The "unprecedented" part of the claim isn't the only thing of interest, but I just couldn't resist matching it with "president" in the title of the post.

It's perhaps even money that PolitiFact will address this statement.  If it does, I'll be interested to see the spin (if any).


Update 4/3/12 (Jeff)-

PolitiFact decided to sort out the truth today of the President's 'unprecedented' claim:
Presidents are paid to be confident about their own laws, but what's up with that "unprecedented"? In Marbury in 1803, Chief Justice John Marshall laid down the doctrine of judicial review. In the 209 years since, the Supreme Court has invalidated part or all of countless laws on grounds that they violated the Constitution. All of those laws were passed by a "democratically elected" legislature of some kind, either Congress or in one of the states. And no doubt many of them were passed by "strong" majorities.

As it happens, probably stronger majorities than passed the Affordable Care Act. Readers may recall that the law was dragooned through a reluctant Senate without a single GOP vote and barely the 60 votes needed to break a filibuster. Despite a huge Democratic majority in the House, it passed by only 219-212.
Oops! My bad. That's not PoitiFact. That was the right-wing kooks over at the Wall Street Journal editorial board. It must not be true. I guess we'll just have to wait until our friends outside the echo chamber decide to weigh in.


Update 2 (4/4/2012):

At about 5 p.m. today PolitiFact published a fact check on this item, finding President Obama's statement "False."  The story isn't without a significant amount of spin, which I'll explore at Sublime Bloviations and connect via hotlink to this update.