Thursday, February 18, 2016

PolitiFact crazy about guns

Is PolitiFact crazy about guns?

Consider the evidence.

Back on Nov. 15, 2013, PolitiFact gave Sen. Ted Cruz (R-Texas) a "Mostly False" rating for claiming gun prosecutions were down 30 percent under President Obama from what they were under President Bush. The rationale PolitiFact used to back its rating was crazy.

PolitiFact preferred measuring the trend in prosecutions using "all charges" instead of "lead charges." PolitiFact used data from the Department of Justice's United States Attorneys Office. Fine. We can work with that:


PolitiFact used the third data column, representing the number of defendants against whom firearm charges were filed. That number is higher than the number of cases in the first data column.

PolitiFact then veered into loony land by claiming that the best way to measure the trend in prosecutions was to compare 2002 under Bush to 2012 under Obama. Seriously:
The most reliable number may be the 10.3 percent increase in gun prosecutions between fiscal years 2002 and 2012. This rate actually indicates the trend moving in the opposite direction from what Cruz asserted.
Why would comparing the numbers for just two arbitrarily chosen years serve as the best method for measuring a trend over a period of about ten years? We have no idea.

PolitiFact argues that the third column represents the most "inclusive" number. But it doesn't follow that cherry-picking figures from the column representing the most inclusive number will yield the most accurate trend line. Rather, that method would appear to represent arbitrary cherry-picking. After all, why should 2002 serve as a representative year of gun prosecutions under Bush? In fact, the figure from 2002 is nothing other than the lowest number in the series PolitiFact presented to its readers.

Add to that the fact that the 2002 figure is at the very start of the series representing Bush, and one cannot accept the figure as any kind of trend for the Bush administration.


Statistics best show trends when we compare year-to-year changes over time. As with a graph:


A federal fiscal year runs from Oct. 1 through Sept. 30. Each two-term president will have complete responsibility for only seven fiscal years. Clinton shared FY 2001 with Bush. Bush shared FY 2009 with Obama. Of Bush's full seven fiscal years, PolitiFact used for its comparison the only one that Obama could have improved on.

PolitiFact finds its most revealing trend in gun prosecutions by comparing Bush's first full fiscal year with Obama's third full fiscal year and ignoring the data from all other fiscal years. PolitiFact's "most inclusive method" excluded most of the data. We can imagine no sensible justifying rationale for that method.

PolitiFact concluded by suggesting Cruz was well off the mark because the best method showed a trend toward increasing gun prosecutions under Obama (bold emphasis added):
Cruz said prosecution of gun crimes under the Bush administration was 30 percent higher than it is under Obama.

It’s possible to get a decline that big by cherry-picking the data, but the most inclusive method actually produces an increase. Cruz also overstates the role of the president in determining prosecution rates. We rate Cruz’s claim Mostly False.
Examples like this should rightly do tremendous damage to PolitiFact's credibility. Remember, PolitiFact's "star chamber" met to collectively consider this fact check. How can such rotten methodology go unnoticed by fact check professionals?


Afters

This is not the first time PolitiFact has pulled shenanigans with gun statistics.

Odds are it will not be the last.

7 comments:

  1. This reminds me of Punditfact's Aug 27, 2015 entry that gave a meme a half true rating because it was based on certain years and left out important information.

    ReplyDelete
  2. This comment has been removed by the author.

    ReplyDelete
  3. Alright, I'm a believer now in politifact's bias. The same Federalists that used to run the GOP are now in charge of the DNC and they really changed things quickly. Expect the DNC to fail much in the same way in the up coming election with Cruz wining the presidency. Gonna be one hell of a show!

    ReplyDelete
  4. You're really reaching here. They gave all of the data available for the relevant period and said that there were four different ways you can examine it. All of the ways can be seen as having their own merits, they just have different starting points and compare different estimations of gun crime prosecution. From that stance, since there are four different ways of looking at the numbers, it is pointed out that Cruz cherry-picked the one that was most supportive to his stance. It is just as logical to pick the other estimation, and merely state that there was a spike in prosecutions during Bush's term which tapered off. In fact, it strikes me as fairly remiss to claim a 30% decrease from Bush to Obama, and then leave out the fact that nearly 100% of that decrease happened during FYs that Bush was at least partially responsible for.

    For that matter, why shouldn't we use 2008 as a starting point? It's the last FY that Bush was fully responsible for, it eliminates the possibility of taking into account trends from previous years that had nothing to do with Obama, and doesn't wrongfully attribute increases or decreases that happened wholly under the Bush administration to Obama.

    Oh, and the red line DOES actually indicate the most inclusive method. Most inclusive = including as much as possible. So including more years of relevant data is more inclusive, and starting from 2002 is more inclusive than 2005. "Lead crimes" is not as inclusive as "all firearm crimes". So the most "inclusive" measure of this would be starting from 2002 using "all firearm crimes" and going to 2012. This is not an arbitrarily chosen endpoint, but merely the last year for which said statistics are available.

    As someone that is a collegiate statistician, I can honestly say you just don't understand either the math or logic involved. It makes it really difficult to take your stance as anything other than shallow pseudo-political grandstanding.

    ReplyDelete
    Replies
    1. James Oakes, sometimes I make a post out of my response to a comment. I was tempted to do that with yours, but I'm leaning toward reserving that practice for criticisms with merit.

      I triple-dog dare you to write a paper for your statistics classes supporting PolitiFact's position, that tracking the change from 2003 to 2011 is "perhaps the most inclusive method" for comparing gun prosecutions under Bush compared to Obama. You can subtitle that paper "Why I should not receive a passing grade in this class."

      I'll address your comment point by point.

      **They gave all of the data available for the relevant period and said that there were four different ways you can examine it.**

      1A) The Clinton years are relevant in seeing how the Bush presidency itself affected the number of gun prosecutions. That's obvious, isn't it? If gun prosecutions went way up under Bush compared to Clinton, then that's part of the relevant trend line. PolitiFact obscured that, and you're following suit.

      1B) There are more than four ways to approach the data, including various types of averaging. Averaging is a more inclusive method than cherry-picking one pair of years for comparison for a series of more than two years. You can confirm that by running it past any competent statistics professor.

      **All of the ways can be seen as having their own merits, they just have different starting points and compare different estimations of gun crime prosecution.**

      The fact that PolitiFact considered no type of averaging of any kind instantly discredits the proffered menu of four choices.

      **From that stance, since there are four different ways of looking at the numbers, it is pointed out that Cruz cherry-picked the one that was most supportive to his stance.**

      Yes, Cruz cherry-picked. But what Cruz did was not as bad as what PolitiFact did. It is absurd to recommend the 2003/2011 comparison as the best method for comparing the gun prosecution numbers for the two presidents.

      **It is just as logical to pick the other estimation, and merely state that there was a spike in prosecutions during Bush's term which tapered off.**

      The other estimation doesn't make sense even pointing out the so-called "spike." The "spike was" a years-long trend higher than the performance of the Obama administration. PolitiFact's readers would be better served by presenting average figures and a graph such as the one I provided. Assuming truth is the goal instead of falsely burnishing Obama's image, of course.

      **In fact, it strikes me as fairly remiss to claim a 30% decrease from Bush to Obama, and then leave out the fact that nearly 100% of that decrease happened during FYs that Bush was at least partially responsible for.**

      Good for you. I don't recall defending Cruz's method except to say PolitiFact's recommended method was worse.

      **For that matter, why shouldn't we use 2008 as a starting point?**

      Is it inclusive? Is it properly representative of Bush's record?

      Delete
    2. (continued)
      **It's the last FY that Bush was fully responsible for, it eliminates the possibility of taking into account trends from previous years that had nothing to do with Obama, and doesn't wrongfully attribute increases or decreases that happened wholly under the Bush administration to Obama.**

      Why would we want to eliminate trends from previous years that have nothing to do with Obama? Aren't trends under Bush relevant when comparing gun prosecutions under Bush to the same types of prosecutions under Obama?

      You're right that comparing Obama's record to 2008 would eliminate attributing trends under Bush to Obama. But we'd pay a price by failing to attribute trends under Bush to Bush. Is it desirable to ensure that Bush receives no real credit for raising gun prosecutions during his administration to a high point that Obama has not approached? In the midst of a comparison between the two presidents? Doesn't that strike you as a little bit unfair?

      **Oh, and the red line DOES actually indicate the most inclusive method. Most inclusive = including as much as possible.**

      That's silly. Suppose gun prosecutions spiked to 3 trillion per year for 2004-2010. Or dropped to zero for each of those years. The "most inclusive" method completely ignores all the years in between (as our article pointed out, albeit not well enough for you to notice). A most inclusive method should not skip huge amounts of relevant data between the start and end points. Inclusive methods like that exclude most of the relevant data.

      **So including more years of relevant data is more inclusive,**

      Comparing 2003 to 2011 includes two years of data and skips all the data from 2004-2010. Am I supposed to be impressed?

      **and starting from 2002 is more inclusive than 2005.**

      No, it's two years of data either way. It would be more inclusive if the data in the intervening years wasn't ignored.

      **"Lead crimes" is not as inclusive as "all firearm crimes".**

      Right, and I think that's where PolitiFact tied its own logic in knots. It's a more inclusive data figure, but it can't make comparing 2003 to 2011 into anything other than comparing just two years of data when there's seven more years of ignored data in between.

      **So the most "inclusive" measure of this would be starting from 2002 using "all firearm crimes" and going to 2012.**

      That all depends on whether you ignore 2003-2011, doesn't it? You're saying that ignoring two extra years makes your method even more inclusive than PolitiFact's?
      :-)

      **This is not an arbitrarily chosen endpoint, but merely the last year for which said statistics are available.**

      Right, but what about the starting point? What makes 2002 properly representative of gun prosecutions under Bush?

      **As someone that is a collegiate statistician, I can honestly say you just don't understand either the math or logic involved.**

      As someone who has had to suffer through reading your argument, you aren't qualified to judge. I'll trust that you honestly don't know what you're talking about.

      Delete
  5. Good post about gun. Thank for share this Bryan
    Charllote

    ReplyDelete