PolitiMath from PolitiFact New Hampshire

What's new with PolitiMath?

PolitiFact New Hampshire, lately the Concord Monitor's partnership with PolitiFact, gives us a double dose of PolitiMath with its July 2, 2015 fact check of New Hampshire's chief executive, Governor Maggie Hassan (D).

Hassan was the only Democrat to receive any kind of false rating ("False" or "Pants on Fire") from PolitiFact New Hampshire in 2015. PolitiFact based its ruling on a numerical error by Hassan and added another element of interest for us by characterizing Hassan's error in terms of a fraction.

What type of numerical error earns a "False" from PolitiFact New Hampshire?

PolitiFact summarizes the numbers:

In her state of the state address, Hassan said that "6,000 people have already accessed services for substance misuse" through the state’s Medicaid program.

There is no question that substance abuse in the state is a real and pressing problem, and the statistics show that thousands have sought help as a result of the state’s expanded Medicaid program. But Hassan offered (and later corrected) a number that simply wasn’t accurate. The real total is closer to 2,000 -- about one-third the amount she cited.

We rate her claim False.

Describing Hassan's mistake as a percentage error using PolitiFact's figures, Hassan exaggerated her figure by about 230 percent. PolitiFact gave Hassan no credit for her underlying point.

In our PolitiMath series we found the closest match for this case from PolitiFact Oregon. PolitiFact Oregon said conservative columnist George Will exaggerated a figure--by as much as 225 percent by our calculations. The figure PolitiFact Oregon found was uncertain, however, so Will may have exaggerated considerably less using the range of numbers PolitiFact Oregon provided.

In any case, PolitiFact Oregon ruled Will's claim "False." PolitiFact Oregon gave Will no credit for his underlying argument, just as PolitiFact New Hampshire did with Gov. Hassan.

Percent Error and Partisanship

One of our research projects looks in PolitiFact's fact checks for a common error journalists make. We reasoned that journalists would prove less likely to make such careless errors for the party they prefer. Our study produced only a small set of examples, but the percentage of errors was high and favored Democrats.

PolitiFact New Hampshire's fact check of Gov. Hassan draws some consideration for this error, giving us the second mathematical element of note.

PolitiFact could have expressed Hassan's mistake using a standard percentage error calculation like the one we used. We calculated a 230 percent error. But PolitiFact New Hampshire did not use the correct figure (1,800) as the baseline for calculating error. Instead, the fact checkers used the higher, incorrect figure (6,000) as the baseline for comparison: "about one-third the amount she cited."

Using the number "one-third" frames Hassan's error nearer the low end. "One-third" doesn't sound so bad, numerically. Readers with slightly more sophistication may reason that the "one-third" figure means Hassan was off by two-thirds.

Sometimes using the wrong baseline makes the error look bigger and sometimes it makes the error look smaller. In this case the wrong baseline frames Hassan's mistake as a smaller error. The Democrat Hassan gains the benefit of PolitiFact's framing.

PolitiMath at PolitiFact New Hampshire

PolitiFact New Hampshire provides us an example of PolitiMath with its Sept. 19, 2014 rating of Sen. Jeanne Shaheen's ad attacking Republican challenger Scott Brown.

The ad claims Brown ranked first in receiving donations from "Wall Street," to the tune of $5.3 million.

PolitiFact New Hampshire pegged the reasonably "Wall Street" figure lower than $5.3 million:

Brown’s total haul from these six categories was about $4.2 million, or about one-fifth lower than what the ad said.

Note that national PolitiFact's Louis Jacobson, writing for PolitiFact New Hampshire, figures the difference between the two figures with the errant figure as the baseline. That method sends the message that Shaheen's ad was off on the number by about one-fifth, or in error by about 20 percent. Calculated properly, the figure in Shaheen's ad represents an exaggeration (that is, error) of 26 percent.

Curiously, PolitiFact doesn't bother reaching a conclusion on whether it's true that Brown ranks number one in terms of Wall Street giving. Jacobson says Brown led in four of the six categories he classified as Wall Street, but kept mum about where Brown ranked with the figures added up.

That makes it difficult to judge whether the 26 percent error implied by PolitiFact New Hampshire's $4.2 million figure accounts for the "Mostly True" rating all by itself.

Capricious.

For comparison, we have a rating of President Obama where the PolitiFact team made a similar mistake, calculating the error as a percentage of the errant number. In that case, Obama gave a figure that was off by 27 percent and received a rating of "Mostly True."


Afters

After a little searching we found a "Mostly True" rating of a conservative where the speaker used the wrong figure. Conservative pundit Bill Kristol said around 40 percent of union members voted for the Republican presidential candidate in 2008. The actual number was 37 percent. Kristol was off by about 8 percent. So "Mostly True."