PolitiFact, unfortunately, provides us a prime example of partisan poll interpretation with a pair of relatively recent fact checks. Within the past week, PolitiFact ruled President Obama "Mostly True" on the claim that his position on the budget deficit negotiations--using a mix of spending cuts and tax increases to address the problem--has popular support. In February 2012, PolitiFact revised a ruling on Sen. Marco Rubio (R-Fla.) from "Mostly True" to "Half True." Rubio claimed a conservative majority in the United States.
PolitiFact's rulings serve as a useful case study showing the pitfalls of misusing poll data.
Where did PolitiFact go wrong on majority support for the president's "balanced approach"?
PolitiFact fell for a fairly obvious piece of misdirection from the White House.
President Obama wants it to look like the public supports his position compared to that of his Republican opposition. To accomplish that, he gives a murky description. His actual position would try to address the budget deficit by using spending cuts and "increased revenues" (read as "tax increases") in roughly equal measure. To the general public, Mr. Obama proclaims he is simply in favor of a "balanced approach," which he hints is defined as a combination of spending cuts and "increased revenues."
The president's rhetoric draws an imbalanced line between his position and the Republican position.
The Republicans originally were unwilling to go along with tax increases. Increased revenues were fine so long as they occurred because of increased economic activity instead of by changing the tax code. When the Republicans fared poorly in the 2012 election they adopted a compromise position. They would go along with increased revenues through the closing of tax loopholes. When a deal on the "fiscal cliff" was finally struck, the Republicans compromised further, allowing tax rates to increase on the wealthy by allowing some of the Bush tax cuts to lapse in addition to a graduated phase-out of tax deductions.
The Republicans, then, compromised their position and allowed for increased revenues in the form of higher taxes. But in 2013 the president has continued to say the Repubicans are unwilling to compromise on increased revenues because they will not compromise a second time on increased revenues. The president identifies the Republican position as refusing all revenue increases.
In reality, the Democratic Party position consists of the approximately 1:1 ratio between spending cuts and revenue increases. The real-world Republican position has allowed some revenue increases via higher taxation, but refuses a second compromise.
Now suppose we have a person who wants $4 in spending cuts for every $1 increase in revenue. Does that person agree with the Republicans or with the president? President Obama's rhetoric suggests the answer: One agrees with the Republicans if one wants only cuts in spending. Everyone in the middle ground between only cuts to spending and a 1:1 ratio between spending cuts favors the president's position. And that's just the way PolitiFact does its fact check.
Where did PolitiFact go wrong on Marco Rubio's conservative majority?
During a Feb. 2012 speech, Rubio said he knew America had a conservative majority because people don't like to wear the label "liberal."
To some extent, Rubio was doing the same thing we looked at just above. He was defining two groups in a way that would bolster his claim that most Americans are conservative.
In this case, PolitiFact never bothered to use the speaker's measuring stick. PolitiFact chose its own: If a person self-identified as a conservative then the person was a conservative. Not otherwise. PolitiFact simply ignored Rubio's attempt to claim an expanded middle ground and chose its own measure.
Rubio tried to make the point that conservatives could win elections by appealing to moderates who will not take accept the "liberal" label. To measure his accuracy, one needs to pressure the moderates to make a choice between liberal and conservative. Liberal writer Kevin Drum did just that (I've addressed this topic before at Sublime Bloviations).
Straw men on the loose
Addressing and defeating a weak version of a person's argument instead of a better version counts as a straw man fallacy.
In our first example, the president used a straw man version of support for Republicans on deficit reduction. PolitiFact, in testing his claim, bought it. People only agree with Republicans on deficit reduction if they will not allow for a dime of new revenue. Otherwise they support the president's "balanced approach."
In the second example, PolitiFact came up with its own straw man version of Rubio's argument. Rubio said he knew that most Americans are conservatives because they don't like the label "liberal." PolitiFact did not measure conservatives by dislike of the "liberal" label but by preference for the label "conservative."
Does PolitiFact mislead its readers on purpose? Probably not. But the end result is the same either way.