Case in point: Donald Trump says that many Americans are paying more for health care than to pay their rent or mortgage for the first time in history. PolitiFact finds it "Mostly False."
This rating drew our attention right away because of the ambiguity of Trump's claim. How does a fact-check go about making a truth distinction about "many instances"? How many is "many"? And if pinning down "many" poses a challenge, how does one go from that challenge to finding out whether it's happening for the first time in history?
After getting into the text of the fact check, it was a matter of trying to control laughter over the way PolitiFact approached the problem.
Cray-cray methodologyTo figure out whether many Americans were paying more for health care than for their mortgages or rent, PolitiFact focused on national average rent figures. PolitiFact reasoned that if average rent was higher than average health care expense, then Trump was largely wrong even if some individuals were paying more for health care than for rent/mortgage.
Rents and mortgages vary widely according to region. The following chart, even if we discount the reliability of the source, illustrates the problem:
These cities with very high median rent distort the average national rent figures. The same type of distortion occurs with average mortgage figures. Median national rent will tend to be lower than the average national rent. And half of renters pay less than the median, right? The same goes for median mortgages.
Fact check spinPolitiFact even tries to spin expert testimony that supports Trump to try to make it look like it undercuts Trump (bold emphasis added):
For recent home buyers and the self-insured under Obamacare, Dean Baker, co-director of the left-leaning Center for Economic Policy and Research in Washington, said it's important to remember that while health costs have been rising, homeowners have been benefiting from "the extraordinarily low mortgage interest rates of the last seven years."PolitiFact makes the first two paragraphs read as though recent low mortgage costs thanks to low interest rates make it less likely that people are paying more for health care than for their mortgage. The opposite is true: Lower mortgage costs relative to rising health care costs make it more likely that health care costs will exceed mortgage costs.
"A 30-year mortgage at 4 percent on $200,000 (80 percent of the median house price of $250,000) would cost around $11,500 a year," he said in an email. When it comes to health insurance, "a family of four getting a silver plan without any subsidy would pay a bit over $11,000. If the mortgage rate was 5 percent, the annual cost of the mortgage (just interest and principle, no insurance or taxes) would be just under $13,000. If it was 6 percent, then the cost would be $14,400," so Obamacare would cost less.
On the other hand, someone buying Obamacare now might find it far more expensive than the mortgage payment set up 20 years ago when housing prices were much lower.
Rather than "on the other hand," PolitiFact should have led the third paragraph with "In addition" or even "Also supporting Trump's point."
And need we remind PolitiFact that half the people paying mortgages pay less than the median figure?
PolitiMostlyFalse on convergenceAnd, though it's a nitpick compared to the other problems in the story, does PolitiFact know what "converging" means?
Out-of-pocket health care costs have been rising, according to the BLS data, but they're not close to converging, even though prices have been rising steadily as premiums have gone up, deductibles and co-pays have increased, and employers have shifted more of those rising costs to their workers.PolitiFact offered the above observation right under a chart showing health care costs relative to (average) rent and mortgage costs. PolitiFact notwithstanding, the line for health care costs shows a converging trend with the other two lines. So they're not just close to converging. They are converging. PolitiFact apparently thinks the lines haven't converged unless they intersect. That's not true. Two lines may converge over an infinity without intersecting, for example.
It was folly for PolitiFact to think it could pin down "many" and then figure out when "many" might have happened for the first time. " Many" is an indeterminate number. If an appreciable number of persons pay more for their health care insurance than for their rent, then Trump's claim is plausible. And if the median cost of health insurance is closing the gap with the median cost of rents or mortgages then it follows that an increasing number of people are probably paying more for their health insurance than for their rents or mortgages.
Trump's claim that this is happening for the first time ever was effectively meaningless, given Trump was citing an indeterminate number. It's fair for PolitiFact to point out that it doesn't mean much, but absurd for PolitiFact to find it false.
PolitiFact's fact check was far below the standard we should ever see from professional fact-checkers.
Yet PolitiFact often produces fact checks of similarly low quality.
Correction Nov. 6, 2016: Fixed backward cost relationship in the post title.