As I read the birthday wishes Duke University Professor Bill Adair wrote for PolitiFact, the fact-checking enterprise he helped create, I was floored by its spurious reasoning. Adair looked to validate his creation by telling readers how it has met its primary goal:
The mission of fact-checking has always been to inform, not to change behavior. And by that measurement, fact-checking this year is a smashing success. There is more fact-checking than ever — more than 100 sites around the world, according to the latest tally by the Duke Reporters’ Lab. That’s up more than 60 percent in the last year.What's wrong with Adair's reasoning? I shall explain in a moment, but whether I end up informing anybody about Adair's mistake will depend on my audience.
Indeed, the fact that we know the politicians are using so many falsehoods is actually proof that fact-checking is working. In debates and interviews, the candidates are being questioned about the claims the fact-checkers have found were untrue.
Consider that foreshadowing.
When Adair considers the purpose of fact-checking (informing readers), he chooses a poor means of measuring whether readers are informed. Adair proposes measuring the degree to which readers are informed based on the number of fact-checking websites.
After telling readers that the aim of fact-checking is not merely to change behavior, Adair offered readers a metric that amounts to a change in behavior. Journalists in the media changed their behavior by producing 60 percent more fact-checking sites this year than last. And by asking politicians about fact checked claims.
Were people successfully informed though that increase in fact-checking sites? Maybe. But confirming that would take, for example, a scientific survey of readers that would measure the effects of fact-checking. It does not automatically follow that an increase in the number of fact-checking sites leads to a more informed public. And it should concern us that a principal figure in the fact-checking movement makes an argument that misses that fact.
Adair segues from this stinker of an argument to a second one: He says "The fact that we know politicians are using so many falsehoods is actually proof that fact-checking is working." Adair has simply assumed what he wants to prove. He offers not a shred of evidence that "we know politicians are using so many falsehoods." If Adair tried to offer evidence in support of his claim, it would probably rely on the findings of fact-checkers. Using the findings of fact-checkers to confirm that people informed by the fact-checkers know the truth assumes what Adair seeks to prove.
If readers learn the truth from fact checks, let us see that outcome in the form of hard data. Instead, Adair commits the fallacy of begging the question.
For fact checkers to successfully inform their readers of the truth, the fact checkers have to have a message capable of communicating the truth (unproven) as well as an audience that successfully receives that message (also unproven).
Yet here we have Bill Adair, fact checker, telling people, despite the lack of evidence, that fact-checking is working to inform people.
Adair's argument is naive and this naive approach to knowledge broadly infects Adair's fact-checking movement.
Here's one more plug for Uscinski & Butler's "The Epistemology of Fact Checking," which makes the case that fact-checkers take a naive approach to epistemology.
Update Aug. 23, 2016:
PolitiFact highlighted Adair's birthday gloat, using one of the statements we criticized as the pull quote.
Self-validation the easy way!