We contributed a comment noting that fact checkers commit one of the worst abuses of fact checks by publishing aggregated fact check ratings for individuals and groups. Regular readers know Jeff and I have harped on that issue for years.
Stencel, to our delight, drew comments from PolitiFact editor Angie Drobic Holan and embedded them in his reply:
As you said, I don't think any fact-checkers I know suggest their fact-checks are meant to be a random sample. But I just asked PolitiFact editor Angie Drobnic Holan to explain the "report cards" that appear on that site. As she put it, the person-by-person and group-by-group tallies PolitiFact offers are a "snapshot of what we've chosen to fact-check," presented as a "reader service" that helps PolitiFact's audience navigate their reporting.We replied that readers regularly respond on Facebook to PolitiFact's "report card" articles in ways that indicate they are misled. We asked if we're supposed to believe Holan doesn't know that.
"I think people understand that the Truth-O-Meter is not a scientific instrument," Holan added, noting that PolitiFact's editors also provide a page on the site where they explain their methods and principles (http://bit.ly/1AbmKCz). "There's not a lot of reader confusion out there," she said. And the feedback and metrics she sees tell her it's a popular feature.
Stencel didn't reply to our second comment. And we don't blame him. He's not responsible for what PolitiFact does, after all.
His article on responses to fact checking has some good material in it, and his comment has given us a useful new tag to use when we highlight the way PolitiFact misleads its readers with "report card" stories.