Thursday, October 11, 2018

This Is How Selection Bias Works

Here at PolitiFact Bias we have consistently harped on PolitiFact's vulnerability to selection bias.

Selection bias happens, in short, whenever a data set fails to serve as representative. Scientific studies often simulate random selection to help achieve a representative sample and avoid the pitfall of selection bias.

PolitiFact has no means of avoiding selection bias. It fact checks the issues it wishes to fact check. So PolitiFact's set of fact checks is contaminated by selection bias.

Is PolitiFact's selection bias influenced by its ideological bias?

We don't see why not. And Taylor Swift will help us illustrate the problem.


PolitiFact looked at Swift's claim that Sen. Marsha Blackburn voted against the Violence Against Women Act. That fact check comes packed with the usual PolitiFact nonsense, such as overlooking Blackburn's vote in favor of VAWA in 2012. But this time our focus falls on PolitiFact's decision to look at this Swift claim instead of others.

What other claims did PolitiFact have to choose from? Let's have a look at the relevant part of Swift's statement:
I cannot support Marsha Blackburn. Her voting record in Congress appalls and terrifies me. She voted against equal pay for women. She voted against the Reauthorization of the Violence Against Women Act, which attempts to protect women from domestic violence, stalking, and date rape. She believes businesses have a right to refuse service to gay couples. She also believes they should not have the right to marry. These are not MY Tennessee values.
 Now let's put the different claims in list form:
  • Blackburn voted against equal pay for women.
  • Blackburn voted against the Reauthorization of the Violence Against Women Act
  • Blackburn believes businesses have a right to refuse service to gay couples
  • Blackburn also believes they should not have the right to marry
PolitiFact says it checks claims that make it wonder "Is that True?

The first statement regarding equal pay for women makes a great candidate for that question. Congress hasn't had to entertain a vote that would oppose equal pay for women (for equal work) for many years. It's been the law of the land since the 1960s. Lilly Ledbetter Fair Pay Act? Don't make me laugh.

The second statement is a great one to check from the Democratic Party point of view, for the Democrats made changes to the VAWA with the likely intent of creating voter appeals based on conservative opposition to those changes.

The third statement concerns belief instead of the voting record, so that makes it potentially more challenging to check. On its face, Swift's claim looks like a gross oversimplification that ignores concerns about constitutional rights of conscience.

The fourth statement, like the third, involves a claim about belief. Also, the fourth statement would likely count as a gross oversimplification. Conservatives opposed to gay marriage tend to oppose same-sex couples asserting every legal advantage that opposite-sex couples enjoy.

PolitiFact chose its best candidate for finding the claim "True" instead of one more likely to garner a "False" rating. It chose the claim most likely to electorally favor Democrats.

Commonly choosing facts to check on that type of basis may damage the election prospects of those unfairly harmed by partisan story selection. People like Sen. Blackburn.

It's a rigged system when employed by neutral and nonpartisan fact checkers who lean left.

And that's how selection bias works.


4 comments:

  1. s PolitiFact's selection bias influenced by its ideological bias?

    We don't see why not. And Taylor Swift will help us illustrate the problem.

    Isn’t this your own bias talking? Aren’t you doing what you accuse PolitiFact of doing?

    ReplyDelete
  2. John Paul Suchecki wrote:

    **Isn’t this your own bias talking?**

    I don't think so. Do you?

    **Aren’t you doing what you accuse PolitiFact of doing?**

    We accuse PolitiFact of claiming neutrality while leaning left. We claim to lean right while making arguments based on neutral evidence while avoiding subjective rating scales like the "Truth-O-Meter."

    We don't see much of a parallel. But if you think you can make the case we're glad to read what you write.

    Recommended:

    https://www.politifactbias.com/2014/05/politifact-bias-is-biased.html

    ReplyDelete
  3. Sorry, I'm not trying to be rude. It wasn't designed to be an attack. I don't even have a problem of your frisk of the actual fact check. I've been following your work quite regularly,and I always assumed you leaned right, but attempted to remain neutral through the best of your ability. After all, you are right in that no one can be completely without bias.

    I choose to write about this one, because while the fact check has its flaws as you pointed out, I fail to see how it prove selection bias based on ideology. In itself, it just seems anecdotal. Perhaps you meant it as smaller part of the larger picture.

    ReplyDelete
  4. **I choose to write about this one, because while the fact check has its flaws as you pointed out, I fail to see how it prove selection bias based on ideology.**

    If we thought this counted as surefire proof we would have called it as much.

    Our presentation is more the scientific method of making a prediction (what would ideological selection bias look like if PolitiFact was left-leaning) and then using this case study to show that PolitiFact plays the left-leaning role pretty much to perfection. And our purpose for the post was to show how a pattern of similar cases would represent a left-leaning bias even if PolitiFact avoided other errors favoring the left.

    It's another piece in the mosaic showing PolitiFact's lean to the left.

    Thanks for taking the time to comment and to follow up.

    ReplyDelete