Was Rasmussen Right? Is it OK to be White?
Note: this article is a work in prosgress. Still needs more research and editing. Suggestions welcome. Thanks.
There’s lies, d@mn lies, and statistics, according to Mark Twain. And then there’s biased polls, such as the recent Rasmussen poll which suggested that a significant percentage of Black folks said that it’s not OK to be white. The one that Dilbert creator Scott Adams took and ran with — and we know the rest of the story. Strangely enough, I could not find this report on the Rasmussen site, however I did find it referenced in a few other places, such as this article in the Washington Post.
I teach statistics at my local community college and one of the things we discuss is recognizing whether or not a poll is valid. This is important, because poll results shape public opinion as well as report it. Moreover, those in power often make policy decisions based on poll results. So here’s a review of how to recognize valid / invalid polls. I promise, there will be no math!
This poll has a leading / ambiguous question, one of the ways polls can be wrong. There are a few other ways polls can be wrong:
2. invalid sampling techniques leading to non-representative samples
3. invalid math used to process the data, and
4. invalid (biased, inaccurate, etc) reporting of the poll results
The question asked of a small sample (n=117) of Black people regarded as likely voters, was — do you agree with “it’s OK to be white”? According to the report, 53% of the responders said they agree, 26% disagreed, and 21% said they were not sure.
The problem: this question is ambiguous. It could mean:
1. “Is it ok for some folks to have pale skin and slick hair?” Or,
2. “Do you agree with a white supremacist slogan?”
To further complicate things, the white supremacist slogan began in social media spaces, while Rasmussen conducts most of its polling via telephone land lines. The problem: it seems reasonable to surmise that people who still use land lines are more likely to be older and less likely to be active on social media. These may therefore be more likely to interpret the question the first way. This could explain the 56% who agreed. After all — none of us get to choose our parents, and we all come to this planet in bodies resembling those of our parents. How could it not be ok to be in the skin you’re in? We’re all how God made us. What’s more, for older folks who survived Jim Crow, agreeing with this statement may look like the safer option.
Likewise, the 26% who disagreed, and the 21% “Unsure”, were probably thinking of the second interpretation. I visited a few media sites with this slogan as their headline and there was nothing about cool things that folks of Euro descent can feel good about. No French cuisine, no Beethoven or Mozart, no Swiss technology, not even cute pictures of the kids and the pets. It was nothing but hateful memes and posts about Black folks. The denizens of those sites struck me as ignorant and hateful folks who can feel better about themselves only by putting other folks down.
Most people of any color — aside from consciously committed white racists — would disagree with a white supremacist slogan if they recognized it as such. But did they? Garbage in / garbage out.
And then there’s leading questions, that is, questions that suggest that the pollster is looking for a certain answer. For example, “Agree or disagree: the current president is a disgrace?” vs “On a scale of 1 to 5, where 1 is worst and 5 is best, how would you rate the current president?”
To be honest, I had to think a bit about a neutral form for this question. If you try with some other topic, you’ll probably find that it’s a bit harder to phrase a question in a neutral way than in a leading way. That’s a quirk of the human mind: we tend to shade our thoughts with our emotions. And there’s nothing wrong with that — as long as we take it into account. Emotions can be shorthand for the subconscious to distill the tons of information that it constantly takes in, and summarize this info for the conscious mind. That’s “going with your gut.” On the other hand, emotions can be distracting, as the subconscious mind is very energetic but not rational. Too many people dont realise this. They seem to think they’re always 100 percent rational, and mistake the rumblings of their subconscious minds for solid verifiable fact. (Fact = that which stands up to evidence.) So, when writing poll questions, pollsters should make the extra effort to make questions neutral, otherwise it’s garbage in / garbage out.
In general, samples should be representative of the population. Here, the population means, everyone in the group that we’re interested in (for example, all Black voters in the USA), while the sample is the people who actually get polled (for example, the 117 folks who answered the poll). Representative means that the trend in the sample is the same as the trend in the population. For example, if 56% of the sample agrees, then it should be reasonable to conclude that about 56% of the population feels the same way. How to get a representative sample? The best way is to use a large random sample, that is, everyone in the population is equally likely to be included in the sample. That’s not always practical, though, so there are a few other ways.
Convenience sampling, such as calling people on the phone, is popular because it’s easy. You’re just asking whoever is on hand. Convenience sampling is not representative because it does not include those who aren’t on hand. Other examples are mail-in surveys, standing on the corner with a clipboard, asking all your friends, etc.
Any history buffs out there? Remember when the Chicago Tribune announced — based on telephone polls — that Dewey beat Truman in 1948? Back then, landline phones were cutting edge technology. Now, they’re mostly popular with older and / or rural folks. There’s no reason to expect that people who use landlines are likely to trend the same way as the general population, and a few good reasons to expect otherwise.
Same with mail-in surveys, street corner surveys, or any other form of convenience sampling. There’s a built-in bias, because people who answer convenience-sampled polls tend to have stronger opinions than those who decline to answer. Otherwise they wouldn't bother. Answers skew toward the edges, while most people incline toward the middle.
Rasmussen uses convenience sampling — mostly telephone polling and a few online surveys. That’s why their polling base is biased conservative. Looking at the website, it seems to be a watering hole for right-wing folks who want confirmation of their biases. As a math teacher, I cringed when I read their methods and reports.
The next point, using inappropriate math, gets real wonky and I promised there would be no math in this essay. So I’ll just say that collapsing categories, eg, combining the “not sure’s” with the “agree’s” — or with the “disagree’s” — is a common feature in biased polls. That’s what Scott Adams did when he combined the 26% who disagreed with a racist slogan with the 21% who declined to state an opinion. There’s a lot more to it than that, but not for here. It’s pretty interesting, though, if you like nerdy stuff. So anyone interested can look into Statistics by Jim, an intuitive introduction to some of the math involved.
Next up, invalid reporting. Language should be neutral — the point of the report is to convey information, not to provoke emotions. Some poll reports look like something you’d see in a supermarket tabloid. Inflammatory language has no place in a poll report. If a poll report reads more like an ad for a certain viewpoint than like an informational piece, it’s probably just that — an ad. Also, poll report should include a few technical points such as the sample size, the sampling and data processing methods, and caveats.
Polls are very influential in today’s information-based society so it’s good to know how to recognize which ones are worthwhile, which need to be taken with a grain of salt, and which need to be tossed in the bin. I hope this essay will be a good start.
Comments? Have you ever seen a poll that made you feel a certain way, or just didn’t seem right? Have you ever tried to learn statistics and got so bogged down in the math that you missed the point of it all? Have you ever wondered about bias in the media, and how to detect this bias? This article is a work in progress so I’m interested in any and all comments. Thanks!
