Survey Says...

I'm sure most people have seen and been appalled by the latest crop of stupid American surveys - the world is flat,  evolution is a conspiracy by Godless scientists, President Obama practics voodoo and eats babies, etc... Maybe I'm too optimistic about the general populace, but sometimes I wonder about the sampling methods that produce these alarming statistics. Some of the organizations behind these surveys are very open about their methodologies, others aren't, but either way it seems possible that there's some sampling bias going on.

I mean, sampling errors are behind some pretty infamous polling mistakes. The 1936 Literary Digest Presidential Poll incorrectly predicted that Alf Landon would defeat FDR, which was shocking not only because FDR went on to carry all but two states, but also because the Literary Digest had correctly predicted the outcome of every presidential race since 1920. The problem in 1936 was that they drew their "random" sample from three seemingly-random-but-not-really sources: their own subscription list, lists of registered automobile owners, and the telephone registry - all of which contained a disproportionate amount of affluent people in the 1930s. Affluent people were more likely to vote for the Republican Landon so he was predicted to win, but when the rest of America (which didn't subscribe to Literary Digest or own automobiles or phones) actually voted, they overwhelmingly chose the Democratic Roosevelt. A similar telephone-related sampling error was behind the infamous 1948 "Dewey Defeats Truman" debacle.

I also remember learning about a study carried out more recently to ascertain what percentage of American wives cheat on their husbands. The researchers sent surveys out to a large, random sample of married American women, collected the results, and found that some shockingly high percentage of them were cheating. It wasn't until later that they realized their mistake: the women who were most likely to return their surveys were those sitting home all day, alone and bored, and therefore also most likely to cheat. The women who had happy, fulfilling lives and marriages were less likely to take the time to fill out some random survey about their love life that showed up in their mailbox. I know I was taught this example in a stats class in college, but of course now I can't find the syllabus, and googling "cheating American wives survey" is getting me nowhere, so I apologize for the lack of sources. Just take my word that I didn't completely make it up.

Anyway, I guess my point is that I like to hope all these stupid American surveys are actually contaminated by sampling bias and not an accurate reflection of the idiocy levels in our country. It totally seems possible that the type of person who has nothing better to do than answer random survey phone calls in the middle of the day may not be representative of the general population, which is too busy with their jobs/schooling/functional brain cells to sit around telling strangers on the phone that they believe the President was born in Kenya. Because if those numbers are an accurate indication of the state of the union today, then we're in trouble...