A History of Polling - 1936 - Landon Defeats Roosevelt?

While perhaps the most famous blunder in presidential election polling is the consensus that Dewey would defeat Truman in 1948 (which we will get to), a great one for our purposes is the Literary Digest's prediction that Republican would defect incumbent Franklin Roosevelt in 1936.

Throughout the 19th century, newspapers conducted straw polls from their readers to predict presidential elections. In the early 20th century, magazines, often with a wider reach, began polling as well. The Literary Digest, beginning in 1916, became a market leader in polling, predicting the victory of Woodrow Wilson. Unlike newspapers, which were limited to their reading audiences, the Literary Digest sent out postal ballots to representative samples of the population. They acquired lists for their samples from sources such as car registration records and telephone directories. By 1936 they were sending out approximately 10 million ballots and receiving over 2 million responses (Moon, 1999, pp. 7-8).

In 1936 The Literary Digest predicted that Alf Landon would defeat incumbent Franklin Roosevelt, winning the popular vote 57% to 43%. However, the actual result was a Roosevelt victory, with a popular vote margin of 62% to 37% - a substantial error (The Oxford Math Center).

There were two main problems with The Literary Digest's polling methodology. In the 1936 election they obtained their address lists from phone numbers, drivers’ registrations, and country club memberships. However, in the height of the Great Depression, all three of these were more available to wealthier voters. Adding to this problem, the economy and Roosevelt’s New Deal were the central campaign issues, issues attractive to lower income voters underrepresented in the sample set. This made the sample extremely biased towards wealthier voters – as can be seen by how badly the poll failed to predict the results (The Oxford Math Center).

The second problem with their polling methodology was it had what is called a voluntary response bias. In Chicago, instead of using samples eschewed towards the wealthy, the Literary Digest sent a poll to every third registered voter. This removed the bias towards the wealthy. However, they still predicted a Landon victory in Chicago, an incorrect prediction, as Roosevelt won an overwhelming victory there. This indicates that Roosevelt supporters were less likely to respond to the survey than Landon supporters - which is just what the voluntary bias warns of (The Oxford Math Center).

In that same election, Doctor George Gallup successfully predicted a Roosevelt victory. Gallup made use of much smaller sample sizes, though with a much greater focus on getting a representative sample. This included taking into account factors such as location, race, education, profession, and party affiliation. He also had to consider the “intensity” of voters and the likelihood that those polled would actually vote. He predicted The Literary Digest was very wrong in its predictions while simultaneously confirming that using its methods would indeed result in the prediction it came up with (Time Magazine, 1948, pp. 21-23).

What this shows us is a large sample isn't necessarily a good thing. Note that this is a relative definition of large - 10 million ballots sent out by the The Literary Digest and they received 2 million responses - there were 80 million people of voting age in the United States.

The Literary Digest itself didn't last as a concern much longer, ceasing publication in 1938.

Works Cited

Moon, N. (1999). Opinion polls - History, theory and practice. Manchester: Manchester University Press.

The Oxford Math Center. (n.d.). Famous Statistical Blunders in History. Retrieved from The Oxford Math Center: http://www.oxfordmathcenter.com/drupal7/node/251

Time Magazine. (1948, May 3). The Black & White Beans. Time Magazine, pp. 21-23.