A History of Polling - The Straw Polls of 1824
I'd like to write a bit about the history of polling in United States Presidential Elections. I'm excerpting this from a research paper I wrote on Big Data in United States Presidential Elections at Brandeis University. With that in mind, you'll see some academic citations in these blog posts - which lead to some interesting sources you might want to pursue - especially if you have good library resources. This post itself is sourced primarily from a 1990 article from The Public Opinion Quarterly and goes a bit beyond what I wrote in the paper to give a bit more background for the layperson.
Big Data is one of those terms that gets thrown around a lot but whose definition is a bit nebulous. We'll focus on politics when working on a definition. Political predictions usually make use of polls. The trick with polls is making them either truly random or making certain you properly account for various demographics. With a truly random poll you can survey a surprisingly low number of people and get an accurate prediction. The problem is in making it a truly random poll. You need to essentially be able to reach a certain number of people in your survey area - for presidential elections, usually a state - but make certain the people who you hear back from is actually random. That unto itself is a problem because when people don't respond to your survey you can run into what is called a response bias - does the mere fact a person responds or doesn't respond indicate a likelihood of favoring a certain candidate. I could write a whole post on these problems - and likely will.
Big Data essentially bypasses this problem. In understanding political inclinations, for example, you would gather as much data as you can from as many places as you can. And you would not be simply looking for survey information - you'd be reading the text of Facebook and Twitter posts, you'd decipher whether the image in a meme posted is for or against a candidate, etc. When I say "you" I really mean some massive computing power, as you are gathering amounts of data that would make a spreadsheet application like Excel cry.
Before getting to Big Data in elections I wanted to look at what was used prior to Big Data (and is still used for that matter). In the United States that took us far back, all the way to the early 19th century. The election of 1936, with a famous incorrect result from The Literary Digest, is often considered to be the first presidential election poll in the United States. However, George Gallup himself considered the election of 1824 to be the first instance of presidential election polls (Smith, 1990, p. 21).
It is important to understand the significance of the election of 1824. This was the first contested election in the United States since the 1800 election, which, as all fans of Hamilton will know, saw Thomas Jefferson defeat both incumbent John Adams and the man who he wound up in an electoral college tie with, Aaron Burr. After this election, Jefferson's Democratic-Republican party began over two decades of electoral dominance. During these decades the Democratic-Republican nominee was selected by the congressional party caucus. This nominee, for all intents and purposes, ran unopposed. However, the Democratic-Republican party, its factionalism reaching a breaking point, lost its single nominee and had four candidates running against each other in the 1824 election - Andrew Jackson, John Quincy Adams, William Crawford. Also note that in 1800, only 5 of the 16 states choose their presidential electors by popular vote, making polling of limited value. In 1824, 18 of the 25 states chose their presidential electors by popular vote. There was therefore great interest in predicting the results of this election (Smith, 1990, pp. 22-24).
With the congressional caucusing system having collapsed, some states saw both preexisting organizations as well as new groups endorsing candidates and electors, but these were often akin to mass rallies. Straw polls became important in understanding the electorate. Militia musters often conducted straw polls as to their preference for presidential candidate. Militias offered a good way of sampling the electorate, as they often consisted of all white males between the ages of 18 and 45 (i.e. the majority of the voting population at the time). The results of these polls were reported by the press, with some newspapers presenting the results of straw polls organized by region and state, updated on a regular basis (Smith, 1990, pp. 24-27).
In the end, the election wound up being a bit of a mess. No one managed to get over 50% of the electoral college, though Andrew Jackson got the most electoral votes. Congress voted the presidency to John Quincy Adams and Andrew Jackson spent the next four years screaming at the top of his lungs that Adams won the presidency by a "corrupt bargain". And probably would have claimed that Adams had been born in Kenya had the idea come to him...
Smith, T. W. (1990, Spring). The First Straw?: A Study of the Origins of Election Polls. The Public Opinion Quarterly, pp. 21-36.