Suddenly, Prediction Markets are in vogue. No longer a fringe phenomenon, they are headlining articles in all the major financial media outlets – The NY Times, the Wall Street Journal, The Financial Times… They are beginning to penetrate the middlebrow journalism of cable news. They are drawing attention from politicians and podcasters and hedge funds. What is happening here?
The adaptation of financial theory and practice to other fields is becoming more common. In his best seller Moneyball, Michael Lewis described how ideas like “value investing” (searching for underpriced assets) and “portfolio construction” (combining the assets in a structured way) have been applied in baseball to help managers select the right players and build a balanced team. Similar thinking spread to other sports and gave rise to the growing field of “sports analytics” (now being taught at many universities).
Prediction Markets are another finance-theory-inspired innovation, bringing the power of a market mechanism to bear on questions concerning the likelihood of future events of all sorts – but especially in politics. While simple betting on political outcomes has a long history, the introduction of modern market technologies can potentially improve the accuracy and offer a new way of looking at the American political landscape, murky and confused as it is this election cycle.
Which is apparently much needed – as traditional polling methods seem to be breaking down.
The Polling Crisis
The headline problem is that the polls, and the analytics based on the polls, failed in several recent elections, often spectacularly. The New York Times predicted on the very morning of the 2016 Presidential election that Hillary Clinton had an 85% chance of winning “based on the latest state and national polls.” It was another embarrassing “Dewey Defeats Truman” moment. Burned by such errors, and uncertain about the validity of the data, almost every new poll today comes with a caveat about the ways in which it could be just wrong.
The under-the-hood problems with the polling process are even more disturbing. Response rates are dismal. For example, Gallup reported in 2017 that it just 7% of its polling attempts succeeded (down from 28% from 20 years earlier). The Pew Research Center saw its response rate drop to just from 36% to 6% in the same time frame.
There is a general sense of disruption and even crisis in the polling industry, or as one source politely put it, “polling has entered a period of unprecedented diversity in methods.” Live telephone surveys – which used to be the mainstay of the industry – have been phased out in favor of other sample-construction methods, including paid panels and online “opt-in” sampling (now the most popular solution by far), which may entail new forms of statistical bias. Pew has shifted from phone surveys to address-based panels and yet still reports that “the cumulative response rate accounting for nonresponse to the recruitment surveys and attrition is 3%.” Another leading pollster paints an even bleaker picture of the process deterioration.
- “Whereas once I could extract one complete interview from five voters, it can now take calls to as many as 100 voters to complete a single interview, even more in some segments of the electorate.”
Such low response rates mean that sound statistical sampling cannot be sustained.
- “A few missing observations are a minor nuisance, but a large amount of missing data is a major threat to a study’s integrity.”
- “There is no satisfactory statistical solution to deal with missing data that may not be random.”
If the data intake (the actual sampling and polling process) is so flawed, it is no wonder that the higher-level forecasts are unreliable.
In summary, the same expert concluded that
- “There’s a dirty little secret that we pollsters need to own up to: People don’t talk to us anymore, and it’s making polling less reliable.” – The Washington Post (November 19, 2020)
Are Prediction Markets The Answer?
Are Prediction Markets a better way to access public sentiment and forecast event outcomes? What are these new platforms exactly, and are they truly superior to the existing poll and survey methods?
The underlying concept of these platforms is the “Wisdom of Crowds” (WOC) – the idea that collective opinion is structurally superior to individual judgment. As the modern popularizer James Surowicki puts it: “Groups are remarkably intelligent, and are often smarter than the smartest people in them.” (I have written about the origin story of this idea in a previous column, here.)
Wow.
This simple-sounding proposition actually goes quite deep, embodying a number of assumptions that need to be understood and examined critically. But if it is true, and if we can find a way to access this group intelligence, it could be a way of improving many kinds of forecasts.
One of the design principles for WOC to work is size, or volume. The larger the group, the more accurate and reliable the outcome is expected to be. But very large groups will by nature be dispersed rather than gathered physically in one place. And the collection of group information is not once and done. It should be ongoing, adjusting to new information that may affect the probability of the event in question. (This is something that polls have a hard time doing, as each polls a frozen snapshot in time.) Accessing a large and dispersed group, with continuous updating, requires elaborate technology. It requires a market. This is the conceptual link between WOC and the finance-theoretic concept known as the Efficient Market Hypothesis (EMH).
Essentially, classical finance theory views a financial market as a huge information processing engine. It sucks in vast quantities of fact and opinion from millions of investors, in the form of buy and sell orders for a given asset – stock, bond, option – and boils it all down to a single standardized output: the price. This single metric is said to incorporate all known information about that asset that could affect its value. The EMH further claims that in a well-designed and properly functioning market this price becomes the accurate valuation measure for that asset. Price=Value. And price is the product of group intelligence. In a market, no single investor sets the price. Which means that for EMH believers it is theoretically superior to any individual analysis.
WOC and EMH are parallel ideas, and since EMH comes along with decades of experience and a ton of technology and design knowledge about how to put together a well-functioning market, the next step is obvious.
How Do Prediction Markets Function?
Prediction markets should work more or less like financial markets. Participants (investors? traders? bettors? – what shall we call them?) buy or sell financial instruments or contracts which Polymarket – a leading Prediction Market – refers to as “shares.” Others have called them “idea futures” or “event derivatives.” All of these terms are metaphorical. These contracts are really more like wagers that pay off based on a particular outcome. However, unlike most simple wagers, they are tradable, and are sometimes referred to as “options” (though that is also inaccurate, strictly speaking).
Tradability is what transforms the casino into a market. You can buy a Prediction Market “share” to hold until the event in question (e.g., a presidential election) takes place and the option pays off – or you can buy it at today’s price and hope to resell it at higher price to another buyer tomorrow. As in a financial market, a participant can be an “investor” or a “speculator.” Tradability opens up other possibilities for hedging, shorting, and price arbitrage (although not all of these may yet be operational features of current Prediction Markets). Presumably other features of financial markets may soon follow, such as margin lending and leverage, swaps, and true derivatives such as futures and options properly speaking. Perhaps a tradable volatility index of its own. Indexes? ETFs? It may eventually be possible to trade across different platforms.
The market prices of these “shares” fluctuate with news and sentiment. Most “shares” are technically binary options, which pay off either $1.00 or $0.00 depending on whether the outcome in questions occurs or not. As the price fluctuates over that range, it is interpreted as equivalent to the percentage probability (as collectively assessed by the crowd) that the event will yield a particular outcome.
The emergence of prediction markets into the political discourse occurred in the 1988 Presidential election, where one of the earliest of these markets (the Iowa Electronic Market, at the University of Iowa) startled many observers by outperforming the polls, and continuing to do so for the next several Presidential races.
This prompted interest. People began to consider whether Prediction Markets could be superior forecasting tools. Before long, the idea migrated out of academia and became a business model. In the last several decades a number of companies have ventured into this space with various offerings.
The development of these markets was somewhat constrained by its association with online gambling models, which tended to become embroiled in scandals from time to time. Many of the early participants were simply outgrowths of conventional betting parlors – publishing “lines” on elections along with those on sporting events. They lacked the features of tradability that characterize a true financial market.
The other important constraint has come from the regulators of the financial sector. As Prediction Markets began to resemble true financial markets more closely, as the dollar volumes and trader numbers grew, the regulators were drawn in to consider whether and how to apply traditional financial standards for regulating and in some cases prohibiting certain behaviors and operations. In the United States, the Commodity Futures Trading Commission (CFTC) exercised a heavy hand, and eventually took the position that Prediction Markets should essentially be banned from offering contracts related to elections. Among other arguments, the CFTC claimed that “betting” on elections could distort the electoral process itself. This ban made it difficult at best, and likely illegal, for U.S. citizens to participate in Prediction Markets. Most of these platforms were located outside the U.S., to escape these constraints.
Suddenly, however, this has all changed. On October 2, the District Of Columbia Circuit Appeals Court ruled against the CFTC (in a case brought by Kalshi, one of the Prediction Market ventures that has lobbied for deregulation of the sector); the Court decided to allow Kalshi to offer election contracts freely in the U.S.
The floodgates swung open. In the three weeks since the ruling, volumes have surged on Kalshi, Polymarket, PredictIt and other Prediction Markets. Kalshi says its business has doubled every day since the ruling. Polymarket – which is the largest platform, and uses cryptocurrency for its trades – has a seen a total volume of $3.2 Billion in value (this year?), with almost 250,000 traders, with a surge following the October 2 decision. (This according to According to LayerHub, a data aggregator for cryptocurrency businesses.)
In addition to the volume surge, the “directionality” of the market changed dramatically. After months of neck-and-neck trending between the Presidential candidates, the spread blew out. Here is the chart of the price action on Polymarket.
Other platforms show the same general pattern. It seems clear that a wave of new participants has jumped into these markets, unleashed by the court ruling and energized by the momentum shifts in the Presidential race. They are betting big. The world is paying attention.
The polls right now are at odds with these new Prediction Markets – and suddenly everyone wants to know which signal is right.
The essential problem with polling is summarized by star pollster Nate Silver, writing in today’s New York Times:“nonresponse bias.”
- “Nonresponse bias can be a hard problem to solve. Response rates to even the best telephone polls are in the single digits — in some sense, the people who choose to respond to polls are unusual. Trump supporters often have lower civic engagement and social trust, so they can be less inclined to complete a survey from a news organization. Pollsters are attempting to correct for this problem with increasingly aggressive data-massaging techniques, like weighting by educational attainment (college-educated voters are more likely to respond to surveys) or even by how people say they voted in the past. There’s no guarantee any of this will work.”
Polling is in trouble, and in transition. If Prediction market pan out, they could even render traditional polling obsolete for certain problems.
But what about the Prediction Markets? These systems are new and more or less unregulated. There have a very limited track record. What sorts of biases might they contain? What failure modes might emerge, especially as they ramp up rapidly to trading volumes not seen before? What loopholes in their architectures may exist, to be exploited by traders whose skills have been honed on Wall Street?
Drawing on the experience of financial markets, there are at least two points that should raise concern about the accuracy of these markets.
First, even the largest and oldest financial markets have experienced stresses and glitches on a disturbingly regular basis. The financial model is not flawless either.
Second, the EMH is not an accurate picture of the stock market. Hardly anyone believes in strict market efficiency anymore. It is quite clear that in all too many cases, Price≠Value . There are many mispricings in the market, some fleeting, some strangely persistent. There are bubbles and crashes and meme stocks, and front-running (legal and otherwise), and high-frequency algorithmic trading designed to exploit volatility rather than the fundamentals, hedge funds that trade numbers rather than companies, and so on.
Part 2 of this column will take up several of these questions:
- Do these Markets truly offer better accuracy than the polls?
- What are the known strengths and weaknesses of these Prediction Markets?
- What is the actual record of Prediction Markets recently?
- What factors might skew the Prediction Markets away from the “fundamentals”?
The looming election will be a major test for this new technology. Prediction markets will have to show they can weather the chaos of American political brawl.
Further reading: