Nate Silver is a clever man. Using “quants” he correctly predicted the outcome of the US presidential race in every single state bar none. Interest in “business analytics” that uses algorithms that detect trends and key relationships in vast amounts of apparently unstructured data is on the rise. According to a report in Bloomberg, UBS bank which is shedding 10,000 staff, will shortly be replacing the head of its credit-default swaps index trading unit with an algorithm. This could well lead to a public demand for political leaders to be similarly replaced. Folk could vote for the algorithm of their choice.
In practice that is what most of us do already. Following the victory of President Obama, stock markets plummeted for a day or two as wrong bets on stock options came unstuck. According to standard economic myth, financial markets are near perfect, so why did professional punters get it so wrong? Because they voted for the wrong algorithm. They believed what they wanted to believe and tweaked the analysis of the data accordingly. Man over-rides machine? Well maybe Man subconsciously (or not) programs the machine in a way that produces the desired results. Mitt Romney’s political supporters read the tea leaves and gave their opinions. They thought Mitt Romney was onto a winner when evidently he was not. Ironic really that Mr. Romney was a former hedge fund manager.
In 1980 Mrs. Margaret Thatcher, Britain’s indefatigable Prime Minister was so convinced that the more pliable Joshua Nkomo, leader of the ZAPU in Zimbabwe, would win the election she was happy to see it through. Practically everyone in Zimbabwe at the time knew that Robert Mugabe of ZANU-PF would win by a landslide. Simple demographics were all that was needed, not an algorithm in sight, but wishful thinking in Britain trumped what would have been an algorithmic truth.
President Obama’s team was more astute. Behind locked doors they ran a Big Data analytics operation that was running 66,000 simulations a night using four data streams from each state, often with sample sizes of 1% of the population (which is a huge number) (https://swampland.time.com/2012/11/07/inside-the-secret-world-of-quants-and-data-crunchers-who-helped-obama-win/2/). Every step of the campaign from raising funds through personalized emails and recommendations by social media, to getting out the voters in the swing states was carefully calibrated and acted upon. The Democrats knew they had this election in the bag, as did Nate Silver, even as the fatal binge of betting on stock options by, among others, hedge fund managers, was in full flood.
Of course, in theory Big Data analytics has a very high probability of being correct, but it is monitoring and analyzing events that are beyond its own control. If President Obama had decided to pick his nose during a TV debate or send roses to Mr. Mahmoud Ahmadinejad, President of Iran, he might have seen the election slip from his grasp; which seemed almost the case after the first TV debate. So in theory Big Data can do a lot, in practice it can contribute to an outcome but not determine it absolutely. As Albert Einstein is believed to have said: “In theory, theory and practice are the same. In practice, they are not.”
