Can social media be used to predict election results?

Many of the polls got it wrong when it came to predicting who would be the next president of the United States.

After Hillary Clinton had led throughout most of the campaign, she was also ahead in the BBC poll of polls on Tuesday with 48% of the votes to Donald Trump's 44%.

Number cruncher Nate Silver, of statistical analysis website FiveThirtyEight, wrote that morning that Mrs Clinton had a 71.4% chance of winning.

The results of course were quite different.

Whist pollsters' reputations may have taken another hit, newer methods of prediction based on data from social media appear to have had more success.

BrandsEye, a tool that looks at people's tweets, correctly predicted both the vote to leave the EU in June's referendum and a Trump victory in the US election, but how did it do this?

For both elections, Brandseye measured which side had more tweets in its favour on Twitter - predicting that the most popular of the two on the social media platform would win.

They compared the volume of pro-Trump and pro-Clinton tweets in key battleground states. As the chart above shows, between 1 October and 7 November Trump's popularity in key states on Twitter was greater than most of the traditional polls were suggesting.

It is easy to count the volume of tweets but how do you measure sentiment? The answer is using lots and lots of people.

The company took a sample of 200,000 tweets. These tweets were then checked by a crowd-sourced army logging in from their homes. Their task was simple; when presented with a tweet, decide if it was pro-Trump or pro-Clinton.

Using vast numbers of people is expensive but Jean Pierre Kloppers, the founder of BrandsEye, insists it is a better option than using an algorithm, saying: "Algorithms find it hard to see how people feel. Using humans to read the tweets makes it far more accurate."

For example, says Klopper, an algorithm would find it harder to understand the meaning of a tweet like "I'm supporting Clinton to see her wearing striped pyjamas", whereas a human can detect the irony or sarcasm.

But is social media really a good source of data for predicting how people are going to vote? After all, there are far fewer people on Twitter than in the US electorate so is it really representative? And there are many other social media sites, so only looking at that one platform may not be a true reflection of how people feel.

Tech philosopher Dr Tom Chatfield believes that it can offer valuable insights that are being missed by pollsters.

"Traditional polls are cumbersome and not that data-rich. It is interesting that for all the sophistication of traditional polling, for me the story of the night was seeing political experts have their expectations destroyed across the board.

"It appears polls didn't reflect the passionate mobilisation of emotion of some people. People are less shy on social media than they are on the phone to pollsters."

Artificial Intelligence

It was not only Brandseye that used social media posts to predict the election result.

An Artificial Intelligence system called MogIA called it for Trump as well. According to US TV channel CNBC, MogIA was developed by innovator Sanjiv Rai and has correctly predicted all US elections since 2004.

Mogia measures engagement data from sites like Google, Facebook and Twitter. For example how many people read a Trump tweet or watched a Trump Facebook Live?

However, Mr Rai told CNBC that while it can measure the volume of engagements with a particular candidate, the sentiment of those engagements is not easy for an AI system to analyse - the point made by BrandsEye about using algorithms.

False information

There is also a danger of using social media when a lot of the information on it is unverified.

And as the BBC reported, some of the campaigning for both Trump and Clinton was helped on social media by thousands of automated Twitter accounts.

According to research led by Prof Philip Howard, from the University of Oxford, these so-called bots accounted for about 1.8 million pro-Trump tweets and 613,000 pro-Clinton posts, making it hard to infer public opinion by this method.

Although real humans can try to spot bots, the sheer volume of them suggests it is unlikely they can all be discovered.

And it is worth considering that if measuring social media sentiment becomes a more established way of predicting elections, there will be a great incentive for each side to create bots to give the impression they will win.

Craig Silverman, from social news and entertainment company Buzzfeed, reports that there are groups of teenage boys in the Balkans making money through creating and sharing fake pro-Trump news stories. For each click they get on their story they get money from advertisers on their websites.

Chatfield remarks: "It is concerning for politics that social media platforms are more interested in whether people share than what they share. The world of social media is at right angles to truth. So true or untrue doesn't matter but emotional resonance does matter."

And let's not forget that Michael Moore,The Simpsons and others also got it right.

by Patrick Evans, UGC & Social News team