The idea that the Twitter feed is an open window on the future is convincing. But is it true?
The idea that social media such as Twitter sites can predict the future has a controversial history. In recent years, various groups have claimed to be able to predict all of the outcome of the elections for the recipes of ticketing for new movies.
It is fair to say that these claims have generated their fair share of criticism. It is therefore interesting to see a new claim to come to light.
Today, Nathan Kallus at the Massachusetts Institute of Technology in Cambridge said it has developed a way to predict crowd behavior using statements made on Twitter. In particular, it analyzed the tweets associated with the 2013 coup in Egypt and wrote that civil unrest linked to this event was clearly predictable days in advance.
It is not difficult to imagine how the future behaviour of the crowds may be incorporated into the Twitter stream. People often report their intention to respond in advance and even to coordinate their behavior by using social media. This social media activity is therefore a leading indicator of future behaviour of crowd.
That makes it seem clear that predict the future behaviour of crowds, just crochet this indicator on noise.
Kallus said this is possible by mining tweets for any mention of events to come and then analyze trends associated with them. "The gathering of crowds in a single action can often be seen through trends that appear in these data a long time in advance," he said.
It turns out that exactly this type of analysis is available from a company called recorded future based in Cambridge, which scans 300 000 different sources web in seven different languages, from anywhere in the world. It then extracts mentions of upcoming events for later analysis.
It is these data which Kallus analyzed to predict major events. "We find that the mass of information available to the public online has the power to reveal the future actions of the crowd ', he said.
First of all, Kallus defines an important event like the one that receives a much larger than usual media coverage.
It then analyzes the mainstream coverage to see where really important protests and looks for activity occur in the Twitter stream that precedes the events. If they are predictive indicators, then it is possible to find similar types of activity and assumes that it is too predictive.
Kallus tested this idea by studying the tweets associated with the 2013 coup in Egypt, that was focused on the anniversary of the rule of president Morsi, triggering major protests during which he rejected from power by the army Egyptian.
Kallus said that proof of the protests was clearly visible in the Twitter account in advance, given advanced protests that occurred before the anniversary. In addition, the content of social media predicted that protests would go for weeks beyond the anniversary.
Conclusion of Kallus tweets can accurately predict significant protests in advance is interesting. There is no doubt that the evidence is to be found in social media with the benefit of hindsight. There's no shortage of people who do this kind of predictions on historical events using historical data.
The big question is whether it is possible to select in advance the evidence. In other words, is possible to make predictions before the events actually occur?
This is not so clear, but there are good reasons to be cautious. Firstly, it is possible to correlate the Twitter activity to actual protests, it is also necessary the possibility of false positives. There may be significant trends of Twitter which do not lead to major demonstrations in the streets. Kallus does not adequately address the question of how say these things apart.
There is the question of whether tweets are trustworthy. It is not difficult to imagine that when it comes to issues of great national consequence, propaganda, rumor and irony can play an important role. Then how to deal with that?
There is also the issue of demographics and know if the tweets truly represent the intentions and activity of the population as a whole. Those who tweet is extremely likely to be young, but there's another silent majority who plays the extremely important role. If can the firehose Twitter really represents the intentions of this part of the population too?
The final of the challenge is in the nature of the prediction. If the Twitter account is predictive, then there is evidence that it can be used to make actual predictions on future, not only historical predictions about the past.
We looked at some of these problems with the predictive power of the social media before and the challenge is clear: If there is a demand to be able to predict the future, this request must be accompanied by convincing proof of a real prediction on an event until it happens.
In the meantime, it would certainly make sense to be circumspect on the predictive powers of Twitter and other forms of social media.
Ref: arxiv.org/abs/1402.2308: predict the behaviour of the crowd with Big public data