Many past research efforts have tried to exploit human-generated content posted on Social Media platforms to predict the result of an election1,2 or of various sociopolitical polls including the ones targeting voting intentions3,4. Most papers on the topic, however, received waves of criticism as their methods failed to generalise when applied to different case studies. For example, a paper5 showed that prior approaches1,3 did not predict the result of the US congressional elections in 2009.
One may also spot various other shortcomings of these approaches. Firstly, the modelling procedure is often biased towards specific sentiment analysis tools6,7. Therefore, the rich textual information is compressed to quite a few features expressing different sentiment or affective types. Apart from their ambiguity and overall moderate performance, those tools are also language-dependent to a significant extent and in most occasions machine-translating them creates problematic outputs. Furthermore, Social Media content filtering is performed using handcrafted lists of task-related terms (e.g., names of politicians or political parties) despite the obvious fact that the keywords of interest will change as new political players or events come into play. Most importantly, methods are mainly focusing on the textual content only without making any particular effort to model individual users or to jointly account for both words and user impact. Finally, predictions are one-dimensional meaning that one variable (a party or a specific opinion poll variable) is modelled each time. However, in a scenario where political entities are competing with each other, it would make sense to use a multi-task learning approach that will incororate all political players in one shared model.
As part of our latest research☉, we propose a method for text regression tasks capable of modelling both word frequencies and user impact. Thus, we are now in the position to filter (select) and weigh not only task-related words, but also task-related users. The general setting is supervised learning performed via regularised regression that, in turn, favours sparsity — from tens of thousands of words and users in a Social Media data set, we are selecting compact subsets that ideally will be related to the task at hand.
Consequently, instead of solving a linear regularised regression problem where only words are being modelled, we are also including users by learning an impact score for each user per political party, making our model bilinear. To exploit all the bits of information available and learn something more generic, we also formulate a bilinear multi-task learning optimisation task. The idea is simple, although the technical details may get a little bit complicated; if you want to find out more, you probably have to read the paper.
Figure 1. Predicted (left) and actual (right) voting intention percentages for the three major parties in the UK.
The method is not based on a sentiment analysis tool, thus, apart from being able to use all the rich textual information without compressing it to a few features, it automatically becomes language-independent. The performance results on the two case studies we investigated — voting intention prediction in the UK (Figure 1) and Austria (Figure 2) based on Twitter content — were quite encouraging. In both scenarios, the error in predicting voting intention percentages (obtained from professional pollsters) was on average smaller than 1.5% for all the political parties involved.
Figure 2. Predicted (left) and actual (right) voting intention percentages for the four major parties in Austria.
Furthermore, by using the model parameters (word and user weights), we were able to score single tweets and identify the ones with high positive and negative impact per party (see paper's Table 3); for example a journalist's tweet "PM in friendly chat with top EU mate, Sweden's Fredrik Reinfeldt, before family photo" had a positive score of +1.334 for the Conservatives (UK), whereas the tweet "Have Liberal Democrats broken electoral rules? Blog on Labour complaint to cabinet secretary" (posted again by a journalist) had a negative score of -0.991 for the same party.
Voting intention has been the main focus here, but the bilinear modelling idea is clearly generic and could be applied on text regression tasks for other domains such as health or socioeconomics.
☉ Lampos, V. et al. Proc. of ACL 51, 993‒1003 (2013) | Article
1. Tumasjan, A. et al. Proc. of ICWSM 4, 178‒185 (2010) | Article
2. Bermingham, A. and Smeaton, A. F. Proc. of SAAIP, 2‒10 (2011) | Article
3. O’Connor, B. et al. Proc. of ICWSM 4, 122‒129 (2010) | Article
4. Lampos, V. CoRR (2012) | Article
5. Gayo-Avello, D. et al. Proc. of ICWSM 5, 490‒493 (2011) | Article
6. Pennebaker, J. W. et al. (2007) | Article
7. Esuli, A. and Sebastiani, F. Proc. of LREC 5, 417‒422 (2006) | Article