#weakandwobbly: how Twitter revealed the mood that led to a hung parliament

The political events of recent few weeks in the UK have thrown up all sorts of interesting questions. Was Theresa May doomed as soon as she called an election? What drove the young people who voted Labour? How can we make better election predictions?

The latest research by the University of Central Lancashire uses machine-learning techniques to shed new light on a complicated picture that defied pollsters. We trained a computer to recognise similar messages on Twitter, thus allowing rapid examination of huge sets of data. Researchers manually categorised a sample of 5,000 tweets sent as @messages to MPs as hostile, disagreement, neutral or positive, then used machine-learning software built by Dr Martin Bateman to categorise more than 700,000 @messages sent from December 22 to the election.

Examining the content of this huge dataset of messages directed at party leaders allows us to see the depth of emotion and range of concerns behind the voters’ decisions that ultimately led to a hung parliament.

After categorising the tweets, hostile tweets were then further analysed to identify the most frequent terms used over time, excluding common words.

The tide turns

The figures show the prime minister’s fortunes turned from the moment she called an election. In the months before April 18, about 9% of all messages aimed at May were hostile, only topping 10% when she signed Article 50. Within two weeks of announcing the election, the percentage of hostile messages had risen to 12%, continuing to climb slowly through the teens until shooting up to 25% on June 8.

The launch of her campaign slogan at the start of May was much mocked online, inspiring hashtags such as #strongandstablemyarse and #weakandwobbly. In hostile tweets aimed at the prime minister, the most common word mentioned between December 2016 and April 2017 was “Brexit”, until the launch of the Conservative’s slogan, when “strong and stable” immediately topped the list.

The manifesto launch heralded a flood of articles about the funding for the Tories’ social care policy in the mainstream media, but Twitter seemed unconcerned, perhaps reflecting a demographic reflecting that 74% of users are under 44. Instead, “fox” and “hunting” shot to the top of the most used keywords in hostile tweets due to a provision to allow a free vote on repealing the ban on hunting which was barely covered by some newspapers.

But although the Conservative plans to force older people to pay more for social care didn’t seem to concern the Twitterverse, the U-turn less than a week later certainly did. The words “manifesto”, “lies”, “lying” and “liar” all appeared high up, along with the first mention of “care” and “old”. By May 31, Corbyn’s last-minute decision to appear in a television debate hit May hard. The top three words in hostile tweets directed at the PM were “debate”, “Corbyn” and “weak”.

Corbyn reborn

For Corbyn, hostility on Twitter showed a very different picture. The early months of the year had seen a slightly higher proportion of hostility in @messages directed at him compared to May, bouncing up to 14% when Article 50 was signed, reflecting Labour’s refusal to oppose the vote.

But although hostility levels varied towards him during the campaign, they were generally below those for May. Just as it had for the prime minister, the announcement of the election marked a turning point for Corbyn. The word “resign”, common throughout the early months of the year in messages directed at the Labour leader, all but disappeared as soon as the election was called. “Brexit” and “EU”, which had appeared frequently for many weeks surrounding the signing of Article 50 also practically vanished.

However, Diane Abbott, shadow home secretary, regularly appeared in messages to the Labour leader for two weeks after an embarrassing interview in which she made a series of mistakes about funding for police. The terms “IRA” and “terrorist” also became common during the month preceding the election, suggesting messages about Corbyn’s sympathies in the 1980s made at least some impression.

For both leaders, hostile messages became more virulent as the election neared, and profanities began to appear much more frequently in the final three weeks.

These are early results for this long-term research project, – and further analysis of the data is needed. The plan is to introduce further keyword filters of profanities, misogynistic and racist abuse to get a clearer picture of strength of feeling.

June 8 2017 showed that once again, pollsters failed to predict the election accurately. PA, CC BY-SA

The demographics of Twitter mean older people are less well represented, along with less educated and people based in the countryside, but this type of research supplies an interesting alternative to traditional polling data, showing strength of feeling and issues of concern rather than voting intention. Corbyn consistently received far more @messages than May, illuminating the way his campaign dominated social media with positive messages aimed at younger voters.

The Conservatives have generally wooed older voters with sweeties such as the pension triple lock and winter fuel payments, because they are more likely to turn out to vote. As a party they have been able to ignore younger people’s concerns, such as tuition fees, but this election showed an increased turn-out among younger people that clearly benefited Labour.

Pollsters have been heavily criticised for once again miscalling the election result – though YouGov did noticeably better – but polls have been inaccurate for the past two general elections and the referendum.

This type of research using social media listening and machine-learning tools allows for a much subtler, more in-depth picture than the binary questions of pollsters. It not only gives us an indication of how young people may vote, but can tell us what they care about and, crucially, how much. It was this depth of feeling that drove an election turnout that flummoxed pollsters once again.

Amy Binns does not work for, consult, own shares in or receive funding from any company or organisation that would benefit from this article, and has disclosed no relevant affiliations beyond the academic appointment above.