Beyond academia: social media analysis in social and market research

1. Beyond academia: social media analysis in social and market research Bobby DuffyManaging Director, Ipsos MORI Social Research InstituteVisiting Senior Research Fellow, King’s College London

2. To manage expectations… What we’re doing is not that unusual/special… …but illustrative of issues faced …and how we’re trying to develop approaches: social media research in researchers’ control

3. Ipsos MORI… Public Affairs The Social Research & Corporate Reputation Specialists Media The Media, Content, & Technology Research Specialists Marketing The Innovation & Brand Research Specialists Advertising The Advertising Research Specialists Loyalty The Customer & Employee Research Specialists Political polling = 0.16% of our business

4. Has been quite a traditional industry… Neuro-science Passive data analysis SOCIAL MEDIA ANALYSIS FACE TO FACE FOCUS GROUPS QUALITATIVE DEPTH TELEPHONE WORKSHOPS ONLINE POSTAL ETHNOGRAPHY DELIBERATION

5. “Traditional” social media analysis allows us to capture basic metrics for evaluating campaigns or identifying trends 1. Define the search term Define which phrases are to be included and which should be excluded. Including definition of word combinations and distance between sets of words. 2. Define the base • This can be set by: • Country • Type of media/site • Timeframe • Sample of users (eg Twitter users that follow a specific account) • All mentions of the search term are then pulled into one single database. The database is then used to explore different metrics: • Number of mentions • Where (eg. blog, news, Twitter) • History of mentions over time • Topic (word cloud over time 3. Dashboard metrics • Mentions by individual site • Location (world map) • Automated sentiment • All metrics can be filtered. 4. Site specific metrics • Specific metrics are set up for Twitter (and news sites), exploring: • Top stories • Top hashtag • Top tweeters • Number of ‘impressions’ (including followers/re-tweets) • Most mentioned accounts 5. Manual coding Individual entries can be viewed at any time (based on any of the filters above). These can be coded to consider context, tone, manual sentiment.

6. Public attitudes to science evaluated engagement in science related topics over the course of a year Climate change (reaction to IPCC report) Horsemeat Measles Badger cull GM food Animal research Fracking Meteor Source: Public Attitudes to Science 2014, BIS/ Ipsos MORI http://www.ipsos-mori.com/Assets/Docs/Polls/pas-2014-social-listening-climate-change-and-animal-research.pdf

7. What do people write to MPs about? Q Which of the subjects on this list, if any, do you receive most letters about in your post bag, or receive most approaches about from individuals in clinics or other ways? All MPs | % Top mentions Source: Ipsos MORI MPs survey Base: All MPs (143), Conservative MPs (58), Labour MPs (66) asked, Summer 2014:

8. Or much more specific: how particular audiences discuss issues – eg child abuse 1. Top tweets related to: campaigns & awareness raising, toolkits and applied information, and sharing of information of general professional interest. 2. We searched on 61 pre-specified terms of interest across four different samples of interest Research into reporting of child abuse How to deal with instances of child sexual abuse

9. Or use network analysis to identify relationships and key influencers @Individual @CharityX These maps show the influence of a node or user in a network. In this case the network is the @CharityX account. So, in the case of @CharityX we can see that it’s connected to everyone else because everyone else is following @CharityX. @Educationgovuk is the second most influential user in this network… We identified the top influencers in the sample. They tended to be either in senior / high profile positions, have some training or consultancy role, be academics, practitioners or some combination of all of these. @Educationgovuk

10. A lot of event-based analysis…

11. The speech 23rd September

12. 36,124 tweets >800 p/m

13. 12,145 Unique voices

15. How did Ed do?

16. Devolution/ constitutional reform/votes for 16-17 year olds Tweets over time How did Ed do?

17. Who tweeted, and how? How did Ed do?

18. 41 Positive 1367 Negative 1598 Positive 5644 Negative Who tweeted, and how? How did Ed do?

19. The white haired lady in the audience just flashed Nigel a serious come to bed smile • 8 x more –ve than +ve tweets • 72% on personality, 22% on the political debate Farage proving what a weapons grade doucheknuckle he is on subject of gay marriage. What a tit. Like Ben Swain, Farage is looking more like a sweaty octopus trying to unhook a bra.

20. But lots of challenges and limitations

21. Some of the challenges… Calibration: understanding what good looks like Sampling: A number of proxies can be established for identifying types of user. But, these remain assumptive and subjective. Limited profiling: profiling data is limited. Small proportion of tweets have GPS tag, gender algorithms are around 80% accurate, but age profiling is considerably lower. Quality of the data: most current tools are based on creating large search terms to identify relevant entries on social media. Limited ability to quality check and refine the data. Analysis: no comprehensive single solution, requires using a number of different software packages. Resource intensive / expensive to understand sentiment or conduct text analysis. Guidance on ethical reporting is limited. Representativeness… 21

22. Twitter is a bit weird…

24. Trends In Social Networking Sites Visited % Visited in last 3 months Base: circa 1,000 GB adults aged 15+ per wave Source: Ipsos MORI

25. Profile of Twitter Users Base: circa GB adults (1,000) / All visiting / using Twitter in last 3 months (151): Q3 2014 Source: Ipsos MORI Twitter users are young: Nearly two thirds are agedunder 35. They are also more likely to be AB or C1 social grade and quite mobile: 88% of them own a Smartphone, 58% a Tablet. Twitter users are young: Nearly two thirds are agedunder 35. They are also more likely to be AB or C1 social grade and quite mobile: 88% of them own a Smartphone, 58% a Tablet.

26. Social Networking – accessing Twitter in past 3 months 2014 Base: circa 4,000 GB adults aged 15+: Q4 2013 / Q1 / Q2/ Q3 2014 Source: Ipsos MORI 40-100% 20-39% 0-19%

27. Has penetration started to flatten out? In which of these ways have you used the Internet in the last three months? % To visit social networking sites (such as Facebook, Twitter etc), or to look at or/and to take part in discussion forums or blogs Source: Ipsos MORI Observer

28. Challenge: Identifying relevant discussion can be difficult Map of how IPCC report from Sept 2013 was discussed online Armed police officer reinstated because sex on duty is 'like a tea break' - MIRROR http://bit.ly/1eHvK6R #ipcc

29. Challenge: Identifying ‘public opinion’ isn’t easy either… Media agencies, charities, environmental organisations and politicians all have voices on social media The Guardian Newspaper Blogs 14% British Medical Journal Forums 8% Twitter 35% Horse-meat European Commissioner for Climate Action Meteor Traditional news 43% Fracking Badger cull GM Measles Animal testing Source: Public Attitudes to Science 2014, BIS/ Ipsos MORI http://www.ipsos-mori.com/Assets/Docs/Polls/pas-2014-social-listening-climate-change-and-animal-research.pdf

30. Challenge: Automating sentiment 200 sample Automated sentiment: positive, neutral, negative Same 200 sample Manual coding: pro, anti, neutral climate change Only a 55% accuracy rate when trying to use machine learning to automate manual coding RT @[name]: Way to go, elected officials! #idiots House Votes To Deny Climate Science And Ties Hands On Climate Change http://tu2026

31. Developing our approach

32. Partnership

33. Developing our approach…

34. Developments over the next 12 months… • Drawing new insights: demographic profiling (age, gender, location); can we produce aggregated profiles of the demographic split of gathered social media datasets? • Credible research: • a framework for understanding the representativeness of social media attitudes through tests against conventional research • a confidence scoring system with which to judge the performance of social media analysis • a corrective weighting programme from which to generalise social media attitudes onto wider constituencies/ social groups • Ethical research: best practice ethics guide for social media research conducted by the social/market research community

35. Thank you bobby.duffy@ipsos.com @BobbyIpsosMORI