Conducting social research

1. Conducting social research Carolyn Black

2. Ipsos MORI – More than just opinion polls • Leading, world-wide research company • Work with the world's leading businesses, governments and institutions • We specialise in researching: • Advertising (brand and communications); • Loyalty (customer and employee relationship management); • MediaCT (media, content and technology research); • Marketing (understanding markets and building brand portfolios) and • Public Affairs (social & political issues) • Ipsos MORI Scotland focuses on Public Affairs research

3. Who are our clients? Public sector

4. My background • Higher Modern Studies • Degree in Psychology, University of Edinburgh (but employees have diverse academic backgrounds e.g. Politics, History, Journalism) • Started as a Trainee Research Executive at Ipsos MORI in March 2008 • There is no one path!

5. Behaviour in Scottish Schools Research 2012 – Scottish Government Evaluation of the ‘You First’ programme – Scottish Government Glasgow City Council Staff Survey – Glasgow City Council Some current projects

6. Research methods

7. Characteristics of Qualitative Research Characteristics • In-depth, open-ended interviewing with a discussion or topic guide • Relatively small samples Benefits • Respondents define issues • identify the full range of issues, • Probe views in-depth (including unconscious associations) • understand why people think/ behave as they do Limitations • Small sample sizes so not statistically reliable • Can never be representative

8. Characteristics of Quantitative Research Characteristics • Relatively large samples of people (aims to be representative) • Mainly ‘closed’ questioning Benefits • Measures the prevalence of attitudes/behaviours etc • Estimate for whole population • Identify clear priorities in a reliable way • Useful for tracking Limitations • Tells you ‘what’ and ‘how many’ but not always ‘why?’ • The researcher defines the range of answers given

9. Different types of data What is primary data vs. secondary data? Primary data is something that you interact with – if ‘you’ didn’t instigate it, the data would not exist Secondary data is something that exists despite ‘you’, for example desk research (market reports, sales data etc) Qualitative Research Quantitative Research Desk Research 9

10. Qualitative research

11. Methodologies available – Qualitative Research • Standard • Mini • Discussion groups • Deliberative workshops Conflict/ Krisis Paired-depths Qualitative research Online Bulletin boards Testing concepts/ads • In-depth interviews • Ethnography 11

12. Group discussions Exposes participants to other views (ideas/consensus) Multiplier effect Less time intensive than depths • Exposes participants to other views (may defer/change) • One participant can dominate • Easier for participant to ‘coast’ & not engage fully • Not good for busy/dispersed audiences • Social desirability bias 12

13. One to one - depths For busy professionals (B2B research) Widely dispersed May speak more freely Sensitive topics Detailed case studies • More time intensive than groups (expense per person) • Often re- scheduled/cancelled • No deliberation/ consensus (ideas?) • Telephone - harder to build rapport 13

14. Quantitative research

15. Methodologies available - quantitative In street On site In-home Postal Face-to-face Self-completion Online On site Telephone surveys Quantitative surveys 15

16. Face to face research Use of visual aids/show cards Sensitive questions Longer (more detailed) interview possible Personal rapport Good response rates Know you are speaking to the right person • Expensive • Takes time • Cluster sampling (sample bias) • Interviewer bias • Social desirability bias 16

17. Telephone research Speed Cost effective Easy to control quotas and sample Especially appropriate for B2B Good for widely dispersed audiences Client can observe Perceived anonymity • Difficult to ensure representative and might exclude mobile owners and/or those who are ex-directory • No visual aids (unless posted) / show cards • Difficult to explain complicated concepts or products • Higher refusal/drop out rate • Shorter interview length (less detail) 17

18. Postal research Can be much cheaper Large number of people can be invited to take part Best for high interest/ involvement studies No interviewer bias Can show basic visual material Can reach dispersed sample Anonymity Complete in own time – more considered answers? • Low response rates are common (representative?) • Self-selecting - more critical/those with strong opinions likely to take part • No check on who actually responds • Poor qualitative information • Less sophisticated questions • Hard to control use of prompt materials • Cannot use complex routing 18

19. On-line Cheap (No interviewer costs) Can be extended internationally Large samples Best for high interest/ involvement studies Particularly suitable for employee studies Complex routing and questions possible No interviewer bias Anonymity Complete in own time – more considered answers? Many of the same characteristics as postal • Sample issues: Internet penetration Self selection (panels) Out of date / incorrect email addresses • Response rates can be low 19

20. New methodologies – Social listening • What is Social Listening? • Capturing online ‘buzz’, conversations and opinions • We use a tool that trawls through the internet daily to monitor mentions of particular words or phrases • It sorts through blogs, social networking sites, forums, wikis, news sites, and video and image sites and then complies the data into a manageable format • How does it work? • The volume of mentions • What is being said • Where it is being said • Who is saying what on Twitter

21. Questionnaire design • Things you should do: • As a rule, keep questions short and simple • Use familiar words and concepts • Start the questionnaire with easy questions they can all answer that are relevant to what respondents have been told the survey is about • Ask sensitive and demographic questions at the end of the questionnaire

22. Questionnaire design B. Things to avoid: • Avoid ambiguity • Avoid leading questions • Avoid double barreled questions • Avoid double-negatives (usually with agree-disagree questions):

23. Question Validity • A valid question is one that measures what we think it does • A question is of little use if people measure it one way one day and another the next • E.g. Self rated health question: How healthy are you? Does this measure health or does it measure something else, such as optimism or happiness? • E.g. What is your national identity? Measure of subjective feelings or emotions towards a nation or an objective measure of citizenship or legal status?

24. Checking validity – cognitive testing • Cognitive testing allows an investigation into the way people understand, mentally process and respond to the question under scrutiny to explore whether the question, response options and accompanying instructions are interpreted in the way they are intended. • Is there a common understanding between researcher and respondent (which will result in valid data)? • How? Test in true-to-life conditions, small samples, in-depth discussions • 2011 Census – health, household composition, qualifications, national identity, ethnicity, language, Scots!

25. Sampling – ensuring the results are generalisable

26. Definition of sampling • Learning about the views of a large group of people by speaking to a smaller number of them • On the assumption that the characteristics of the few we have interviewed (the sample) match those of the population i.e. they are to REPRESENT the population of interest

27. Representative samples The usual goal in sampling is to produce a representative sample (i.e. a sample that is similar to the population on all characteristics, except that it includes fewer people because it is a sample rather than the complete population)  In other words, a representative sample is a "mirror image" of the population from which it was selected. If a sample is representative of a known population inferences can be made to the population as a whole i.e. we can generalise the results Unlikely to be perfectly representative – use statistical theory to help estimate how close the true population figure is likely to be to the figure obtained in the any particular sample

28. Types of samples Random Probability Simple Random Stratified Clustered Quota ‘Snowballing’ - Specific groups not easy to find

29. Random probability samples Every member of the population has an equal chance of inclusion in the sample Purest form of sampling (in theory) Most credible method Relatively easy process Usually used in postal, telephone and face to face surveys

30. Quota sampling Aims to control for variables that could affect the accuracy of the sample. For example: total of 12 per Output Area 5 men, 7 women 2 aged 16- 34, 4 aged 35-54, 6 aged 55+ generally reflecting profile of local population Fast / easy Mainly used for face to face surveys

31. Random or quota - which is best? Random sample theoretically most pure Everyone has an equal chance of selection BUT costly (to do well f2f) non-response bias • Quota sample • cheaper than random (f2f) • can achieve representative samples (on key factors controlling for) BUT • interviewers have freedom of selection • quotas not met for certain target groups Generally, Random Sampling is statistically purer, but Quota Sampling is much cheaper and quicker to administer

32. Thank you! Thank you for listening!If you need any further information, please get in touch:carolyn.black@ipsos.com