Unobtrusive Methods for Social Science Research

1. Unobtrusive Methods forSocial Science Research A Neglected Methodological Approach in the Social Sciences Yoram M Kalman

2. Resources

3. Methodological Weaknesses of Interviews & Questionnaires • Responders provide answers that are in line: • with self-image • with researcher expectations • We only get answers from those who are accessible and willing to respond • Hawthorne effect, etc.

4. Justifications for Unobtrusive Measures • Complementary to direct elicitation techniques • Adaptable in situations where eliciting is difficult and/or dangerous • Approach that promotes creative methodologies

5. Categories of Unobtrusive Data • Found data • Captured data • Retrieved data • Running records • Personal and episodic records

6. Found Data – Erosion • Floor tiles around chick exhibit • Popularity of gym apparati by chalk consumed • Popularity of library books by smudges, finger marks etc. • Leaflets in different languages in tourist sites • Postcards in museum exhibits • Paper tissue and cough medicine in campus store correlated with class attendance records • Fun, but relatively rare in research

7. Found Data – Accretion • Before and after electric waste disposal units: change in fly population measured on car grilles • Graffiti: • Racial tensions in Hawaii • Youth relationships in high schools • Correctional facility for male juveniles • Garbology: • Survey data on alcohol consumption vs. number of empty alcohol containers appearing in garbage cans. The survey data underestimated consumption due to high refusal rate and exclusion of teenage drinkers • Others: Condom wrappers1976-1984, and beyond; Beer tabs vs. beer cans as evidence for recycling. • In US, 4th amendment issues

8. Found Data – Disadvantages • Conservative estimate • Socially dependent • Takes time to accumulate • Inferentially weak

9. Captured Data • Exterior physical signs: head and facial hair, tattooing, clothing and adornments • Expressive movement: demeanor, eye gaze, touching, verbal latency • Physical location: proxemics, spatial arrangement • In-situ conversation: pronunciation • Time related behavior: often neglected. Duration as proxy for importance, time of day influencing behavior

10. Retrieved Data – Running Records • Running records vs. episodic and private records: e.g.: mass media, reference works, records of proceedings • Lengthy periods of time • Ubiquitous • Lower cost • Less amenable to self report • Allow the exploration of trends and temporal patterns • Limitation: collected by others

11. Retrieved Data – Running Records • Mass media: news stories, advertisements, photographs, obituaries/weeding announcements • Reference works: directories, almanacs, yearbooks • Records of proceedings: discussions and decisions

12. Retrieved Data – Running Records • Actuarial records: births, deaths, marriages • Personal ads: • Jagger (1998) and gender conceptions in 1st and 2nd halves of 20th century • Race relations • HIV status • Marriage announcements and obituaries: • Class vs. church denomination • Gender vs. occupation • Gender vs. length of obituary • Others: job ads, book lists, phone books, etc.

13. Retrieved Data – Disadvantages • Often can’t be used “as is”. Should consult with those who produced them (reactivity!) • Quality issues, especially when more extensive • Selectivity: exclusion and inclusion criteria • Statistics might reflect more the organization collecting the data, than the sources of the data • Confidentiality

14. Retrieved Data – Personal and Episodic Records • Best example is personal documents: • Letters • Diaries/ daily journals • Autobiographies • CV’s • Wills • Photo albums

15. Key Principles of Unobtrusive Measurement • Construct and impose multiple indices that converge • Assume noise is rare • Investigators believe in amortization • Find foolishness functional • Ponder the variance rather than the mean • Investigators use expectancy as a control Webb & Weick, 1983 in Lee, 2000

16. Example: What email Latency Constitutes Silence? • Response latencies in Enron emails • Research on CRM in hotel industry • Chain letters • OOO messages • Published corporate policies • Blog postings discussing online responsiveness + • Questionnaires • EVT experiments

17. Key Challenges • Unconventional measures stand out and could receive less respect • Often, unchartered ground • Privacy: • When does observation intrude on privacy • Informed consent, etc. are reactive • Challenge is multiplied in online settings • Falling in love with methodology

18. Unobtrusive Measures Online • Plenty of sources • Searchable • Digitized, and ready for processing • Logs • Sharing with other researchers • Demographics of online users are no longer unrepresentative • Less limited geographically

19. Conclusion • Originality and innovation • Complementary, triangulation • Do the impossible: • Sensitive issues • Limited resources • ICT revolution a significant influence • Beware of: • Ethical challenges • Methodological pitfalls

20. Yoram Kalman www.kalmans.com Center for the research of the Information Society yoram@kalmans.com