Qualitative Inquiry in Social and Cultural Contexts The Critical Incident Technique

1. CoLIS, Copenhagen, Denmark August 22, 2013 Lynn SilipigniConnaway, Ph.D. Senior Research ScientistOCLC @LynnConnaway connawal@oclc.org Qualitative Inquiry in Social and Cultural ContextsThe Critical Incident Technique

2. Critical Incident Technique (CIT) • Flanagan (1954) • Qualitative technique • Focuses on most memorable event/experience • Allows categories or themes to emerge rather than be imposed (Flanagan, 1954)

3. Origins of CIT • Sir Francis Galton • Aviation Psychology Program of US Army Air Forces in WWII • Analysis of failure in learning to fly • Used in proceedings of elimination boards • Research for selecting pilots • Reasons for failures of bombing missions (Flanagan, 1954, p. 2)

4. Origins of CIT • Combat leadership studies • Gather specific incidents of behavior • Helpful or inadequate in accomplishing mission • “Describe the officer’s action. What did he do?” • Resulted in “critical requirements” of combat leadership (Flanagan, 1954, 2)

5. CIT in LIS • Radford (2006) Tice (2001) • Evaluate CLASP (Connecting Libraries and Schools Project) • YA attitudes toward public libraries • Fisher & Oulton (1999) • Staff development needs • Library decision-making • Tool for librarian entering management positions • Hamer (2003) • Information seeking of LGBT youth coming out • Wilkins & Leckie (1997) • Investigate information needs & information-seeking behavior of university staff • Used in addition to questionnaire

6. Critical Incident Technique Examples: Our Research

7. Seeking Synchronicity: Evaluating Virtual ReferenceServices from User, Non-User & Librarian Perspectives • Studied habits & needs of virtual reference services (VRS) users and non-users to identify characteristics for informing library system and service development • Four phases: • Focus group interviews • Analysis of 850 QuestionPoint live chat transcripts • Online surveys of VRS • 137 VRS Users • 173 VRS Librarians • 134 VRS Non-users • Telephone interviews • 76 VRS Users • 100 VRS Librarians • 107 VRS Non-users • Generalizable through large sample sizes, multiple methods of data collection, and triangulation of results. • http://www.oclc.org/research/activities/synchronicity/default.htm

8. Example: Seeking Synchronicity CIT VRS User Online Survey Questions Think about one experience using VRS in which you felt achieved (or did not achieve) a positive result • Please describe the circumstances and nature of your question. • Describe why you felt the encounter was successful (or unsuccessful). • Did the chat format help your experience to be successful (or unsuccessful)? If yes, how? (Connaway & Radford, 2011)

9. Example: Seeking Synchronicity CIT VRS Potential User Online Survey Questions Think about one experience in which you felt you achieved (or did not achieve) a positive result after seeking library reference services in any format. • Think about one experience in which you felt you did (or did not) achieve a positive result after seeking library reference services in any format. • Describe each interaction. • Identify the factors that made these interactions positive or negative. (Connaway & Radford, 2011)

10. Example: Seeking Synchronicity CITVRS User Result “The librarian threw in a cordial sign-off and encouraged me to pursue the reading. It was like talking to a friendly librarian in person.” –VRS User (online survey)

11. Digital Visitors and Residents: What Motivates Engagement with the Digital Information Environment? • Investigate theory of digital residents & visitors with students in the various stages of education • Increase understanding of learners’ online engagement • How can educational services & systems attract & sustain new group of lifelong learners? • Trans-Atlantic partnership supports comparison of students' digital learning strategies in different cultural contexts. (Connaway and White for OCLC Research, 2012.) http://www.oclc.org/research/activities/vandr/default.htm

12. Example: Digital Visitors and ResidentsTriangulation of Data • Several methods: • Semi-structured interviews (qualitative) • Diaries (qualitative) • Online survey (quantitative) • Enables triangulation of data (Connaway et al., 2012)

13. Diaries • Ethnographic data collection technique • Get people to describe what has happened • Center on defined events or moments (Connaway & Powell, 2010)

14. Example: Digital Visitors and ResidentsDiaries

15. Example: Digital Visitors and Residents CITDiarist Follow Up Interviews Think of a time when you had a situation where you needed answers or solutions and you did a quick search and made do with it. You knew there were other sources but you decided not to use them. Please include sources such as friends, family, teachers, coaches, etc.

16. Example: Digital Visitors and Residents CITDiarist Follow Up Interviews • Explain a time in the past month when you were SUCCESSFUL in completing an ACADEMIC assignment. What steps did you take? • Think of a time fairly recently when you struggled to find appropriate resources to help you complete an ACADEMIC assignment. What happened? • Explain a time in the past month when you were successful in getting what you needed in a PERSONAL situation. What steps did you take? • Explain a time in the past month when you were NOT successful in getting what you needed in a PERSONAL situation. What steps did you take?

17. Cyber Synergy: Seeking Sustainability through Collaboration between Virtual Reference & Social Q&A Sites • New model for VRS • Collaboration with Social Q&A (SQA) community • Provide evidence for modeling new library collaborative services • Three phases • Analysis of transcripts • 500 QuestionPoint • 1000 Yahoo Answers Q & A pairs • 200 QuestionPointlive chat • 200 QuestionPointQwidget session transcripts • Telephone interviews & analysis • 50 librarian interviews, 50 user interviews • Construct design specifications http://www.oclc.org/research/activities/synergy/default.htm

18. Example: Cyber Synergy CIT User Phone Interviews Please recall one specific VR interaction...that you would consider successfuland describe. Please recall one specific VR interaction...that you would consider unsuccessfuland describe. (Radford, Connaway, & Shah, 2011-2013)

19. “I was looking for books on bilingual communication in the setting of sports teams and I needed at least three book references and I wasn't able to get them on time, so I think it was unsuccessful because I ended up not getting the information I needed for research I was working on.” Cyber Synergy(VS43, Male, Age 19-25)

20. Using CIT in YOUR Research • Can be applied to a range of studies • Allows categories or themes to emerge rather than be imposed • Provides clear examples that prove the value of your services • Users speak for you

21. References Connaway, L. S. & Radford, M. L. (2011). Seeking Synchronicity: Revelations and recommendations for virtual reference. Dublin, OH: OCLC Research. Retrieved from http://www.oclc.org/reports/synchronicity/full.pdf Connaway, L. S., Lanclos, D., White, D. S., Le Cornu, A., & Hood, E. M. (2012). User-centered decision making: A new model for developing academic library services and systems. IFLA 2012 Conference Proceedings, August 11-17, Helsinki, Finland. Connaway, L. S., & Powell, R. R. (2010). Basic research methods for librarians. Santa Barbara, CA: Libraries Unlimited. Fisher, S., & Oulton, T. (1999). The critical incident technique in library and information management research. Education for Information, 17(2), 113–126. Flanagan, J. C. (1954). The critical incident technique. Psychological Bulletin, 51(4), 327–358. Geertz, Clifford. (1973). The interpretation of cultures: selected essays. New York: Basic Books, 6. Hamer, J. S. (2003). Coming-out: Gay males’ information seeking. School Libraries Worldwide, 9(2), 73–79. Radford, M. L. (summer, 2006). The critical incident technique and the qualitative evaluation of the Connecting Libraries and Schools Project. Library Trends 54(1), 46-64. Radford, M. L. (1999). The Reference Encounter: Interpersonal Communication in the Academic Library. Chicago: Association of College & Research Libraries, A Division of the American Library Association. (Publications in Librarianship #52). Radford, M. L., Connaway, L. S., & Shah, C. (2011-2013). Cyber Synergy: Seeking Sustainability through Collaboration between Virtual Reference and Social Q&A Sites. Funded by the Institute of Museum and Library Services (IMLS), Rutgers University, and OCLC. Retrieved from http://www.oclc.org/research/activities/synergy/default.htm Tice, M. (2001). Queens Borough Public Library and the Connecting Libraries and Schools Project. Journal of Youth Services in Libraries, 14(3), 11–13 White, D., & Connaway, L. S. (2011). Visitors and residents: What motivates engagement with the digital information environment. Funded by JISC, OCLC, and Oxford University. Retrieved from http://www.oclc.org/research/activities/vandr/ Wilkins, J. L. H., & Leckie, G. J. (1997). University Professional and Managerial Staff: Information Needs and Seeking. College & Research Libraries, 58(6), 561-74.

22. Lynn SilipigniConnaway, Ph.D.connawal@oclc.org Questions?