Survey and Interview Techniques

1. Information Systems Research Methods Survey and Interview Techniques

2. Survey Methodologies • Personal Interviews • Group Interviews • Phone Interviews • Mail-out Questionnaires • Hand-out Questionnaires • Clip-and-Mail Questionnaires • Survey of Secondary Data

3. Survey Methodologies Reasoning The goal of our research is to describe or relate the behavior, opinions, impressions, knowledge, etc. of people in situations. If we cannot directly observe them, their behavior, etc., we must rely on their own (“subjective”) observations of themselves. Individuals and their actions and opinions are unique, however. No single self-observation can be generalised to everyone.

4. Survey Methodologies Reasoning-2 In order to describe the general situation, we aggregate a series of measurements, observations or judgments about individual (“subjective” impressions of) events. The “objective” description is the average of all these “subjective” descriptions. We assume all biases “average out” and compensate for individual differences.

5. Survey Methodologies Background The strength of a survey is its generalizability. Since it samples stimuli from the REAL world to be applied in the REAL world, it stands a good chance of having REAL world responses. Issues are these: Validity: Do the stimuli create the desired responses? Reliability: Are the responses generated in a dependable way?

6. Survey Methodologies General Method Select Respondent Sample (Sampling #1) Create Stimulus Sample (Sampling #2) Motivate Respondent Participation (Sampling #3) Induce Responses From Respondent To Stimulus* Select Response Sample (Sampling #4) *Uncontrolled sampling

7. Friendly, familiar, cheap Relatively quick Useful if survey questions are clear Ideal for Descriptive research Uses common skills Hard to control Samples are often very large Has hidden flaws that can be easily ignored Cannot easily be used to draw causal inferences Retrospective Strengths...and…Weaknesses

8. A(1). Personal Interviews • Researcher or team member schedules a meeting with a respondent (subject, participant) • The respondent is asked questions • The respondent replies with answers (responses) • Responses are recorded, then later coded and transcribed if necessary

9. What is an Interview? An interview is a structured conversation between an active agent and a respondent; both are trying to find out what the other thinks, feels, knows, suspects, fears, desires, or respects. Often the result is quite remarkable! @*# ? !

10. Interviewing: General Interviewers must be trained Watch out for observer effects Rapport, empathy VERY important Easy to do VERY badly Each hour “costs” about 3 to 4 hours labor Recording is a problem Useful in uncovering in depth

11. Interviewing: The Four Samples Respondents: Whom you approach for interview access [sampling frame, permissions] Stimuli [“Schedule”]: Questions you intend to ask (or sample from) Participation: Times and venues at convenience of all parties Responses: What you chose to hear and record

12. Interviewing Procedure • Develop questions pertinent to theory • Create Interview “Schedule” • Pretest Questions on Schedule • Develop a sampling frame • Sample potential interviewees • Schedule interviews (2 - 4 per day) • Perform interviews • Write up data • Code/Store/Secure data`

13. Interviewing Issues Structure of Interview (Typology, constr’n) Flow of Interview Choice of language and level, jargon Degree of intervention with interviewee Questions and open vs. closed questions Using tape recorders Non-verbal cues, dress, style, accent Power relationships

14. Degree of Structuring • Structured: Interviewer is a mechanical questioner, available to clear up ambiguity • Semi-structured: Interviewer has fixed list but can deviate to follow tangents and get clarification, elaboration • Unstructured: No preset questions, just follows subject’s ideas

15. Interview Flow Structures Specific FUNNEL TIME FLIP- FLOP FAN Content experts Managers Generalists General

16. Language, Level, Jargon • Your language can tell a lot about you, the research, the answers! • The criterion is understanding • Select appropriate levels of language for your subjects • Avoid your own jargon unless interviewing your own kind of person

17. Open vs. Closed Questions Open: Answers aren’t predetermined, researcher has no control over answer other than by question “What was your first impression of the interpersonal skills of your new CIO?” Closed: Only a limited (usually small) number of possible responses “Which of the following 3 statements best describes your impression of your new CIO’s interpersonal skills?”

18. Allows subject to answer in his/her own terms Makes for more “friendly” interview May be hard to code Subject might not know how to answer Useful for probing Makes sure subject answers in researcher’s terms Makes for more systematic interview Easy to record/code Subject may answer incorrectly Might miss important responses Questions:Open Closed

19. Intervention • “Intervention” is a general term implying what you will do for (or with or to) the respondent • In action research, you are going to make a change in the respondent’s environment • In other kinds of research you may appear as a change agent or blocker. • Watch out for “halo effects” wrt technology or development outcomes.

20. Role of Non-Verbals • Non-verbals “modify” your speech and tell subject “who” and “what” you are • Includes these characteristics: Dress Posture Proxemics Accent Paralanguage Encouragers Tone of Voice Sexism/racism Time Mgmt. Mannerisms Facial expression Gaze Body Odor Gender Body image

21. Can theoretically record all responses Frees interviewer to concentrate on managing the process Provides a record of what was actually said Can be used to check your impressions Sometimes fails or is cumbersome to use Interviewer might forget to probe or pursue thinking “it’s on tape” May be intrusive and actually shape responses May inhibit immediate response formation and hence bias the research. Tape RecordersPros Cons

22. Degree of Structuring • Structured: Interviewer is a mechanical questioner, available to clear up ambiguity • Semi-structured: Interviewer has fixed list but can deviate to follow tangents and get clarification, elaboration • Unstructured: No preset questions, just follows subject’s ideas

23. Interviewing Example 175 handicapped people were interviewed concerning their use of the Internet. Their names were obtained from various societies of the disabled. Questions pertained to when, how, why they used the Internet and what they found difficult to use. Some problems occurred in interviewing deaf people and severely disabled persons. Each interview took about 1.5 hours.

24. Interviewing Diagnostic-1 Street traders are to be interviewed about their impressions of the value of computers in their work. A team of students, in business dress, shows up in Detroit at 17h00 and ask complex questions about hardware and software. What problems should have been anticipated?

25. Interviewing Diagnostic-2 A team of 4 interviewers is trained from the OU basketball team to conduct 100 interviews at a local girls’ prep school concerning students’ need for computer training. One week is allowed for data collection. Interviewers show up in uniform and ask 25 open-ended questions about training needs. What problems might occur?

26. Interviewing Diagnostic-3 You are interviewing managing directors on their views about IT and company strategy. For one interview you show up in sandals and shorts, tardy by half an hour, late on a Friday afternoon. Most of your 18 closed questions concern technology. Do you expect any problems with the data collection?

27. A2. Group Interviews • As with personal interviews, but the interviewer speaks with a group brought together at the same time • Person who runs meeting is called a “facilitator”. • Meetings may run for an hour or for hours • There might be others there: observers or recorders; it might be video or audio taped

28. Group Interviews: General • Useful, lowers cost • Hard to get groups together • Facilitation skills are not easy to acquire • May help simulate real work environment • Beware domination, kidnapping, S&M • Locale, interruptions • List of attendees is critical

29. Group Interview Benefits Synergy, group interaction Simulation of an almost real or normal situation Multistreaming, parallelism

30. Group Interview Example Five groups each of managers, professionals, and secretaries of five to sixteen people were interviewed in a conference room concerning their perceptions of their role and participation in computerization and how this affected their subsequent use. A facilitator, a recorder, and an observer were present. A video tape recorder was used. The meetings lasted from 2 to 3 hours. Some of the groups were lively, others not. Most corroborated one another’s experiences.

31. A3. Phone Interviews • Like personal interviews but… • Much shorter (10-30 minutes MAX) • Level of trust is quite low • Nonverbals are minimized, potential problem with control, empathy • Cuts down on travel time • Questions must be VERY succinct & clear

32. Phone Interview Example A team of student interviewers called 88 CIOs and conducted twenty-minute interviews on their role in corporate strategy. Each had been mailed a copy of the questions in advance. The interviews had been scheduled with their secretaries. Ten CIOs could not keep their appointments and two were interrupted during the interviews. The interviews were supplemented with biographical data obtained from the secretaries.