Data Gathering Procedures

1. Data Gathering Procedures • Two dimensions on which we can distinguish data gathering procedures • The degree of contact between the researcher and the participant • The extent to which participants are given an opportunity to respond

2. Data Gathering Procedures • Dimensionalizing direct contact / response-gathering procedures

3. Direct Contact / Direct Response • Different methods of interaction • Whether or not measure was disguised in its intent • The degree of structure imposed on participants’ responses

4. Direct Contact / Direct Response • Cross-tabulation of interaction types between researcher and participant More Structured Less

5. Direct Contact / Direct Response • The status of disguised measures • Peak of popularity was in 1950s and 1960s • A period when social sciences were obsessed with negative attitudes and behaviors like prejudice, obedience, conformity • Techniques emphasized control and structure, couple with deception • Techniques were probably oversold by researchers of the day • Less useful and predictive than non-disguised measures • “If you want to know something about someone, ask them; they might just tell you” (Kelly, 1955)

6. Direct Contact / Direct Response • Non-Disguised measures • Questionnaires and interviews • Means of questionnaire administration • Self-administered • Group-administered • Mail out • Means of interview administration • In-person interview • Focus-group interview • Telephone interview • Degree of structure • Open-ended questions • Close-ended/forced choice questions

7. Direct Contact / Direct Response • Advantages and disadvantages to interviews and questionnaires Disadvantages Advantages

8. Direct Contact / Direct Response • Rating and Likert Scales • Rating scale: All things considered, how satisfied or dissatisfied are you with your life as a whole right now? 1 2 3 4 5 6 7 Completely Completely Dissatisfied Satisfied • Likert (Likert-type) Scale: All things considered, I believe that the death penalty should be reinstated ___ Strongly disagree ___ Moderately disagree ___ Slightly disagree ___ Neutral ___ Slightly agree ___ Moderately agree ___ Strongly agree

9. No Contact / No Response • Unobtrusive measures • Physical trace measures • Erosion and accretion • Selective deposit and selective survival • Archival measures • Allow for longitudinal analysis • Archival sources actually exist • Lower cost • Less influenced by reactivity • Selective deposit and survival

10. Direct Contact / No Response • Observational techniques • What is an observation? • “The selection, provocation, recording, and encoding of that set of behaviors and organisms in situ which is consistent with empirical aims” (Weick, 1968)

11. Direct Contact / No Response • Methods of observational coding • What, who, and when to observer • What – Behavior sampling • Who – Subject sampling • When – Time sampling

12. Direct Contact / No Response • Methods of observational coding • Behavior sampling • Composite behavior • Syndrome • Simultaneous behaviors • Subject sampling

13. Direct Contact / No Response • Methods of observational coding • Time sampling • Why is time important? • Behavior is expressed as a sequence of events • Observational data is gathered over time • Repeated observations are necessary • The need to be able to differentiate individual differences

14. Direct Contact / No Response • Time sampling techniques • Semi-continuous sampling • Partial-interval sampling • Divide observation period into equal time intervals • Score behavior interval by interval • 1 for occurrence, 0 for non-occurrence • Overestimates prevalence • Whole-interval sampling • 0 for non-occurrence over entire interval; 1 for any occurrence • Underestimates prevalence

15. Direct Contact / No Response • Time sampling techniques • Discrete sampling • Ad libitum sampling • Record behaviors on a non-systematic, informal basis • Susceptible to observer bias • Some behaviors attract attention • Momentary sampling • Observe behavior at specific, predetermined point in time • 1 if behavior occurred at that time, 0 if it did not occur • Unbiased estimate of prevalence

16. Reliability of Observational Coding • Interobserver agreement indices • Smaller / Larger Index • Divide smaller of two values by larger of two values • Observer 1: 9 vocalizations • Observer 2: 10 vocalizations • Smaller/Larger Index = 9/10, 90% • Problem: • Observer 1: 0 1 0 1 0 1 • Observer 2: 1 0 1 0 1 0 • S/L = 3/3 = 100% • No individual observation agrees

17. Reliability of Observational Coding • Interobserver agreement indices • Percent agreement • Number of agreements divided by the number of agreements plus the number of disagreements • Observer 1: 0 1 0 1 1 1 0 1 0 1 • Observer 2: 0 1 1 1 1 0 0 0 0 1 • # of agree = 7, disagree = 3 • % agreement = 7 / (7+3) = 70% • Problem – inflation by chance

18. Reliability of Observational Coding • Interobserver agreement indices • Cohen’s Kappa • Discounts expected chance agreements • (Po – Pc) / (1 – Pc) • Po = proportion agreement • Pc = proportion expected by chance

19. Reliability of Observational Coding Percent agreement & Cohen’s Kappa S: Success in barrier crossing F: Failure in barrier crossing R: Refusal in barrier crossing Percent agreement: # agree / (# agree + # disagree) Example: # agree: 7 (S) + 4 (F) + 3 (R) = 14 # disagree: 0( S) + 2 (F) + 3 (R) = 6 Percent agreement: 14 / (14 + 6) = 70%

20. Reliability of Observational Coding Percent agreement & Cohen’s Kappa S: Success in barrier crossing F: Failure in barrier crossing R: Refusal in barrier crossing Percent agreement: 70% Cohen’s Kappa: (Po – Pc)/(1 – Pc) Po: Proportion agreement observed Pc: Proportion agreement expected by chance

21. Reliability of Observational Coding Percent agreement & Cohen’s Kappa Percent agreement: 70% Cohen’s Kappa: (Po – Pc)/(1 – Pc) Po: (7 + 4 + 3) / 20 = .70 Pc: ((7*7) + (6*8) + (7*5) / (20*20) = (49+4*+35)/400 = .33 Kappa: (.70 - .33) / (1 - .33) = .5522