Quantitative Research

1. Quantitative Research

2. March 8 • Quiz • Assignments • Experimental Research • Quantitative Research • Emotional Intelligence Survey • Validity, Reliability, Error • Characteristics of Surveys

3. Laboratory Experiments Artificial – low realism Few extraneous variables High control Low cost Short Duration Subjects aware of participation Field Experiments Natural – high realism Many extraneous variables Low control High cost Long Duration Subjects unaware of participation Experimental Research

4. Quantitative Research • Emotional Intelligence Survey http://quiz.ivillage.co.uk/uk_work/tests/eqtest.htm

5. Validity • Extent to which a measurement instrument actually measures the attribute it was intended to measure. • Validity can be examined from a number of different perspectives, including: • Face, content, criterion-related, and construct validity

6. Assessing Validity Face validity Researchers judge the degree to which a measurement instrument seems to measure what it is supposed to. Content validity The degree to which the instrument items represent the universe of the concept under study. Criterion-related The degree to which a measurement instrument can validity predict a variable that is designated a criterion. (a) Predictive ability; (b) Concurrent validity Construct validity The degree to which a measure confirm a hypothesis created from a theory based upon the concepts under study. (a) Convergent validity; (b) Discriminate validity

7. Reliability • Reliability • Degree to which measures are free from random error and, therefore, provide consistent data. • There are three ways to assess reliability • Test-retest, equivalent forms, and internal consistency

8. Assessing the Reliability of a Measurement Instrument Test-retest reliability: Use the same instrument a second time under as nearly the same conditions as possible. Equivalent form reliability: Use two instruments that are as similar as possible to measure the same object during the same time period. Internal consistency reliability: Compare different samples of items being used to measure a phenomenon during the same time period.

9. Reliability and Validity Situations in Measurement Situation 1 Situation 2 Situation 3 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Neither reliable nor Valid Highly reliable but not valid Highly reliable and valid

10. Error

11. Types of Error in Survey Research • Random Sampling Error (Random error) • Error that results from chance variation • Impact can be decreased by increasing sample size and through statistical estimation (confidence interval) or “rule of thumb” • Systematic Error (non sampling error) • Error that results for the research design or execution.

13. Types of Systematic Error 1. Administrative Error • Error that results from improper execution. • Data Processing Error • Quality of data depends on quality of data entry. • Use of verification procedures can minimize

14. Sample Selection Error • Systematic error resulting from improper sampling techniques either in design or execution. • Interviewer Error • Data recorded incorrectly (error or selective perception). • Interviewer Cheating • Mitigate by random checks

15. 2. Respondent Error • Humans interviewing humans... • Non-response error • Statistical difference between a survey that includes only those who responded and a survey that also includes those who failed to respond. • Non-respondent: person not contacted or who refuses to participate • Self selection bias: extreme positions represented

16. Response bias • Errors that result from tendency to answer in “a certain direction”. • Conscious or unconscious misrepresentation • Types: • 1. Deliberate falsification (why?)

17. Why would people deliberately falsify data • Appear to be what they are not • Don’t trust confidentiality • Protect • To end the interviewer quicker • “Average man effects”

18. Types of response bias continued: • 1. Deliberate falsification • 2. Unconscious misrepresentation

19. Reasons for unconscious misrepresentation: • Question format • Question content • Misunderstanding of question leading to biased answer • Lack of time to consider answer fully • Communication or semantic confusion • other

20. Types of response bias • Acquiescence bias: individuals have a tendency to agree or disagree with all questions or to indicate a positive/negative connotation • Extremity bias: results for response styles varying from person to person; some people tend to use extremes when responding to questions

21. Types of response bias continued... • Interviewer bias: Bias in the responses of the subject due to the influence of the interviewer • Auspices bias: respondents being influenced by the organization conducting the study • Social desirability bias: caused by respondents’ desire, either consciously or unconsciously to gain prestige or to appear in a different social role

22. Measurement

23. The Concept of Measurement and Measurement Scales • Measurement • Process of assigning numbers or labels to things in accordance with specific rules to represent quantities or qualities of attributes. • Rule: A guide, method, or command that tells a researcher what to do. • Scale: A set of symbols or numbers constructed to be assigned by a rule to the individuals (or their behaviors or attitudes) to whom the scale is applied.

24. Types of Measurement Scales • Nominal Scales • Scales that partition data into mutually exclusive and collectively exhaustive categories. • Ordinal Scales • Nominal scales that can order data.

25. Interval Scales • Ordinal scales with equal intervals between points to show relative amounts; may include an arbitrary zero point. • Ratio Scales • Interval scales with a meaningful zero point so that magnitudes can be compared arithmetically.

26. Nominal Show Win Place Ordinal Interval 1 length 2 lengths Ratio 40 to 1 long-shot pays $40

28. Selecting appropriate univariate statistical method

32. Surveys

33. Survey data • Survey data are obtained when individuals respond to questions asked by interviewers or when the individual responds to questions that he has read • Quantitative and qualitative information • Census or sample?

34. Survey Sampling • Sample Survey • Sample of the population (e.g., sample of Alberta CEOs) • Census Survey • Complete population (e.g., all CEOs in Lethbridge)

35. Functions • Descriptive • Causal (limited function) • Exploratory (limited function)

36. Information Provided by Surveys • respondent’s knowledge of facts • how many times a month do you buy cookies? • respondent’s attitudes • “chocolate chip is my favorite variety of cookie.” • May describe processes undertaken by respondent • Stages in the purchase decision, for example • usually self-report data

37. The type of information gathered depends on a survey’s objectives. • Surveys typically have multiple objectives

38. Classifications • Method of Communication • Telephone • Mail • Personal interview • Degree of Structure • Unstructured • structured • Degree of Disguise Limitations: Structure and disguise are not clear categories; most surveys are hybrids

39. Temporal classifications • Cross sectional studies • Longitudinal studies • Trend Studies • Samples general population at each point; complete turnover in who is actually sampled • Cohort studies (tracking studies) • Samples from one group over time; e.g., sample of a graduating class • Panel studies • sample cohort, same specific respondents each sample

40. Method of Communication • Media • Personal Interviews • Telephone Interviews • Self administered Questionnaires

41. Questionnaire design A survey is only as good as the questions it asks

42. What should you ask? • The questions asked are a function of previous decisions • The questions asked are a function of future decisions (such as statistical analysis)

43. Key criteria • Questionnaire relevancy • No unnecessary information is collected and only information needed to solve the problem is obtained. Be specific about your data needs; tie each question to an objective • Questionnaire accuracy • Information is both reliable and valid

44. Phrasing Questions • Open ended response versus fixed alternative questions “?” • Decision criteria: type of research; time; method of delivery; budget; concerns regarding researcher bias

45. Avoid • Leading questions • Overly complex questions • Use of jargon • Loaded questions (can use a counterbiasing statement) • Ambiguity • Double barreled questions • Making assumptions

46. Order? • Order bias results from an alternative answer’s position in a set of answers or from the sequencing of questions • Funneling technique: general to specific helps understand the frame of reference first • Anchoring effect: the first concept measured tends to become a comparison point from which subsequent evaluations are made

47. Types of questions

48. Types of fixed alternative questions… • Single dichotomy or dichotomous-alternative questions • “Are you currently registered in a course at the University of Lethbridge? • Yes____ No____” • Respondent chooses one of two alternatives (yes/no; male/female) • What scale would this data create?

49. Types of fixed alternative questions… • Multi-choice alternative • Respondent chooses from several alternatives • Many types…

50. Multi-choice alternative questions… • Determinant choice • Choose only one from several possible responses “Which faculty are you currently registered in at the University of Lethbridge? Management ___ Education ____ Arts/Science____ Health sciences____ Combined degree____ • What type of scale would these data create?