Data & Data Collection

1. Data & Data Collection by Miles Hamby, PhD Principle, Ariel Training Consultants drhamby@cox.net Source:Wadsworth,Thomson (2005). So you want to use a Likert scale? from the Learning Technology and Babbie, Earl R. (2005). The Basics of Social Research. p. 174. ISBN 0534630367. and Likert, Rensis (1932), "A Technique for the Measurement of Attitudes", Archives of Psychology 140: pp. 1-55

2. Problem Statement drives the research question Research Question drives the methodology Methodology drives the data collection

3. Driven by the statistical or analysis procedure intended (descriptive, correlation, regression, ANOVA, non-parametric, t-test. etc) Type of data (nominal, ordinal, interval, ratio) Scale and values Data Collection Instrument

4. RATIO Meaningful ‘0’ e.g., age, height { Quantitative __ Meaningful X e.g., Fahrenheit scale INTERVAL Qualitative or Quantitative e.g., Order of finish in a race ORDINAL e.g., eye color, jersey numbers NOMINAL Qualitative only Levels of Data

5. Variables Before composing a survey, identify the dependent variable and independent variables.

6. Variables Dependent Variable – the characteristic (variable) of an hypothesis that is dependent upon the action of other variables. Independent Variables – those characteristics (variables) that effect the dependent variable. e.g. “The purpose of this research is to determine if daily exercise will reduce a person’s weight.”

7. Variables e.g. “The purpose of this research is to determine if pay is a significant factor in job satisfaction.” e.g. “The purpose of this research is to determine if increasing oil prices has changed people’s lifestyles.”

8. To collect specific data for testing anhypothesis Purpose of a Survey

9. Data Measurement Scales - Likert • Pronounced 'like-urt‘ • Named after Rensis Likert, who published a report describing its use (Likert, 1932). • Type of psychometric response scale • Widely used in survey research. • Respondents specify level of agreement to a statement.

10. Likert Scale – cont’d • Respondent asked to indicate his or her degree of agreement with the statement or any kind of subjective or objective evaluation of the statement. • The five response categories represent an Interval level of measurement. • Traditionally a five-point scale is used, • Many psychometricians advocate using a seven or nine point scale - some recommend even-numbered

11. Likert Scale – Cont’d Example. Please indicate the strength of your agreement or disagreement with the following statement. “Pay is important to my job satisfaction.” Strongly Neither Agree Strongly Agree Agree Disagree Disagree Disagree 5 4 3 2 1

12. Likert Scale – Scaling • Bipolar - measures positive or negative tendencies toward a statement. • Even-point scale used to force respondent ‘off the fence’ • e.g., middle option "Neither agree nor disagree" not available.

13. Likert Scale – Cont’d Example. Please indicate the strength of your agreement or disagreement with the following statement. “Pay is important to my job satisfaction.” Strongly Strongly Agree Disagree 6 5 4 3 2 1

14. Likert Scale – Scaling • Likert scales subject to distortion • central tendency bias – Respondent avoids extreme response categories • acquiescence bias – respondent agrees with statements as presented • social desirability bias – respondent tries to be portrayed in favorable light

15. Likert – Analysis • Summative Analysis • Items summed to create score for a group • Treated as interval data measuring a latent variable • With more than 5 responses, summed responses fulfil relevant assumptions, parametric statistical tests such as the Analysis of Variance (ANOVA) can be applied. • Nominal data - combining all agree and disagree responses into two categories of "accept" or "reject". • Commonly used statistical procedures - Chi-Square, Cochran Q, or McNemar

16. Ipsative Scale • Rank-order scale - particular rank used only once - opposite of Normative. • Example – “Rank the following ten movies from best to worst, in your opinion.” • With multiple raters , mean, median, and standard deviation is the same.

17. Normative Scale With multiple raters, the mean, median, and standard deviations usually different from one iteration to the next. Vogt, W. Paul (2005). Dictionary of Statistics and Methodology: A Non-technical Guide for the Social Sciences, Sage Publications, Inc. ISBN 0761988556

18. Survey Example

19. Satisfaction Survey This survey is being conducted as a part of coursework for a graduate degree at XYZ University and is in no way connected to your place of employment. Your responses are anonymous and information will be confidential. 1) Overall, how satisfied are you with your current job? VERY STAISFIED 4 VERY SIGNIFICANT 4 SOMEWHAT SIGNIFICANT 3 SATISFIED 3 SIGNIFICANT 2 NOT SATISFIED 2 NOT SIGNIFICANT 1 VERY DISSATISFIED 1 2) How significant a factor is pay in your job? Please indicate your response to the following questions: Please indicate the strength to which you agree or disagree with the following statements: 3) Management keeps me informed. 4) The work is interesting. 5) My pay is commensurate with my job requirements. 6) My work schedule is flexible. 7) The environment is conducive to work.   8) Please indicate your work status. 9) Please indicate your age. (write in )______________ 10) Please indicate your ethnic origin. 11) Please indicate your gender. 12) Please indicate your family status. 13) Please indicate your income level. Strongly Agree Strongly Disagree 6 5 4 3 2 1 6 5 4 3 2 1 6 5 4 3 2 1 6 5 4 3 2 1 6 5 4 3 2 1 6 5 4 3 2 1 WHITE HISPANIC AFRICAN AMERICAN AFRICAN ASIAN/PACIFIC ISLANDER NATIVE AMERICAN/ALASKAN MALE FEMALE MARRIED, NO CHILDREN MARRIED W/ CHILDREN SINGLE, NOT SUPPORTING SINGLE PARENT < $10,000 $10,000 to $30,000 $30,000 to $50,000 $50,000 to $80,000 >$80,000 Please return the completed survey in the SASE to: Researcher, PO Box 459, Alexandria, Va, 22306

20. Survey Guidelines • Compose questions with research DV, statistical procedure, and ease of data entry in mind • Group questions by type ~ eg, Strength of Agreement, Level of Perception, Demographics • State questions in the affirmative • Code higher numbers affirmative, low numbers unfavorable • Insure visually asthetic ~ e.g., alignment, font size & style, consistency in capitalization, numbering • Strive for 1 page max

21. Survey Hosts • MS Excel • Survey and questionnaire forms can be composed on an MS Excel spreadsheet and emailed or sent electronically to potential respondents. The respondent can open the Excel spreadsheet, simply click on the appropriate response (Check-box or Option button). • 1. Open Excel, click on View/Toolbars/Forms • Write the survey question in a cell • To insert a response item, click on ‘Check Box’ or ‘Option Button’ – position the box to the appropriate place next to the question and type in the text for the appropriate response. Continue to add and position response boxes as required. ‘Group’ the question and response boxes with the ‘Select Objects’ arrow (from the Draw toolbar) and Draw/group.

22. Online Survey Hosts • Many providers/hosts of online surveys • Search keywords ‘online survey providers’ • Many offer limited use free or full short-term use for reasonable cost. • Before selecting an online survey host - identify the research dependent and independent variables, measurement scales, statistical method, and specific questions.

23. Data & Data Collection

24. Likert Scale – Analysis Single Item Analysis • Responses to single item treated as ordinal data • With only 5 levels, cannot assume respondents perceive difference between adjacent levels as equidistant. • As ordinal data, Likert responses analyzed using non-parametric tests, e.g., Mann-Whitney test, the Wilcoxon Signed-rank test, or Kruskal-Wallis test.

25. Likert – CBA • Consensus Based Assessment (CBA) • Used to create objective standard for Likert scales in domains where no generally accepted standard or objective standard exists • CBA can be used to refine or validate generally accepted standards.

26. Likert – Rasch Model • Used to obtain interval level estimates on a continuum • Polytomous Rasch model permits testing of hypothesis that the statements reflect increasing levels of an attitude or trait, as intended. • e.g., application of the model often indicates neutral category does not represent a level of attitude or trait between the disagree and agree categories.