ARL Workshop on User Surveys - PowerPoint PPT Presentation

slide1 n.
Skip this Video
Loading SlideShow in 5 Seconds..
ARL Workshop on User Surveys PowerPoint Presentation
Download Presentation
ARL Workshop on User Surveys

play fullscreen
1 / 27
Download Presentation
ARL Workshop on User Surveys
Download Presentation

ARL Workshop on User Surveys

- - - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript

    1. ___________________________ ARL Workshop on User Surveys Measurement Scales and Types of Questions Brinley Franklin

    3. ___________________________ Importance Of This Section Reliable and valid survey questions are a cornerstone of effective survey research Done correctly, questions can produce valuable information Done incorrectly, questions can gather inappropriate information, introduce bias, or even invalidate the user survey

    4. ___________________________ Topics To Be Covered The effects of attitudes, behavior, knowledge, and demographics on question construction Understanding the surveys context Guidelines for asking survey questions Open or closed questions? Response choices/measurement scales

    5. ___________________________ The Effects of Attitudes, Behavior, Knowledge and Demographics on Question Construction Attitudes Behavior Knowledge Questions Demographic Considerations

    6. __________________________ Attitudes Attitudes are complex and are difficult to define and measure Attitude questions often have two components: how respondents feel; and how strongly they feel

    7. ___________________________ Behavior Questions refer to what respondents actually do Generally concerned with identification, quantity, time, duration, or frequency (what, how many, when, how long, how often) Asking specific questions about a long period of time yields imprecise answers Lists help respondents recall things they might have done

    8. ___________________________ Knowledge Questions Can determine if respondents know enough about a topic to warrant asking their opinion May identify gaps in knowledge that need to be addressed Sometimes help to explain attitudes and behavior

    9. ___________________________ Demographic Considerations Knowing something about the surveys targeted respondents can help frame the questions and determine an appropriate level of specificity If specific responses are sought and the range of answers is large, ask for the exact information; dont present a multiple choice question Use current words and terms Decide if comparability is sought. If so, standardize questions

    10. ___________________________ Understanding the Surveys Context Focus the questions on the surveys specific purpose(s) or information sought Agree upon the precise data being sought in advance Clarify or define all potentially imprecise or ambiguous terms Terminology, language level, and time frame should be appropriate for respondents Stick with the least number of questions needed for adequate measurement

    11. ___________________________ Understanding the Surveys Context (Continued) Standardize survey(or)s Use anonymous surveys to ask sensitive questions about personal behaviors or attitudes

    12. ___________________________ Guidelines For Asking Survey Questions Ask concrete questions that are precise and unambiguous use time periods that are related to the importance of the question Use conventional grammar, punctuation, and spelling Use the proper context when constructing survey questions

    13. ___________________________ Using Conventional Grammar, Punctuation, and Spelling Use complete sentences Avoid abbreviations, slang and colloquialisms Avoid jargon and technical expressions Have the questions reviewed by experts in survey question writing Have the questions reviewed by potential respondents Adopt or adapt questions used successfully in other surveys

    14. ___________________________ Using Proper Context Shorter questions take less time and often lead to shorter answers Use longer questions with lead-ins to provide background information and collect data on sensitive concerns (e.g., journal cancellations) Use loaded questions cautiously Avoid biasing words or phrases Avoid two-edged questions (avoid and) Avoid negative questions or ones that require an exercise in logic

    15. ___________________________ Open Or Closed Questions? Open questions are used to solicit answers in the respondents own words Closed questions, where responses are pre-selected, are considered more efficient and reliable Use a checklist to decide between open and closed questions

    16. ___________________________ Open Questions Are useful in getting unanticipated answers Describe the respondents, not the surveyors world Provide quotable material Are primarily used to make decisions about individuals, not groups may be difficult to compile, compare and interpret

    17. ___________________________ Closed Questions Provide standardized data that can be analyzed statistically Answers have a better chance of relevance and consistency over time Are more difficult to write and validate May overlook things respondents might volunteer in open questions

    18. ___________________________ Checklist For Deciding Between Open And Closed Questions

    19. ___________________________ Response Choice/Measurement Scales Nominal/Categorical response choices Ordinal response choices Numerical response choices

    20. ___________________________ Nominal/Categorical Response Choices Are not numerical or preferential values Often used to categorize respondents characteristics Findings are usually reported as percentage groupings

    21. ___________________________ Suggestions For Constructing Nominal/Categorical Responses Generally try to be exhaustive Try to make choices mutually exclusive Try to use categories that are meaningful to respondents and will make sense when results are reported

    22. ___________________________ Ordinal Response Choices Respondents are asked to rate or rank choices Are often used more than the other types of questions Are typically used to measure satisfaction Survey findings are usually reported as an average score or percentage

    23. ___________________________ Suggestions For Constructing Ordinal Responses Consider five types of response options: Endorsement/Frequency/Intensity/Influence/ Comparison Determine through pre-testing if a neutral response is called for Use a 5point to 7point scale (either odd or even) Know when to put the negative end of the scale first

    24. __________________________ Numerical Response Choices Ask respondents to quantify a response Provide completely objective response Allow many statistical analyses to be performed and reported

    25. ___________________________ Suggestions For Constructing Numerical Responses Decide if discrete (i.e. exact) numbers work best Consider using intervals if precision is not needed Decide if it is most appropriate to offer numerical responses along a continuum

    26. ___________________________ Conclusion - You Should Now Have a Better Understanding Of: Why survey questions are important Characteristics of the user population that influence the question asked The context for framing survey questions Guidelines for asking survey questions The difference between open and closed questions, and when to use each type The three types of response choices: nominal/categorical; ordinal; and numerical

    27. ___________________________ Related Readings Babbie, Earl R. Survey research methods. Belmont, CA: Wadsworth Press, 1990. Bradburn, Norman M. and Sudman, Seymour. The current status of questionnaire design. In Paul P. Biemer et al, Measurement errors in surveys. New York: John Wiley, 1992. Converse, Jean M. Survey questions: handcrafting the standardized questionnaire. Beverly Hills, CA: Sage Publications, 1986. Fink, Arlene and Kosecoff, Jacqueline. How to ask survey questions. Volume 2 of The survey kit. Thousand Oaks, CA: Sage Publications, 1985. Fowler, Floyd J. Survey research methods. Newbury Park, CA: Sage Publications, 1993. Kosecoff, Jacqueline and Fink, Arlene. Evaluation basics: a practitioners manual. Beverly Hills, CA: Sage Publications, 1982. Schuman, Howard and Presser, Stanley. Question and answers in attitude surveys. New York: Academic Press, 1981. Sudman, Seymour and Bradburn, Norman M. Asking questions. San Francisco, CA: Jossey Bass, 1982.