1 / 21

Building a Better Survey

Building a Better Survey. Katie Busby, PhD Director of Student Affairs Assessment & Planning University of Alabama. Keston Fulcher, PhD Director of Assessment, Evaluation and Accreditation Christopher Newport University. International Assessment & Retention Conference 2007. Survey.

van
Download Presentation

Building a Better Survey

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Building a Better Survey Katie Busby, PhD Director of Student Affairs Assessment & Planning University of Alabama Keston Fulcher, PhD Director of Assessment, Evaluation and Accreditation Christopher Newport University International Assessment & Retention Conference 2007

  2. Survey Please take a few minutes to complete this survey. It has some errors, note them when you read.

  3. Overview of Session • Introduce the assessment cycle as a foundation upon which to build assessment efforts • Complete an exercise that will identify strong and weak components of a survey • Discuss, in detail, the steps that are necessary to develop a good survey instrument • Share practical techniques that can be applied to overcome common missteps in survey development

  4. Definition of Assessment • The systematic process of determining educational objectives, gathering, using, and analyzing information about student learning outcomes to make decisions about programs, individual student progress, or accountability - T. Dary Erwin (1991)

  5. Assessment Model Conceptualize and Establish Intended Learning Outcomes Report and Use Results Select Assessment Methods Select Assessment Methods Analyze and Interpret Collect Data

  6. Rules for Developing Survey • Avoiding the don’ts and doing the do’s

  7. Rules for Developing Surveys • Be sure a survey is the way to go • Is there existing data you can use? • Is the survey the best method of collecting data? • Is there an existing survey you can use? • Design your assessment project first • Identify your research questions • Determine what you want to know • Clarify your goals

  8. Rules for Developing Surveys • Survey development is a team sport • Include other experts in the process • Involve relevant decision makers • Combine efforts with others who are developing a survey instrument

  9. Rules for Developing Surveys • You never get a second chance to make a first impression • Keep survey as short as possible • Have a logical flow to questions • Ensure a good layout design • Requests for participation, title, and instructions should be clear and concise

  10. Rules for Developing Surveys • It’s all about the respondent • Keep the respondent in mind during the development phase • Make it easy for the respondent to respond • Consider offering incentives • Pilot the survey

  11. Specific Hint 1 • Keep sensitive items toward the end of survey (i.e., warm up the responder). • Note: Do you need IRB approval?

  12. Specific Hint 2 • If intended, make sure response options are mutually exclusive. • Bad: • 0-15k • 15k-30k • Better: • Under $15,000 per year • $15,001 - $30,000 per year

  13. Specific Hint 3 • Group items with similar content and/or similar item types together. • Notice that Item 2 and Item 4 are needlessly separated on Bogus Survey.

  14. Specific Hint 4 • “Don’t Know” or “Not Applicable” options should be included at the end of or aside from other response options (READ: NOT IN THE MIDDLE)

  15. Specific Hint 5 • Avoid Double-Barreled Items • Bad: • (4) I enjoy working in the field of assessment and analyzing data. • Better • (4) I enjoy working in the field of assessment. • (5) I enjoy analyzing data.

  16. Specific Hint 6 • Response options should accommodate all reasonable responses. • On Bogus Survey, response options to Item 5 do not cover all responses.

  17. Specific Hint 7 • Attempt to maximize variability with responses • For Item 6, the vast majority of attendees would mark the middle option.

  18. Specific Hint 8 • Proofread • As with any document, errors connote a lack of professionalism.

  19. Questions about survey development • What challenges have you faced? • How can we improve our “survey”?

  20. Contact us Keston H. Fulcher, PhD Director of Assessment, Evaluation, and Accreditation Christopher Newport UniversityMcMurran Building Annex, Room 101-A1 University PlaceNewport News, Virginia 23606 (757) 594-7392  keston.fulcher@cnu.edu Katie Busby, PhD Director, Student Affairs Assessment and Planning University of Alabama Box 870116 Tuscaloosa, AL 35487-0016 205-348-7939 kbusby@sa.ua.edu

  21. Resources for Survey Design Alreck, P. L., & Settle, R. B. (1995). The survey research handbook, 3rd edition. New York: McGraw Hill. Bradburn, N. M., Sudman, S., & Wansink, B. (1996). Asking questions: The definitive guide to questionnaire design – for market research, political polls, and social and health questionnaires. San Francisco: Jossey Bass.. Dillman, D. A. (2006). Mail and Internet surveys: The tailored design method 2007 update with new Internet, visual, and mixed-mode guide, 2nd edition. New York: John Wiley & Sons. Rea, L. M., & Parker, R. A. (1997). Designing and conducting survey research: A comprehensive guide. San Francisco: Jossey-Bass. Sudman, S., Bradburn, N. M., & Schwartz, N. (1996). Thinking about answers: The application of cognitive processes to survey methodology. San Francisco: Jossey Bass.

More Related