1 / 28

DATA COLLECTION METHODS

DATA COLLECTION METHODS. Conducting Assessment in Student Affairs-Session FOUR . Session Overview. Revisit Assessment Cycle Overview of Qualitative and Quantitative Paradigms Qualitative Data Collection Methods Interviews Focus groups Rubrics Other Methods. Assessment Cycle.

neena
Download Presentation

DATA COLLECTION METHODS

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. DATA COLLECTION METHODS Conducting Assessment in Student Affairs-Session FOUR

  2. Session Overview • Revisit Assessment Cycle • Overview of Qualitative and Quantitative Paradigms • Qualitative Data Collection Methods • Interviews • Focus groups • Rubrics • Other Methods

  3. Assessment Cycle Defining Assessment Purpose Context: Politics, Ethics, etc. Analyzing and Reporting Data Collection

  4. Qualitative and Quantitative Assessment

  5. Overview of “Paradigms”

  6. Not a Dichotomy Open-Ended Survey Questions Measurements Semi-Structured Interviews Naturalistic Observation Close-Ended Survey Questions

  7. The Qualitative vs. quantitative “debate” • The approach you choose should depend on the questions being asked. • Both paradigms lend themselves to quality research, to answering complex questions. • One is not “harder” than the other. • You are not “smarter” if you do quantitative work. • Reality bites: one paradigm may be valued over another in a given context.

  8. Discuss… • What approach is appropriate for your assessment question? • Do you know enough to create good close ended questions? • Is your question about process and how or why things happen?

  9. Qualitative Data Collection methods

  10. Semi-structured Interviewing • Type of interview in which the structure is predetermined • The researcher explores a few general topics to help uncover views on specific issues, but does not interfere with how the respondent frames their response • The participants perspective should unfold as the participant views it, not as the researcher views it

  11. Purpose and Limitations • Purpose • Yields ample data in short period of time • Immediate follow-up and clarification • Allows researcher to understanding the meaning held by the participant • Develop basis for future quantitative exploration • Limitations • Breadth vs. depth • Not observing people in a natural setting • Communication style/characteristics of participants

  12. Researcher Role • The researcher is the instrument • Negotiating entry • Reciprocity • Trust • Unequal power/coercion

  13. Interviewing in Action • Explain the nature of the research and obtain consent • If a participant has answered one of your questions prematurely, do not ask them to answer it again • Follow the protocol but do not let it derail a natural conversation flow • Don’t go too far off topic • Emphasize the factors that the respondent seems to want to talk about most

  14. Focus Group Interviewing • A discussion among a small group where the facilitator supplies the topics and monitors the discussion • The purpose is to gather information about a specific (or focused) topic in a group environment. • A group environment allows for discussion and interaction among participants. The interaction involved in focus groups can provide additional information to the researcher.

  15. Purpose of Focus Groups • Understanding the why? and how? Questions • Examine perceptions, beliefs, or opinions of participants • Identify strengths and weaknesses of a program, service, or experience • Build on existing information or contribute to the development of future studies

  16. Advantages of the Focus Group • Members respond to one another resulting in potentially rich data • “Synergistic group effect”: Interactions among and between group members stimulate discussions in which one group member reacts to comments made by another (Stage & Manning, 2003) • Flexible and dynamic • Relatively low cost as compared with technology driven methods • Participants may feel more comfortable in a group setting with others who are sharing similar opinions

  17. Limitations • Difficult to assemble • Groups can influence individual opinions • Strength of data drawn from focus group is largely dependent on the strength of the facilitator(s) • Lack of control over discussion

  18. Developing an Interview protocol

  19. Developing an Interview protocol • List of questions to ask participants • Ask concrete, specific, simple, and open-ended rather than complex, loaded, or close-ended questions.

  20. Types of Questions • Experience and behavior questions • What resources on campus do you use? • Opinion and values questions • What could UCLA administrators and staff do to provide a nurturing environment for LGB students? • Feelings questions • When you first came to UCLA, what did it feel to be out on campus? • Knowledge questions • What are the resources available to members of the LGB community on campus? • Sensory/environmental questions • What makes the UCLA campus welcoming or not welcoming to LGB students? • Background and demographic questions • If you feel comfortable, please tell us with which gender you identify.

  21. Example: LGBQ Experiences

  22. Activity: Protocol Worksheet

  23. Rubrics What is Rubric? • A scoring tool that lists the criteria for a piece of work, or “what counts” • Articulates gradations of quality for each criterion, from excellent to poor. Why Use Rubric? • Powerful tools for teaching and assessment • Helps guide self and peer-assessment to become more thoughtful judges of the quality of work • Reduce the amount of time to spend evaluating work • Easy to use and explain

  24. Rubrics Rubric Development • Articulate the outcome(s) being assessed • List criteria • Articulate gradations of quality • Practice on models • Use self- and peer-assessment • Revise • Use as assessment

  25. Common components • Scale: represents established levels of achievement, performance, or mastery • Common Scales: • Exemplary, Competent, Developing • Distinguished, Intermediate, Beginner • Excellent, Competent, Needs Work • Expert, Intermediate, Novice • Excellent, Good, Fair, Poor

  26. EXAMPLES • Your assessment plan rubric (Tab 10) • ACPA Competencies Rubrics • http://www2.myacpa.org/images/professional-development/docs/professional-competencies-rubrics2013.pdf • AACU VALUE Rubrics • http://www.aacu.org/VALUE/rubrics/index_p.cfm?CFID=18491496&CFTOKEN=32182917 • Baseline Rubric Tool (Templates)* • http://ucla.campuslabs.com Rubrics Tab *Need a Baseline Account--approval through your director; email Kristen

  27. Other Methods • Observation • Record situations as they happen • Participant Observation • You become part of the situation • Network Research • Understand the diffusion of information or behaviors • Content analysis of secondary text or visual data • Elicitation of themes in a body of written or visual media (e.g. photos) • Spatial mapping • Observe data across spacial units

  28. Example: Photovoice

More Related