270 likes | 402 Views
Join the Innovation Network's enlightening session on advanced data collection methods tailored for nonprofits. Learn about five major data collection techniques including document review, observation, interviews, focus groups, and surveys. Discover how to choose the right method, identify sources, and assess effort levels while maintaining a participatory approach. Gain insights into effective evaluation practices and harness data as a tool for community empowerment. Together, we can enhance evaluation capacity to create lasting change in our communities.
E N D
“Advanced” Data Collection January 27, 2005
Who We Are: Innovation Network • National nonprofit organization • Committed to evaluation as a tool for empowerment • Build evaluation capacity of nonprofits and funders so they can better serve their communities • Practice a participatory approach
Who We Are: Innovation Network www.innonet.org/services/LSC_Train
Objectives of This Session • Review the five major groups of data collection methods • Discuss some of the subtleties of web-based data collection
Data Collection Program Plan Evaluation Plan Data Collection Strategy
Data Collection: 3 Steps • Choose the method • Decide which people or records will be the source of the information • Determine the level of effort involved in using that method with that population
Data Collection Steps 1 & 2:Choose Method, Identify Source • Review documents • Observe • Talk to people • Collect written responses • Pictorial/multimedia
Data Collection Step 3:Level of Effort • Instrument development • Cost/practicality of actually collecting data • Cost of analyzing and presenting data Also consider: • Communication Power • Proxy Power
Document Review Examples: • Attendance records • Application information • Feedback forms • Web statistics • Curricula or content
Document Review • Efficient and cost-effective • Dependent on the existence of documentation • Better for process evaluation than outcome evaluation
Observation • When conducting and evaluation the observer: • Does not intervene in any way • Does not stand out or draw attention • Does not record names or any identifying information • It is important that observations be held over time (and at different places/days/times)
Observation • Inexpensive way to obtain an outside perspective • Highly susceptible to bias • Subjective; relies on anecdotal data
Interviews • Structured • Follow a predetermined interview protocol • Identify broad evaluation questions and specific interview questions in advance • Unstructured • Begin with one or two open-ended questions • Proceed based on the responses received
Interviews • Provides the story behind someone’s experiences • Useful as a follow-up to a survey to further investigate responses • Time consuming (both in terms of collection and analysis) • Respondents susceptible to “leading” by interviewers
Interview Tips • Make the respondent feel comfortable • Introduce yourself and the evaluation • Start with easy, short answer questions – end with more revealing, long answer questions • Record the interviews • Important for capturing great quotes
Focus Groups A small group of people (who have something in common) who are led through a conversation by a moderator and discuss their opinions and provide suggestions on a topic.
Focus Groups • Efficient and cost-effective way to gather qualitative data • Requires a skilled moderator/facilitator • Collects consensus rather than individual perspectives
Focus Group Tips • Ideally 8-10 person groups • Maximizes amount of information you can gather without the suppression of voices • Don’t get bogged down • Rewards • Breakfast/Lunch/Dinner • Gift certificates
Surveys • Delivery options: • By mail • In person • Online • By telephone
Surveys • Collect quantitative information • Efficient • Open-ended questions (qualitative data) can be cumbersome for respondents and data entry
Survey Tips • Introduce the survey • Describe the purpose and what will be done with the results • Reinforce that it is confidential and that you request they respond candidly • Lead with easy, closed-ended questions • Group categories of questions together • Consider how long it will take for someone to respond rather than the length of pages
Web Statistics • # of hits • Most popular pages • Most popular downloads • Common “drop off” points • Where are they coming from?
Don’t Be Afraid to Ask More! • Why are they there? • Who are they? What do they do? • What are they looking for? • Did they find it? • What are they going to do with it? • Are they willing to help you with future evaluation projects?
Usability Testing We can help… ereed@innonet.org 202.728.0727 x 109
Next Steps • We can do more! • Online, instructor-led trainings • Online, self-paced modules • Individual technical assistance www.innonet.org/services/LSC_Train
Next Steps • We can do more! • Survey Builder • Survey Library • Data Analysis Tool
Thanks for Your Participation! Measure results. Make informed decisions. Create lasting change. Innovation Network, Inc. 1625 K St. NW, 11th Floor Washington, DC 20006 (202) 728-0727 www.innonet.org Ehren Reed: ereed@innonet.org www.innonet.org/services/LSC_Train