Senior Corps Annual Assessments: The Basics

Senior Corps Annual Assessments: The Basics July 20, 2010 WV Faces of Leadership Senior Corps pre-conference

Before we begin consider this question… “What impactdoes your Senior Corps Program have on your communities?

Today’s Discussion • What is the annual assessment? • Where do we begin? • Planning Exercise • Implementation tips • How do we use the results?

What is the annual assessment? • § 2551.23 (Sponsor's Responsibilities) • (j) Annually assess the accomplishments and impact of the project on the identified needs and problems of the client population in the community Source: Code of Federal Regulations 45 §2551.23(j) WV Senior Corps Programs must submit Assessment information/ results with 1st year of 3 year renewal application.

What is the annual assessment? • Should involve key stakeholders • Should complement (without duplicating) current work plan efforts • Can make use of existing data • Should inform future efforts • Format and approach are up to you! Source: “Reference Guide to Accompany the Senior Corps Fiscal Year 2009 Field Guidance”

Many types of assessments… Program Cost/ Efficiency Program Outcome/Impact Program Process/Implementation Program Design/Theory Assessment of Need for the Program

The annual assessment is not.. • Satisfaction survey of volunteers or advisory council • Annual Report • Community assessment conducted by an outside organization • Summary list of activities

Why do an annual assessment? • Examine aspects of your project not necessarily addressed by work plans • Focus moredeeplyon key aspects of your project • Serves as an avenue for continuous improvement • Information on program impact = stakeholder support and funding opportunities

Getting Started • Involve Stakeholders! • Before choosing your focus area • Input will strengthen your approach • Ensures buy-in and commitment to take action on results • Failure to do so: • May alienate key stakeholders • Focus on the wrong questions/waste time

First questions to ask • How do we know our project is meeting critical community needs? • What kind of information can improve our project? • Where are the gaps in our current system of performance measurement? • What ongoing challenges can we glean from available data? • How do we keep the process manageable?

Example: Disaster Preparedness Question: Are there important impacts of our program not being captured in our work plan? Players: Project staff, advisory council, volunteer station manager and staff, volunteers, beneficiaries Data: Review outcome surveys, interview volunteers and beneficiaries to identify additional outcomes. Reporting: List of players, CNCS Use: Revamp outcome measurement to capture significant outcomes. Use new results to promote program and apply for funding.

Exercise: Developing a plan20 minutes • Use the worksheet to identify a possible focus area for your annual assessment. • Fill out all six steps to develop a plan for the assessment. • Compare and discuss your plan with a neighbor. • Debrief on the exercise.

Implementation Tips • Data Collection • Surveys • Some specific issues • Other ways to strengthen your assessment

Methods of Data Collection Logs & Tally Sheets Survey Existing Data Interview Rubric Focus Group Info collected by agency as part of operations (student grades, crime stats, etc.) Documents participation or things completed You get to define the questions Must choose wording carefully Not always available in desired format Does NOT document impacts/outcomes Detailed scale to assess written material or behavior Moderated discussion with small group Ask questions via phone or face-to-face Does NOT generate statistical data Requires training; can be time-consuming Not easy to develop, requires training to use Good for in-depth exploration of issues

Ensuring Quality Data • Design good instruments • Minimize bias • Pilot test instruments • Ensure representative data • Train data collectors • Clear/Realistic deadlines

Survey Exercise Match the issues/ common survey errors with specific examples

Survey Tools • http://nationalserviceresources.org • STAR (Don Pratt) • Survey Research Methods (Fowler) • Evaluation (Rossi) • Online Survey tools: • Zoomerang • Survey Monkey

Sharing your results • Data must be digested/translated into meaningful terms for your audience • Ask trusted advisors to review • Connect the dots for your stakeholders: • Implications of the results? • How can we build on this data? • Follow through! You (& stakeholders) felt the question was important-- be prepared to take action.

Some final tips • Always consider the counterfactual: what would ‘X’ look like if our program didn’t exist? • Evidence Based Practice • Describe your impact in the context of other contributing factors/ gaps • Visualizing/Sharing the data is half the battle: • Edward Tufte: Visual Display of Information • PP Alternatives: Prezi, Open Office Impress

For more information… • Contact CNCS State Office for further assistance with annual assessment • http://nationalserviceresources.org • Project STAR: 1-800-548-3656 • http://evidencebasedprograms.org

Thank You! Questions?

Senior Corps Annual Assessments: The Basics