Mary Beth Loucks-Sorrell

1. National Assembly on School-Based Health Care Outcome Training Session Mary Beth Loucks-Sorrell

2. The language of outcomes… goal outcome output milestone targets result measurable short term change objective impact differences long term change achievements benchmark indicator input benefits

3. Key Words • Accountability • Compliance • Monitoring • Reporting • Evaluation

4. Results Mindset For Today • Performance • Success • Supporting • Information Use • Learning & Verification (Accountability) (Compliance) (Monitoring) (Reporting) (Evaluation)

5. Result Thinking How you approach work, not a component

6. Historic Shift Occurring Investor Funder • distributes funds to the types of organizations that align with Foundation mission • solicits and grades proposals based on alignment with guidelines, completeness and clarity • sees the proposal as the predictor of project success • forgets what you promise and adds demerits for late reports • distributes funds to those that clearly contribute to foundation defined results • asks three questions: • What results are we buying? • What are the chances we’ll get them? • Is this the best use of our dollars? • knows there is a difference between what you write to get the money and how you plan to use it. • remembers what you promised, and is focused on their return on investment

7. Funder Investor Historic Shift Occurring $25,000 to XYZ Program: Organize weekly mentoring sessions for up to 500 low-income 7th and 8th grade students in 3 separate middle schools for a 6 month period $25,000 to XYZ Program: 325 of the 500 students we mentor will be at or above grade level in reading at term end.

8. Three Questions for You • How do you define success – the results of your services? • How do you know for sure when success has been achieved? • How do you know, throughout your program, that you have enough time and money left to achieve the success you have defined?

9. Lace up the Boots Outcomes matter most on the “shop floor” It took us 30 years to make it simple! Start with a single program Leader must be involved

10. How our Foundation Clients Think • What project results are you committed to achieving — meaning outcomes from your efforts? • How will you know when your results have been achieved? What information or evidence will you use to verify success? • What are the critical project steps and participant milestones you will use to manage to success?

11. Simple Terms we use, Choose your own Targets: • Specific changes in core participant behavior, condition or status that defines ‘success’ for the program • Tangible, verifiable, and within the influence of an implementor Milestones: • Interim behaviors, condition or status that define progress toward meeting the target.

12. Where to aim I have changed! I see the difference I keep using it It worked I remember I tried it • I like it I got it I came Start

13. Creating Targets 4 Steps • Identify the change in behavior you seek for your core participants • Specify the degree of change you consider a success • Estimate how many participants will be impacted or changed and by when • Express your result in a structured statement

14. Step 1Identify the kind of change you seek for your core participants Core Participants Your Program Change you Seek • Underachieving students • Children at risk for disease • Uninformed policy makers • Students complete high school • Children at risk for disease get health needs met • Policies support access to healthcare

15. Step 2Specify how much change is required for you to consider it a success

16. Step 2Specify how much change is required for you to consider it a success • Examples: • Students that complete high school on time and have the financial, social and academic achievement to enter post secondary education • Children with asthma, high blood pressure, or diabetes learn to manage their disease correctly • Policy change that supports increasing school based health clinics to one per county

17. Step 3 Estimate how many of your participants will change and by when 50 Where to look: • Past experience with other participants • Level of difficulty of participant • New and existing participants Make your projection!

18. Step 4 Structuring your target statement • Single Focus: • By December 31, 2012, 350 of the 500 children who have been using the E.R. to manage their chronic disease symptoms, will instead manage those symptoms successfully at home for at least 9 months after receiving support from a school based health clinic. • Individualized: • 50 of 100 schools enrolled in our advocacy training program will launch at least one successful youth led policy effort to removes a barrier to improved health by September, 30, 2012. • Menu Focus: • Of the 250 teenagers enrolled in Grades 11-12 at Bedford Valley High School,195 will achieve at least 2 of the following by May 31, 2012: • Graduate with their class • Successfully complete pre-college testing • Submit required financial aid documentation/applications • Apply for admittance to 3 or more colleges or universities

19. How our Foundation Clients Think • What project results are you committed to achieving — meaning outcomes from your efforts? • How will you know when your results have been achieved? What information or evidence will you use to verify success? • What are the critical project steps and participant milestones you will use to manage to success?

20. Verifying Results Verification Sources • Existing data or measuring instruments • Easy to obtain documents • Observations and reports by others OR in some cases self-reported behaviors • Multiple weak sources can create a strong verification

21. How our Foundation Clients Think • What project results are you committed to achieving — meaning outcomes from your efforts? • How will you know when your results have been achieved? What information or evidence will you use to verify success? • What are the critical project steps and participant milestones you will use to manage to success? Will you have enough time and dollars to finish the job?

22. Answer is in the Room Make mistakes as fast as you can mistakes mistakesmistakes

23. Milestones – tracking to success Consider this basic flow of participant milestones (M) that predict success: M1 Participants show up and commit to the program M2 Participants engage and begin to benefit M3 Participants demonstrate initial success M4 Participants demonstrate stronger success Target: Participants achieve the program target

24. Milestones – Tracking to Success Example: A youth led policy and advocacy training initiative that focused on removing barriers to access. M1 Sign up for program and set policy goals M2 Demonstrate new skills & use for 1st time within 30 days M3 Achieve at least one short-term goal within 3 months M4 Achieve all short-term goals and at least 1 long-term goal within 6 months Result: Engage a minimum of 50 youth who successfully engage in policy and advocacy efforts with a minimum of 5 actual school level or community level policy changes attributed to their work.

25. Milestones – what makes good ones? Common Elements: • Reflect key achievements as a result of the major activities being undertaken • Happen over time – not simultaneously • Reflects a ‘response’ or achievement progress in regular intervals • Are time bound • Defines all key accomplishment that are anticipated prior to the result being achieved for the time period

26. Putting it all Together Senior Citizen Volunteer Program Day 7 Day 14 Day 21 Day 30 Day 120

27. Just another evaluation framework - right? • Evaluation Framework • External to program • Focus: Assessment of results • Understand logic of program delivery • Establish measurable objectives • Result Framework • Imbedded in program • Focus: Improving results before assessment • Understand predictive milestones • Define clear results and targets • Considerations • Standard of evidence • Cost and timeliness • Effect on energy • Timing of decision

28. Logic Models: Lost in the translation exhausted?

29. What goes in What comes out Missions/Beliefs Programs/Services Goals Work Plans Organization Job Description Budgets Data Base Strategic Planning Financial Audits Evaluation Core Know-How Result Cards Targets Milestones Key Persons Result Description Accounted Costs Data Use Strategic Mapping Energy Audits Rapid Assessments

30. Capacity Building Capacity Building efforts are intended to strengthen the efficiency or effectiveness of organizations to achieve results for those they serve

31. New Capacity (New Model, Strategic Plan, Programmatic Assessment, Board Training, New Software, Fund Development, etc.) Capacity Building and Planning Result Trailfor Organizations & TA Providers Obtains new capacity Identifies & updates systems, policies, materials, practices Implements change & demonstrates ability to successfully use new capacity Uses new capacity and confirm they are on track to achieve organizational results Realizes reduced costs or increased revenue (increased efficiency) Improves results for those served (increased effectiveness)

32. New Capacity (New Model, Strategic Plan, Programmatic Assessment, Board Training, New Software, Fund Development, etc.) Capacity Building and Planning Result Trailfor State Associations of SBHC Leadership identified School District agrees/approves creation of clinic Acquire Equipment and Space Health clinic opens and begins to address needs Students get immediate health care needs met Clinic confirms improved student health outcomes

33. Convening, Advocacy, Research & Dissemination Piloting Best Practice Integration Systems Change Result Path for Policy, Advocacy, Research Practice Change Policy Change Stakeholders agree to address issue Best practices defined Create action plan Best practices piloted Propose changes to decision makers Best practices defined and packaged Changes agreed to by Decision makers Practitioners agree to adopt best practice Practitioners integrate best practices Changes made in policies or resource allocations Long Term Result Improved results for populations served (foundation results)

34. How does applying a result framework impact board development and engagement? • Clarity allows you to communicate more effectively what the organization is trying to achieve • Helps to identify who/what skills are needed at this stage of development • Energy is higher when targets are defined • Easier for an organization to determine when to say “No”

35. Set Your Board Recruitment And Engagement Targets • Focusing on what you would like to achieve in the next 2 years, what skill sets are you missing from the board? • What are the tasks you can give to the three most effective board members currently on your board to achieve success? • What are the first three things you will do when you get back to your community to achieve the success you outlined earlier?