Resources for State Migrant Directors

1. Resources for State Migrant Directors Webinar: Program Evaluation Toolkit August 9, 2012 Toolkit Series from the Office of Migrant Education

2. Toolkit Series • Comprehensive Needs Assessment (CNA) Toolkit • Service Delivery Plan (SDP) Toolkit • Program Evaluation Toolkit

3. Program Planning Process

4. Evaluation Toolkit • Third in a series of three documents • Purpose is to provide guidance and resources to state directors to conduct useful Program Evaluations (PE) • Draws upon good practices gleaned from Program Evaluation field • Aims to help fill gaps between available guidance and OME’s expectations for high-quality evaluation reports.

5. Evaluation Toolkit (cont.) Designed to help state directors align their Program Evaluation with the requirements of the Federal legislation and Guidance Adaptable to programs of different sizes Can be used as a “gauge” on how much outside expertise on Program Evaluation you may need or as a “guiding document” for your current program evaluator.

6. Key Features of the Toolkit Resource Document Step-by-step approach Flexible Tool – can use the whole thing or just certain sections Customizable tools and charts in appendices Includes questions for application Web and Word based document

7. Example Resources Online Documentation Code of Federal Regulations, Title 34, Section 200.83 http://ecfr.gpoaccess.gov/cgi/t/text/text-idx?c=ecfr&tpl=/ecfrbrowse/Title34/34tab_02.tpl The Code of Federal Regulations (CFR) is the codification of the general and permanent rules published in the Federal Register by the executive departments and agencies of the Federal government. Title 34, Section 200.83 specifies that it is the responsibility of the state education agency to implement projects through a Comprehensive Needs Assessment (CNA) and a comprehensive state plan for service delivery. Elementary and Secondary Education Act (ESEA), Section 1306 http://www2.ed.gov/policy/elsec/leg/esea02/index.html ESEA is the federal statute overseeing primary and secondary education. Section 1306 refers specifically to the CNA and Service Delivery Plan (SDP), as well as other authorized activities. Non-RegulatoryGuidance for Title I, Part C Education of Migratory Children http://www.ed.gov/programs/mep/mepguidance2010.doc The Guidance (found in the ESEA) is designed to help state education agencies and local operating agencies use migrant education program funds to develop and implement supplemental educational and support services to assist migratory children. Chapter IV of the document refers specifically to the procedures for conducting a CNA and the development of an SDP.

8. Example Tool CHECKLIST FOR STATE MIGRANT EDUCATION PROGRAM EVALUATION

9. Knowledge Check What is a Continuous Improvement Cycle? What is the purpose of the MEP Evaluation Toolkit? What flexible features of the Toolkit will be most useful to you? Please dial *1 on your telephone keypad to provide a brief response to any of the above questions.

10. Organization of the Toolkit • Section A: Introduction & Overview • Section B: Overview of Statutes, Regulations, and Non-Regulatory Guidance • Section C: Planning the Evaluation • Section D: Collecting Evaluation Data • Section E: Analyzing and Interpreting Data • Section F: Communicating Evaluation Findings • Section G: Using Evaluation Findings Preview: http://center.serve.org/nche/ome_toolkits/pe.html

11. Setting MPOs • MPOs are outcomes for the MEP to enable migrant students to achieve the state migrant student performance targets. • Focused • Detailed • Quantifiable • Clearly define what you would consider a "success" in meeting a particular need   • In addition, MPOs clearly define: • Which students will participate • What will happen in the program • What is expected to happen as a result of participation in the migrant program • In what time frame this will occur

12. Setting MPOs From 2013-2015, students who parents attended at least 50% of the reading and homework support workshops will perform ,on average, 10 percentage points higher on the state reading test than children of non-participants All students will reach high standards, at a minimum attaining proficiency or better, in reading/language arts In grades 3 & 8, the percentage of migrant students attaining proficiency or advanced in reading & language arts was 73.2% compared to the range for all students of 86.7%. Strategy 1.1: Provide parents of migrant students with a menu of support services to help them support their child’s academic accomplishments

13. Establishing Evaluation Questions Strategy: Provide parents of migrant students with a menu of support services to help them support their child’s academic accomplishments. MPO: From 2013-2015, students who parents attended at least 50% of the reading and homework support workshops will perform, on average, 10 percentage points higher on the state reading test than children of non-participants What are some relevant Implementation Questions? To what extent were programs delivered as intended? What are some relevant Results Questions? How much change occurred as a result of strategy?

14. Planning Tools

15. Knowledge Check What is a Measurable Program Outcome (MPO)? What are specific components of a strong MPO? What is a implementation question? What is a results question? Please dial *1 on your telephone keypad to provide a brief response to any of the above questions.

16. Analyzing & Interpreting Data • Gives overview of main types of data and statistical analyses • Provides 2 Key Appendices: • Using Microsoft Excel to analyze Quantitative data • Using Microsoft Access to analyze Qualitative data • Meant as a primer and resource • The concepts that underlie really high quality data analysis take expertise and training and just a single topic has volumes

17. Communicating & Using Findings • What are aspects of effective Evaluation Reports? Knowledge check: • What detracts or takes away from an effective Report? Please dial *1 on your telephone keypad to provide a brief response to any of the above questions.

18. Communicating & Using Findings • Communicating Findings • Pointers • Reflection questions • Online resources • Using Findings • Protocol for using findings in Continuous Improvement Cycle • Online resources

19. Knowledge Check With whom should you share the MEP evaluation report? What are aspects of effective evaluation reports? How can an evaluation report be used as part of the continuous improvement cycle? Please dial *1 on your telephone keypad to provide a brief response to any of the above questions.

20. Follow Up • Finalized document will be available on the RESULTS website • Questions can be directed to Irene Harwarth: Irene.Harwarth@ed.gov