Developing an Effective Evaluation to Check for Understanding

1. PARK Teachers Developing an Effective Evaluation to Check for Understanding Susan E. Schultz, Ph.D. Evaluation Consultant

2. PARK Teachers Developing an Effective Evaluation Plan Overview Part 1: Components of Program Development Part 2: Goals and Types of Evaluation Part 3: Creating an Effective Evaluation Plan

3. PARK Teachers PART 3: Creating an Effective Evaluation Plan

4. PARK Teachers Developing an Effective Evaluation Plan Pilot and Revise Assessments Use variety of Assessment Techniques Select Assessment Strategies Create an Evaluation Matrix Identify Program Goals

5. PARK Teachers Identify the program goals and essential question “What are the three key ideas or messages that I want all participants to know when they complete the program?” “What overarching question or statement brings all of your goals together?”

6. PARK Teachers Identify the program goals and essential question • The PARK program goals focused on pre-service teachers being able to: • Improve their geology content knowledge, • Engage in quality instructional practices, • Experience national park as a research education site, • Learn about educational resources in national parks, • Transfer their knowledge to their local community. • The essential question was, “How do you recognize evidence of geologic change in your environment?”

7. PARK Teachers Check for Understanding Task 3: Outcomes Part 1: Using your group’s essential question and program goals, write your group’s outcomes on a piece of poster paper. Part 2: Share your group’s outcomes with the whole group.

8. PARK Teachers Create an evaluation matrix based on program goals • An evaluation matrix is an outline of the evaluation plan. • The matrix includes the: • Program Outcomes, • Task to achieve the outcomes, and • Assessment strategies to measure the outcomes. • Let’s look at some specific examples from the PARK program on the next two slides.

9. PARK Teachers Sample of PARK Program Evaluation Matrix

10. PARK Teachers Sample of PARK Program Evaluation Matrix

11. PARK Teachers Check for Understanding Task 4: Evaluation Matrix Part 1: Complete your group’s evaluation matrix for your project A. Break down each program outcome and generate a list of tasks that need to be completed to achieve the specific outcome. B: Identify how you plan to measure or evaluate if you have achieved the outcome. Part 2: Present your evaluation matrix to the whole group.

12. PARK Teachers Select the Appropriate Assessment Strategies • It is imperative that you select the appropriate assessment strategy when developing your evaluation plan. • The type of assessment used must match the: • program outcomes and • type of knowledge being evaluated.

13. PARK Teachers Assessing Different Types of Knowledge • Essential to select the appropriate assessment instrument to evaluate different types of knowledge. • Most programs focus on a combination of content and performance-based knowledge. • Content Knowledge • Performance or Skill Knowledge 13

14. PARK Teachers Check for Understanding Task 5: Selecting Assessments Part 1: Select the Appropriate Assessments A. Identify how you plan to measure or evaluate if you have achieved the outcome. B: Think about the desired outcome and the type of knowledge being evaluated when selecting the assessment instrument. Part 2: Present your assessment selection to the whole group.

15. PARK Teachers Pilot and Revise Assessment Instruments Piloting and Revising the Assessment Instruments is Essential! The revision process helps you to: clarify what you are trying to assess, make revisions to other program components (content, instructional activities, or course planning)