Utilizing Kirkpatrick’s “Levels of Evaluation”

1. Utilizing Kirkpatrick’s “Levels of Evaluation”

2. Why Evaluate? • Evaluation is the engine the powers the vehicles of innovation and progress • Without evaluating the effectiveness of a product or a program improvements will never be made • Utilizing Kirkpatrick’s “four levels of evaluation” best ensures an instructor is meeting the educational needs of his or her students in an object-driven, constructivist classroom.

3. Setting • 6th Grade Social Studies Classroom in Virginia • Constructivist Classroom • Objective-driven : VA DOE Standards of Learning • United States History to 1865 • American Indians to the Civil War

4. Level 1 - Reaction • Reaction measures how those who participated in the program react to it • The instruction needs to be interesting and relevant so the learner will have a favorable reaction to it • Kirkpatrick believes positive reaction may not single-handedly ensure learning, but negative reaction almost certainly reduces the possibility of it occurring.

5. Classroom Example • Standard USI.9b • “the student will demonstrate knowledge of the causes, major events, and effects of the Civil War by explaining how the issues of states’ rights and slavery increased sectional tensions” (Virginia Department of Education, 2008) • Compromises on slavery in the western states

6. Negative Reaction • Too much info in too little space • Picture is faded and difficult to interpret • When a discussion question is posed there is not enough room for the learner to write his or her response

7. Positive Reaction • Considers good message design principles • Simplistic and structured • An adequate amount of information is presented • A later activity could focus on the map visualization

8. Level 2 - Learning • Learning is defined as the extent to which participants change attitudes, improve knowledge, and/or increase skill as a result of instruction • Lessons can focus on knowledge, skills, attitudes, or a combination of each.  In a school setting a combination of the three is the focus for most lesson plans.

9. Starting Point - Preassessments • Taken before each new unit of study • Not graded • Use of ActivExpressions • Results can be stored for future comparisons

10. Along the Way – Formative Assessment • “Checkmarks” • Immediate feedback • Homework, classroom activities, class discussions

11. At the End – Achievement Tests • The test should cover the concepts presented to the students • The test should be consistent with the standards set forth by the State Department of Education • Comparison of preassessments and achievement tests will show if learning took place

12. Level 3 - Behavior • Behavior is defined as the extent to which change in behavior has occurred because the participant during instruction • Difficult to measure behavior • The learning shown in the classroom does not guarantee the student has mastered the knowledge, skill, or attitude.

13. What Can the Teacher Do? • In order to create an ideal learning environment a teacher needs to create either an encouraging environment or a requiring environment.   • An encouraging environment is the teacher encouraging the student to use new knowledge, skills, and attitudes in the classroom.  • A requiring environment is the teacher monitoring and requiring the use of new knowledge, skills, and attitudes in the classroom

14. Classroom Example • Standard USI.8a • states “the student will demonstrate knowledge of westward expansion and reform in America from 1801 to 1861 by describing territorial expansion and how it affected the political map of the United States, with emphasis on the Louisiana Purchase, the Lewis and Clark expedition, and the acquisitions of Florida, Texas, Oregon, and California” (Virginia Department Of Education, 2008) • End of the Unit Project showing change in behavior

15. Western Territories Project

16. Western Territories Project

17. Western Territories Project

18. Level 4 - Results • Results is defined as the final results that occurred because the student attended instruction • Best evaluated by the experience of a student from one grade to the next • Many extraneous factors other than training can influence results outcomes so it’s difficult to validate the relationship between training and results • Student participation, parental expectations, and resource availability among other things

19. Results • The results level in a school setting is creating a better student who uses the knowledge, skills, and attitudes he or she has learned during instruction to be a productive member of society • The easiest way to measure results is whether or not a student graduates from high school or a higher level of education • Determining whether or not a student has become a productive member of society is so much harder to evaluate

20. Critiques of Kirkpatrick • Kaufman and Keller’s Expansion

21. Conclusion • Kirkpatrick’s “four levels of evaluation” best ensure an instructor is meeting the educational needs of his or her students in an object-driven, constructivist classroom. • If followed effectively each level will force the qualified instructor to reflect and determine what changes could lead to better performance by students at different stages of learning. • Experience can play an important role in the effectiveness of evaluation. • The most important attribute for an effective instructor is the ability to realize that evaluation of instruction has the biggest effect on the progress and success of the educational system.