Evaluating professional development

1. Evaluating professional development Jan Coleman 26 September, 2006

2. Basic evaluation questions These are: • oWhat are you trying to evaluate and why? • o What information do you need to make valid and reliable judgments? • o What sources should you tap for this information? • o How will you gather the information you need? • o How will you analyse the information you gather? • o How will you use the information? (Hall, C. 2002).

3. The Kirkpatrick Model This model has four key stages: • Level 1 - Reaction • o How well did the participants like the programme? • Level 2 - Learning • o What principles, facts and techniques were learned? • Level 3 - Behaviour • o What changes in job behaviour results from the programme? • Level 4 - Results • o What were the tangible results of the programme in terms of cost, quality etc?

4. Kraiger’s Model

5. Guskey’s Model Level 1 • Participants’ reactions • How do the participants perceive the course? Level 2 • Participants learning • Did participants acquire the intended knowledge and skills? Level 3 • Organisational Support and Change • Did the organisation, its characteristics, culture etc support and encourage or hinder and prevent changes as a result of the professional development.

6. Guskey’s model continued Level 4 • Participant’s use of new knowledge and skills • Did the new knowledge and skills the participants learned make a difference to their professional practice? Level 5 • Student learning outcomes • What was the impact on students? Is the impact on them positive?

7. Levels of Professional Development • At each level we must consider: a. What questions will be addressed? b. How will the information be gathered? c. How will the information be recorded? d. How will the information be used?

8. Ask…. What do I want them to learn ? What evidence will I accept to prove they have done this? What knowledge, skills and support do they need to achieve this?

9. Focus on the students Ask the tough questions regularly: Who is not learning? Why? What can we do about it? Avoiding deficit thinking - blaming the students, school culture… Focus on the results!

10. Affective Evidence: • We might seek to gather evidence about students • More positive attitude to the subject, • Higher levels of student engagement • Psychomotor skills. • Students’ability to perform skills and behaviours • Higher level thinking, a workshop skill using tools, cooperative learning skills…

11. Consider: There is a difference between evidence and proof. If you have evidence of change is this evidence the professional development is successful? Is it evaluation or documentation? What is evidence?

12. Evidence of student achievement Evidence can be at 3 levels: Cognitive – student performance and achievement Affective (attitudes and dispositions) Skill and Behaviour

13. Sources of information • Brainstorm the possibilities • Quantitative v qualitative • Inspiration • Software for Learning

14. Tools for evaluation • Post box • Questionaire • Discussion groups • Surveys • Observation • Other???

15. Follow up Follow up is critical in re-enforcing the learning. Teachers must see results quickly (preferably 2 weeks) Failure to see results will result in the teacher reverting to previous methods.

16. Teachers will be no different from students! Students are motivated when they are achieving success but will avoid learning when there is no success! Things can get worse before they get better and teachers need to be coached and encouraged through the bumps. Apply the 3 N rule Nurture-Nudge-Nag!

17. Recognise that change is an individual and organisational process. Both must be addressed to be successful!