Evaluation and Reporting extension activities

1. Evaluation and Reporting extension activities Jeanette Long Ag Consulting Co

2. Trials – why bother? • R, D or E?

3. Why evaluate??

4. Why Evaluate? • Measure practice change • Ensure accountability • Measure cost to benefit • Continuously improve • Account for unintended effects • Report to funding body

5. Evaluation starts at the beginning of the project!

6. Evaluation Steps • Clarify the evaluation request • Focus the evaluation – program logic • Identify the information you will need • Identify information sources and methods • Decide on methods of analysis and reporting • Prepare a timeline for the evaluation – its not all at the end

7. Program logic Achievement of the desired outcome Requires a change in practice – what people do Which requires appropriate levels of Motivation, attitudes, knowledge, and skills Created by activities that engage people and create technologies (trials, field days, farm walks, case studies, communications, new products etc) Which require resources (money, time, expertise, infrastructure, human resources)

8. Types of data • Quantitative – where responses can be easily measured • “like it” scale 1-5, numbers of people etc • Qualitative – responses from open ended questions • What changes did you make in your stubble management as a result of attending the workshop?

9. Collecting the data • Numbers of people, hits on website etc • Surveys – happy sheets • Dart boards • Facilitated discussion • Survey monkey • Case studies • Observation • Stories of significant change • Keypads • Narratives

10. Golden rules of questions • Each question has ONE request in it • Use open questions • Ask exactly what you need to know and no more • Ask about attitudes, beliefs, behaviour and motivation • Avoid testing knowledge • Test your questions before using them

11. To what extent have you learnt new things relevant to your business ? Overall how beneficial has the day been for you? How likely are you to take some action from what you saw? How well did the process work for you?

12. Multiple choice questions • As a result of your participation in the field days for the BWY project, what actions have you taken with respect to BWY? • Followed up options with my consultant • Commenced IPM • Changed variety • Use seed dressing • Nothing – don’t have the resources at this stage • Other: Practice Change question

13. Narrative • Who – Farmer X in district Y farms X • Activity – he attended … • Reaction – was impressed by …asked lots of questions • Action – what he/she did (practice change) • Impact – what impact had on business and what next

14. Observations watching and listening • What is the goal • Who is involved • Activities • Actions • Language • Feelings Be aware of your own bias’s

15. ORID questioning approachFocused conversation • Objective – facts • What did we do today? • Reflective /reactive – emotions, feelings • What surprised you? How did you feel? • Interpretive – value, meaning, purpose • What were the main messages? • Decisional – future steps • What will you apply on your farm?

16. M & E data collection Data analysis Develop Program logic and monitoring & evaluation plan Summarise against performance measures Report against plan