1 / 30

Partners in Evaluation

Partners in Evaluation. Co-constructing educational evaluation at the local level. Ruth Pritchard and Carol McSporran Presentation to ANZEA 2013. Nāu te rourou, nāku te rourou, ka ora te iwi. With your food basket and my food basket, people will thrive.

waseem
Download Presentation

Partners in Evaluation

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Partners in Evaluation Co-constructing educational evaluation at the local level Ruth Pritchard and Carol McSporran Presentation to ANZEA 2013

  2. Nāu te rourou, nāku te rourou, ka ora te iwi. With your food basket and my food basket, people will thrive. ‘Nga Kite WhaiMatauranga’ by Mihi Williams and Roimata Emery

  3. The Notion of Partnership Te Tiriti o Waitangi • - Partnership • - Protection • - Participation embedded in evaluation principles, standards and competencies

  4. Serious engagement with difference engagement with diversity and difference in evaluation is both substantive and a moral commitment (Hood, Hopson & Frierson, 2005, p.9) • Values-engaged approach evaluators must assume responsibility for explicating and justifying the values being advanced in their work, in ways that respect other values (Hall, Ahn & Greene, 2012, p. 206)

  5. Tukutuku: partnership and participation From national library and te papa archive collections

  6. Te Whare Runanga, Waitangi House

  7. Co-constructing the big picture ERO’s complementary evaluation Weaving on a 2-dimensional framework • Ko te tamaiti te putake o te kaupapa The child is the heart of the matter • Evaluation– complementary review

  8. Co-construction Lincoln and Guba (1985); Stake (1975; 1983) Morris (2002) • Multiple perspectives build towards ‘a commonly accepted reality’ • “Stakeholder involvement using the principles of social constructivism should lead to a more valid evaluation that empowers stakeholders and increases the likelihood of the utilization of results as stakeholder investment is greater than would otherwise be the case”(p. 51)

  9. Participatory evaluation as partnership(e.g. Patton, 2008, p. 224; Cousins & Earl, 1992; Cousins & Whitmore, 1998; Greene, 1988; Ayers, 1987). • Complex dynamic operating at three levels: individual, group, organisational (Smits & Champagne, 2008) • Comprising five interactive processes (Weaver & Cousins, 2004)

  10. Cousins & Whitmore, 1998; Cousins & Chouinard, 2012 “this shifting responsibility is a prerequisite for the success of an evaluation” Themessl-Huber & Grutsch, 2003, p. 98.

  11. Complementary evaluation(Mutch, 2012; Nevo, 1994, 2002) • Appreciative Inquiry approach (Cooperrider, 1982) • Deeper participation, fewer/primary users (Taut & Alkin, 2010) • Transparent processes and values • Complementary relationships

  12. Hopkins & West (2002)

  13. ERO’s approach Enacting partnership at the local level • contextually responsive design • negotiated values • dialogic basis for engagement • space for shared sense-making

  14. ERO (2013)

  15. Opportunities for weaving threads • Reciprocity of learning bridge the gap between theory and practice…and create opportunities for mutual learning” (MacLellan-Wright. et al., 2007, p. 102). • Enhancing validity the experiences and expertise of stake-holders enlarge and enrich the scope of the evaluation (Abma, 2000) acknowledged as ‘natural experts in their field’(Themessl-Huber & Grutsch, 2003, p. 94). • Evaluation capabilitybuilding (Compton, Baizerman & Stockdill, 2002) • Utilization process use (Greene, 1988; Patton, 1998) contributes to a learning phase that reinforces understandings and the ownership of results and, eventually, a greater sense of obligation to follow through on the results (Smits & Champagne, 2008, p. 429).

  16. Position of the evaluator putting ourselves into practice (Collins 1994) it is as much about who we are, and where we position ourselves in relation to others, as it is about what we do (ANZEA, 2011, p. 10) • a shift from the primary investigator and participant observer to becoming a facilitator (Garaway, 1995) • a “high tolerance for ambiguity” (Patton, 2011, p. 26) • cultural responsiveness with “careful attention to one’s place and perspective” (Letiecq & Bailey, 2004, p. 32)

  17. Repositioning evaluators to: • use diverse methods of inquiry • suspend judgement and sense-making • create time and space for participants to: - articulate - explore - co-construct knowledge - develop their evaluative capacity

  18. Challenges • Complex contexts and competing agendas - growth in reliance on evaluation - diversity in the sector - political implications

  19. Competence and understanding of practice/theory - moving from apprenticeship to a professional model - variability

  20. Evolving, dynamic process and expectations - high expectations for creativity, innovation and cultural responsiveness - we can be deliberate and responsive in the evaluation approaches we use and they can co-exist : participatory, empowerment, appreciative, transformative, collaborative, deliberately democratic, developmental, responsive, and utilization focused

  21. Enabling partnership - what seems to work? • Communication and information sharing • Inviting participation at different levels • Shared team and context understandings • Professional Practice Model (PPM) initiation

  22. Enabling partnership - what seems to work? (contd.) • Review Officers’ understanding of expected practices and principles for complementary practices:- clarity of roles and responsibilities - agreed evaluation framework - recognition of the contribution each party makes - respect for different perspectives (Mutch, 2012) • Synergy between internal and external evaluation – making connections at the local level

  23. Pūawaitanga o te Ringa - Fruits of our busy hands, 2001 Christchurch Central Library

  24. References Abma, T. A. (2000) .‘Stakeholder Conflict: A Case Study’, Evaluation and Program Planning, 23(2): 199–210. ANZEA (2011). Evaluator Competencies 2011 Retrieved from www.anzea.org.nz/.../05/110801_anzea_evaluator_competencies_final.pdf · Ayers, T. D. (1987). Stakeholders as partners in evaluation: A stakeholder-collaborative approach. Evaluation and Program Planning, 10, 263-211. Collins, M. (1994). Putting ourselves into practice: New prospects for program planning and evaluation. In M. Brown (Ed.), A collection of original essays on curriculum for the workplace, 94–131. Victoria, Australia: Deakin University Press. Compton, D. W., Baizerman, M., & Stockdill, S. H. (2002). The art, craft, and science of evaluation capacity building. New Directions for Program Evaluation, 93. Cooperrider, D & Srivastva, S. (1987). Appreciative Inquiry in organizational life. Research in Organizational Change and Development 1, 129-169. Cousins, J.B. & Whitmore, E. (1998). Framing participatory evaluation. In E. Whitmore (Ed.), Understanding and practicing participatory evaluation. New Directions for Evaluation, 80, 2-23. Cousins, J.B. & Chouinard, J.A. (2012). Participatory evaluation up close: An integration of research-based knowledge. Charlotte, NC: Information Age Publishing. Cousins, J. B., & Earl, L. M. (1992). The case for participatory evaluation. Educational Evaluation and Policy Analysis, 14, 397–418. ERO (2013). He PouTātaki:How ERO reviews early childhood services. Wellington, New Zealand: Education Review Office. ERO (2012). Statement of Intent 2012-2013. Wellington, New Zealand: Education Review Office. Garaway, G. B. (1995). Participatory practice. Studies in Educational Evaluation, 21,. 85-102, Greene, J.G. (1988). Stakeholder participation and utilization in program evaluation. Evaluation Review, 12, 91–116. Hall, J. N., Ahn, J. & Greene, J. C. (2012). Values engagement in evaluation: Ideas, illustrations, and implications. American Journal of Evaluation, 33, 195-207. Hawkins, P. (2012). Successful evaluation management: Engaging mind and spirit. The Canadian Journal of Program Evaluation, 25 (3) 27–36.

  25. Hood, S., Hopson, R. & Frierson, H. (2005). Introduction : this is where we stand S. Hood, R. Hopson, and H. Frierson (Eds.). The role of culture and cultural context: a mandate for inclusion, the discovery of truth and understanding in evaluative theory and practice . Greenwich, CT: IAP Hopkins, D., & West, M. (2002). Evaluation as school improvement. In D. Nevo (Ed.) School-based Evaluation: An International Perspective. Oxford, England: Elsevier Science. Letiecq, B. L. & Bailey, S. J. (2004). Evaluating from the outside. Evaluation Review, 28(4), 342-357 Lincoln, Y. & Guba, E. (1985). Naturalistic Inquiry. Thousand Oaks, CA.: Sage MacLellan-Wright, M.F., Patten, S., de la Cruz, A.M. & Flaherty, A. (2007). A participatory approach to the development of an evaluation framework: process, pitfalls, and payoffs. The Canadian Journal of Program Evaluation 22(1), 99–124. Morris, D.B. (2002). The inclusion of stakeholders in evaluation: benefits and drawbacks. The Canadian Journal of Program Evaluation, (17)2, 49-58. Mutch, C. (2012). Complementary evaluation: the development of a conceptual framework to integrate external and internal evaluation in the New Zealand school context. Policy Futures in Education, 10(5), 569-586. Nevo, D. (1994). Combining internal and external evaluation: A case for school-based evaluation. Studies in Educational Evaluation, 20(1), 87-98. Nevo, D. (2002). Dialogue evaluation: combining internal and external evaluation. In D. Nevo (Ed.) School-based Evaluation: An International Perspective. Oxford, England: Elsevier Science. Patton, M. Q. (1998) ‘Discovering Process Use’, Evaluation 4(2): 225–33. Patton, M. Q. (2008). Utilization-Focused Evaluation (4th ed.).Thousand Oaks, CA; Sage Publications Inc. Smits, P.A. & Champagne, F. (2008). An assessment of the theoretical underpinnings of practical participatory evaluation. American Journal of Evaluation 29, 427-442. Stake, R. E. (1975). Evaluating the Arts in Education. Colombus, OH: Merrill.  Stake, R. E. (1983). Program evaluation, particularly responsive evaluation. In G. F. Madaus, M. Scriven, & D. L. Stufflebeam (Eds.), Evaluation Models. Boston: Kluwer-Nihoff. Taut, S. & Aiken, M. C. (2010). The role of stakeholders in educational evaluation. Educationalevaluation: Contexts of Educational Evaluation, 629-635. Themessl-Huber, M.T. & Grutsch, M.A. (2003). The shifting locus of control in participatory evaluations. Evaluation 9, 92-111. Weaver, L., & Cousins, J. B. (2004). Unpacking the participatory process. Journal of MultiDisciplinary Evaluation 1, 19-40.

More Related