1 / 22

A Tool for Designing Evaluations of Paradigm Shifts in Complex System Interventions

A Tool for Designing Evaluations of Paradigm Shifts in Complex System Interventions. Beverly Parsons InSites www.insites.org bparsons@insites.org. Marah Moore I2i Institute www.insites.org marah@i2i-institute.com. Pat Jessup InSites www.insites.org pjessup@insites.org.

inge
Download Presentation

A Tool for Designing Evaluations of Paradigm Shifts in Complex System Interventions

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. A Tool for Designing Evaluations of Paradigm Shifts in Complex System Interventions Beverly Parsons InSiteswww.insites.org bparsons@insites.org Marah Moore I2i Institutewww.insites.org marah@i2i-institute.com Pat Jessup InSiteswww.insites.org pjessup@insites.org Handouts and PowerPoint presentation available at: www.insites.org/AEA2010

  2. Session Objective To build capacity of evaluators to design evaluations of interventions in complex systems

  3. Four Phases of Evaluation Design Evaluation Shape Practice Collect Data MakeMeaning from Data

  4. Tool’s Role in Evaluation Design Tool represents the intervention’s theory of change and relationship of intervention to larger system/situation Evaluation designers also need to know: Whether evaluation is part of intervention (internal) or is separate from intervention (external) If evaluation is part of intervention, does it participate through Evaluation processes and/or Evaluation findings

  5. Session Agenda Introduction to tool for designing evaluations of interventions in complex systems Work in small groups to begin tool development Reconvene to discuss completion of tool and its use

  6. The Tool

  7. Theory of Change in Paradigms, Structures, and Conditions of Complex Systems Example from Cross-Site Evaluation of Quality ImprovementCenter on Early Childhood

  8. Tool Characteristics Tool focuses on deep patterns of change in systems (i.e., changes in structures and paradigms) Tool recognizes multiple system dynamics at play in complex systems

  9. Differences from Typical Logic Model Will almost always depict more aspects of the system than just the intervention Depicts phases of change over time with variation in rates and patterns across components of the system Likely to depict the intervention through a different lens

  10. Background Concepts

  11. What is a System? A group of interacting, interrelated, and interdependent elements forming a complex whole A configuration of parts connected and joined together by a web of relationships The whole is greater than (different than) the sum of its parts

  12. The Systems Iceberg What is happening now? Events and Behaviors How do patterns play out over time and space? Patterns StructuresParadigms Conditions What are the drivers? How are they related?

  13. Deep System Change Deep system change involves shifts in paradigms and structures of system boundaries, relationships, perspectives, and dynamics over time and space. Change is evident in mid-level patterns through data analysis/synthesis. Change is evident in observable events and behaviors.

  14. Paradigm Shift: Risk to Protective Factors • Protective Factors • Parental resilience • Social connections • Concrete support in time of need • Knowledge of parenting and child development • Social and emotional competence of child • Risk Factors • Substance Abuse • Poverty • Stress

  15. Dynamics of a Social System and Its Context Context Unorganized (random) Low Agreement Self-Organizing (complex, adaptive) Perspectives Organized (simple, complicated) High Agreement High Predictability Low Predictability Relationships

  16. Group Task Succinctly describe the paradigm shift. Identify the ultimate beneficiary of the paradigm shift. Identify the likely key points of leverage to shift the system. Order the likely points of leverage along left side of page beginning with the ultimate beneficiaries of the system and those close by.

  17. Phases of Shift Baseline understanding of fundamentals and system dynamics Trying out interventions in system Tipping point to new fundamentals and system dynamics balance Sustainable adaptive balance of new fundamentals and system dynamics in a shifting context

  18. Theory of Change in Paradigms, Structures, and Conditions of Complex Systems Example from Cross-Site Evaluation of Quality ImprovementCenter on Early Childhood

  19. Other Key Evaluation Design Questions Is the evaluation internal or external to the intervention? What are the boundaries of the intervention? What are the focal points of the evaluation? How will the evaluation attend to the larger system (beyond the intervention)?

  20. Example of Evaluation Design

  21. QIC-EC Cross-Site Analysis Framework

  22. A Tool for Designing Evaluations of Paradigm Shifts in Complex System Interventions Beverly Parsons InSiteswww.insites.org bparsons@insites.org Marah Moore I2i Institutewww.insites.org marah@i2i-institute.com Pat Jessup InSiteswww.insites.org pjessup@insites.org Handouts and PowerPoint presentation available at: www.insites.org/AEA2010

More Related