1 / 13

Implementation and Fidelity within a Theory-Driven Framework

Implementation and Fidelity within a Theory-Driven Framework. Greg Roberts, PhD. Vaughn Gross Center & National Center for Instruction The University of Texas at Austin. General Sources. Theory-driven program evaluation Theory types Program modeling Implementation theory and research

alexia
Download Presentation

Implementation and Fidelity within a Theory-Driven Framework

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Implementation and Fidelity within a Theory-Driven Framework Greg Roberts, PhD. Vaughn Gross Center & National Center for Instruction The University of Texas at Austin

  2. General Sources • Theory-driven program evaluation • Theory types • Program modeling • Implementation theory and research • Innovation diffusion, knowledge utilization, NIRN, CBAM, etc. • Predicting & managing change • Measurement theory

  3. Programs, Implementation, & Fidelity • Programs, initiatives, funding mechanisms • Adoption, implementation, use, evaluation • Fidelity • Coverage • Dosage • Preserving causal mechanism • Scaling up • Lipsey meta-analysis (1999)

  4. Notes on Programs and Fidelity • Intended model: Implemented model • Normative theory (program model) • The “what” and “how” • Activities, outputs, outcomes, moderators • Causative theory (theory of change) • The “why” • Mediating factors

  5. Treatment Theory Normative Treatment Evaluation Normative Theory Implementation Environment Theory Normative Implementation Environment Evaluation Outcome Theory Normative Outcome Evaluation Program Theory Impact Theory Impact Evaluation Causative Theory Intervening Mechanism Theory Intervening Mechanism Evaluation Generalization Theory Generalization Evaluation Theory-Driven Evaluation (Chen, 2004)

  6. Program Models • Models can not substitute for reality • Pattern matching • Models can support: • Assessment of “evaluability” • Client needs identification • Program development, refinement, etc. • Monitoring and evaluation

  7. Program Models • Models can change • Formative processes • Standards & protocols regulating change • Embedded flexibility • Innovation

  8. Notes on Implementation • Models of systems change v. models of change inducement/management • Stage-like nature • Trends across theories • Change can be conceptualized at the individual, group, and social levels • Change is not an event • Change can not be mandated • Change can, at a very general level, be anticipated and managed

  9. More Notes on Fidelity • Implementation and evaluating fidelity • Context for evaluating fidelity • Considerations for conceptualizing fidelity • Multilevel nature of many interventions • Level and intensity of measurement aligned with need • Capacity for monitoring fidelity • Burden of monitoring fidelity • Alignment with desired outcomes

  10. More Notes on Fidelity • Tools for gathering fidelity data • Observation protocols • User logs • User self reports • Unobtrusive measures • Systemic indicators • Extant data

  11. More Notes on Fidelity • Analysis considerations • Structural models and measurement models • Latent variable models • Item response theory • Data quality

  12. Questions?

More Related