Loading in 2 Seconds...
Loading in 2 Seconds...
Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.
Implementation and Fidelity within a Theory-Driven Framework Greg Roberts, PhD. Vaughn Gross Center & National Center for Instruction The University of Texas at Austin
General Sources • Theory-driven program evaluation • Theory types • Program modeling • Implementation theory and research • Innovation diffusion, knowledge utilization, NIRN, CBAM, etc. • Predicting & managing change • Measurement theory
Programs, Implementation, & Fidelity • Programs, initiatives, funding mechanisms • Adoption, implementation, use, evaluation • Fidelity • Coverage • Dosage • Preserving causal mechanism • Scaling up • Lipsey meta-analysis (1999)
Notes on Programs and Fidelity • Intended model: Implemented model • Normative theory (program model) • The “what” and “how” • Activities, outputs, outcomes, moderators • Causative theory (theory of change) • The “why” • Mediating factors
Treatment Theory Normative Treatment Evaluation Normative Theory Implementation Environment Theory Normative Implementation Environment Evaluation Outcome Theory Normative Outcome Evaluation Program Theory Impact Theory Impact Evaluation Causative Theory Intervening Mechanism Theory Intervening Mechanism Evaluation Generalization Theory Generalization Evaluation Theory-Driven Evaluation (Chen, 2004)
Program Models • Models can not substitute for reality • Pattern matching • Models can support: • Assessment of “evaluability” • Client needs identification • Program development, refinement, etc. • Monitoring and evaluation
Program Models • Models can change • Formative processes • Standards & protocols regulating change • Embedded flexibility • Innovation
Notes on Implementation • Models of systems change v. models of change inducement/management • Stage-like nature • Trends across theories • Change can be conceptualized at the individual, group, and social levels • Change is not an event • Change can not be mandated • Change can, at a very general level, be anticipated and managed
More Notes on Fidelity • Implementation and evaluating fidelity • Context for evaluating fidelity • Considerations for conceptualizing fidelity • Multilevel nature of many interventions • Level and intensity of measurement aligned with need • Capacity for monitoring fidelity • Burden of monitoring fidelity • Alignment with desired outcomes
More Notes on Fidelity • Tools for gathering fidelity data • Observation protocols • User logs • User self reports • Unobtrusive measures • Systemic indicators • Extant data
More Notes on Fidelity • Analysis considerations • Structural models and measurement models • Latent variable models • Item response theory • Data quality