Scaling Up: Lessons for Fundações

1. Scaling Up: Lessons for Fundações Dr. Richard Kohl, Leading and Learning for Large Scale Change Sao Paolo, Brazil 28 September 2012

2. Summary of the Presentation • Successful large scale implementation as important as new efficacious interventions • Successful scaling up usually not spontaneous • Chances for successful scaling up can be increased by pro-active strategic management • Requires having • a clear scaling up strategy identifying large scale delivery system and funding • understanding intervention does not exist apart from the political, social context and delivery system • Explicitly managing political process (advocacy) and implementation (system strengthening and governance )

3. Scaling up historically taken for granted: “Build it and they will come” • Key is the intervention itself, emphasis on technical content vs. process and delivery • Proof of effectiveness is sufficient • Scaling up assumed to be spontaneous • Activities limited to Dissemination, Research Utilization, or RIP; peer-reviewed journals, conferences, dissemination workshops

4. Spontaneous Scaling Up based on information alone is empirically false • Ignores ALL of new neuroscience and behavioral economics research on how decision making happens • Reinforced by foundation attitudes/incentives • Limited resources • Institutional incentives to be innovative, leveraging, and catalytic (the reactant is?) • Strong ties (staffing) from universities; research mentality – Academic culture of publishing, “theory of change” • Result is neither no scaling, policy adoption only, or scaling (quantity) without quality and impact

7. New Approach to Scaling Up • Recognizes the greater importance of politics and implementation/delivery systems capabilities and capacities i.e. & org. culture (5/80%, vs. 10/10%) • Develops pro-active strategies for each issue; strategic management of scaling up process • New innovations in delivery mechanisms, systems, quality; Supply, Demand and Delivery • Technical support for either • growth (expansion) to create new delivery mechanisms • System strengthening of existing delivery mechanisms • Monitoring, learning and feedback of delivery at scale

8. Two Key Concepts • Capacity(Reach or Scale) – ability of an organization to delivery and implement the programmeat the targeted or desired scale • Capability: ability of an organization to delivery and implement the program according to the way it was designed (fidelity), with quality achieving the expected impact • Delivery and Implementation Organizations have to have both Capacity and Capability to achieve the scaling up goals

9. Definitions of Scaling Up (2) • Scaling up Organizational Delivery and Implementation Capacityto achieve greater reach and deliver additional contents • Expansion: creating capacity from zero, pilot organization gets bigger • Replication: hand off from one organization to another, usually grafting onto an organization that already has large scale delivery capacity • Collaboration: partnership between one or more organizations to leverage either reach in specific places or capacity in specific content

10. Defining Successful Scaling Up • Policy Adoption: integrated into policy/strategy • Reach: the scale to be attained, number of people, locations, demographic, % coverage • Cost and Efficiency: meets a budget/financial constraints • Impact: impact on key outputs and outcomes • Sustainability: political, financial and institutional • Equity: reaching the hardest to reach; marginalized, poor indigenous, and rural areas

11. Six Elements of Scaling Up • Intervention: proven or promising model/program • Evidence: demonstrating effectiveness, efficient, compatibility, and robustness and relevance • Advocacy: convincing institutions to adopt, fund/resource and implement the model at large scale • Modifying and Strengthening Organizations: • Creating the capacity and/or capabilities (HR, infrastructure, systems and supply) to implement at scale and • Alignment of rules/procedures, incentives and org. culture • Implementation at Scale: • Coordination and cooperation among multiple stakeholders • Monitoring and feedback for fidelity, qualityand adaptation • Sustainability: political, financial and institutional

12. The 8 Common Breakdowns in Scaling Up • Missing Key Invisible Elements of the Model • Lack of Generalizability • M&E not Designed for Scaling • Lack of Large Scale Delivery Capacity and Capabilities/Compatibility • Limited, Narrow Advocacy • Training ≠ Capability Building • No Provision for Monitoring and Feedback • Dropping the Baton: Resources and Responsibility for Going to Scale

13. Missing Elements of the Model • Mistake: neglect key invisible elements • Focus only on technical contents; drugs, equipment, procedures • Ignore preparations, preconditions, processes and follow-up • Favor technical elements over tacit and intangibles • Solution: • Identify intangible, tacit elements at the pilot phase • Ensure use of qualitative methods in pilot design, process tracing • Comprehensive documentation of the model

14. Pilot Design, OR/M&E not designed for Scaling • Mistake: measure impact/efficacy • Ignores role of social environment and organizational context in pilot phase • For scaling, not what works, but what works where and why (use Realistic Evaluation approach) • Ignores that information for policy makers not the same as information for research • Assumes whole model will go to scale as is • Solution: • Identify and test key social/org. assumptions at the pilot phase through M&E for scaling up • Design into pilot/OR anticipated variations in social environment and organizational context • Measure cost and impact and disaggregate by components • M&E for advocay Scaling Up March 2010 Chalkboard Project

15. Lack of large scale delivery system: capacity and capability • Mistake: • Large scale delivery system not identified up front and there exists no system that can deliver at scale (lack of CAPACITY) • no attempt to ensure compatibility with large scale systems (lack of CAPABILITY) • Solutions: • Clearly identify goals for scale (who/where) and who the large scale implementer will be up front • Identify capabilities needed for the model • Design the model to be compatible with the implementer’s capabilities and capacities • Anticipate needed capabilities and capacities building

16. Limited and Narrow Advocacy • Mistake: • Believe impact results necessary and sufficient • Ignore political considerations of policy makers • Exclusive focus on policy champions (patrons) • Solution: • Identify decision makers, stakeholders, their interests • Develop messaging & evidence to address interests • Build advocacy into pilot design and evaluation • Organize broad-based multi-stakeholder coalition across party lines and beyond usual suspects • Extend advocacy beyond adoption and strategy to programming, funding and implementation

17. No Matter What the Problem, Training is the Solution • Mistake: Ignore complete set of capabilities needed for successful implementation • aligning incentives • Ensuring adequate financial and esp. human resources, • infrastructure, supply, logistics and systems, • organizational culture; rules, regulations and procedures • Adding programs to systems w/o adding capacity • Training is short-term, not institutionalized • Solution: Comprehensive capability building for entire system with dedicated MT resources

18. Modifying and Strengthening Organizations POLICIES/REGULATIONS RESOURCES & INFRASTRUCTURE SYSTEMS Capacity to Build Capacity INCENTIVES(Internal Politics) Skills ORG. CULTURE

19. No Monitoring of Performance and Fidelity at Scale • Mistake: model will be implemented as designed with quality and fidelity • Key components are often dropped, especially the more tacit, intangible or process oriented • Emphasis on quantitative targets and achievements, outputs not outcomes • Tensions between dilution or distortions of the model versus necessary adaptations • Solution • Principles vs. procedures, local adaptation • Build in a monitoring system to ensure fidelity and quality, learning and adaptation

20. Passing or Dropping the Baton: The Missing Link? • Developing a Promising Innovation • Generating Evidence of Efficacy (Internal Validity) • Generating Evidence of Effectiveness and Generalizability (External Validity) • Dissemination • Adoption into Strategies < Policies < Programs < Funding < Buy-In in District & Front-Line • Creating the Pre-Conditions for Large Scale Implementation: Training, Capability Building, Incentives, Org. Culture, Coordination etc. • Implementation at Large Scale • Ensuring Impact on Key Outcomes (monitoring) • Sustainabiility

21. Summary (1) • Build scaling up into discovery, design and pilot testing from the very beginning • Identify processes and intangibles as part of model • Test for generalizability and robustness (social and organizational) • Disaggregated evidence on cost and effectiveness by component, able to simplify • Identify capabilities needed and cost • Identify large scale delivery system with capabilities and cost constraints up front and ensure alignment • Generate evidence for advocacy in pilot design

22. Summary (2) • Advocacy • beyond dissemination and RIP, champions to broad-based advocacy (coalitions) for adoption , funding and implementation (sustainability) • Anticipate political and institutional opposition • Capability building beyond training to aligning entire system (health, education, etc) • Coordination and monitoring for fidelity and adaptation to local circumstances • Responsibility and resources for going to scale