Innovations in planning evaluating system change ventures
Download
1 / 78

Innovations in Planning & Evaluating System Change Ventures - PowerPoint PPT Presentation


  • 94 Views
  • Uploaded on

Innovations in Planning & Evaluating System Change Ventures. Roles for System Dynamics Simulation Modeling. Bobby Milstein Syndemics Prevention Network Centers for Disease Control and Prevention [email protected] Diane Orenstein Division for Heart Disease and Stroke Prevention

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about ' Innovations in Planning & Evaluating System Change Ventures' - dylan-potter


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
Innovations in planning evaluating system change ventures
Innovations in Planning & Evaluating System Change Ventures

Roles for System Dynamics Simulation Modeling

Bobby Milstein

Syndemics Prevention Network

Centers for Disease Control and Prevention

[email protected]

Diane Orenstein

Division for Heart Disease and Stroke Prevention

Centers for Disease Control and Prevention

[email protected]

NCCDPHP Cross-Division Evaluation Network

Atlanta, GA

January 29, 2008


Framework for program evaluation
Framework for Program Evaluation

“Both a synthesis of existing evaluation practices

and a standard for further improvement.”

Left Unexamined…

  • Singular “program” as the unit of inquiry (N=1 organizational depth)

  • Dynamic aspects of program effectiveness (e.g., better-before-worse patterns of change)

  • Democratic aspects of public health work (e.g., alignment among multiple actors, including those who are not professionals and who may be pursuing other goals)

  • Evaluative aspects of planning(e.g., defining problems, setting priorities, developing options, selecting strategies)

Milstein B, Wetterall S, CDC Evaluation Working Group. Framework for program evaluation in public health. MMWR Recommendations and Reports 1999;48(RR-11):1-40. Available at <http://www.cdc.gov/mmwr/PDF/RR/RR4811.pdf>.


Imperatives for protecting health
Imperatives for Protecting Health

Typical Current State“Static view of problems that are studied in isolation”

Proposed Future State“Dynamic systems and syndemic approaches”

“Currently, application of complex systems theories or syndemic science to health protection challenges is in its infancy.”

-- Julie Gerberding, CDC Director

Gerberding JL. Protecting health: the new research imperative. Journal of the American Medical Association 2005;294(11):1403-1406.


Rationale for innovation
Rationale for Innovation

  • Enormity of the challenges (problems of greater scale, speed, diversity, novelty)

  • Appreciation for the effectiveness as well as the limits of narrowly-bounded approaches

  • Potential for comprehensive changes(global, multi-sectoral, infrastructural, intergenerational, root-causes)

  • Threat of policy resistance

  • Mismatch with conventional methods for planning/evaluating


Seeing beyond the probable
Seeing Beyond the Probable

“Most organizations plan around what is most likely. In so doing they reinforce what is, even though they want something very different.”

-- Clement Bezold

  • PossibleWhat may happen?

  • PlausibleWhat could happen?

  • ProbableWhat will likely happen?

  • PreferableWhat do we want to have happen?

Bezold C, Hancock T. An overview of the health futures field. Geneva: WHO Health Futures Consultation; 1983 July 19-23.


Public health systems science addresses navigational policy questions
Public Health Systems Science Addresses Navigational Policy Questions

2010

2025

2050

Where?

What?

17% increase

How?

Why?

Who?

Centers for Disease Control and Prevention. Health-related quality of life: prevalence data. National Center for Chronic Disease Prevention and Health Promotion, 2007. Accessed October 23, 2007 at <http://apps.nccd.cdc.gov/HRQOL/index.asp>.

Milstein B. Hygeia's constellation: navigating health futures in a dynamic and democratic world. Atlanta, GA: Centers for Disease Control and Prevention; Draft, 2007.


Broad dynamics of the health protection enterprise
Broad Dynamics of the Health Protection Enterprise

-

Health

B

Protection

Taking the Toll

Efforts

B

B

-

Responses

Prevalence of Vulnerability, Risk, or Disease

Obstacles

to Growth

R

Resources &

-

Resistance

Drivers of

R

Growth

Reinforcers

Broader Benefits

& Supporters

Prevalence of Vulnerability, Risk, or Disease

100%

Values for Health & Equity

Size of the Safer, Healthier

Population

PotentialThreats

0%

Time

The concepts and methods of policy evaluation must engage the basic features of this dynamic and democratic system


Serious challenges for planners and evaluators
Serious Challenges for Planners and Evaluators

  • Locating categorical disease or risk prevention programs within a broader system of health protection

  • Constructing credible knowledge without comparison/control groups

  • Differentiating questions that focus on attribution vs. contribution

  • Balancing trade-offs between short- and long-term effects

  • Avoiding the pitfalls of professonalism (e.g., over-specialization, arrogance, reinforcement of the status quo)

  • Harnessing the power of intersectoral and citizen-led public work

  • Defining standards and values for judgment

  • Others…






Essential elements for system change ventures limitations of conventional alternatives
Essential Elements for System Change VenturesLimitations of Conventional Alternatives


Essential elements for system change ventures limitations of conventional alternatives1
Essential Elements for System Change VenturesLimitations of Conventional Alternatives


Essential elements for system change ventures limitations of conventional alternatives2
Essential Elements for System Change VenturesLimitations of Conventional Alternatives


Essential elements for system change ventures limitations of conventional alternatives3
Essential Elements for System Change VenturesLimitations of Conventional Alternatives


Looking through the macroscope
Looking Through the Macroscope

“A symbolic instrument made of a number of methods and techniques borrowed from very different disciplines…The macroscope filters details and amplifies that which links things together. It is not used to make things larger or smaller but to observe what is at once too great, too slow, and too complex for our eyes.”

Can SD simulation models provide practical macroscopes for planning and evaluating health policy?

-- Joèl de Rosnay

Rosnay Jd. The macroscope: a book on the systems approach. Principia Cybernetica, 1997. <http://pespmc1.vub.ac.be/MACRBOOK.html


System dynamics was developed to address problems marked by dynamic complexity
System Dynamics Was Developed to Address Problems Marked By Dynamic Complexity

Origins

  • Jay Forrester, MIT, Industrial Dynamics, 1961 (“One of the seminal books of the last 20 years.” -- NY Times)

  • Public policy applications starting late 1960s

  • Population health applications starting mid-1970s

Good at Capturing

  • Differences between short- and long-term consequences of an action

  • Time delays (e.g., developmental period, time to detect, time to respond)

  • Accumulations (e.g., prevalences, resources, attitudes)

  • Behavioral feedback (e.g., reactions by various actors)

  • Nonlinear causal relationships (e.g., threshold effects, saturation effects)

  • Differences or inconsistencies in goals/values among stakeholders

Sterman JD. Business dynamics: systems thinking and modeling for a complex world. Boston, MA: Irwin McGraw-Hill, 2000.

Homer JB, Hirsch GB. System dynamics modeling for public health: background and opportunities. American Journal of Public Health 2006;96(3):452-458.


AJPH Systems Issue

Science Seminars and Professional Development Efforts

SD Identified as a Promising Methodology

Hygeia’s Constellation

CDC Evaluation Framework Recommends Logic Models

SDR 50thIssue

System Change Initiatives Encounter Limitations of Logic Models and Conventional Planning/Evaluation Methods

Syndemics Modeling*

Neighborhood Grantmaking Game

Diabetes Action Labs*

Fetal & Infant Health

Obesity Overthe Lifecourse*

CVH in Context*

Upstream-Downstream Dynamics

National Health Economics & Reform

Health System Transformation Game*

Milestones in the Recent Use of System Dynamics Modeling at CDC

2006

2007

2008

1999

2000

2001

2002

2003

2004

2005

* Dedicated multi-year budget


Learning in and about dynamic systems
Learning In and About Dynamic Systems

Strategy, Structure,

Mental

Decision Rules

Models

Real World

  • Unknown structure

  • Dynamic complexity

  • Time delays

  • Impossible experiments

Virtual World

  • Known structure

  • Controlled experiments

  • Enhanced learning

  • Implementation

  • Game playing

  • Inconsistency

  • Short term

  • Selected

  • Missing

  • Delayed

  • Biased

  • Ambiguous

Information

Decisions

Feedback

  • Inability to infer dynamics from mental models

  • Misperceptions

  • Unscientific

  • Biases

  • Defensiveness

Sterman JD. Learning in and about complex systems. System Dynamics Review 1994;10(2-3):291-330.

Sterman JD. Business dynamics: systems thinking and modeling for a complex world. Boston, MA: Irwin McGraw-Hill, 2000.


A model is
A Model Is…

It helps us understand, explain, anticipate, and make decisions

“All models are wrong, some are useful.”

-- George Box

An inexact representation of the real thing


Simulations for learning in dynamic systems
Simulations for Learning in Dynamic Systems

Dynamic Hypothesis (Causal Structure)

Plausible Futures (Policy Experiments)

Obese fraction of Adults (Ages 20-74)

50%

40%

30%

Fraction of popn 20-74

20%

10%

0%

1970

1980

1990

2000

2010

2020

2030

2040

2050

Multi-stakeholder Dialogue

Morecroft JDW, Sterman J. Modeling for learning organizations. Portland, OR: Productivity Press, 2000.

Sterman JD. Business dynamics: systems thinking and modeling for a complex world. Boston, MA: Irwin McGraw-Hill, 2000.


Learning in and about dynamic systems1
Learning In and About Dynamic Systems

“In [dynamically complex] circumstances simulation becomes the only reliable way to test a hypothesis and evaluate the likely effects of policies."

-- John Sterman

Benefits of Simulation

  • Formal means of evaluating options

  • Experimental control of conditions

  • Compressed time

  • Complete, undistorted results

  • Actions can be stopped or reversed

  • Tests for extreme conditions

  • Early warning of unintended effects

  • Opportunity to assemble stronger support

  • Visceral engagement and learning

Complexity Hinders

  • Generation of evidence (by eroding the conditions for experimentation)

  • Learning from evidence (by demanding new heuristics for interpretation)

  • Acting upon evidence (by including the behaviors of other powerful actors)

Sterman JD. Learning from evidence in a complex world. American Journal of Public Health (in press).

Sterman JD. Business Dynamics: Systems Thinking and Modeling for a Complex World. Boston, MA: Irwin McGraw-Hill, 2000.


Tools for policy planning evaluation
Tools for Policy Planning & Evaluation

Events

Time Series Models

Describe trends

  • Increasing:

  • Depth of causal theory

  • Robustness for longer-term projection

  • Value for developing policy insights

  • Degrees of uncertainty

Multivariate Stat Models

Identify historical trend drivers and correlates

Patterns

Dynamic Simulation Models

Anticipate new trends, learn about policy consequences, and set justifiable goals

Structure


Different modeling approaches for different purposes
Different Modeling Approaches For Different Purposes


Look Reasonable, But How Much Will it Take, and What’s the Expected Benefit? When?

Milstein B, Chapel T, Renault V, Fawcett S. Developing a logic model or theory of change. Community Tool Box, 2002. Accessed April 9, 2003 at <http://ctb.ku.edu/tools/en/section_1877.htm>.


Model uses and audiences
Model Uses and Audiences

  • Set Better Goals (Planners & Evaluators)

    • Identify what is likelyand what is plausible

    • Estimate intervention impact time profiles

    • Evaluate resource needsfor meeting goals

  • Support Better Action (Policymakers)

    • Explore ways of combining policies for better results

    • Evaluate cost-effectivenessover extended time periods

    • Increase policymakers’ motivation to act differently

  • Develop Better Theory and Estimates (Researchers)

    • Integrate and reconcile diverse data sources

    • Identify causal mechanisms driving system behavior

    • Improve estimates of hard-to-measure or “hidden” variables


An inter active form of policy planning evaluation
An (Inter) Active Form of Policy Planning/Evaluation

System Dynamics is a methodology to…

  • Map the salient forces that contribute to a persistent problem;

  • Convert the map into a computer simulation model, integrating the best information and insight available;

  • Compare results from simulated “What If…” experiments to identify intervention policies that might plausibly alleviate the problem;

  • Conduct sensitivity analyses to assess areas of uncertainty in the model and guide future research;

  • Convene diverse stakeholders to participate in model-supported “Action Labs,” which allow participants to discover for themselves the likely consequences of alternative policy scenarios


Expanding public health science
Expanding Public Health Science

Syndemic Orientation

Problems Among People inPlaces

Over Time

Boundary

Critique

“Public health imagination involves using science to expand the boundaries of what is possible.”

-- Michael Resnick

EpidemicOrientation


Boundary critique
Boundary Critique

Creating a new theory is not like destroying an old barn and erecting a skyscraper in its place. It is rather like climbing a mountain, gaining new and wider views, discovering unexpected connections between our starting point and its rich environment.

-- Albert Einstein

Ulrich W. Boundary critique. In: Daellenbach HG, Flood RL, editors. The Informed Student Guide to Management Science. London: Thomson; 2002. p. 41-42. <http://www.geocities.com/csh_home/downloads/ulrich_2002a.pdf>.

Ulrich W. Reflective practice in the civil society: the contribution of critically systemic thinking. Reflective Practice 2000;1(2):247-268. http://www.geocities.com/csh_home/downloads/ulrich_2000a.pdf


The Weight of Boundary Judgments

Forrester JW. Counterintuitive behavior of social systems. Technology Review 1971;73(3):53-68.

Meadows DH. Leverage points: places to intervene in a system. Sustainability Institute, 1999. Available at <http://www.sustainabilityinstitute.org/pubs/Leverage_Points.pdf>.

Richardson GP. Feedback thought in social science and systems theory. Philadelphia, PA: University of Pennsylvania Press, 1991.

Sterman JD. Business dynamics: systems thinking and modeling for a complex world. Boston, MA: Irwin McGraw-Hill, 2000.


Implications for policy planning and evaluation
Implications for Policy Planning and Evaluation

Insights from the Overview Effect

  • Maintain a particular analytic distance

  • Not too close to the details, but not too far as be insensitive to internal pressures

  • Potential to anticipate temporal patterns (e.g., better before worse)

  • Structure determines behavior

  • Potential to avoid scapegoating or lionizing

Richardson GP. Feedback thought in social science and systems theory. Philadelphia, PA: University of Pennsylvania Press, 1991.

Richmond B. Systems thinking: critical thinking skills for the 1990s and beyond. System Dynamics Review 1993;9(2):113-134. Available at <http://www.clexchange.org/ftp/documents/whyk12sd/Y_1993-05STCriticalThinking.pdf>.

White F. The overview effect: space exploration and human evolution. 2nd ed. Reston VA: American Institute of Aeronautics and Astronautics, 1998.


Civic Participation

Forces Outside the Community

  • Social cohesion

  • Responsibility for others

  • Macroeconomy, employment

  • Food supply

  • Advertising, media

  • National health care

  • Racism

  • Transportation policies

  • Voluntary health orgs

  • Professional assns

  • University programs

  • National coalitions

Health Care & Public Health Agency Capacity

  • Provider supply

  • Provider understanding, competence

  • Provider location

  • System integration

  • Cost of care

  • Insurance coverage

Personal Capacity

Local Living Conditions

  • Understanding

  • Motivation

  • Social support

  • Literacy

  • Physio-cognitive function

  • Life stages

  • Availability of good/bad food

  • Availability of phys activity

  • Comm norms, culture (e.g., responses to racism,

  • acculturation)

  • Safety

  • Income

  • Transportation

  • Housing

  • Education

Health Care Utilization

  • Ability to use care (match of patients and providers, language, culture)

  • Openness to/fear of screening

  • Self-management, monitoring

Metabolic Stressors

  • Nutrition

  • Physical activity

  • Stress

Population Flows

  • Percent of patients screened

  • Percent of people with diabetes under control

  • Baseline Flows

We Convened a Model-Scoping Group of 45 CDC professionals and epidemiologists in December 2003 to Explore the Full Range of Forces Driving Diabetes Behavior over Time


Diabetes model overview
Diabetes Model Overview

Burden of

Diabetes

Costs

(per person with diabetes)

Total Prevalence

Unhealthy Days

(people with diabetes)

(per person with

diabetes)

Developing

Prediabetes

Diabetes

People with

Deaths

People with

Diagnosis

People with

Onset

Onset

People with

Normal

e

Undiagnosed

Diagnosed

a

b

c

Prediabetes

Blood Sugar

Diabetes

Diabetes

Levels

Recovering from

Prediabetes

Deaths

Obesity in the

Prediabetes

Diabetes

Diabetes

General

Detection &

Detection

Management

Population

Management

Developing

d

Data sources: NHIS, NHANES, BRFSS, Census,

Vital statistics, Clinical studies, Cost studies


Diabetes model overview1
Diabetes Model Overview

Burden of

This larger view takes us beyond standard epidemiological models and most intervention programs

Diabetes

Costs

(per person with diabetes)

Total Prevalence

Unhealthy Days

(people with diabetes)

(per person with

diabetes)

PreDiabetes

People with

Deaths

People with

Diagnosis

People with

Onset

Normal

e

Undiagnosed

Diagnosed

a

b

Blood Sugar

Diabetes

Diabetes

Levels

Recovering from

PreDiabetes

Deaths

Obesity in the

Prediabetes

Diabetes

Diabetes

General

Detection &

Detection

Management

Population

Management

Standard boundary

Developing

Developing

Diabetes

Onset

People with

c

Prediabetes

d

Data sources: NHIS, NHANES, BRFSS, Census,

Vital statistics, Clinical studies, Cost studies


Expanding public health science1
Expanding Public Health Science

Syndemic Orientation

Governing Dynamics

CausalMapping

Dynamic

Modeling

Problems Among People inPlaces

Over Time

Plausible Futures

Boundary

Critique

“Public health imagination involves using science to expand the boundaries of what is possible.”

-- Michael Resnick

EpidemicOrientation


Selected cdc projects featuring system dynamics modeling 2001 2008
Selected CDC Projects Featuring System Dynamics Modeling (2001-2008)

  • Grantmaking ScenariosTiming and sequence of outside assistance

  • Upstream-Downstream EffortBalancing disease treatment with prevention/protection

  • Healthcare ReformRelationships among cost, quality, equity, and health status

  • Chronic Illness DynamicsHealth and economic scenarios for downstream and upstream reforms

  • SyndemicsMutually reinforcing afflictions

  • DiabetesIn an era of rising obesity

  • ObesityLifecourse consequences of changes in caloric balance

  • Infant HealthFetal and infant morbidity/mortality

  • Heart Disease and StrokePreventing and managing multiple risks, in context

Milstein B, Homer J. Background on system dynamics simulation modeling, with a summary of major public health studies. Atlanta, GA: Syndemics Prevention Network, Centers for Disease Control and Prevention; February 1, 2005. <http://www2.cdc.gov/syndemics/pdfs/SD_for_PH.pdf>.


Preventing and Managing Risk Factors for Heart Disease and Stroke Modeling the Local Dynamics of Cardiovascular Health

Homer J, Milstein B, Wile K, Pratibhu P, Farris R, Orenstein D. Modeling the local dynamics of cardiovascular health: risk factors, context, and capacity. Preventing Chronic Disease (in press).


What is best allocation of resources to eliminate the burden, disparity & costs of preventable CVD, recognizing the spectrum of opportunities in particular places & settings? Over what time frame?

Guiding Questions


Contributors

Core Design Team burden, disparity & costs

CDC: Michele Casper, Rosanne Farris, Darwin Labarthe, Marilyn Metzler, Bobby Milstein, Diane Orenstein

Austin: Cindy Batcher, Karina Loyo, Ella Pugo, Rick Schwertfeger, Adolfo Valadez, Josh Vest,

NIH: David Abrams, Patty Mabry

Consultants: Jack Homer, Justin Trogdon, Kristina Wile

Contributors

Organizational Sponsors

  • Austin/Travis County Health and Human Services Department

  • CDC Division for Heart Disease and Stroke Prevention

  • CDC Division of Adult and Community Health

  • CDC Division of Nutrition, Physical Activity, and Obesity

  • CDC Division of Diabetes Translation

  • CDC Office on Smoking and Health

  • CDC NCCDPHP Office of the Director

  • Indigent Care Collaborative (Austin, TX)

  • NIH Office of Behavioral and Social Science Research

  • RTI International

  • Sustainability Institute

  • Texas Department of Health


Model purpose and rationale
Model Purpose and Rationale burden, disparity & costs

  • Purpose

    • How do multiple risk factors and social factors combine to affect cardiovascular disease (CVD) endpoints and costs?

    • How should we focus our policy efforts given limited resources?

  • Rationale for systems modeling

    • Capturing intermediate links so that possible “confounding factors” are included explicitly rather than ignored

    • Non-additive effects when multiple risk factors are combined

    • Time delays from change in incidence to change in prevalence (accumulation or “bathtub” effects)

The model described here is a work in progress funded by the CDC’s

Division of Heart Disease and Stroke Prevention. We plan to finalize the

model’s equations and parameter values by February 2008.


Intervention Approaches from burden, disparity & costs “Upstream” to “Downstream”

Our model focuses on the prevention and control of

risk factors that can lead to a first-time CVD event.


Crafting effective intervention strategies for upstream prevention in context
Crafting Effective Intervention Strategies for burden, disparity & costs Upstream Prevention in Context

  • Concentrate on “upstream” challenge of minimizing risk, rather than the better understood “downstream” task of post-event care

  • Local conditions affect people’s health status and their responses to perceived problems

  • Local social and physical factors may be critical when characterizing the history—and plausible futures—of cardiovascular disease in a given city or region

  • These aspects of local context are difficult to measure and too often excluded when planning and evaluating policies or programs

The CDC is partnering on this project with the Austin (Travis County), Texas,

Dept. of Health and Human Services. The model is calibrated to represent the

overall US, but is informed by the experience and data of the Austin team,

which has been supported by the CDC’s “STEPS” program since 2004.


Local capacity for leadership & organizing burden, disparity & costs

LOCAL ACTIONS

COSTS (CVD & NON-CVD)

NUTRITION, PHYSICAL

ATTRIBUTABLE TO RISK FACTORS

ACTIVITY & STRESS

  • Salt intake

  • Saturated/Trans fat intake

  • Fruit/Vegetable intake

  • Net caloric intake

  • Physical activity

  • Chronic stress

LOCAL CONTEXT

  • Eating & activity options

  • Smoking policies

  • Socioeconomic conditions

  • Environmental policies

  • Health care options

  • Support service options

  • Media and events

RISK FACTOR ONSET,

PREVALENCE & CONTROL

  • Hypertension

  • High cholesterol

  • Diabetes

  • Obesity

  • Smoking

  • Secondhand smoke

  • Air pollution exposure

Modified Anderson

Risk Calculator

UTILIZATION OF SERVICES

ESTIMATED FIRST-TIME FATAL

AND NON-FATAL CVD EVENTS

  • Behavioral change

  • Social support

  • Mental health

  • Preventive health

  • CHD (MI, Angina, Cardiac Arrest)

  • Stroke

  • Total CVD (CHD, Stroke, CHF, PAD)

Preventing and Managing Risk Factors for CVDSector Diagram

Homer J, Milstein B, Wile K, Pratibhu P, Farris R, Orenstein D. Modeling the local dynamics of cardiovascular health: risk factors, context, and capacity. Preventing Chronic Disease in press.

DRAFT: October, 2007


Data sources for cvd risk modeling
Data Sources for CVD Risk Modeling burden, disparity & costs

  • Census

    • Population, deaths, births, net immigration, health coverage

  • AHA & NIH statistical reports

    • CVD events, deaths, and prevalence (CHD, stroke, CHF, PAD)

  • National Health and Nutrition Examination Survey (NHANES)

    • Risk factor prevalences by age (18-29, 30-64, 65+) and sex (M, F)

    • Risk factor diagnosis and control (hypertension, high cholesterol, diabetes)

  • Behavioral Risk Factor Surveillance System (BRFSS)

    • Diet & physical activity

    • Primary care utilization

    • Lack of needed emotional/social support

  • Research literature

    • CVD risk calculator, and relative risks from SHS, air pollution, obesity, and inactivity

    • Medical and productivity costs of CVD and risk factors

  • Questionnaires for CDC and Austin teams (expert judgment)

    • Potential effects of social marketing

    • Potential effects of expanded access to healthy food, activity, and behavioral services

    • Effects of behavioral services on smoking, weight loss, stress reduction

    • Relative risks of stress for high BP, high cholesterol, smoking, and obesity


Cvd risk factors and linkages
CVD Risk Factors and Linkages burden, disparity & costs


Improving primary care
Improving Primary Care burden, disparity & costs


Reducing risk factor prevalence
Reducing Risk Factor Prevalence burden, disparity & costs


Adding up the disease costs
Adding Up the Disease Costs burden, disparity & costs


Developing a status quo scenario
Developing a “Status Quo” Scenario burden, disparity & costs

Obese % of non-CVD popn

  • A straightforward base case

    • Assume no changes after 2000 in contextual factors or in risk factor inflow/outflow rates

    • Any changes in risk prevalences after 2000 are due to “bathtub” adjustment and population aging

  • Result: Past trends continue after 2000, but decelerate and level off

    • Increasing obesity, high BP, and diabetes

    • Decreasing smoking

    • High cholesterol mixed bag by age and sex, flat overall

Uncontrolled hypertension %

of non-CVD popn

Smoking % of non-CVD popn

The model is calibrated to reproduce data from NHANES 1988-94 and 1999-2004 on risk factor prevalences in the non-CVD population by age and sex.


Testing alternative scenarios
Testing Alternative Scenarios burden, disparity & costs

Obesity prevalence

  • Policy Tests

    • What if this intervention had been fully implemented by 1997?

  • Sensitivity Tests

    • How would the effects of a particular policy change if we vary a more uncertain assumption across its plausible range?

0.4

0.3

0.2

1. Base Case

2. Increase access to PA

0.1

0

1990

2005

2015

2030

2040

Time (Year)

Obesity prevalence

0.4

0.325

0.25

Varying RR of Obesity w/o PA

0.175

0.1

1990

2003

2015

2028

2040

Time (Year)


1 Reductions in smoking may lead, in turn, to some increase in eating and obesity; 2 Primary care improves diagnosis and control of affected conditions, not their prevalence; 3 Due to stress-eating; 4 Affects chronic stress; 5 Affects secondhand smoke; 6 Affects nutrition, PA, smoking; 7 Affects use of available services


Broader categories of policy change
Broader Categories of Policy Change in eating and obesity; 2 Primary care improves diagnosis and control of affected conditions, not their prevalence; 3 Due to stress-eating; 4 Affects chronic stress; 5 Affects secondhand smoke;

  • Policies that decrease socioeconomic gaps

    • Educational policies

    • Fiscal policies

    • Skills training policies

  • Policies that mitigate adverse conditions

    • Policies affecting the environment

    • Polices affecting the workplace

    • Policies enabling healthier behaviors

    • Policies affecting the medical system

Adapted from: Adler N, Stewart J. Reaching for a healthier life: facts on socioeconomic status and health in the USA.San Francisco, CA: John D. and Catherine T. MacArthur Research Network on Socioeconomic Status and Health 2007


Simulation framework
Simulation Framework in eating and obesity; 2 Primary care improves diagnosis and control of affected conditions, not their prevalence; 3 Due to stress-eating; 4 Affects chronic stress; 5 Affects secondhand smoke;


Simulation framework and policy space
Simulation Framework and Policy Space in eating and obesity; 2 Primary care improves diagnosis and control of affected conditions, not their prevalence; 3 Due to stress-eating; 4 Affects chronic stress; 5 Affects secondhand smoke;

POLICIES AFFECTING

THE ENVIRONMENT

POLICIES AFFECTING

THE MEDICAL SYSTEM

POLICIES AFFECTING

THE WORKPLACE

EDUCATION

POLICIES

FISCAL

POLICIES

SKILLS

TRAINING

POLICIES

POLICIES ENABLING

HEALTHIER BEHAVIORS


Simulation control panel
Simulation Control Panel in eating and obesity; 2 Primary care improves diagnosis and control of affected conditions, not their prevalence; 3 Due to stress-eating; 4 Affects chronic stress; 5 Affects secondhand smoke;


Three illustrative policies
Three Illustrative Policies in eating and obesity; 2 Primary care improves diagnosis and control of affected conditions, not their prevalence; 3 Due to stress-eating; 4 Affects chronic stress; 5 Affects secondhand smoke;

  • Expand Access and Social Support

    • Provide full access for all to healthy food, safe physical activity, primary care, and behavioral services

    • Provide social supports to mitigate stress, reducing it 50%

  • Strengthen Primary Care and Promote Healthy Living

    • Transform primary care to meet highest standards for prevention and control activities and referrals

    • Strongly promote healthy eating, activity, no smoking, and use of primary care and behavioral services

  • Fight Tobacco and Air Pollution

    • Tobacco control package: Raise taxes, police sales to minors, and ban smoking in workplaces and public places

    • Reduce particulate (PM 2.5) air pollution by 50%

The interventions are tested retroactively with implementation starting in 1995

and ramping up to full effectiveness by 1997, continuing unabated through 2040.


Annual Disease Costs in 5 Illustrative Scenarios in eating and obesity; 2 Primary care improves diagnosis and control of affected conditions, not their prevalence; 3 Due to stress-eating; 4 Affects chronic stress; 5 Affects secondhand smoke;

Total Annual Risk Factor Complication Costs per Capita Among the Never-CVD Population

2,000

Work in progress - for illustration only

1,750

dollars/(Year*person)

1,500

1,250

1,000

1990

1995

2000

2005

2010

2015

2020

2025

2030

2035

2040

  • Base

  • Access & social support

  • Strengthen primary care & promote healthy living

  • Fight tobacco & air pollution

  • All of the above


Obesity, Uncontrolled Hypertension, and Smoking in eating and obesity; 2 Primary care improves diagnosis and control of affected conditions, not their prevalence; 3 Due to stress-eating; 4 Affects chronic stress; 5 Affects secondhand smoke; Five Illustrative Scenarios

0.3

0.4

Obese % of non-CVD popn

Uncontrolled hypertension % of non-CVD popn

0.3

0.2

0.2

0.1

0.1

0

0

1990

1995

2000

2005

2010

2015

2020

2025

2030

2035

2040

1990

1995

2000

2005

2010

2015

2020

2025

2030

2035

2040

0.3

Smoking % of non-CVD popn

  • Base

  • Access & social support

  • Strengthen primary care & promote healthy living

  • Fight tobacco & air pollution

  • All of the above

0.2

0.1

0

1990

1995

2000

2005

2010

2015

2020

2025

2030

2035

2040


What are we learning
What are We Learning? in eating and obesity; 2 Primary care improves diagnosis and control of affected conditions, not their prevalence; 3 Due to stress-eating; 4 Affects chronic stress; 5 Affects secondhand smoke;

  • Literature on risk factors and social determinants poses a challenge for modeling

    • Many studies skip causal links or don’t quantify effect sizes

  • BRFSS offers reasonable proxies for tricky variables like stress and access

  • Health departments are practically oriented and can help refine concepts and estimate effect sizes

  • Policy analysts want us to model broadly despite the numerical uncertainties

    • Give more attention to how effectiveness of social interventions may change over time (erosion, bandwagon effects)

  • Take audience background into account when presenting concepts and intervention approaches


Conceptual and methodological features of system dynamics modeling

Thinking dynamically in eating and obesity; 2 Primary care improves diagnosis and control of affected conditions, not their prevalence; 3 Due to stress-eating; 4 Affects chronic stress; 5 Affects secondhand smoke;

Move from events and decisions to patterns of continuous behavior over time and policy structure

Thinking in circular causal /feedback patterns

Self-reinforcing and self-balancing processes

Compensating feedback structures and policy resistance

Communicating complex nonlinear system structure

Thinking in stocks and flows

Accumulations are the resources and the pressures on policy

Policies influence flows

Modeling and simulation

Accumulating (and remembering) complexity

Quantification (distinct from measurement)

Rigorous (daunting) model evaluation processes

Controlled experiments

Reflection

Conceptual and Methodological Features of System Dynamics Modeling

Richardson GP, Homer JB. System dynamics modeling: population flows, feedback loops, and health. NIH/CDC Symposia on System Science and Health; Bethesda, MD: August 30, 2007. Available at <http://obssr.od.nih.gov/Content/Lectures+and+Seminars/Systems_Symposia_Series/Systems_Symposium_Four/SEMINARS.htm>.


A specific set of thinking skills
A Specific Set of Thinking Skills in eating and obesity; 2 Primary care improves diagnosis and control of affected conditions, not their prevalence; 3 Due to stress-eating; 4 Affects chronic stress; 5 Affects secondhand smoke;

Karash R. The essentials of systems thinking and how they pertain to healthcare and colorectal cancer screening. Dialogue for Action in Colorectal Cancer; Baltimore, MD; March 23, 2005..

Richmond B. Systems thinking: critical thinking skills for the 1990s and beyond. System Dynamics Review 1993;9(2):113-134.

Richmond B. The "thinking" in systems thinking: seven essential skills. Waltham, MA: Pegasus Communications, 2000.


Revisiting the framework
Revisiting the Framework in eating and obesity; 2 Primary care improves diagnosis and control of affected conditions, not their prevalence; 3 Due to stress-eating; 4 Affects chronic stress; 5 Affects secondhand smoke;

“Steps in the framework are starting points for tailoring an evaluation to a particular public health effort at a particular time.”

Simulation Modeling Offers

  • Support for multi-stakeholder dialogue

  • A larger conception of the “program” context

  • Another avenue for experimentation and visceral learning, with the need for comparison or control groups

  • Ability to track interrelated indicators (both states and rates)

  • An emphasis on pragmatism (learning through action)

Milstein B, Wetterall S, CDC Evaluation Working Group. Framework for program evaluation in public health. MMWR Recommendations and Reports 1999;48(RR-11):1-40. Available at <http://www.cdc.gov/mmwr/PDF/RR/RR4811.pdf>.


An alternative philosophical tradition
An Alternative Philosophical Tradition in eating and obesity; 2 Primary care improves diagnosis and control of affected conditions, not their prevalence; 3 Due to stress-eating; 4 Affects chronic stress; 5 Affects secondhand smoke;

Positivism

  • Begins with a theory about the world

  • Learning through observation and falsification

  • Asks, “Is this theory true?”

Pragmatism

  • Begins with a response to a perplexity or injustice in the world

  • Learning through action and reflection

  • Asks, “How does this work make a difference?”

"Grant an idea or belief to be true…what concrete difference will its being true make in anyone's actual life?

-- William James

We are not talking about theories to explain, but conceptual, methodological, and moral orientations: the frames of reference that shape how we think, how we act, how we learn, and what we value

Shook J. The pragmatism cybrary. 2006. Available at <http://www.pragmatism.org/>.

Addams J. Democracy and social ethics. Urbana, IL: University of Illinois Press, 2002.

West C. The American evasion of philosophy: a genealogy of pragmatism. Madison, WI: University of Wisconsin Press, 1989.


How Should We Value Simulation Studies? in eating and obesity; 2 Primary care improves diagnosis and control of affected conditions, not their prevalence; 3 Due to stress-eating; 4 Affects chronic stress; 5 Affects secondhand smoke;

  • All models, including simulations, are incomplete and imprecise

  • But some are better than others and capture more important aspects of the real world’s dynamic complexity

  • A valuable model is one that can help us understand and anticipate better than we do with the unaided mind

“All models are wrong, some are useful.”

-- George Box

Artist: Rene Magritte

Sterman JD. All models are wrong: reflections on becoming a systems scientist. System Dynamics Review 2002;18(4):501-531.

Meadows DH, Richardson J, Bruckmann G. Groping in the dark: the first decade of global modelling. New York, NY: Wiley, 1982.

Forrester JW. Counterintuitive behavior of social systems. Technology Review 1971;73(3):53-68.


“Simulation is a third way of doing science. in eating and obesity; 2 Primary care improves diagnosis and control of affected conditions, not their prevalence; 3 Due to stress-eating; 4 Affects chronic stress; 5 Affects secondhand smoke; Like deduction, it starts with a set of explicit assumptions. But unlike deduction, it does not prove theorems. Instead, a simulation generates data that can be analyzed inductively. Unlike typical induction, however, the simulated data comes from a rigorously specified set of rules rather than direct measurement of the real world. While induction can be used to find patterns in data, and deduction can be used to find consequences of assumptions, simulation modeling can be used as an aid to intuition.”

What?

Where?

Prevalence of Obese Adults, United States

Why?

How?

Who?

2020

2010

Data Source: NHANES

Simulation ExperimentsOpen a Third Branch of Science

“The complexity of our mental models vastly exceeds our ability to understand their implications without simulation."

-- John Sterman

-- Robert Axelrod

Axelrod R. Advancing the art of simulation in the social sciences. In: Conte R, Hegselmann R, Terna P, editors. Simulating Social Phenomena. New York, NY: Springer; 1997. p. 21-40. <http://www.pscs.umich.edu/pub/papers/AdvancingArtofSim.pdf>.

Sterman JD. Business Dynamics: Systems Thinking and Modeling for a Complex World. Boston, MA: Irwin McGraw-Hill, 2000.


What s on the horizon for system science health
What’s on the Horizon for System Science & Health? in eating and obesity; 2 Primary care improves diagnosis and control of affected conditions, not their prevalence; 3 Due to stress-eating; 4 Affects chronic stress; 5 Affects secondhand smoke;

2007

  • Extramural funding for methodology and technology (NIH Roadmap)

  • Symposia series on system science and health (NIH/OBSSR and CDC/SPN; ~6,000 participants)

  • Conference on complexity approaches to population health (Univ of Michigan; ~250 participants)

  • NIH monograph, “Greater Than the Sum”

  • CDC monograph, “Hygeia’s Constellation”

  • CDC to hire directors for preparedness modeling and public health systems research

  • Concept mapping of public health policy resistance (NIH/OBSSR and CDC/SPN)

  • Historical examples of health system transformation (CDC Public Health Practice Council)

  • Methodology to support CDC’s focus on “health protection…health equity” (PriceWaterhouseCoopers)

    2008

  • Summer training institute for system science and health (NIH/OBSSR and CDC/SPN)

    2009

  • Extramural funding for “Health System Change” (NIH and CDC?)


For further information
For Further Information in eating and obesity; 2 Primary care improves diagnosis and control of affected conditions, not their prevalence; 3 Due to stress-eating; 4 Affects chronic stress; 5 Affects secondhand smoke;

  • CDC Syndemics Prevention Network http://www.cdc.gov/syndemics

  • NIH/CDC Symposia on System Science and Healthhttp://obssr.od.nih.gov/Content/Lectures+and+Seminars/Systems_Symposia_Series/SEMINARS.htm

  • Recommended Reading

    • AJPH theme issue on systems thinking and modeling (March, 2006)http://www.ajph.org/content/vol96/issue3/

      • Sterman JD. Learning from evidence in a complex world. AJPH 2006;96(3):505-514.

      • Midgley G. Systemic intervention for public health. AJPH 2006;96(3):466-472.

      • Homer JB, Hirsch GB. System dynamics modeling for public health: background and opportunities. AJPH 2006;96(3):452-458.

    • Sterman JD. A skeptic's guide to computer models. In: Barney GO, editor. Managing a Nation: the Microcomputer Software Catalog. Boulder, CO: Westview Press; 1991. p. 209-229.http://web.mit.edu/jsterman/www/Skeptic%27s_Guide.html

    • Meadows DH. Leverage points: places to intervene in a system. Sustainability Institute, 1999. http://www.sustainabilityinstitute.org/pubs/Leverage_Points.pdf

    • Meadows DH, Robinson JM. The electronic oracle: computer models and social decisions. System Dynamics Review 2002;18(2):271-308.


Forthcoming report

Forthcoming Report in eating and obesity; 2 Primary care improves diagnosis and control of affected conditions, not their prevalence; 3 Due to stress-eating; 4 Affects chronic stress; 5 Affects secondhand smoke;

Milstein B. Hygeia's constellation: navigating health futures in a dynamic and democratic world. Atlanta, GA: Centers for Disease Control and Prevention 2008.


EXTRAS in eating and obesity; 2 Primary care improves diagnosis and control of affected conditions, not their prevalence; 3 Due to stress-eating; 4 Affects chronic stress; 5 Affects secondhand smoke;


Cdc diabetes system modeling project charting plausible futures for hp 2010

CDC Diabetes System Modeling Project in eating and obesity; 2 Primary care improves diagnosis and control of affected conditions, not their prevalence; 3 Due to stress-eating; 4 Affects chronic stress; 5 Affects secondhand smoke; Charting Plausible Futures for HP 2010

Milstein B, Jones A, Homer J, Murphy D, Essien J, Seville D. Charting plausible futures for diabetes prevalence: a role for system dynamics simulation modeling. Preventing Chronic Disease 2007;4(3):1-8. Available at <http://www.cdc.gov/pcd/issues/2007/jul/06_0070.htm>


Cdc diabetes system modeling project understanding population dynamics

CDC Diabetes System Modeling Project in eating and obesity; 2 Primary care improves diagnosis and control of affected conditions, not their prevalence; 3 Due to stress-eating; 4 Affects chronic stress; 5 Affects secondhand smoke; Understanding Population Dynamics

Jones AP, Homer JB, Murphy DL, Essien JDK, Milstein B, Seville DA. Understanding diabetes population dynamics through simulation modeling and experimentation. American Journal of Public Health 2006;96(3):488-494.


Cdc diabetes system modeling project discovering dynamics through state based action labs models

CDC Diabetes System Modeling Project in eating and obesity; 2 Primary care improves diagnosis and control of affected conditions, not their prevalence; 3 Due to stress-eating; 4 Affects chronic stress; 5 Affects secondhand smoke; Discovering Dynamics Through State-based Action Labs & Models


Cdc obesity dynamics modeling project exploring historical growth and plausible futures

CDC Obesity Dynamics Modeling Project in eating and obesity; 2 Primary care improves diagnosis and control of affected conditions, not their prevalence; 3 Due to stress-eating; 4 Affects chronic stress; 5 Affects secondhand smoke; Exploring Historical Growth and Plausible Futures

Homer J, Milstein B, Dietz W, Buchner D, Majestic D. Obesity population dynamics: exploring historical growth and plausible futures in the U.S. 24th International Conference of the System Dynamics Society; Nijmegen, The Netherlands; July 26, 2006.

Centers for Disease Control and Prevention. The state of the CDC, fiscal year 2006. Atlanta, GA: CDC 2007. <http://www.cdc.gov/about/stateofcdc/index.htm>


Cdc syndemics modeling neighborhood transformation game

CDC Syndemics Modeling in eating and obesity; 2 Primary care improves diagnosis and control of affected conditions, not their prevalence; 3 Due to stress-eating; 4 Affects chronic stress; 5 Affects secondhand smoke; Neighborhood Transformation Game

Homer J, Milstein B. Optimal decision making in a dynamic model of poor community health. 37th Hawaii International Conference on System Science; Big Island, HI; January 5-8, 2004. <http://csdl.computer.org/comp/proceedings/hicss/2004/2056/03/205630085a.pdf

Homer J, Milstein B. Syndemic simulation. Forio Business Simulations, 2003. <http://broadcast.forio.com/sims/syndemic2003/>.


Sd society health policy dynamics modeling upstream and downstream reforms

SD Society Health Policy Dynamics Modeling in eating and obesity; 2 Primary care improves diagnosis and control of affected conditions, not their prevalence; 3 Due to stress-eating; 4 Affects chronic stress; 5 Affects secondhand smoke; Upstream and Downstream Reforms

Homer J, Hirsch G, Milstein B. Chronic illness in a complex health economy: the perils and promises of downstream and upstream reforms. System Dynamics Review 2007 (in press).


Cdc leadership on health system transformation
CDC Leadership on Health System Transformation in eating and obesity; 2 Primary care improves diagnosis and control of affected conditions, not their prevalence; 3 Due to stress-eating; 4 Affects chronic stress; 5 Affects secondhand smoke;

Gerberding JL. CDC: protecting people's health. Director's Update; Atlanta, GA; July, 2007.

Gerberding JL. Health protectionomics: a new science of people, policy, and politics. Public Health Grand Rounds; Washington, DC: George Washington University School of Public Health and Health Services; September 19, 2007. Available at <http://www.kaisernetwork.org/health_cast/hcast_index.cfm?display=detail&hc=2349>

Centers for Disease Control and Prevention. Health system transformation: Office of Strategy and Innovation; September 28, 2007. <http://intradev.cdc.gov/od/osi/policy/healthSystems_overview.htm>.

Time 100: the people who shape our world. Time Magazine 2004 April 26.


Mapping the Dynamics of Upstream and Downstream: in eating and obesity; 2 Primary care improves diagnosis and control of affected conditions, not their prevalence; 3 Due to stress-eating; 4 Affects chronic stress; 5 Affects secondhand smoke; Why is So Hard for the Health System to Work Upstream?

Milstein B, Homer J. The dynamics of upstream and downstream: why is so hard for the health system to work upstream, and what can be done about it? CDC Futures Health Systems Work Group; Atlanta, GA; December 3, 2003.

Jackson DJ, Valdesseri R, CDC Health Systems Work Group. Health systems work group report. Atlanta, GA: Centers for Disease Control and Prevention, Office of Strategy and Innovation; January 6, 2004. <http://intranet.cdc.gov/od/futures/wrkgroup/stage_i/hswg.htm>

Milstein B, Homer J. Health system dynamics: mapping the drivers of population health, vulnerability, and affliction. Atlanta, GA: Syndemics Prevention Network; June 27 (work in progress), 2006.

Milstein B. Hygeia's constellation: navigating health futures in a dynamic and democratic world. Atlanta, GA: Centers for Disease Control and Prevention 2008.


ad