Phases of research and development for health promotion programs
This presentation is the property of its rightful owner.
Sponsored Links
1 / 16

PHASES OF RESEARCH AND DEVELOPMENT FOR HEALTH PROMOTION PROGRAMS PowerPoint PPT Presentation


  • 81 Views
  • Uploaded on
  • Presentation posted in: General

PHASES OF RESEARCH AND DEVELOPMENT FOR HEALTH PROMOTION PROGRAMS. Brian R. Flay, D.Phil., FAAHB Health Research and Policy Centers University of Illinois at Chicago. Presented at the Annual Conference of the American Academy of Health Behavior St. Augustine, Florida, March 16-19, 2003.

Download Presentation

PHASES OF RESEARCH AND DEVELOPMENT FOR HEALTH PROMOTION PROGRAMS

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Phases of research and development for health promotion programs

PHASES OF RESEARCH AND DEVELOPMENT FOR HEALTH PROMOTION PROGRAMS

Brian R. Flay, D.Phil., FAAHB

Health Research and Policy Centers

University of Illinois at Chicago

Presented at the Annual Conference of

the American Academy of Health Behavior

St. Augustine, Florida, March 16-19, 2003

“Phases of Research and Development for Health Promotion Programs” AAHB Annual Conference, March 17 2003


Phases of research an historical view

Phases of Research: An Historical View

  • FDA (19??)

  • OTA (1978)

  • Medical Innovations (McKinley, 1981)

  • NHLBI (1983)

  • NCI (Greenwald & Cullen, 1984)

  • Smoking Prevention (Flay & Best, 1982; Flay, 1986)

  • Expedited FDA (for AIDS) (1990)

  • Alcohol Prevention (Holder, Boyd, et al, 1995; Holder, Flay, et al, 1999)

“Phases of Research and Development for Health Promotion Programs” AAHB Annual Conference, March 17 2003


Phases of research in intervention development flay 1986

Phases of Research in Intervention Development (Flay, 1986)

  • Basic Research

  • Hypothesis Development

  • Component Development and Pilot Studies

  • Prototype Studies of Complete Programs

  • Efficacy Trials of Refined Programs

    • Well controlled randomized trials

      6. Treatment Effectiveness Trials

    • Generalizability of effects under standardized delivery

      7. Implementation Effectiveness Trials

    • Effectiveness with real-world variations in implementation

      8. Demonstration Studies

    • Implementation and evaluation in multiple whole systems

“Phases of Research and Development for Health Promotion Programs” AAHB Annual Conference, March 17 2003


Limited use of phases

Limited Use of Phases

  • Few researchers or research groups have followed the phases sequence systematically

  • Sussman comes closest at the earliest phases

    • See Sussman et al., 1995 monograph

    • Used theory-driven program development

    • Conducted small-scale pilot studies of program components

    • Altered or added new activities as a result

    • Piloted full-scale intervention

  • Botvin comes closest at the efficacy-effectiveness phases

    • Has conducted multiple efficacy trials

    • Has conducted multiple treatment effectiveness trials

    • Has conducted multiple implementation effectiveness trials

  • To my knowledge, no one research group has done it all -- the substance abuse prevention field almost has.

“Phases of Research and Development for Health Promotion Programs” AAHB Annual Conference, March 17 2003


Reasons for limited use of phases

Reasons for limited use of Phases

  • Interventions are becoming more complex

    • Multi-component, involving multiple stake-holders

  • Higher stages require large, rigorous studies

    • Often too costly for standard research grant funding

  • Different research skills and interests are needed at the different phases

  • Many interventions are developed outside the research environment

    • Developed by professionals in the field, communities or legislators

    • Holder and colleagues have suggested modifications to incorporate these types of interventions

“Phases of Research and Development for Health Promotion Programs” AAHB Annual Conference, March 17 2003


Complex interventions

Complex Interventions

  • Always thought of as curricula, or whole programs, not separate components

    • Few field-based tests of efficacy of separate components to date

    • But curricula/programs based on basic and hypothesis-driven research

  • Programs have grown more complex over the years

  • Multiple outcomes are becoming the norm

    • E.g., multiple behaviors + Character + Achievement

  • Also multiple ecologies are involved

    • School-wide

    • Involvement of parents/families

    • Involvement of community

  • Therefore, multiple mediators, both distal and proximal

    • Distal: Family patterns, school climate, community involvement

    • Proximal: Attitudes, normative beliefs, self-efficacy, intentions

“Phases of Research and Development for Health Promotion Programs” AAHB Annual Conference, March 17 2003


School based prevention promotion studies are large and complex

School-based Prevention/Promotion Studies are Large and Complex

  • Large randomized trials

    • With multiple schools per condition

  • Comparisons with “treatment as usual”

  • Measurement of implementation process and program integrity

  • Assessment of effects on presumed mediators

    • Helps test theories

  • Multiple measures/sources of data

    • Surveys of students, parents, teachers, staff, community

    • Teacher and parent reports of behavior

    • School records for behavior and achievement

  • Multiple, independent trials of promising programs

    • At both efficacy and effectiveness levels

  • Cost-effectiveness analyses

“Phases of Research and Development for Health Promotion Programs” AAHB Annual Conference, March 17 2003


Holder et al 1999 modifications

Holder et al (1999) modifications

  • For field-developed programs and policy interventions

    • Often skip efficacy trials and use “natural experiments” and quasi-experimental effectiveness trials.

  • To distinguish between research-driven and program-driven interventions

    • Program-driven programs are not researcher-initiated

    • They are program- or policy-driven

  • To allow for natural experiments or other quasi-experiments

    • Time-series analyses with natural variations

    • Comparison of states that adopt a policy with those that don’t

“Phases of Research and Development for Health Promotion Programs” AAHB Annual Conference, March 17 2003


Phases of research for program driven prevention interventions holder et al 1999

Phases of research for program-driven prevention interventions (Holder et al. 1999)

1. Foundational Research

  • Epidemiology, etiology and behavior change models

    2. Developmental (Preliminary Effectiveness) Studies

  • Documentation and preliminary evaluation of an existing intervention

  • Should address safety and cost issues

    3. Efficacy Studies

  • Quasi-experimental test of an optimally implemented intervention

  • Acceptance (compliance/readines) is also maximized when possible

  • Experimental or quasi-experimental test of well implemented variations of the intervention

    4. Effectiveness Studies

  • Evaluation of the quality and levels of acceptance, implementation and overall effectiveness under real-world conditions

    5. Diffusion Studies

  • Acceptance Evaluation -- of levels of acceptance and effects of

  • Implementation Evaluation -- of levels of implementation and effects of

  • Monitoring -- Continued monitoring of program maintenance and diffusion

“Phases of Research and Development for Health Promotion Programs” AAHB Annual Conference, March 17 2003


Examples from alcohol see holder et al 1999 for more examples and references

Examples from AlcoholSee Holder et al. (1999) for more examples and references

  • DUI Enforcement

    • Ross evaluated the effects of DUI enforcement with time-series analysis of archival data.

    • Short-term results approach an estimate of maximum effects (I.e., efficacy). Long-term results (decay) represent effectiveness under on-going real-world conditions.

  • Minimum drinking age

    • Natural variation in time of legislation and implementation allowed for evaluation of the effects of changes in the minimum drinking age on youth alcohol use, DUI, crashes and other outcomes.

    • Studies of high versus low implementation/enforcement states.

  • Pricing Policies

    • Early cross-sectional studies led to time-series econometric analyses.

    • Studies in “early change” States where enforcement was high

    • Studies comparing high and low enforcement states

    • Other national-level studies

  • “Phases of Research and Development for Health Promotion Programs” AAHB Annual Conference, March 17 2003


    Much of prevention is stuck we re spinning our wheels

    Much of Prevention is STUCK … We’re Spinning our Wheels

    • At the Efficacy Trial phase

      • How can we get more programs into effectiveness trials

    • At the Effectiveness Trial phase

      • How can we get more proven programs adopted

    • At the “Model Program” phase

      • How can we ensure the ongoing effectiveness of model programs

    “Phases of Research and Development for Health Promotion Programs” AAHB Annual Conference, March 17 2003


    What does the future hold

    What does the future hold?

    • Where are the Dissemination Trials?

      • Deliberate studies of planned variations in implementation/delivery, training, etc.

    • Where are the Demonstration Studies?

      • Large scale studies of effects in the real world

      • On-going assessments of sustained effectiveness

    • Alternatives: Where do we go next?

    • Conclusions

    “Phases of Research and Development for Health Promotion Programs” AAHB Annual Conference, March 17 2003


    Glasgow et al 20023

    Glasgow et al., 20023

    • Pointed out that Greenwald & Cullen (1985 and Flay (1986) assumed that the best the candidates for effectiveness trials are interventions that have proven effective in efficacy trials

    • And argued that this assumption, or the way it was operationalized, often led to to the development of interventions with low probability of success in the real world

      • Because of the strict standardization of the intervention (to maximize internal validity)

      • And the limited conditions under which efficacy trials are conducted (limited external validity)

    “Phases of Research and Development for Health Promotion Programs” AAHB Annual Conference, March 17 2003


    Re aim

    RE-AIM

    • REACH

    • Efficacy or Effectiveness

    • Adoption

    • Implementation

    • Maintenance and cost

    • Strange order!

    • How about:

      • Efficacy, Effectiveness, Dissemination, Implementation, Reach, Maintenance of effects and delivery

    “Phases of Research and Development for Health Promotion Programs” AAHB Annual Conference, March 17 2003


    Glasgow et al recommendations

    Glasgow et al Recommendations

    • Increased attention to moderating factors

      • I.e., for whom and conditions under which it works

    • Realize that impact involves more than just efficacy

      • Impact = Reach x Efficacy x Implementation

      • Or Impact = Effectiveness x Dissemination x Maintenance

    • Include external validity criteria in author guidelines

    • Increase funding for research focused on moderating variables, external validity, and robustness

      • All of these get to generalizability

    “Phases of Research and Development for Health Promotion Programs” AAHB Annual Conference, March 17 2003


    Selected references

    Selected References

    Flay BR (1986) Efficacy and effectiveness trials (and other phases of research) in the development of health promotion programs. Prev Med, 15, 451-474, 1986.

    Flay BR, Best JA (1982) Overcoming design problems in evaluating health behavior problems. Evaluation and Health Prof., 5, 43-69.

    Food and Drug Administration (1990) From test tube to patient: New drug development in the United States. DHHS Pub. No. (FDA) 90-3168.

    Greenwald P, Cullen JW (1984) The scientific approach to cancer control. CA Cancer J. Clin. 34, 328-332.

    Holder H, Boyd G, Howard J, Flay B, Voas R, Grossman M (1995) Alcohol-related prevention research policy: The need for a phases research model. J of Public Health Policy, 16(3), 324-346.

    Holder H, Flay B, Howard J, Boyd G, Voas R, & Grossman M (1999) Phases of alcohol problem prevention research. Alcoholism: Clinical and Experimental Research, 23(1), 183-194.

    McKinlay JB (1981) From “promising report” to “standard procedure”: Seven stages in the career of a medical innovation. Milbank Memorial Fund Quarterly/Health Sot. 59, 374-411.

    National Heart, Lung, and Blood Institute (1983) Guidelines for Demonstration and Education Research Grants. NHLBI, Washington, D.C.

    Office of Technology Assessment (1978) Assessing the Efficacy and Safety of Medical Technologies. U.S. Govt. Printing Office, Washington, D.C.

    “Phases of Research and Development for Health Promotion Programs” AAHB Annual Conference, March 17 2003


  • Login