Reforming Children’s Mental Health Services – Lessons Learned from the Past and Suggestions for ...
Download
1 / 73

Leonard Bickman and Manuel Riemer Center for Mental Health Policy Vanderbilt University - PowerPoint PPT Presentation


  • 125 Views
  • Uploaded on

Reforming Children’s Mental Health Services – Lessons Learned from the Past and Suggestions for the Future. Leonard Bickman and Manuel Riemer Center for Mental Health Policy Vanderbilt University. Presentation Overview. Lessons Learned from the Past:

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Leonard Bickman and Manuel Riemer Center for Mental Health Policy Vanderbilt University' - siegfried


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
Slide1 l.jpg

Reforming Children’s Mental Health Services – Lessons Learned from the Past and Suggestions for the Future

Leonard Bickman

and

Manuel Riemer

Center for Mental Health Policy

Vanderbilt University


Presentation overview l.jpg
Presentation Overview Learned from the Past and Suggestions for the Future

Lessons Learned from the Past:

Children’s Mental Health Reform in the US

Professionalism

System Level

Evidence Based Practice

Suggestions for the Future:

CQI and the Role of Measurement

and Feedback

Measurement

Feedback Intervention


Slide3 l.jpg

Learning from History - Learned from the Past and Suggestions for the Future A Critical Evaluation

of Three Major Approaches

to Children’s Mental Health Reform


Defining the problem l.jpg
Defining the Problem Learned from the Past and Suggestions for the Future

Two reports, almost 20 years past by, but …

J. Knitzer, 1982: Unclaimed Children

  • Limited access

  • Little funding

  • Little coordination across agencies

  • Very few staff

  • Inappropriate services

  • Provides little information about effectiveness of services

Surgeon General, 1999: Report on Mental Health

  • More funding

  • More coordination

  • More staff

  • More and different services

  • Limited access

  • Still, we know little about effectiveness of services


Purpose of this analysis l.jpg
Purpose of this Analysis Learned from the Past and Suggestions for the Future

Analyze 3 main reformmodels to find leveragepoints for improvement

Reform


Three approaches to reform l.jpg

Professionalism Learned from the Past and Suggestions for the Future

System Level

Evidence-BasedPractice

Three Approaches to Reform

Clinical

Outcome

SYSTEM

PROFESSION

TREATMENT


Professionalism l.jpg

PROFESSION Learned from the Past and Suggestions for the Future

Professionalism

Professionalism

Clinical

Outcome

Professionalism:The conduct, aims, or qualities that characterize or mark a profession or a professional person(Merriam Webster Dictionary, 2001)


Professionalism approaches l.jpg
Professionalism: Approaches Learned from the Past and Suggestions for the Future

1) Instruction

  • 3) Selection

Accreditation of providers

Preservice training

Inservice training

Licensing of

clinicians

Supervision

2) Experience

Requirement of practice experience


Is professionalism effective in improving clinical outcomes l.jpg

PROFESSION Learned from the Past and Suggestions for the Future

Professionalism

Is Professionalism Effective in Improving Clinical Outcomes?

?

Clinical

Outcome


Finding the evidence l.jpg
Finding the Evidence Learned from the Past and Suggestions for the Future

  • Instruction:

  • Preservice Training:

  • Christensen & Jacobson (1994)

  • Stein & Lambert (1995)

  • Shadish et al. (1993)

  • Inservice Training:

  • Beaudry (1989)

  • Davis (1998); Davis et al. (1992); Davis et al. (1995);

  • Bickman (1999)

  • Supervision:

  • Holloway & Neufeldt (1995)

  • Steinhelber et al. (1984)

  • Ellis et al. (1996)


Finding the evidence cont l.jpg
Finding the Evidence (cont.) Learned from the Past and Suggestions for the Future

  • Experience:

  • Sechrest, Gallimore, & Hersch (1967)

  • Dawes (1994)

  • Luborsky et al. (1980)

  • Smith & Glass (1977)

  • Stein & Lambert (1984)

  • Strupp & Hadley (1979)

  • Dush, Hirt, & Schroeder (1989)

  • Lyons & Woods (1991)

  • Stein & Lambert (1995)


Finding the evidence cont12 l.jpg
Finding the Evidence (cont.) Learned from the Past and Suggestions for the Future

  • Selection:

  • Accreditation:

  • Hadley & McGurrin (1988)

  • Bravo et al. (1999)

  • Licensing:

  • 173 publications on licensing (1967–1999) but no investigation of association between licensure and quality of professional services


Results l.jpg

PROFESSION Learned from the Past and Suggestions for the Future

Professionalism

Results

  • Currently there is not enough empirical evidence to make reliable conclusion whether professionalism is effective or not

?

Clinical

Outcome


What are the logic and the underlying assumptions of professionalism l.jpg

Profes- Learned from the Past and Suggestions for the Future

sionalism

What are the Logic and the Underlying Assumptions of Professionalism?


Instruction l.jpg

Educator/supervisor Learned from the Past and Suggestions for the Future

knows how to teach

best practice

  • Clinicians’ characteristics

  • Environmental characteristics:

    • Organization

    • System

    • Society

Educator/supervisor

knows best

practice

Instruction

Transportability

Assessment & Diagnosis

Clinician

receives

instructions

about best

practice

and how

to apply it

Clinician

Is in the

position

to learn

best

practice

Clinician

is

motivated

to

use best

practice

Clinician

Is

capable

of using

best

practice

Clinician

uses

best

practice

Clinician

can match

best

practice

to client

problem

Improved

clinical

outcome

Best practice

has been empirically

identified to be effective as well as efficacious

Professional’s and organizational context

Evidence base of treatment


Experience l.jpg

  • Clinician Learned from the Past and Suggestions for the Future

  • knows outcome of treatment

  • receives quick, continuous, and correct feedback

  • is aware of the processes, tactics, or strategies of their treatment

  • knows the relationships between processes and outcomes

  • is able to contextualize their knowledge to fit individuals

  • is able to generalize from individual cases to establish principles

  • is able to apply the knowledge

  • Clinician is motivated

  • to change his/her behavior

  • organizational barriers

Experience

Clinician

works

in a clinical

setting

Clinician obtains

experience

Clinician

learns

Clinician

positively

changes his/her

behavior

Improved

clinical

outcome

Professional’s andorganizational context and conditions

Process of learning

through experience


Selection l.jpg

Authority has Learned from the Past and Suggestions for the Future

empirical evidence

what kind of

organizational

or clinician

characteristics are

ineffective or bad

Note:

Conditions under which the bad apple strategy is effective

are unclear

Selection

Authority makes

selection

Ineffective or bad

organizations or

clinicians

are sorted out

The remaining

group of

organizations and

clinicians

are more effective

or better

Improved

clinical

outcome

Process of change


Three approaches to reform18 l.jpg

SYSTEM Learned from the Past and Suggestions for the Future

PROFESSION

Professionalism

System Level

Three Approaches to Reform

Clinical

Outcome

System of care:

Delivering coordinated but diverse services on an individualized basis using case management and interdisciplinary treatment teams to integrate and facilitate transition between services.


System level approaches l.jpg
System Level: Approaches Learned from the Past and Suggestions for the Future

  • Increasing access

    • Community-based services

    • Adding new services

    • Using the existing services more effectively

  • Organizing and systematizing

    • Cross-collaboration and coordination of services

  • Tailoring

    • Individualized services “wrapped” around the child and his/her family

    • Community-based services

    • Continuum of care

    • Diversification of services


Is system level reform effective in improving clinical outcome l.jpg

SYSTEM Learned from the Past and Suggestions for the Future

System Level

Is System Level Reform Effective in Improving Clinical Outcome?

?

Clinical

Outcome


Finding the evidence21 l.jpg
Finding the Evidence Learned from the Past and Suggestions for the Future

  • Fort Bragg Study (1995; Continuum of Care)

  • Stark County Study (1998; System of Care)

  • Wraparound Project Study (2001; Wraparound)

  • Comprehensive Community Mental Health System Reform (1993 – present; System of Care)


Results22 l.jpg

SYSTEM Learned from the Past and Suggestions for the Future

System Level

Results

  • System changes have been shown to

    • Increase satisfaction

    • Increase access

    • Increase costs

    • But not improve clinical outcomes

?

Clinical

Outcome


What are the logic and the underlying assumptions of system level reform l.jpg

System Learned from the Past and Suggestions for the FutureLevel

What are the Logic and the Underlying Assumptions of System Level Reform?


Increasing access l.jpg
Increasing Access Learned from the Past and Suggestions for the Future

Increase

access to

mental health

services

Services

are used

by

children

In need

More children

will receive

services

Improved

clinical

outcome

Client Context

Effective

treatments

have been

identified

Effective treatments

are offered

and implemented

correctly by

services

Evidence base of treatment

Transportability &Dissemination


Organizing and systematizing l.jpg

The critical Learned from the Past and Suggestions for the Future

assumption is

that saved

resources

are not invested

elsewhere

Same assumptions

as before need

to be met

Organizing and Systematizing

More

children

can

be served

Increased

access

Services are organized and

systematized

Services are used more

efficiently

Improved

clinical

outcome

Costs are

reduced

Saved resources

are invested

into additional

services

Policy context


Tailoring l.jpg

The correct treatment Learned from the Past and Suggestions for the Future

that matches the child

need has been

empirically

identified

Tailoring

Assessment & Diagnosis

Clinician is

able to

assess

child’s need

correctly

Clinician is

able to

match

the correct

treatment

with child’s

need

Services are tailored

to child needs

Child receives

more

appropriate

care

Improved

clinical

outcome

Evidence base of treatment


Three approaches to reform27 l.jpg

SYSTEM Learned from the Past and Suggestions for the Future

PROFESSION

TREATMENT

Professionalism

System Level

Evidence-BasedPractice

Three Approaches to Reform

Clinical

Outcome

EBP:

Use of scientifically evaluated treatments that havebeen shown to be efficacious and/or effective


Ebp approaches l.jpg
EBP: Approaches Learned from the Past and Suggestions for the Future

  • Setting standards

  • Finding and publishing empirically supported treatments (EST)

    • Efficacious studies

    • Effectiveness studies

  • Reviews and meta-analyses

  • Creating a (web-based) data base with ESTs

  • Developing manuals, guidelines, treatment algorithms


Is ebp effective in improving clinical outcome l.jpg

Clinical Learned from the Past and Suggestions for the Future

Outcome

TREATMENT

Evidence-BasedPractice

Is EBP Effective in Improving Clinical Outcome?

?


Finding the evidence30 l.jpg
Finding the Evidence Learned from the Past and Suggestions for the Future

  • Standards:

  • APA’s Division 12 Task Force, Committee on Science & Practice (1995)

  • Interdisciplinary committee on Evidence-Based Youth Mental Health Care

  • FDA

  • International Psychopharmacology Algorithm Project


Finding the evidence cont31 l.jpg
Finding the Evidence (cont.) Learned from the Past and Suggestions for the Future

  • Reviews & Meta-Analyses:

    • APA’s Division 12 Review of evidence-based therapies (1998)

    • Kazdin, Psychotherapy for children and adolescents, Oxford (2000)

    • Weisz & Jensen, Mental Health Services Research (1999)

    • JAACAP special issue on psychopharmacology (1999)

    • Rones & Hoagwood, School-based mental health services,Clinical Child and Family Psychology Review (2000)

    • Burns, Hoagwood, Mrazek; Child Clinical and Family Psychology Review (2000)

    • Chorpita et al., Hawaii Emperical Basis to Services Task Force (2001)

    • Surgeon General’s Mental Health Report (1999)

    • Surgeon General’s Youth Violence Report (2001)

    • Surgeon General’s Supplement on Culture, Race, & Ethnicity (2001)


Finding the evidence cont32 l.jpg
Finding the Evidence (cont.) Learned from the Past and Suggestions for the Future

  • Web-Based Archives:

    • Cochrane Collaboration

    • Campbell Collaboration

    • Center for Evidence-Based Medicine

    • Center for Evidence-Based Mental Health

    • FOCUS

    • Society of Clinical Psychology


Results33 l.jpg

Evidence-Based Learned from the Past and Suggestions for the FuturePractice

Results

?

?

  • Limited number of efficacious treatments

  • Very small number of effective treatments

  • Almost no research findings on transportability, implementation, & dissemination

?

Clinical

Outcome

?

TREATMENT


What are the logic and the underlying assumptions of ebp l.jpg

EBP Learned from the Past and Suggestions for the Future

What are the Logic and the Underlying Assumptions of EBP?


Slide35 l.jpg

Diagnostic Learned from the Past and Suggestions for the Future

approach is the

best match

Transportability

is good

Criteria and

standards for

evidence

are defined,

valid, and

agreed upon

Processes and

outcomes of

therapeutic

change are

assessable

with empirical

methods

EST are

accepted by

clinicians

EST fit

conditions

of real world

practice

(e.g. comorbidity,

feasibility)

Process of

therapeutic

change is

known

EBP

Evidence base of ESTs

is developed

EST’s are

implemented

in practice

Clinicians

provide more

efficacious

treatments

Improved

clinical

outcome

Measurement


What have we learned l.jpg
What have We Learned? Learned from the Past and Suggestions for the Future

We need to pay more attention to the underlyingassumptions of our reform models:

  • What are the real world conditions and context (client, practitioner, organization, policy)

  • Is it enough to intervene at one level only (e.g. professionals, treatment, system, organization)?

  • What are the actual processes of change?

  • How do we improve transportability and dissemination? (e.g. involve practitioners and providers)

  • What are the processes of practitioners’ behavior change (such as learning through experience)

  • How can we match the right treatments to individual clients?

  • How can we assure correct assessment?

  • How can we develop valid measures for processes and outcomes?


Continuous quality improvement as a reform the roles of measurement and feedback l.jpg
Continuous Quality Improvement As a Reform – Learned from the Past and Suggestions for the Future The Roles of Measurementand Feedback


Previous reform efforts had limitations l.jpg
Previous Reform Efforts Had Limitations Learned from the Past and Suggestions for the Future

  • They did not make all their assumptions explicit

  • They did not involve all levels of an organization

    • Professionalism deals with clinician

    • System level reform deals with system

    • EBP focuses mainly on treatment

  • They are not focused on improving the quality of treatment

  • They are not sensitive to the real world

  • They are externally implemented and not internally innovative


Another suggested reform l.jpg
Another Suggested Reform Learned from the Past and Suggestions for the Future

Implement continuous quality improvement (CQI) with an integrative concurrent consumer measurement system


Continuous quality improvement cqi l.jpg
Continuous Quality Improvement (CQI) Learned from the Past and Suggestions for the Future

  • Originally developed for industrial and manufacturing applications

  • CQI involves the use of assessment, feedback, and application of information to improve services

  • CQI relies on a continuous evaluation of processes and outcomes

  • CQI involves a dynamic interplay of assessment, feedback, and application of information

  • Typically requires changes in the organization and thus is multilevel

  • CQI can empower organizations, clinicians and clients


A cqi change process model l.jpg
A CQI Change Process Model Learned from the Past and Suggestions for the Future

System

Organization

DATA

SERVICES

  • process

  • outcome

INTERPRETATION

ACTION

  • guidelines

  • training

  • feedback to practitioner

  • system modification


Cqi was developed for simple processes l.jpg
CQI Was Developed for Simple Processes Learned from the Past and Suggestions for the Future

  • Highly sensitive process measures

  • Highly stabilized and replicable processes

  • Good ability to refine and “tweak” production processes

  • Relatively closed system - few external influences on processes and outcome

  • Few unexpected effects

  • Simple descriptive statistics

  • Strong causal linkage of inputs, process, outcome


Cqi is more difficult in human services l.jpg
CQI Is More Difficult in Human Services Learned from the Past and Suggestions for the Future

  • Few sensitive process and outcome measures

  • No systematic feedback process

  • Highly variable and ill defined processes

  • Low service replicability

  • Clinician behavior difficult to “re-program”

  • Open system - multiple external influences on structure, process, and outcome

  • May be detrimental to program goals:

    • Gaming when a basis for rewards or accountability

    • Goal displacement


Little experience with cqi l.jpg
Little Experience With CQI Learned from the Past and Suggestions for the Future

  • Limited experience in mental health

  • Exception is the Heidelberg-Stuttgart Model

  • Not a sufficient number of validated measures of quality exist

  • Clinician education and resistance are major factors

  • Resources seen as barrier to implementation

  • Few if any evaluations of CQI


What does cqi require l.jpg
What Does CQI Require? Learned from the Past and Suggestions for the Future

  • That we understand the links between process of care and outcome

  • That we systematically collect data on these links and its elements

  • That we feedback how processes affect outcomes

  • That we use information to change clinician behavior

  • That we create an atmosphere of change that supports treatment based on data


A cqi change process model46 l.jpg
A CQI Change Process Model Learned from the Past and Suggestions for the Future

System

Organization

Measurement

DATA

SERVICES

  • process

  • outcome

INTERPRETATION

ACTION

  • guidelines

  • training

  • feedback to practitioner

  • system modification

  • Feedback intervention


Measurement of processes and outcomes l.jpg
Measurement Learned from the Past and Suggestions for the Futureof Processes and Outcomes


Mental health measurement needs for cqi l.jpg
Mental Health Measurement Needs For CQI Learned from the Past and Suggestions for the Future

  • Sensitive, accurate, real time estimates of meaningful clinical change over time

  • A comprehensive and integrated system that uses consumer outcomes – e.g. symptoms, functioning

  • Sensitive and informative characterization of process indicators (therapeutic alliance, treatment modality, readiness to change, etc.)

  • Reduction in the paperwork burden for clinicians, youth and families


Components of the child adolescent measurement system cams l.jpg
Components of the Child Adolescent Measurement System (CAMS) Learned from the Past and Suggestions for the Future

  • Measures – concurrent and baseline

    • Functioning - strengths and impairment

    • Symptom severity

    • Hopefulness/satisfaction with life

    • Satisfaction with services

    • Victimization

    • Acuity

    • Substance use

    • Violent/aggressive behavior


Feedback intervention model to change clinician behavior l.jpg
Feedback Intervention Model to Change Clinician Behavior Learned from the Past and Suggestions for the Future

X

M

Y


What is feedback l.jpg
What is Feedback? Learned from the Past and Suggestions for the Future

  • General Feedback:The American Heritage Dictionary:“The return of a portion of the output of a process or system to the input, especially when used to maintain performance or to control a system or process.”

  • Feedback Intervention:Kluger & DeNesi (1996):“Actions taken by (an) external agent(s) to provide information regarding some aspect(s) of one’s task performance.”


Feedback interventions can improve performance l.jpg
Feedback Interventions can Improve Performance Learned from the Past and Suggestions for the Future

  • Possible to intervene so that professionals can learn more effectively from experience

  • Feedback intervention in several settings have been shown to have a positive effect on performance and client outcomes

  • Meta analysis show medium size effects in non-mental health settings


Feedback interventions in psychotherapy l.jpg
Feedback Interventions in Psychotherapy Learned from the Past and Suggestions for the Future

  • Little research on the effects of feedback to clinicians about client progress

  • Lambert has several studies showing feedback increased length of time in treatment as well as improved clinical outcomes

  • Feedback was simple colors – red, yellow, green on progress

  • Can’t tell how feedback operated but provides support for CQI


The effects of feedback interventions often vary l.jpg
The Effects of Feedback Interventions often Vary Learned from the Past and Suggestions for the Future

  • Feedback is not consistently an effective learning technique

  • In 1/3 of the studies the feedback decreased the quality of the performance

  • Effect sizes from differentstudies were very different

  • Feedback is not a simplephenomenon that is easilyimplemented


Previous research on feedback is limited l.jpg
Previous Research on Feedback is Limited Learned from the Past and Suggestions for the Future

  • Most research is applied and mainly atheoretical

  • The theoretically grounded research is artificial


Four step approach to developing a research program l.jpg
Four Step Approach to Developing a Research Program Learned from the Past and Suggestions for the Future

Theory application

Theory refinement

Theory testing

Theory construction


Step 1 theory construction l.jpg
Step 1: Theory Construction Learned from the Past and Suggestions for the Future

  • Used an interdisciplinary team of experts

  • Research synthesis through traditional literature reviews, meta-analytic procedures, etc.

  • Findings from cognitive psychology, learning theory, social psychology, organizational psychology, management, and other related fields

  • Work has already been started on this phase

  • Need to “drill down” into details

  • Move from macro perspective to micro perspective.


Our preliminary theory l.jpg
Our Preliminary Theory Learned from the Past and Suggestions for the Future


1 existing moderating conditions l.jpg
1) Existing Moderating Conditions Learned from the Past and Suggestions for the Future

  • Task characteristics:

  • Complexity

  • Level of task generated feedback

  • Workplace atmosphere

  • Leadership structure

  • Learning structure

  • Organization goal standard

  • Competitiveness among peers

Setting specific

Client

Characteristics

Practice

behavior

Organizational

characteristics

Practitioner

Characteristics

Implicit theory /

working model

  • Personality

  • Regulatory focus (prevention or promotion)

  • Cultural background (e.g collectivistic or individualistic)

  • Values

  • Knowledge

  • Competence

  • Personal goal standard

Competing demands

Organizational

goal standard


2 practice process chain l.jpg

  • For example: Learned from the Past and Suggestions for the Future

  • Normative FI

  • Velocity FI

  • Corrective FI

FI cues

2) Practice Process Chain

FEEDBACK

INTERVENTION

(FI)

PROCESS & OUTCOME

Practice

behavior

Process

mediators

Concurrent

outcome

Proximal

outcome

Ultimate

outcome


3 feedback evaluation part i l.jpg

  • Moderators: Learned from the Past and Suggestions for the Future

  • Previous feedback experiences

  • source of feedback

  • quality of feedback

  • Moderators:

  • organizational leadership structure

  • peer competitiveness

  • organizational learning structure

3) Feedback Evaluation Part I

Feedback is

disregarded

NO

Feedback is

considered valid

FEEDBACK

INTERVENTION

(FI)

YES

Feedback

information

Perceived

important

feedback

standard

discrepancy

Feedback-

goal standard

comparison

Goal is clear

and important

to practitioner

Personal

goal standard

YES

NO

Not attend to

Feedback


4 feedback evaluation part ii l.jpg

Moderators Learned from the Past and Suggestions for the Future

Moderators

4) Feedback Evaluation Part II

Motivation to

overachieve

goal standard

YES

Motivation to

keep or

increase effort

YES

(feedback

> standard)

Motivation to

reduce effort

Perceived

important

feedback

standard

discrepancy

NO

Current behavior

is considered

as successful

NO

YES

(feedback

< standard)

Disregard

feedback

Choosing

discrepancy

reduction

strategy

Motivation to

reduce

discrepancy

Motivation to

change

behavior

Abandon or

change goal

standard

Abandon or

change goal

standard

Practice

behavior


5 behavior change processes l.jpg

Moderators: Learned from the Past and Suggestions for the Future

Moderators:

5) Behavior Change Processes

Feedback

initiated

learning

YES

Attribution

of failure

to external

causes

Believe in

success

by learning

NO

Attribution

of failureto self

?

NO

Believe in

success

by increasing

effort or shift

attention

YES

Increase effort

or

shift attention

Motivation to

change

behavior is

eliminated


6 feedback initiated learning l.jpg

Moderators Learned from the Past and Suggestions for the Future

Moderators

Moderators

6) Feedback Initiated Learning

New behavior,

skill,

knowledge

effective?

Acquisition of

new behavior, skill,

knowledge

successful?

Motivation to

learn new

behavior / skill/

to gain new

knowledge

Effort to learn

new

behavior / skill/

to gain new

knowledge

YES

YES

NO

NO

Belief in success

by another trial

of learning

YES

Feedback initiated learning

NO

Attribution

of failure

to external

causes

Application of new skill,

behavior, knowledge

Attribution

of failure to self

?


7 attribution to self l.jpg
7) Attribution to Self Learned from the Past and Suggestions for the Future

Increased

learned helplessness

Decreased

job-satisfaction

Barriers to

leave job?

NO

Decreased

self-efficacy

Drop-out

Increase in

work-related

stress

Increased

burn-out

symptoms

YES

Changes to the self

Decreased work

motivation


Hypothesized permanent learning through feedback l.jpg

Restructuring of Implicit Theory Learned from the Past and Suggestions for the Future

X -> Y

A -> B

C-> Y

Y-> D

X OR C -> Y

Hypothesized Permanent Learning through Feedback

1. Feedback loop

2. Feedback loop

3. Feedback loop

4. Feedback loop


Step 2 theory testing l.jpg
Step 2: Theory Testing Learned from the Past and Suggestions for the Future

  • Whole theory cannot be tested

  • Select key elements to test

  • Primarily experimental and field based tests

  • Will need large number of cooperating professionals


Example of a test of the model l.jpg
Example of a Test of the Model Learned from the Past and Suggestions for the Future

  • Evaluate the impact of a feedback intervention, dissonance arousing academic detailing approach on assessment, diagnosis and treatment of 80 pediatricians for children with ADHD

  • Assess the extent of adoption of the American Academy of Pediatrician guidelines and its affect on children and families


Studies can take place in several settings l.jpg
Studies Can Take Place in Several Settings Learned from the Past and Suggestions for the Future

  • Settings including mental health, health, higher education and primary and secondary education

  • Enhance the generalizability of findings across types of practitioners, settings, and organizations


Step 3 theory refinement l.jpg
Step 3: Theory Refinement Learned from the Past and Suggestions for the Future

  • Based on several studies fromseveral fields the theory is modified

Advantage:

Findings can easily be integrated in dynamic model


Step 4 theory application l.jpg
Step 4: Theory Application Learned from the Past and Suggestions for the Future

  • Ultimate goal is the utilization of the theory in the real world to improve client outcomes

  • The theory generating process will uncover many potential leverage points for interventions that can be used individually or simultaneously across many different settings and professional fields


Conclusions l.jpg
Conclusions Learned from the Past and Suggestions for the Future

  • Previous reform efforts have been based on assumptions that were not explicit

  • There is not sufficient evidence to support these assumptions

  • We need to try alternative approaches to reforming services so as to improve outcomes for children

  • CQI is an attractive alternative to previous reform efforts

  • However we do not have sufficient experience with CQI in the human services

  • Practical measurement is available - technology for real time measurement is advancing


Conclusions cont l.jpg
Conclusions (cont.) Learned from the Past and Suggestions for the Future

  • Need investment in system infrastructure for measurement

  • Need to establish links between interventions and outcomes in real world (Quality)

  • Feedback intervention is potentially an effective approach but it is complex and requires more empirical research