evaluating complex systems of care a logic model approach to ehdi program evaluation l.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
Evaluating Complex Systems of Care A Logic Model Approach to EHDI Program Evaluation PowerPoint Presentation
Download Presentation
Evaluating Complex Systems of Care A Logic Model Approach to EHDI Program Evaluation

Loading in 2 Seconds...

play fullscreen
1 / 33

Evaluating Complex Systems of Care A Logic Model Approach to EHDI Program Evaluation - PowerPoint PPT Presentation


  • 248 Views
  • Uploaded on

National EHDI Conference Atlanta, Georgia, March 3, 2005. Evaluating Complex Systems of Care A Logic Model Approach to EHDI Program Evaluation. Sherry Spence, MA Oregon Department of Human Services Health Services, Office of Family Health.

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Evaluating Complex Systems of Care A Logic Model Approach to EHDI Program Evaluation' - virgo


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
evaluating complex systems of care a logic model approach to ehdi program evaluation

National EHDI Conference

Atlanta, Georgia, March 3, 2005

Evaluating Complex Systems of CareA Logic Model Approach to EHDI Program Evaluation

Sherry Spence, MA

Oregon Department of Human Services

Health Services, Office of Family Health

Supported in part by funding from the Maternal and Child Health Bureau (Title V, Social Security Act), Health Resources and Services Administration, and by the Centers for Disease Control and Prevention, US Department of Health and Human Services

Opinions stated do not represent those of CDC, HRSA, or USDHHS

acknowledgements
Acknowledgements
  • EHDI Staff at CDC and HRSA
  • EHDI Data Committee
  • Oregon’s EHDI Team
    • Claudia Bingham
    • Al Ferro
    • David Laszlo
    • Maureen McNamer
    • Amy Rosenthal
acknowledgements3
Acknowledgements
  • FamilyNet Executive Sponsors
    • Katherine Bradley
    • Lorriane Duncan
  • Office of Family Health Epidemiologists and Research Analysts
  • FamilyNet Project Teams and . . .
foreword
Foreword
  • “If you torture data sufficiently, it will confess to almost anything.”
    • Fred Menger (1937- ), chemistry professor
    • From A Word A Day (AWAD) - wsmith@wordsmith.org
foreword6
Foreword
  • Corollary: “Sometimes you can coerce data by threatening deletion.”
    • Arthur Epstein, professor of chemistry and physics (from daughter, Melissa, responding in AWADmail in response to the Menger quote)
foreword7
Foreword
  • Conclusion: “There are lies, damn lies, and statistics.”
    • Mark Twain (1835-1910), author and humorist
what is our goal for today
What is our goal for today?
  • Disprove the Twain/Menger-Epstein hypothesis
  • Working hypothesis:
    • It is possible, without false testimony, torture, or coercion, to develop an evaluation plan using our program data.
what is our goal for today10
What is our goal for today?
  • Define a logical framework for evaluating EHDI programs
  • Discuss whether our logic model approach avoids the Twain/ Menger-Epstein condition.
  • Figure out where our model fits in identifying newborn hearing screening as a best practice.
what are we currently evaluating

What are we currently evaluating?

Logical framework for EHDI evaluation

slide12

Within 10

Days of

Screening,

Testing or

Referral

Hospitals

Diagnostic

Facilities

Early Intervention

Facilities

Screening Facilities

Demographics, screening, testing and referral information to DHS

Monthly reports, program statistics back to facilities

DHS

DHS Response When:

Child

Is

6 weeks of age

Parent

Healthcare Provider

Reminder letters / program contact only when child has not completed the desired EHDI process by the desired time.

Participating facilities, HCP and LPH able to return updated contact information

6 weeks

after

“Refer”

Parent

Healthcare Provider

Local Public Health

2 weeks

after

Dx Hearing

loss if no

EI referral

Parent

Healthcare Provider

Local Public Health

what are we trying to prove
What are we trying to prove?
  • Success in achieving EHDI goals
    • Early screening of everyone
    • Early identification of some
    • Early preventive or treatment intervention
    • Acquisition of language and communication skills
what else do we need to show
What else do we need to show?
  • Success in achieving EHDI goals
  • Success in achieving health care goals & optimizing resource use
  • Success in all children achieving their full health and development potential
what are the components of an ehdi logic model17
Children interacting and ready to learn

Language and communication skills

Intervention to assure skill acquisition

Early and complete identification and risk monitoring

High level outcomes - population attributes

Intermediate outcomes - client/patient attributes

Outputs - program activities and interventions

What are the components of an EHDI logic model?
what is the approximate timing of this monitoring
All children ready to interact and learn

Language and communication skills

Intervention to assure skill acquisition

Early and complete identification and risk monitoring

4-5 years after birth

12-36 months after birth

3-6 months after birth

1-3 months after birth

What is the approximate timing of this monitoring?
how can we use this logic model in program evaluation
Conduct research

Conduct research

Monitor program outcomes

Monitor program protocols

4-5 years after birth?

12-36 months after birth?

3-6 months after birth

1-3 months after birth

How can we use this logic model in program evaluation?
will system integration help
Will system integration help?
  • Earlier screening and identification
  • More complete screening and identification
  • More timely follow-up
  • Less loss to follow-up
why do we think this is so
Why do we think this is so?
  • About 40% of hearing loss occurs with other conditions or syndromes, resulting in additional disabilities (Gallaudet Research Institute, 2003)
    • Additional issues increase likelihood of other public health program involvement.
  • In Oregon, about 50% of women giving birth are enrolled in WIC and will enroll their babies
    • The WIC program keeps track of its highly mobile population.
what are our intermediate system outcomes
What are our intermediate system outcomes?
  • COORDINATION
  • CONTINUITY
  • EFFICIENCY
  • TIMELINESS
  • ACCURACY & APPROPRIATENESS
how does this fit with the logic model
How does this fit with the logic model?
  • Population success
  • Program success
  • Process success

Program Logic Model

System Logic Model

conclusions
Conclusions
  • One purpose of a system of services is to assure the success of the individual programs and processes it involves.
  • One purpose of a data system is to facilitate and enhance the success of the programs that use it.
what do we have so far
What do we have so far?
  • Solid framework for measuring program processes
  • Solid framework for measuring system processes
what s the key to all this
What’s the key to all this?
  • The logic model works best if the program intervention is an established, evidence-based best practice.
what best practice evidence to we need
Gold standard

Controlled studies by programs

Expert testimony by professionals

Repeated measures and testimony

Immoral or unethical

Good evidence from which we can generalize?

Good evidence or bias?

Eliminates bias and demonstrates generalizability

What best practice evidence to we need?
what evidence do we have
What evidence do we have?
  • Prospective and retrospective studies by programs and researchers?
  • Expert testimony by speech, hearing, and language professionals?
  • Repeated measures of all kinds?
thank you

Thank you.

Thanks to

Amy Rosenthal, Oregon EHDI Coordinator for her participation and to

Scott Grosse, PhD, NCBDDD/CDC, for his participation and for noting an error in slide 32.