Provider perceptions of the child outcomes summary process
This presentation is the property of its rightful owner.
Sponsored Links
1 / 38

Provider Perceptions of the Child Outcomes Summary Process PowerPoint PPT Presentation


  • 85 Views
  • Uploaded on
  • Presentation posted in: General

Provider Perceptions of the Child Outcomes Summary Process. Lauren Barton and Cornelia Taylor October 27, 2012. Measuring and Improving Child and Family Outcomes Conference Minneapolis, MN. Today’s Session. Brief ENHANCE project update Describe provider survey study Share findings

Download Presentation

Provider Perceptions of the Child Outcomes Summary Process

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Provider perceptions of the child outcomes summary process

Provider Perceptions of the Child Outcomes Summary Process

Lauren Barton and Cornelia Taylor

October 27, 2012

Measuring and Improving Child and Family Outcomes Conference

Minneapolis, MN


Today s session

Today’s Session

  • Brief ENHANCE project update

  • Describe provider survey study

  • Share findings

  • Discuss implications


Enhance project update

ENHANCE Project Update


Enhance

ENHANCE

  • Focus

    • Under what conditions are COS data meaningful and useful for accountability and program improvement?

    • Impact of COS on programs/staff

    • Identify needed revisions to COS form and guidance

  • SRI International was funded by

    U.S. Dept. of Education, IES -- July 1, 2009


Four enhance studies

Four ENHANCE Studies

  • Comparison with Child Assessments

  • Team Decision-Making

  • State Data Study

  • Provider Survey


Studies 1 2 4 34 project data collection sites

17 Part B Preschool (3-5)

Illinois

Maine

Minnesota

New Mexico

Texas

South Carolina

Studies 1,2, & 4:34 Project Data Collection Sites

17 Part C (Birth to 3)

  • Illinois

  • Maine

  • Minnesota

  • New Mexico

  • Texas

  • North Carolina

  • Virginia


Comparison with child assessments study

Comparison with Child Assessments Study

Goals

  • Compare COS ratings to BDI-2, Vineland-II scores

    • Program Entry

    • Program Exit

  • Compare conclusions from COS and assessments

    Study Status

  • Final stages of recruiting families

    Find Out More

  • See poster at reception for promising

    findings from preliminary analyses


Team decision making study

Team Decision-Making Study

Goals

  • Learn more about implementation of the COS process.

  • What is team understanding of outcomes and rating criteria?

    Study Status

  • Videos coming in

  • Early stages of coding videos

    Find Out More

    Tabletop discussion tomorrow morning

    • What is quality COS process?

    • Share draft video coding protocol


State data study

State Data Study

Goals

  • Analyze characteristics of COS data

  • Are patterns consistent with those predicted for high quality data?

    Status

  • Analyzing and compiling data

    Find out more

  • See reception poster with early findings

    We are looking for more states

    to participate.


Participating in the state data study

Participating in the State Data Study

Participate at different levels:

  • Share results from analyses you already do for APR OR

  • Share de-identified child level data set used in your APR analyses

  • Work with SRI to either:

    • Share expanded data set* that we analyze and discuss with you OR

    • Conduct extra analyses and share results with project

      * Expanded data set examples:

      include extra child descriptors (race, gender,

      primary disability) or variables describing

      the setting or composition of services


Benefits of participation

Benefits of Participation

  • See how your data fits relative to other state patterns

  • Have more analyses to learn from—extra tables for deeper understanding

  • Technical assistance around unique issues in your data

  • Make a difference, generating new knowledge about a topic that matters

For more information, contact [email protected](650) 859-5314


Enhance provider survey

ENHANCE Provider Survey


Provider survey

Provider Survey

Question:

How is COS implementation going?

Method

  • Online survey sent to providers in program participating in the assessment and video studies

    Study Status

  • Completed data collection

  • Early stages of analysis


Provider survey sample

Provider Survey Sample

  • 856 providers in 8 states

  • Primary population

    • EI: 472 (55%)

    • ECSE: 302 (35%)

    • Mix: 82 (10%)

  • Roles

    • 50% early interventionists/teachers

    • 38% therapists and asst. (SLP, OT, PT)

    • 9% coordinators/psychologists

    • 3% other

  • 75% previously worked with children without disabilities in some capacity.


Provider experience with cos ratings

Provider experience with COS ratings

  • 51% 31 or more COS ratings

  • 21% 10 or fewer COS ratings


Provider training

Provider training

  • Asked about training across a variety of formats (e.g., in person, technology, one-on-one, material review)

  • How many total hours of training do you think most providers reported?

    None?

    1-4 hours?

    5-8 hours?

    9-12 hours?

    13 hours or more?

ECO recommends 8-12 hours of training

to get familiar with the COS process


Provider training on cos process is limited

Provider training on COS Process is limited

  • 90% of providers received some training

  • 68% reported 4 hours or less of training


Cos process time involved

COS Process: Time involved

  • On average, how long does it take to identify a child’s outcome ratings and document it on the form?

    • Identify rating

    • Discuss child’s functioning if directly related to rating decision or exceeds discussions that would have occurred anyway

    • Time to complete information on the form

    • Do NOT include data entry of form, if done after form is completed.

      1-15 min., 16-30 min., 31-45 min., 46-60 min., more than 60 min.


Cos rating process is fairly brief

COS Rating Process is Fairly Brief

  • 36% 1-15 minutes

  • 35% 16-30 minutes

  • EI reported less time than ECSE


What do you think a team process

What do you think?A team process?

Teaming practices vary a lot from program to program.

  • How often are COS ratings actually decided in teams?

  • Do the teams include more than one professional?

  • How often are families involved?


Cos process more than one professional usually was involved

COS Process: More than one professional usually was involved.

75% had most ratings decided by a

team (at least 1 professional in addition to respondent)

79% had at least one other professional provide input about the child’s functioning


Cos process family involvement was limited

COS Process: Family involvement was limited.

34% had family members present for

most of their COS rating decisions.

68% considered input from family members in determining the rating


F unctioning across settings and assessment data was considered in most cos ratings

Functioning across settings and assessment data was considered in most COS ratings.

75% used information about the child’s functioning from

multiple settings and situations

76% used information from one or more assessment tools during rating decisions


Most felt their cos process matched what it was supposed to be

Most felt their COS process matched what it was supposed to be.

78% thought the process used for

deciding COS ratings matched

how it was supposed to be done


Most felt the process was thorough enough to be effective

Most felt the process was thorough enough to be effective.

  • 85% felt there was enough information about each outcome area to determine a COS rating

  • 80% felt there was enough time to review child functioning in each outcome area.

  • 85% in most of the decisions, all involved considered information carefully in an effort to decide an accurate rating.


Most providers felt that most cos ratings given were accurate

Most providers felt that most COS ratings given were accurate.

76% felt confident that most of the ratings given

were accurate

88% reported that NONE of the COS ratings were

selected to make the program look good


Skills needed for cos ratings

Skills Needed for COS Ratings

  • Training should focus on skills essential for COS ratings. For example:

    • Understanding age-expected functioning

    • Comparing specific child’s skills to age-expected

    • Understanding the 3 child outcomes

    • Discussing functioning in outcomes with others

  • To what extent do you think providers report having these kinds of skills?


Providers said they understood the content behind cos ratings

Providers said they understood the content behind COS ratings.

Age-expected functioning

  • 89% understood age expected functioning

  • 92% knew how to compare children’s functioning to what is age expected

    The three child outcomes

  • 85% understood the three child outcomes

  • 83% knew how to discuss functioning in the outcomes with others

* Endorsed statements of understanding

as mostly true or very true.


Understanding the purpose and uses for cos data

Understanding the Purpose and Uses for COS Data

  • How much do providers

    • Understand why COS data are collected,

    • Realize what happens with the data, and

    • Feel they can explain the need for gathering COS data?

      Same or different?


Few providers understood why and how cos data are used

Few providers understood why and how COS data are used.

  • 65% understood why COS data are

    being collected

  • 37% understood what happens with

    the data

  • 52% knew how to explain the need

    for child outcomes data to others


What kinds of feedback and support are available to providers

What kinds of feedback and support are available to providers?

  • What do providers think about the system of feedback and support available related to the COS?

    • Are there people available to help them?

    • Is ongoing support adequate?

    • Are there people reviewing the COS forms?

      Excellent, good, adequate, fair, poor, or a combination??

“Tell me and I forget,

teach me and I may remember,

involve me and I learn.”

- Benjamin Franklin


Limited ongoing support for providers with the cos process

Limited ongoing support for providers with the COS process

82% someone is available to provide support

if I ask for it

50% someone in my program provides support

47% ongoing support related to the COS process

is adequate

37% someone in my program checks

completed COS forms for accuracy


How do providers describe the impact of cos on their practice

How do providers describe the impact of COS on their practice?

  • What do providers say about how the COS process impacts them?

    • Change awareness?

    • Influence assessment or IFSP/IEP outcomes?

    • Influence conversations and relationships with others?

    • Impact on time for other activities?

      Does it seem? Positive? Neutral? Negative?


Neutral impact of cos process on practice

Neutral impact of COS process on practice

Overall impact of COS

on your work with

children and families

Specifics reported about COS Process:

  • 2% had negative impacts on relationships with families

  • 17% improved the assessment process

  • 31% takes time away from other important actives

  • 30% helps focus discussion on the whole child


Summary and implications

Summary and Implications


Summary cos process

Summary: COS Process

  • Most received limited training and support

  • Providers felt comfortable with background content

  • Had limited understanding about what happens

    with the data or how to explain it

  • Didn’t feel like the COS process

    impacts their work much

Got quality

COS data?


Implications and discussion

Implications and Discussion

  • Implications of findings

    • For professional development?

    • For supervision and administrators of programs?

    • For data quality?

  • Learning from providers

    • Have others done surveys or focus groups to learn from providers?

    • Why or why not?

    • How can surveys like this inform improvement activities in your state?


Find out more

Find out more

ENHANCE Website

http://ENHANCE.sri.com

ECO Center Website

http://www.the-ECO-center.org

Contact ENHANCE staff

Email: [email protected]


  • Login