1 / 30

Joint Meeting on Adolescent Treatment Effectiveness Washington, DC – April 10, 2012

R01AA017625 (PI: Garner). Contract #270-07-0191. Opinions are those of the authors and not official positions of the government. Joint Meeting on Adolescent Treatment Effectiveness Washington, DC – April 10, 2012.

skylar
Download Presentation

Joint Meeting on Adolescent Treatment Effectiveness Washington, DC – April 10, 2012

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. R01AA017625 (PI: Garner) Contract #270-07-0191 Opinions are those of the authors and not official positions of the government Joint Meeting on Adolescent Treatment EffectivenessWashington, DC – April 10, 2012 Pay-for-performance as a method to improve delivery of high-quality care: Results from the Reinforcing Therapist Performance (RTP) experiment Bryan R. Garner, Susan H. Godley, Michael L. Dennis, Brooke D. Hunter, Christin M. L. Bair, & Mark D. Godley Chestnut Health SystemsBloomington-Normal, IL

  2. Learning Objectives To learn more about… • Methods to improve the implementation of evidence-based practices • The effectiveness of pay-for-performance methods • The relationship between performance measures and client outcomes

  3. AAFT as the foundation of the Reinforcing Therapist Performance (RTP) Experiment Group of treatment organizations that are: • Receiving the same level of funding. • Implementing the same evidence-based treatment. • Receiving the same comprehensive training model. • Documenting treatment implementation using the same web-based program. • Documenting client outcomes using the same evidence-based assessment instrument. AAFT

  4. A-CRA Clinician Training and Certification Process AAFT Participant in Coaching Calls with Supervisors/Coaches Feedback Monitor Clinician Fidelity Treatment Manual and Training Workshop Record Therapy Sessions Achieve Basic A-CRA Certification In 9 procedures Adapted from Godley, Garner, Smith, Meyers, & Godley (2011)

  5. A-CRA Clinician Training and Certification Process AAFT + A-CRA Training Workshop A-CRA Treatment Manual

  6. A-CRA Clinician Training and Certification Process AAFT Participant in Coaching Calls with Supervisors/Coaches Feedback Monitor Clinician Fidelity Treatment Manual and Training Workshop Record Therapy Sessions Achieve Basic A-CRA Certification In 9 procedures

  7. AAFT Digital Session Recordings (DSRs) A-CRA Clinician Training and Certification Process

  8. Upload DSR to EBTx.org website

  9. A-CRA Clinician Training and Certification Process AAFT Participant in Coaching Calls with Supervisors/Coaches Feedback Monitor Clinician Fidelity Treatment Manual and Training Workshop Record Therapy Sessions Achieve Basic A-CRA Certification In 9 procedures

  10. 15 sites funded in October 2006 AAT 1: First Year AAFT 1: Second Year AAFT 1: Third Year 17 sites funded in October 2007 AAFT 2: First Year AAFT 2: Second Year AAFT 2: Third Year Improving the ImplementationProcess within AAFT How might we improve the process?

  11. Rewarding Provider Performance: Aligning Incentives in Medicare (IOM, 2007) Recommended Pay-for-Performance (P4P) as a promising method to improve the delivery of high quality of care. “The literature evaluating the effectiveness of pay for performance consists of fewer than 20 studies, yielding mixed conclusions on overall impact.”

  12. RTP study funded in October 2008 15 sites funded in October 2006 AAFT 1: First Year AAFT 1: Second Year AAFT 1: Third Year 17 sites funded in October 2007 AAFT 2: First Year AAFT 2: Second Year AAFT 2: Third Year Improving the ImplementationProcess within AAFT How might we improve the process?

  13. RTP Targets: Monthly A-CRA CompetenceTarget A-CRA $50 for each month that a randomly selected digital session recording (DSR) has at least one A-CRA procedure rated at or above the minimum level of competence required for A-CRA certification. $200 for each adolescent who they deliver an empirically derived level of A-CRA treatment that has been shown to be significantly related improved treatment outcomes (Garner et al., 2009, 2010). Aims and Design of RTP Experiment AAFT Aims: Examine the effectiveness and cost-effectiveness of providing monetary incentives to therapists as a method to improve treatment service delivery and subsequent treatment outcomes. Design: Randomly assigned AAFT grantees and their therapists to either: 1) Implementation-As-Usual (IAU) or 2) Pay-For-Performance (P4P)

  14. + + + RTP hypotheses to be tested Assignment to Experimental Pay-For-Performance Intervention A-CRA Competence TreatmentEffectiveness Treatment Implementation Performance Measures TargetA-CRA 6-month Remission Status

  15. Site 1 Therapist 1 Therapist 2 Therapist 3 1 2 3 4 5 6 7 8 9 Sample size & Data structure Level 3: 29 Sites Site 29 … Level 2: 105 Therapists … Therapist 104 Therapist 105 Level 1: 986 Clients 986 983 984 985 … 982

  16. Therapist Characteristics (N=105) a p < .05; b See Donnellan, Oswald, Baird, & Lucas (2006) for more information about Mini-International Personality Item Pool (Mini-IPIP)

  17. Client Characteristics (N=986) a p < .05;

  18. RTP Targets: Monthly A-CRA CompetenceTarget A-CRA $50 for each month that a randomly selected digital session recording (DSR) has at least one A-CRA procedure rated at or above the minimum level of competence required for A-CRA certification. $200 for each adolescent who they deliver an empirically derived level of A-CRA treatment that has been shown to be significantly related improved treatment outcomes (Garner et al., 2009, 2010). Results AAFT Aims: Examine the effectiveness and cost-effectiveness of providing monetary incentives to therapists as a method to improve treatment service delivery and subsequent treatment outcomes. Design: Randomly assigned AAFT grantees and their therapists to either: 1) Implementation-As-Usual (IAU) or 2) Pay-For-Performance (P4P)

  19. 23.7% Odds Ratio = 2.2p = .02 9.2% IAU P4P Results: A-CRA Competence

  20. RTP Targets: Monthly A-CRA CompetenceTarget A-CRA $50 for each month that a randomly selected digital session recording (DSR) has at least one A-CRA procedure rated at or above the minimum level of competence required for A-CRA certification. $200 for each adolescent who they deliver an empirically derived level of A-CRA treatment that has been shown to be significantly related improved treatment outcomes (Garner et al., 2009, 2010). Results AAFT Aims: Examine the effectiveness and cost-effectiveness of providing monetary incentives to therapists as a method to improve treatment service delivery and subsequent treatment outcomes. Design: Randomly assigned AAFT grantees and their therapists to either: 1) Implementation-As-Usual (IAU) or 2) Pay-For-Performance (P4P)

  21. 15.9% Odds Ratio = 5.2p = .01 3.3% IAU P4P Results: Target A-CRA

  22. RTP Targets: Monthly A-CRA CompetenceTarget A-CRA $50 for each month that a randomly selected digital session recording (DSR) has at least one A-CRA procedure rated at or above the minimum level of competence required for A-CRA certification. $200 for each adolescent who they deliver an empirically derived level of A-CRA treatment that has been shown to be significantly related improved treatment outcomes (Garner et al., 2009, 2010). Results AAFT Aims: Examine the effectiveness and cost-effectiveness of providing monetary incentives to therapists as a method to improve treatment service delivery and subsequent treatment outcomes. Design: Randomly assigned AAFT grantees and their therapists to either: 1) Implementation-As-Usual (IAU) or 2) Pay-For-Performance (P4P)

  23. Results: Remission Status

  24. Average Remission Status was 24% in Cannabis Youth Treatment (CYT) study (Dennis et al., 2004) Results: Remission Status ? No Significant Difference 50.8% 41.8% IAU P4P

  25. Poor Follow-up Rates Remission Status Unknown for 35% Remission Status Unknown for 44% 65% 56% Differential Client Attrition! IAU P4P

  26. Significant Differences No Significant Differences Poor Follow-up Rates Remission Status Unknown for 35% Remission Status Unknown for 44% 65% 56% Follow-up group was significantly more severe at baseline IAU P4P

  27. + + Post hoc analyses Assignment to Experimental Pay-For-Performance Intervention A-CRA Competence Not Supported TreatmentEffectiveness Treatment Implementation Performance Measures TargetA-CRA 6-month Remission Status Supported

  28. Take home messages • Modest-sized incentives can have large impacts on improving treatment implementation • Despite the large impact of P4P on the two treatment implementation measures, there remained considerable room for improvement even in the P4P condition.

  29. Next Step Examine the extent to which the improvements in A-CRA Competence and Target A-CRA were Cost-Effective.

  30. Thank You.

More Related