1 / 15

Andrew L. Cherry, DSW, ACSW and Mary Dillon, Ed.D., MSW

Lessons Learned From a Five-Year Evaluation of the Oklahoma Co-Occurring Disorder State Incentive Grant (OK-COSIG) Project. Andrew L. Cherry, DSW, ACSW and Mary Dillon, Ed.D., MSW Anne & Henry Zarrow School of Social Work, University of Oklahoma, Tulsa

aolani
Download Presentation

Andrew L. Cherry, DSW, ACSW and Mary Dillon, Ed.D., MSW

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Lessons Learned From a Five-Year Evaluation of the Oklahoma Co-Occurring Disorder State Incentive Grant (OK-COSIG) Project Andrew L. Cherry, DSW, ACSW and Mary Dillon, Ed.D., MSW Anne & Henry Zarrow School of Social Work, University of Oklahoma, Tulsa CSWE 2010 APM Poster Sessions, Portland Oregon

  2. Science to Service • “It takes well over a decade for proven interventions to make their way into practice” (Hoge, et al., 2007). This poster session reviews lessons learned about success, resistance, and failure based on a five-year statewide evaluation (2004-2009) to provide integrated services for people with the co-occurring disorders of addiction and mental illness.

  3. Background • In 2004, the Oklahoma Department of Mental Health and Substance Abuse Services (ODMHSAS) received a Co-Occurring Disorder State Incentive Grant (COSIG) grant from Substance Abuse and Mental Health Services Administration (SAMHSA).  These grants were designed to fund states with a plan for infrastructure development and enhancement that promote services to people with the co-occurring disorders of addiction and mental illness.  The overarching goal of the OK-COSIG Project was to make those infrastructure changes at the state level and at agencies with which it contracts for treatment services. The OK-COSIG Project was funded for five years (2004-2009).  The award was in the amount of 3.3 million dollars.

  4. Two Major Goals of the OK-COSIG Project • To accomplish infrastructure changes:  • Goal 1 was to develop, implement, and evaluate a standard protocol for the screening and assessing people with a co-occurring disorder at mental health and substance abuse treatment facilities in all State funded programs.  • Goal 2 was to develop, implement, and evaluate an integrated treatment model for persons with co-occurring disorders that would be accessible, culturally competent, and grounded in evidence-based practices.

  5. Overarching Hypothesis • The Overarching Hypothesis for the OK-COSIG Project. • “People with a co-occurring disorder who receive treatment from an OK-COSIG pilot program will make significantly more positive changes than people with a co-occurring disorder who do not receive treatment from an OK-COSIG pilot program.” • This overarching hypothesis served to guide the evaluation of the OK-COSIG pilot program. • In addition, attention was paid to the steps involved in implementing the pilot program, the types of system integration that occurred or did not occur, and how these changes impacted the outcome of people with a co-occurring disorder who received treatment from the pilot programs.

  6. Evaluation Design • To determine if the Goals and Objective were accomplished, the University of Oklahoma, Anne and Henry Zarrow School of Social Work, COSIG Evaluation Team collected data on the Project from numerous sources, including large data files collected by the State Department that consisted of observations of individuals treated at model and control programs.  Data from key informant interviews, focus groups, program fidelity assessments, surveys of clinicians, and direct observation.  • To manage the enormous tasks of collecting qualitative and quantitative data needed to determine outcome, the study was organized into Macro, Mezzo, and Micro domains.  The Macro Level Evaluation was designed to determined infrastructure changes made at the state level.  These changes were based on qualitative data.  The Mezzo Level Evaluation measured Program Level outcomes based on quantitative data from the Individual Client Information System (ICIS) data files for years, FY 2005-2006, FY 2006-2007, and FY 2007-2008.  The total sample of cases from the 32 treatment programs totaled 85,046 cases.  Additionally, Mezzo level data was collected in a systematic way over five years from key informants and focus groups.  The Micro Level evaluation used the ICIS data collected by the state, and the Government Performance and Results Act (GRPA) data collected on individual clients.   

  7. Research Methods • We conceptualized the state of Oklahoma as a natural laboratory, one in which we were going to conduct a quasi-experimental research study over five years.  Using this methodology, we tested for a cause and effect relationship.  In this case, the cause or the intervention employed could be observed in the field for five years.  The effect or goals of the intervention were measurable.  • In quasi-experiential research it is important to measure all influences that might affect change in the dependent variable.  In our study, we were lucky, the cause (the OK-COSIG Project Implementation Team) was easily isolated from other influences that could have caused the change.  There were no other groups or organizations in the state at the time promoting infrastructure change to provide treatment for co-occurring disorders. 

  8. Determining the Impact • To determine the impact of the OK-COSIG Project, the evaluation plan used a mixed approach (qualitative and quantitative) and multi-level measures.  Outcomes were determined in three areas:  1) the result of infrastructure changes made at ODMHSAS to support services for people with co-occurring disorders, 2) the degree of service coordination and networking among mental health and substance abuse model treatment programs to provide coordinated mental health and substance abuse treatment and 3) outcomes of recipients of co-occurring services from the model programs as compared to the control programs (i.e., programs providing typical treatment).

  9. Results • Outcomes, supported by the data. • In Oklahoma today, people with a co-occurring disorder have a much greater chance of being identified and receiving treatment that is responsive to his or her co-occurring disorder.  • Without, the OK-COSIG Project (an effort to move science to service), this level of service and treatment for people with a co-occurring disorder in 2009 would not have been available for years to come. • A Focus Group Question:  What would you say was most responsible for the services that have been developed for people with a co-occurring disorder?  • Unanimous Answer:  “The OK-COSIG Project.” • A Follow-up Focus Group Question:  How long would it have taken to reach this same point without the OK-COSIG Project? Answer:  “Somewhere between 10 and 20 years.”

  10. Putting a Face on People with a Co-occurring Disorder • More likely to be Homeless. • More likely to have a spotty work history. • Less likely to have a high school education. • More likely to be living in community shelters. • More likely to be living in a residential care home. • More likely to have above “average” legal problems including DHS child custody issues. • Less likely to leave “ACA.” • Less likely, if female, “to start treatment.” • More likely to complete treatment The paradox is that patients who presented with an indication of a COD spent about half as many days in treatment than patients who did not present with an indication of a COD. • More likely to be forced into treatment.

  11. Lessons Learned Successes 1. Training clinical staff made a significant difference in positive outcomes of people with a Co-occurring Disorder. 2. Majority of State agencies developed competencies needed to treat Co-occurring Disorders. 3. Changes in state law and rule were made to include the treatment of people with a Co-occurring Disorder. • The goal of infrastructure change at the State and Treatment Program level that supports integrated treatment for people with co-occurring disorders was accomplished. • Developed and disseminated a simple, quick, and free Co-occurring Disorder screen that could be used to screen for people with a co-occurring disorders.

  12. Lessons Learned Resistance 1. Resistance from some State MH/SA Department supported treatment agencies where administrators discouraged clinicians from diagnosing clients with a Co-occurring Disorders. 2. Resistance from State MH/SA Department Leadership to allow the micro-level evaluation for fear of negative findings. 3. Resistance from administrators at the agency level to make the changes needed to be made in their agencies to provide treatment to people with a Co-occurring Disorders. 4. Resistance to the concept that they (the State MH/SA Department) had no control over what I (an outside evaluator from OU SSW) wrote or published. 5. Resistance by the State MH/SA Department to making pay-per-service formula changes in their contracts to help agencies re-tool to better meet the needs of people with a co-occurring disorder.

  13. Lessons Learned Failure 1. Failure to facilitate buy-in from shareholders outside of State supported agency, Racial (African American), Ethnic (Hispanic and Native American), NAMI, Mental Health agencies not providing therapeutic services.. 2. Failure to build a level of collaboration between Mental Health and Substance Abuse Services. For example, a therapeutic process was never developed that allowed clients with co-occurring disorders to move between Mental Health and Substance Abuse treatment agencies. 3. Failure by the State MH/SA Department to support the COSIG initiative beyond the 5 years grant period. 4. Failure by SAMHSAS to support the COSIG initiative beyond an 8 year time frame. SAMHSAS was successful implementing the initiative but lacked follow through. This Federal initiative could have been far more successful if there was more oversight and follow through on outcomes at the state level and during a follow-up period. 5. Failure by the State Department to develop a viable financial policy to promote Co-occurring Disorders treatment.

  14. Was the OK-COSIG Project worth 3.3 million dollars? Using the data collected for this study, the difference in days in treatment can be used to determine a rough estimate of costs and savings. First let’s say (in round numbers) that at least 25,000 people were treated at Model Programs over three years. Also, based on the data collected, we can estimate that people with a co-occurring disorder were in treatment an average of 50 days per case. The average days in treatment for all other treatment groups was at least 100 days. Based on a treatment average cost of $50 a day, we can estimate the difference in cost of treating people with co-occurring disorders. Using these estimates, the saving over five years was at least $16.5 million dollars. Future savings in scarce social dollars and the reduction of human suffering from the struggle with a co-occurring disorder cannot be calculated. The potential savings in dollars statewide is worth the investment of training dollars to increase the Co-occurring Capacity of State treatment programs.

  15. Additional Material • Lessons Learned From a Five-Year Evaluation of the Oklahoma Co-Occurring Disorder State Incentive Grant (OK-COSIG) Project • Andrew L. Cherry, DSW, ACSW and Mary Dillon, Ed.D., MSW Anne & Henry Zarrow School of Social Work, University of Oklahoma, Tulsa • CSWE 2010 APM Poster Sessions, Portland Oregon • The material that this poster session was developed from can be accessed at these websites. • http://faculty-staff.ou.edu/C/Andrew.L.Cherry-1.Jr/okcosig_project.htm • http://faculty-staff.ou.edu/C/Andrew.L.Cherry-1.Jr/AC-CODScreenPg.htm • http://faculty-staff.ou.edu/C/Andrew.L.Cherry-1.Jr/CSWE%20Presentation%20Material.htm

More Related