Consumer-Operated Service Program

1. 12th Annual Conference on State Mental Health Agency Services Research, Program Evaluation & Policy February 11, 2002 Baltimore, Maryland WORKING SCIENCE: Consumer Operated Service Program (COSP) Consumer-Operated Service Program Multi-Site Research Initiative STUDY OVERVIEW Jean Campbell, Ph.D. Principal Investigator COSP Coordinating Center

2. WorkingScience • With the surge of interest in evidence-based systems of care, it is important to “re-search” in a literal sense how scientific work is conducted, and to dialogue about the challenges to knowledge production and practice.

3. Evidence-Based Issues • Relevance for Policy • Efficacy of Evidence • Value of Consumer Involvement • Knowledge Transfer • Closing the gap between research & practice -Domination of Big Science -Cookie-cutter services

4. Panel Contents • Study Overview • Multisite Research Methodology • Tools for Quality Evidence • Preliminary Baseline Findings • Measuring Functioning & Disablement • Do’s & Don’ts of Cost Studies • Definition & Assessment of Programs

5. Value-Driven Research Research ought to and can enhance consumer choice, power, and knowledge* *From Consumer/Survivor Mental Health Research & Policy Work Group (1993)

6. Background • Historical support and promotion of consumer issues and programs by CMHS in the past lead to the funding of research on consumer-operated service programs • Commitment by SAMHSA to multisite studies as one of the most effective research designs for knowledge development

7. State of the Evidence • Prior studies of consumer-run programs suggest that they improve symptoms, promote larger social networks, and enhance quality of life. However, the evidence is limited: -uncontrolled studies -demonstrations of feasibility -preliminary findings

8. Goals of the Multisite Study • Establish the extent to which consumer-operated services when offered as an adjunct to traditional mental health services are effective in improving selected outcomes for people with severe mental illness • Create strong and productive partnerships among consumers, service providers and service researchers • Disseminate the knowledge gained

9. Importance of Multisite Study • Determine cost-effectiveness of consumer-operated programs -What works for whom at what cost? • Such evidence is necessary for consumer-run programs that seek to be partners in the continuum of community care -Funding & employment opportunities • Study results can promote new programs, improve quality of existing programs, expand services for people not easily engaged in traditional services, & reduce costs

10. What is a COSP? We are studying consumer-operated service programs that are administratively controlled and operated by consumers and emphasize self-help as their operational approach

11. Program Models • Drop-in Centers • Educational & Advocacy Training Programs • Peer or Mutual Support Services

12. Connecticut Florida/California Illinois Maine Missouri Pennsylvania Tennessee Participating Study Sites

13. Participating Study Sites

14. Selected Outcomes • Employment • Empowerment • Housing • Service Satisfaction • Social Inclusion • Costs • Well-being

15. Cost Study Questions To what extent does participation in consumer-operated service programs affect costs for the following: • inpatient • crisis intervention • emergency room utilization • offsetting costs in housing, criminal justice, vocational rehabilitation, physical health care, and income support

16. Research Design 4-Year Study Rigorous Methodology • Multisite Design • Random Assignment • Experimental Intervention: Consumer-Operated Program + Traditional Mental Health Services Control: Traditional Mental Health Services Only

17. Research Design Logic Model Common Protocol Data Collection Baseline, 4, 8, 12 months N=1,927

18. Multi-level, Cross-site Analysis • Data quality assessment & development of measures • development of composite scores & scales • missing values analysis • equivalencies • power & effect size • Baseline Data Analysis • measures of central tendency & dispersion • factor analysis • correlations • study population description

19. Multi-level, Cross-site Analysis • Data Clustering & Pooling • Longitudinal Analyses • pattern-mixture modeling • differential loss to follow-up • random regression • hierarchical linear modeling

20. Key Operational Values of Study • Consumer Involvement • Consumer Education • Extensive Technical Assistance • Electronic and Interactive Communications • Collaboration • Cultural Competency

21. EmpoweringConsumer Collaboration SC Consumer Involvement SC Consumer Representatives Consumer Advisory Panel Study Sites Consumer Involvement Site Consumer Advisory Boards Consumer Researchers CC Consumer Research Support Research Glossary Workshops Technical Assistance

22. Building Consumer/Researcher Trust • Easy access to project information • Supportive communications infrastructure • Developing a culture of understanding • Telling our stories • Use of the language “we,” “our” and “us” • Voting, focus groups, and ad hoc subcommittee meetings • Use of conflict resolution and facilitation

23. ACCOMPLISHMENTS Rigorous study design & questionnaire Interviewer training manual & video Common Ingredients & Assessment Tool Research glossary & technical assistance manuals Data entry software program Multisite data repository Quality monitoring tools Extensive listserv network State-of-the-art web page

24. ACCOMPLISHMENTS • Site visits ongoing • Baseline data collection completed & analysis ongoing • 4, 8, 12 month follow-ups ongoing

25. 1,927 Study Participants! Largest Study of Consumer Programs in History

26. CHALLENGES Unexpected effort required for collaboration among study constituents while conducting rigorous research • Unprecedented consumer involvement • Inclusive decision-making process • Extensive communications infrastructure • Professional facilitation and conflict resolution services • Increased technical assistance needs

27. Creating a Learning Community Learning is the core objective and this should guide decision- making. Mike English, CMHS

28. Value of Consumer Involvement • Study has the potential to identify an evidence-based practice for people who are not easily engaged in the mental health system • Offers hope for millions to receive services sensitive to consumers’ expressed needs and dreams of recovery

29. Knowledge Transfer • The COSP Multisite has grown into a productive and collaborative group that is committed to a high quality knowledge development and dissemination effort • 7 manuscripts in progress • Book prospectus under consideration • Numerous conference presentations

30. COME VISIT US ON-LINE! One of the hallmarks of the COSP is the effort made to use technology to facilitate work and disseminate information. Visit our website http://www.cstprogram.org Consumer Operated Service Program (COSP)