Qualitative Evaluation Report for the CTC-RI Integrated Behavioral Health Pilot Program

1. Qualitative Evaluation Reportfor the CTC-RI Integrated Behavioral Health Pilot Program CTC-RI Quarterly IBH Meeting, Aug. 9, 2018 Mardia Coleman, MS Roberta E. Goldman, PhD May Street Consultants Alpert Medical School of Brown University

2. “I mean, when I say how much I love having integrated behavioral health, is that I can't imagine primary care without it. It just makes so much sense to me to have those resources all in the same place because it's so important. So I love it. I can't speak highly enough of it.” (Medical Provider)

3. “One of the things we identified was somebody was going [to the emergency department] almost every other day, and it was due to anxiety. So he was given tools to kind of control that, and it actually empowered him. He felt like he had taken control of this issue. And his ER visits dropped right off. He was being seen here [at the primary care practice] more frequently, but that's okay. We'd rather he come here than go to the ER.” (Practice Coordinator)

4. I think a really big success is when we get called in for suicidal ideation, and in previous to IBH being here, they would just call the rescue. And so now when we get in we dig deeper. We dig deeper. We come up with maybe a safety plan. We sometimes reach out to a family member while a patient is still in there or if there's someone outside that they feel they can talk to about this. And I think we often divert suicidal patients from going to the emergency room. (IBH Provider)

5. “A younger woman I have with diabetes has done better, definitely, with behavioral health services. Like, ‘Oh, you see Dr. G. Like when did you last see Dr. G?’ ‘I see her every four months.’ ‘Did you tell her that you're suicidal?’ ‘No.’ ‘Do you want to come to therapy here?’ So I think for many of those patients, they've pulled their care here coordinated between their PCP and their therapist and their prescriber. And they've gotten a lot better care, a lot, lot better care.’ (Medical Provider)

6. "I just went in to do a warm handoff and that patient is set up with counseling, is set up with psych care. I'm setting her up with a community health worker. So we're able to go in and be a hub and get our patients connected to a few different people and types of services. And they're so grateful because they can just come to one spot and get all their care." (IBH Provider)

7. IBH in Primary Care Pilot • Cohort 1: 5 PCMH practices began IBH program February 2016 • Cohort 2: 5 PCMH practices began IBH program November 2016 • Program requirements: • Universal Screening of depression, anxiety and substance use in primary care for all patients > 18 across two years • Rescreening within 6 months if positive at baseline • Onsite IBH providers offer evidence-based treatment • Range of FQHCs, private group practices and academic settings • 3 PDSAs: • Increase screening rates of depression (PHQ-9 90%), anxiety (GAD-7 70%) , and substance use (CAGE-AID 70%) • High ED utilization with behavioral health • Population health (chronic disease management) focus within behavioral health

8. Overall Qualitative EvaluationResearch Questions • How is the IBH program implemented at the practice sites (including rollout, training, screening, facilitation, PDSAs, communication, IBH and OBH services, care coordination, billing, etc.)? • How and why do these vary among practices? • How do IBH champions and other involved practice staff experience the program at their sites? • Expectations for the program and ability to achieve goals • Assessment of program usefulness • Facilitators and barriers to implementation; recommendations • How do IBH champions and other key practice staff perceive possibilities for sustainability at their practices? • Provide context for the quantitative evaluation findings.

9. Evaluation Methods • Literature review • Document and website reviews • Site visits to each practice • Individual or small group Key Informant qualitative interviews • Open-ended question guides for each participant group • Conducted by Coleman, Goldman, or together • Most were on-site, in-person; some by telephone • Notes taken; audio recorded; professionally transcribed

10. Evaluation Methods, cont. • Interview participant samples • Purposive, criterion-based samples • Key informant interviews with internal and external stakeholders (N=9) - CTC-RI IBH leadership - Health plans - OHIC • Key informant interviews with employees at each pilot practice site (N=49) - Physician champions, other physicians, NCMs, IBH providers, IBH staff assistants, IBH students, practice managers, IBH program coordinators, clinical supervisors

11. Evaluation Methods, cont. • Qualitative data analysis and interpretation • Iterative individual and analysis team approach • Immersion/crystallization method: Review recordings; read transcripts; take notes; repeated discussions of emerging patterns, themes, differences, potential reasons for differences and similarities • Periodic ‘member-checking’ with CTC-RI IBH leadership and practice personnel • Individual site reports sent to practices for review

12. Evaluation Methods, cont. • Qualitative data analysis and interpretation, cont. • Creation of code book to sort and manage data • Data extraction method of coding and documenting • Immersion/crystallization again of coded data documents and return to transcripts as necessary, to analyze by topical categories • Final interpretations reached through team discussion • Draft report written and submitted for review

13. Findings: Themes and Patterns • Practice facilitation, CTC project management were helpful • Systematic programmatic support and oversight • Structured system for grant and incentive funds • Orientation, training • Facilitation by experienced, PsyD facilitator • Educate practices on IBH model • Overcome barriers • Motivate practices • Keep practices progressing with PDSAs • Diversity of practices = practice facilitation could be enhanced if more tailored to each situation, even considering the need to promote uniform program requirements

14. Findings: Themes and Patterns Practice facilitation, CTC project management were helpful “I thought (the facilitator) was fantastic…she always had a lot of insight. I think at first sometimes I'd be like why is she saying that. But it always came full circle, and she provided us with a lot of support. So I personally I think we got a lot out of it.” "I think [the facilitation meetings] were helpful to get everybody on the same page from a number of different perspective from what to do for reporting, how we're going to get things measured and into the program, helping to articulate a vision and helping us figure out how we might execute on that vision.”

15. Findings: Themes and Patterns • Rollout and beyond • Preparation period and providing user-friendly, uniform data tracking platforms would have been beneficial • Critical steps that varied among practices: • Engage practice leadership and staff early, across all professional roles; recognize and work with resistance • Actively foster and maintain staff involvement and commitment • Train and integrate new staff • Plan for effectively communicating program policies and workflow changes

16. Findings: Themes and Patterns Rollout and beyond “And I was having a really tough time because people do what they do, and they get used to what they do, and they don't want to change. Now that was on both sides. BH didn't want to change. Medical didn't want to change.” “It's interesting because when we first looked to do this there was the initial ‘I'm not going to do it because it's going to open a can of worms. And I don't have enough time’‑‑right‑‑was the initial response from providers…We responded, ‘but we need to address this just like we would address if your patient had an elevated blood pressure.’” “So, (the IBH program manager) would come and talk about the program and say, ’Here's what we're doing; how we’re going to staff it. I have trainees. We're going to get social work students.’ Providers loved it.”

17. Findings: Themes and Patterns • EHRs are critical, and can help or hinder IBH • “When you're seeing the volume of patients we see every day, it was a job in and of itself just trying to figure out how to stay on top of it versus trying to figure out how to build tools into our EHR that would start doing that for the teams themselves. Right, things like health reminders for screenings and re-screenings and things like that in the system that made a lot more sense than trying to have this manual registry that was a bear to try to stay on top of.” • Communication between medical and IBH providers and support staff through EHR, messaging, telephone and in-person formal and informal discussion supports implementation of the IBH model • Spatial arrangement of the site is important • Issues of productivity interact with informal discussion

18. Findings: Themes and Patterns Lack of parity in copays for IBH and primary care = patients who refuse IBH or are no shows At some sites, supervised students fill the gap in IBH services for those who cannot afford copay = not billable visits MAs are central to IBH success Systematic screening can produce surprising results "I'm surprised especially with the anxiety screener that there's more out there than I knew about. I was talking to somebody yesterday. You think this wouldn't be useful information. I know the patient pretty well, and the patients, if they had an issue, I'm sure they would tell me. But it comes up on this, on the screener." (Medical Provider)

19. Findings: Themes and Patterns • IBH must pay for itself to be sustainable – critical factors: • Medical providers understand IBH model and have high rate of warm handoff requests and IBH referral • At some sites, IBH providers have high OBH productivity in addition to providing IBH services • Patient population accepts services and understands IBH model • Patient population for whom IBH copay is not an obstacle

20. Findings: Themes and Patterns • Appropriate coding and billing can be a struggle • In some sites interviewees felt coding and billing proceeded smoothly; in others there seemed to be confusion, a sense that there are better codes available, and a strong interest in further training "I honestly would love a hands-on-…With this plan you can bill for this. And break it down by Medicare Medicaid, private, all of our different payers. And work on how to help with the sustainability and how to best utilize the tools that we're probably not using.”

21. Primary Care Practice Processes that Promote Success of an IBH program • Engaged leadership and IBH champions • Cultivation and maintenance of practice culture stronglysupportive of IBH in all work roles • Effective facilitation/internal processes for testing strategies for increasing screening, warm handoff and IBH referral rates, tracking screening and optimizing coding and billing • Creative and efficient systems for warm hand-offs tailored to each site • MAs trained and trusted to support IBH, screening and data entry, warm hand-offs, referrals

22. Primary Care Practice Processes that Promote Success of the IBH program, cont. • EHR that fosters information sharing, care planning, patient tracking “…build it into your system in a way where you can report on and track it in your system. Don't have it be a free text field where you can't really look for it and track. Have it tie into whatever your health reminder system is so it will automatically‑‑you do the screening. If it's negative, great. It's good for the year.” • IBH providers are accepting of and experienced in IBH model • In some types of sites, balanced provision of short-term IBH and longer-term OBH services to meet patients’ needs and generate sufficient income for sustainability • Assistance of students to eliminate IBH copay

23. Data Collection and Tracking Issues • Interviewees appreciated facilitation on data collection, and also described difficulties that persisted: • Creating and using a patient registry/tracking system • Some sites created and used effectively • But was time consuming and labor intensive for many sites • Tracking in a way that makes screening most efficient • Registration desk in some sites cannot access screening status so patients are screened at every visit • This over-screening leads to “screening fatigue” and patient refusals to complete screening tools • An effective tracking system would eliminate this problem • Unclear if sites collect and document data uniformly

24. Data Collection and Tracking Issues, cont. • Sites need access to their own data • Most interviewees did not know their practice trends in screening, BH diagnoses, ED use, cost reduction • When they were able to use their own data it was beneficial to the IBH program in implementing needed modifications “Q. So what turned you around [in your opinion of IBH]?” “A. The data, just the data in terms of…. what you should be reporting out in terms of numbers of diagnosed patients with depression, anxiety and substance abuse. And pulling the UDS data that we submit, based on pulling our diagnostic, or pulling the data that we submit and seeing that we weren't reporting what the world historically reports.”

25. Data Collection and Tracking Issues, cont. • Worth considering how more uniform data collection can: • Aid sites’ abilities to monitor their own progress, identify needed changes, and evaluate changes • Be used as evidence to motivate practice providers and other staff to build and maintain enthusiasm • Facilitate cross-site evaluation and identification of specific factors that contribute or detract from successful IBH outcomes • Facilitate evaluation in future IBH initiatives

26. There is a feel for when IBH is working to enhance disease management, and providers and staff know it when they see it. Though data to prove it would be beneficial! • “I find what I do a lot is when I have somebody who has really serious depression that's out of control, or anxiety that's out of control, or a really big abuse history which I come across a frightful amount, and they have a lot of other health issues, I want to get them into counseling so that they're doing something to address these major issues that are interfering with their ability to manage their blood pressure or their diabetes. But they also feel overwhelmed by all of their mental and physical health issues.   • So having the warm handoff where they can meet one of our IBH people and put a personal face to, like, reach out to our counselors. ‘They're wonderful. They're going to come in. You can meet them right now.’ And sort of get that accountability. I don't have any data to support this, but I suspect that helps me get them in for follow-up with the counseling staff better because we remove that barrier from an abstract follow-up with counseling or engaging counseling, and it's become like a ‘Here's your appointment. It will be with me.’ What's so nice about it is when I do have someone who has serious chronic issues like really uncontrolled hypertension or they're recovering from stroke, and they can't focus on that stuff because their PTSD is out of control.” (Medical Provider)

27. Recommendations • Fund foundational activities and sufficient time for start-up • Test EHR modifications, workflows, patient tracking systems before implementing • Recognize it takes engaged staff at all levels and in all roles to implement IBH • Implementation needs dedicated project management, incentives, quality improvement projects, facilitation • Sites need clear measures to determine success, and help getting set up to report measures • Beyond meeting incentive goals, meeting productivity

28. Recommendations, cont. • EHR can be made more effective, and investment is necessary • Care planning, registry/tracking functions, IBH usability • Create a payment model that categorizes IBH as an essential primary care service and eliminates copays • “The Nurse Care Manager model” • Sites meet performance goals • Payment includes additional staff and the time it takes staff to implement IBH • Recognize non-clinical aspects of IBH implementation; plan and pay for this time • Establish workflow and workloads so IBH does not burn out staff • Address ongoing infrastructure needs

29. Further Questions to Consider both Qualitatively and Quantitatively • How do different IBH staffing structures impact IBH program outcomes, including warm handoffs, IBH referrals, OBH referrals, patient outcomes? • E.g. role of MAs, FTE of IBH staff of different types, presence of or access to behavioral health advocates, presence of or access to psychiatrist, involvement of students • How does turnover of practice employees impact IBH program outcomes? • Compare other sites with WMC that never bills for IBH, and no patient has a copay, no has to meet a deductible

30. Further Questions to Consider both Qualitatively and Quantitatively, cont. • How do different oversight and management structures at the practices impact IBH program outcomes? • E.g. which staff roles are responsible at each site, does more direct oversight lead to better outcomes • How do different IBH service provision structures impact IBH program outcomes? • E.g. IBH providers also provide OBH or do not, non-patients can receive BH services or cannot, level of reliance on students to provide IBH intervention for patients who cannot afford copay

31. Questions? • Mardia Coleman, MS Roberta E. Goldman, PhD • May Street Consultants Alpert Medical School/Brown University • mardcole@aol.comroberta_goldman@brown.edu