Quality Indicator Survey (QIS) Design and Development

1. Quality Indicator Survey (QIS) Design and Development Martha Powell, PhD Assistant Research Professor Division of Health Care Policy and Research September 13, 2012

2. Presentation Overview University of Colorado Anschutz Medical Campus University of Colorado’s role in implementation and analysis of QIS Development and Implementation of the QIS Threshold development and review QIS in Indiana

3. CU’s Role University of Colorado Anschutz Medical Campus Software design and testing Stage 1 and Stage 2 revisions/improvements Desk Audit Reports Threshold analysis Help Desk Onsite support for SA/RO

4. Survey Process University of Colorado Anschutz Medical Campus • Traditional • Paper forms, prelim sample from QM/QI, tour used to supplement sample, comp review then focused reviews • QIS • Computer assisted (tablets), random samples, wide variety of indicators, prelim invest then in-depth for selected areas • Both shaped by OBRA ‘87

5. History of QIS University of Colorado Anschutz Medical Campus Pre-cursor to QIS (Quality Indicator Survey) developed to serve as standard to gauge effectiveness of survey process improvements Development involved Univ. of Wisconsin, Univ. of Colorado, and programming contractors Pilot Tests (field, alpha, beta), then Demonstration and Evaluation National Implementation—phased approach

6. QIS Basics University of Colorado Anschutz Medical Campus • Computer-assisted (tablets), 2 stages • Stage 1: Preliminary Investigations • Random sampling • Structured protocols • Stage 2: In-depth investigations • Facility tasks (e.g., Kitchen, Infection Control & Immunization Review) • Surveyor Initiation option • Typically four surveyors onsite for 5 days

7. Stage 1 University of Colorado Anschutz Medical Campus • Census Sample (N=40): • Resident Interview/Observation • Staff Interview • Record Review • Family Interview (n=3) • Admission Sample (N=30): • recently-admitted residents • Record Review • MDS Sample: assessments from prior 180 days

8. QCLIs University of Colorado Anschutz Medical Campus • Quality of Care and Quality of Life Indictors • Examples of QCLIs: Underweight and No Supplements, Eating Decline Since Admission, Hospitalization within 30 days of Admission • Computed from Stage 1 and MDS samples/data • Facility rate is calculated and compared to pre-set threshold • If facility rate exceeds threshold, the care area (e.g., Nutrition, ADL, Hospitalization) is triggered for a Stage 2 investigation (e.g., 4 residents out of 40 have Stage III/IV pressure ulcers—10% facility rate—exceeds 5% threshold)

9. Thresholds University of Colorado Anschutz Medical Campus • Rates – threshold = 10% (example rate) • For a census sample of 40, 10% would be 4 residents, so 5 would be needed to exceed threshold • For an admission sample of 30, 10% would be 3 residents, so 4 would be needed to exceed threshold • Threshold for Abuse is 0—only 1 resident is needed to exceed the threshold • Not percentiles or benchmarks • Reviewed and reset if appropriate • Based on QIS data • Separate threshold values for smaller census/admission samples

10. Simplified QCLI Threshold Analysis If current threshold is 19% 8 – TRIGGERED and3 – CITED 3/8 or 37.5% citation rate University of Colorado Anschutz Medical Campus

11. Simplified QCLI Threshold Analysis If current threshold is 37% 4 – TRIGGERED and3 – CITED 3/4 or 75.0% citation rate University of Colorado Anschutz Medical Campus

12. Simplified QCLI Threshold Analysis If current threshold is 45% 2 – TRIGGERED and2 – CITED 2/2 or 100.0% citation rate University of Colorado Anschutz Medical Campus

13. Small/Not Small Sample Thresholds University of Colorado Anschutz Medical Campus • Census Sample • Small: 35 or fewer residents • Not Small: 36 – 40 residents • Admission Sample • Small: 9 or fewer residents • Not Small: 10 – 30 residents

14. Stage 2 University of Colorado Anschutz Medical Campus • In-depth investigations of triggered care areas and non-mandatory facility tasks • Critical Element Pathways guide investigation • Additional interviews, observations, and record reviews • Compliance decision • Is facility in compliance with regulations? • Individual and then as team

15. QIS Surveyor Training University of Colorado Anschutz Medical Campus After SMQT Extensive Classroom training (computer basics, QIS process and software) Mock survey—putting training into practice Compliance surveys—performance is evaluated by QIS Trainers Additional classroom and onsite education to become a QIS Trainer

16. QIS Implementation University of Colorado Anschutz Medical Campus

17. QIS in Indiana University of Colorado Anschutz Medical Campus First surveys conducted in January, 2011 As of 7/17/2012—212 QIS ( 500 NH) 15 of 21 teams trained Full implementation can take up to 3 yrs. Following slides/results are based upon QIS from 1/28/2011 – 7/17/2012; National and Indiana QIS results from same time period

18. Triggered Care Areas University of Colorado Anschutz Medical Campus National and Indiana rates are similar for the following: • Frequently Triggered (> 60% of surveys) • Abuse/Abuse Prohibition, Accidents, ADLs, Community Discharge, Dental Status, Nutrition, Personal Property, Pressure Ulcers, Psychoactive Meds • Less Frequently Triggered (< 20% of surveys) • Food Quality, Notification of Change, Privacy, Sufficient Nursing Staff, Social Services

19. Triggered Care Areas cont’d University of Colorado Anschutz Medical Campus • Indiana is a low outlier (compared to National QIS rates) for: • Physical Restraints and Positioning • No high outliers for Indiana for triggered care areas

20. Source of “Negative” Responses University of Colorado Anschutz Medical Campus Indiana QIS rates for: Resident Observation 5% Resident Interview 7% Family Interview 3% Staff Interview 10% Census Clinical Record 5% Admission Clinical Record 5% These rates are similar to Nat’l QIS rates.

21. Citation Rates for Mandatory Facility Tasks University of Colorado Anschutz Medical Campus Liability Notices 25% (↑than Nat’l QIS) Dining 23% Infection Control 46% (↑than Nat’l QIS) Kitchen 27% QA & A 13% (↑than Nat’l QIS) Resident Council 13% (↑than Nat’l QIS) Med. Admin 13% Med Storage 22% Unnecessary Med Use 39%

22. Citation Rates for Non-Mandatory Facility Tasks University of Colorado Anschutz Medical Campus Abuse Prohibition 23% Adm, Tfr, DC 22% Environment 74% Personal Funds 26% Sufficient Nursing Staff 13% Extended Survey 60% Rates = # cited divided by # investigated (not total # of surveys)

23. Citation Rates for Triggered CA/Tasks University of Colorado Anschutz Medical Campus • Indiana QIS similar to Nat’l QIS for most • Indiana QIS low outlier for: • Behavioral and Emotional Status, Rehabilitation, UTIs • Indiana QIS high outlier for: • Physical Restraints and Positioning

24. Scope and Severity University of Colorado Anschutz Medical Campus Percentage of F-tags at each S/S level for Indiana QIS very similar to Nat’l QIS, slightly higher percentages at A & B IN QISNat’l QIS A 3.6% 1.2% B 4.8% 2.7% C 3.9% 3.0% D 64.3% 65.3% E 17.6% 19.9% F 3.2% 4.8% G 2.5% 2.4% H – L 0.2% 0.55%

25. Most Frequently Cited F-tags University of Colorado Anschutz Medical Campus • Nationally in QIS states F441 Infection Control -- 34.9% F329 Drug Regimen free from Unecc. Drugs -- 34.1% F323 Free of Accident Hazards -- 31.3% • Indiana QIS F282 Srvcs by Qualified Persons per Care Plan -- 44.7% F323 Free of Accident Hazards -- 41.6% F279 Develop Comp. Care Plans -- 41.1%

26. Citation Rates for Selected F-tags University of Colorado Anschutz Medical Campus Nat’l QISIN QISdiff F282 21.1% 44.7% -23.6% (Svcs by Qual. Pers per CP) F156 12.1% 28.9% -16.9% (Notice of Rights) F253 19.5% 32.0% -12.5% (Housekeeping)

27. Citation Rates for Selected F-tags University of Colorado Anschutz Medical Campus Nat’l QISIN QISdiff F280 27.8% 22.3% 5.5% (Right to particp. In CP) F281 9.9% 4.1% 5.8% (Srvcs. meet Professional Stds.) F272 24.4% 17.8% 6.5% (Comprehensive Assessments)

28. Citation Rates for Selected F-tags University of Colorado Anschutz Medical Campus Nat’l QISIN QIS F492 3.2% 0% (Comply w/Fed/St/Local Laws/Prof. Stds.) F274 1.6% 0% (Comp. Assess. After Sig Change) F497 1.2% 0% (Nurse Aide Perform Rev-12 hr/yr in-service)

29. Citations and Survey Time University of Colorado Anschutz Medical Campus Nat’l QISIN QIS Average Number of Citations per Survey 6.9 8.4 Percentage of Citation- Free Surveys 6.9% 9.6% Survey Time (Prep, Onsite, Offsite) 139.3 141.5

30. Contact Information University of Colorado Anschutz Medical Campus University of Colorado QIS website http://www.ucdenver.edu/academics/colleges/medicalschool/departments/medicine/hcpr/qis/Pages/default.aspx CMS Project Officers Susan Joslin (Susan.Joslin@cms.hhs.gov) and Tom Kress (Thomas.Kress@cms.hhs.gov)

31. Questions? University of Colorado Anschutz Medical Campus Martha Powell Martha.Powell@ucdenver.edu (303) 724-2434 Sayuri Kelly Sayuri.Kelly@ucdenver.edu (303) 724-2490 For CMS QIS forms and Information: https://www.qtso.com/qisdownload.html