2005 IQLM Conference

1. 2005 IQLM Conference Indicators Workgroup April 29, 2005

2. Workgroup Members Raj Behal, MD (UHC) Lucia Berte, MA (Consultant) Robert Dufour, MD (VAMC) Diane Feeney, BSN, MSc (NQF) Linda Hanold, MHSA (JCAHO) Alan Hoffman, MD (NCQA) Robert Pendrak, MD (Inservco Insurance Services) Paddy Sundararajan, PhD (Quest Diagnostics) Richard Zarbo, MD, DMD (HFHS) Former Workgroup Members: Merilyn Francis, MPP (NQF) David Sundwall, MD (ACLA) Workgroup Co-Leaders: Lee Hilborne, MD, MPH (UCLA/RAND) Frederick Meier, MD (HFHS) CDC Co-liaisons/Indicator Reviewers: Shahram Shahangian, PhD Susan Snyder, PhD Pamela Thompson, MS CDC Supervision: Joe Boone, PhD Devery Howerton, PhD IQLM Thanks The Quality Indicators Workgroup Members

3. Consider the following question • If Congress wanted to quickly know about America’s laboratories, what information should they have? • Laboratory medicine’s contribution to the nation’s health • What is important to the nation should be important to its laboratories • If IQLM goes off by itself, the relevance will be lost

4. But Aren’t Laboratory Tests Relevant To Clinical Monitoring? • Consider 24 common clinical outpatient conditions with explicit evidence for a specific course of evaluation or treatment • In how many of these common conditions are laboratory tests part of the diagnosis or monitoring? • Involved in Diagnosis: 50% • Involved in Treatment Monitoring: 38% • Involved in Diagnosis or Treatment: 63% • No other specialty touches so many clinical situations that frequently impact patients

5. The Indicators Group Was Given A Specific Agenda • Define a core indicator set for laboratory practice • IOM quality domains • Cover total testing process • Practice settings • Be judicial in selection • Perhaps 3-5 domains, 3-5 measures per domain • How should indicators be used? • Surveillance for quality across organizations • Identification of best practices for awards • Ability to monitor through the IQLM Network • Review and incorporate existing evidence • Identify evidence gaps related to core indicator set • Determine strategies to fill the gaps

6. Indicators Fit Into The Institute Vision Develop Network of Laboratories & Partners Awards and Grants Program Network Awards NationalReport LaboratoryIndicators Identify Issues And Best Practices Indicators to Benchmark And Monitor Progress

7. Our Job Was To Provide Initial Direction Regarding Indicators

8. Quality Indicator Characteristics Should Be Broad and Defensible • Evidence-based, practice-tested measures of IOM health care domains • Associated with identifiable health care quality problems • Objective metrics that can be implemented in a standardized and comparable format • Employ standard methods for evaluation • Reliable means of external and internal evaluation of quality performance over time • Address a wide range of laboratory tests and testing sites

9. We Considered How To Prioritize Indicator Efforts • What is important? • Where should we focus? • In what testing venues will indicators be applicable? • How can we avoid reinventing the wheel? • How much can we do at the beginning without biting off too much?

10. Some Important Concepts Emerged • Link to National Quality Report • Define scope of analysis • Information is actionable • Data efforts are doable (practicality) • Feasible • Affordable • Submissions should be auditable

11. While All Phases Of The Total Testing Process Are Important… Post-AnalyticPhase Pre-Analytic Phase Analytic Phase We Concluded Focusing On Pre And Post Analytic Phases Will Have The Greatest Impact

12. IQLM, and Indicators Should Acknowledge All Stakeholders • Laboratory professionals • Accrediting organizations • Standards developers • Administrators • Diagnostic industry • Physicians and others providing direct patient care • Patients and public • Policy makers • Payers and purchasers • Delivery systems • Proficiency testing (?)

13. Indicators Must Be Sensitive To Different Testing Venues • Hospital Laboratories • Physician Office Laboratories • Reference Laboratories • Public Health Laboratories • And Consider Different Perspectives • What the laboratory needs to know • What laboratory users need to know • What patients and consumers need to know to select a laboratory

14. Look To Partners Who Have Considered Questions Before • College of American Pathologists • Q-Probes and Q-Tracks • National Committee on Quality Assurance • HEDIS • Joint Commission on Accreditation of Health Care Organizations • ORYX • National Patient Safety Goals • Veterans Administration • Agency for Healthcare Research and Quality • National Healthcare Quality Report • Natl Quality Measures Clearinghouse/Ntl Guideline Clearinghouse • US Preventive Services Task Force • Centers for Disease Control and Prevention (e.g., MMWR) • Published work (Medline, data mining, others)

15. We Reviewed The Literature And Found Many Potential Indicators

16. Then Focused Down Using Some Guiding Principles • A limited number of indicators relevant to the quality agenda • No more than about 8-12 • If there are too many, the process won’t be deemed credible • Should cover various laboratory venues • If at all possible, use existing, validated indicators • Develop new or test obscure indicators only if absolutely necessary

17. We Identified Two Main Indicator Categories • Systems Indicators • Involve interactions between the laboratory and laboratory customers • Laboratory Quality Indicators • Primary impact is on the provision of care by the laboratory • Specific total testing process subsets • Pre, intra and post analytic components

18. The Following List Encompasses The Highest Priority Items • Diabetes monitoring (system) • Hyperlipidemia screening (system) • Patient identification (preanalytic) • Test order accuracy (preanalytic) • Blood culture contamination (preanalytic) • Adequacy of specimen information (preanalytic) • Accuracy of Point of Care Testing (analytic) • Cervical Cytology/Biopsy Correlation (analytic) • Critical value reporting (postanalytic) • Turnaround time (infrastructure) • Clinician satisfaction (infrastructure) • Clinician follow up (system/general)

19. IQLM’s Initial List Closely Parallels That Recommended by Dr. Howanitz last year Presented at CAP 04

20. There Are Some Key Dimensions That Remain To Be Addressed • We’ve covered issues of underuse and access in several measures • We need to consider a focus on • Overuse (appropriateness) • Misuse

21. We Started By Reviewing The Twelve Selected Indicators • Summarize and describe the evidence so practical healthcare decisions are feasible • Use defined recognized methods for reviews and desirable quality measure attributes • Maintain a practical perspective consistent with the constraints imposed by the limited availability and quality of evidence, producing a comprehensive, objective and reproducible evaluation

22. Indicators Were Evaluated Using A Structured Process • Formulate the problem • Conceptual framework • Search strategy • Quality and validity assessment of studies • Inclusion criteria • Collect and analyze findings • Evaluation criteria/template • Interpret findings and present results • Update findings Cochrane Handbook for Systematic Reviews of Interventions, 3/05

23. Indicator Definition Problem Specification Intermediate Outcomes Health Outcomes Potential Interventions Evaluation Was Approached With This Conceptual Framework Morbidity, Mortality, DALY, QALY, Cost Specific Metric Properties Issue The Indicator Addresses Proximate, Associated w/ desired Health Outcomes Actions To Change Outcome

24. Evaluation Criteria Followed Four Primary Dimensions • Importance • Scientific acceptability • Feasibility • Usefulness

25. Evaluation Criteria: Importance • Health importance • Ability to meaningfully impact populations • Measures an important quality aspect(s) • Common: high prevalence/incidence • Impact: serious impact on health outcomes • Potential for improvement • Need supported by quality variation or substandard quality • Literature or expert opinion support (e.g., effective interventions)

26. Evaluation Criteria: Scientific Acceptability • Strength of evidence based on peer reviewed literature • Quality problem is explicitly defined • Indicator links specifically to the problem • Indicators must be reliable and valid • Findings are consistent among raters • Accurately measures desired attributes • Other sources for future consideration • Professional organizations • ? Expert opinion

27. Evaluation Criteria: Feasibility • Data definitions are sufficiently clear • Abstraction tools can be developed • Data abstraction quality easily standardized • Manual: Trained individuals for consistency • Electronic: Clear data fields easily extractable • Ability to broadly implement indicators • Across multiple similar laboratories • Across different laboratory types • Benefits of measurement exceed financial and administrative burdens • Burdens: need to collect new data, abstraction time, analysis time, health impact of erroneous results • Benefits: health improvement, reduced rework, reduced cost

28. Evaluation Criteria: Usefulness • Relevant • Stakeholder(s) find the indicator useful • Acceptance by laboratories, clinicians and other stakeholders • Relevance extends to the healthcare system (beyond the laboratory) • Opportunity to impact health system • Interventions within stakeholders’ sphere of influence • Actionable findings to guide organizational decisionmaking and inform public policy

29. Here’s An Example of IndicatorEvaluation % of specimensw/ inaccurate orinadequate info Specimens w/ inaccurate or inadequate info • Reporting/treatment error or delay • Patient satisfaction • Cost of Error Morbidity Mortality Cost • Specimen rejection policy • CPOE system • Label bar coding

30. Accuracy And Adequacy Of Specimen Information • Definition: Percent of specimens sent to laboratory with inaccurate or inadequate information • No label, illegible, no patient information • No tissue source or clinical information when needed • Population: Specimens sent to laboratory • Providers: Locations where specimens are collected • IOM domains: Safety, timeliness, efficiency

31. The Degree Of Support For Criteria Domains Is Variable  Current information supporting evaluation criteria appears adequate  Current information supporting evaluation criteria is equivocal/uncertain  Current information supporting evaluation criteria is limited or does not exist

32. Accuracy And Adequacy Of Specimen Information: Importance=  • Prevalence/incidence:  • Surgical specimens: 1.2-2.2% (CAP) • Clinical lab specimens 0.015-0.030% (CAP) • Impact:  • Limited data available regarding impact on outcome • Potential for improvement  • Variation suggests opportunities • Monitoring and feedback insufficient

33. Accuracy And Adequacy Of Specimen Information: Acceptability=  • Strength of evidence:  • Quality problem defined:  • Indicator linked to quality problem:  • Professional society acceptance:  • Reliability and validity:  • Validity: No explicit information:  • Reliability: CAP studies suggest consistency: 

34. Accuracy And Adequacy Of Specimen Information: Feasibility=  • Clear data definitions:  • Standardization of data definitions remains an opportunity • JCAHO has developed standard definitions • Implementable:  • Many studies have been done in multiple institutions • Reasonable Cost: • No data specifically documenting cost to study • Multiple studies suggest cost is not excessive

35. Accuracy And Adequacy Of Specimen Information: Usefulness=  • Relevance to users, stakeholders:  • Useful for performance improvement • Not explicitly on any national metric • Operationalizable: • No specific data on operationalizing this metric • Findings suggest strategies can be developed improve performance • Specimen rejection policy dissemination and enforcement • Bar codes on specimen labels

36. What Do We Currently Know About The First Indictors Evaluated? =Pending Review

37. The Workgroup Raised As Many Challenges As Solutions • Laboratory indicators are limited • Strength of evidence linking laboratory indicators to health outcomes is weak • Lack of evidence for health outcomes • Literature does not directly address many defined quality problems or review questions • Lack of standard definitions limits comparability of findings • Quality of evidence, generalizability and applicability not evaluated • Many laboratory indicators have limited relevance to national health priorities

38. Some General Themes Emerge • System indicators likely are better supported by the evidence on health outcomes • Selected because of their impact on patient outcome • Results are linked to evidence supported specific care interventions • Laboratory indicators are less frequently supported by ties to health outcomes • Most of the work has been done by CAP • Definitions for many are clear or could be standardized • Linked to intermediate outcomes but links to health outcomes are generally inferential

39. Issues For The New IQLM To Consider • Solidify a definition of quality as it relates to laboratory practice • Assure selected/future indicators map to laboratory quality definitions • Focus on test utilization: overuse, underuse and misuse of testing services • Consider whether intermediate outcomes should be sufficient • Identify direct and indirect ways to link intermediate processes to health outcomes • Accept intermediate outcomes as final outcomes for most laboratory services

40. Logical Next Steps for Indicators Once IQLM Direction Is Clear • Begin a broader discussion of indicators with key stakeholders • Link indicators to awards and network groups • Determine the extent to which indicator validation will be studied • Select one or two for more in depth exploration