risk assessment

risk assessment PowerPoint PPT Presentation


  • 223 Views
  • Updated On :
  • Presentation posted in: General

Session Objectives. Discuss the role of risk assessment in the TQM process. Describe the five severity indices presented in the course. Compare and contrast the indices based on the sources of data used, scores produced, and accuracy of predictions. . Why measure severity?. One way to measure quality of health care is to compare outcomes for a group of patients to outcomes observed for other patients with similar severity of illness. When we want to do so, it is important to separate the infl9450

Download Presentation

risk assessment

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


1. Risk Assessment Farrokh Alemi, Ph.D.

2. Session Objectives Discuss the role of risk assessment in the TQM process. Describe the five severity indices presented in the course. Compare and contrast the indices based on the sources of data used, scores produced, and accuracy of predictions. The objectives of this lecture is to help you understand the role of risk assessment in process improvement, to describe the five severity indices presented in the course, to compare and to contrast the indices based on the sources of data used, scores produced, and accuracy of predictions.The objectives of this lecture is to help you understand the role of risk assessment in process improvement, to describe the five severity indices presented in the course, to compare and to contrast the indices based on the sources of data used, scores produced, and accuracy of predictions.

3. Why measure severity? One way to measure quality of health care is to compare outcomes for a group of patients to outcomes observed for other patients with similar severity of illness. When we want to do so, it is important to separate the influence of the patients' severity of illness from the quality of care. There are many ways to evaluate the quality of health care services. One increasingly popular way is to examine the rate of adverse health outcomes. Mortality is frequently used to categorize outcomes for patients who have acute, life threatening illness. Within both the group of patients who live and the group that dies, some patient might receive good care while others receive poor care. Some patients may be so ill that they cannot be saved by the best of care. Other patients may get well despite poor care. One way to measure quality of health care is to examine across a group of patients the rates of mortality and decide that these rates are lower or higher than what would be expected given the patients' severity of illness. When we want to do so, it is important to be able to separate the influence of the patients' severity of illness from the quality of care. There are at least two ways for doing so. One is to randomly assign patients to different providers, making sure that providers have equal chance of treating patients of high or low severity. The other is to measure patient's severity of illness and to statistically explain the variance in mortality with these measures first and then attribute any remaining variance to quality of care. Because it is difficult to randomly assign patients to providers, the second approach is the most common approach for measuring quality of care and is known as severity adjusted assessment of outcomes. There are many ways to evaluate the quality of health care services. One increasingly popular way is to examine the rate of adverse health outcomes. Mortality is frequently used to categorize outcomes for patients who have acute, life threatening illness. Within both the group of patients who live and the group that dies, some patient might receive good care while others receive poor care. Some patients may be so ill that they cannot be saved by the best of care. Other patients may get well despite poor care. One way to measure quality of health care is to examine across a group of patients the rates of mortality and decide that these rates are lower or higher than what would be expected given the patients' severity of illness. When we want to do so, it is important to be able to separate the influence of the patients' severity of illness from the quality of care. There are at least two ways for doing so. One is to randomly assign patients to different providers, making sure that providers have equal chance of treating patients of high or low severity. The other is to measure patient's severity of illness and to statistically explain the variance in mortality with these measures first and then attribute any remaining variance to quality of care. Because it is difficult to randomly assign patients to providers, the second approach is the most common approach for measuring quality of care and is known as severity adjusted assessment of outcomes.

4. What is severity? What is severity? Severity is the progression of the disease when left untreated. Severity is prognosis under ideal levels of care. Some patients are further along in their illness than others. It is difficult to define what is severity because it is difficult to separate the influence of quality from severity. One definition is that severity is the progression of the disease when left untreated. Since no one in his right mind leaves diseases untreated, it is difficult to observe the progression of the disease. Another definition is that severity is the progression of the disease given customary treatment. This definition is also flawed because it customary treatment of a disease includes both poor and good quality care. These difficulties of observing severity of illness within one person has led many investigators to suggest that severity is a comparative concept, meaning that a reasonable way to observe severity is to examine people at different stages of the disease. These individuals have different risks for mortality or adverse health outcomes. While it is difficult to observe progression of a disease, it is easy to see that some patients are further along in their illness than others. There are many measures of severity, some commercial and others available through academic journals. Because severity measurements is the reason for attribution of adverse outcomes to poor or good quality, it is important to use a severity measure that is well accepted by the people whom we are evaluating. Hence, many select commercial severity indices. In this lecture we review a sample of these indices and their major differencesIt is difficult to define what is severity because it is difficult to separate the influence of quality from severity. One definition is that severity is the progression of the disease when left untreated. Since no one in his right mind leaves diseases untreated, it is difficult to observe the progression of the disease. Another definition is that severity is the progression of the disease given customary treatment. This definition is also flawed because it customary treatment of a disease includes both poor and good quality care. These difficulties of observing severity of illness within one person has led many investigators to suggest that severity is a comparative concept, meaning that a reasonable way to observe severity is to examine people at different stages of the disease. These individuals have different risks for mortality or adverse health outcomes. While it is difficult to observe progression of a disease, it is easy to see that some patients are further along in their illness than others. There are many measures of severity, some commercial and others available through academic journals. Because severity measurements is the reason for attribution of adverse outcomes to poor or good quality, it is important to use a severity measure that is well accepted by the people whom we are evaluating. Hence, many select commercial severity indices. In this lecture we review a sample of these indices and their major differences

5. How to measure severity? Statistical analysis of outcomes of care Sample may not be relevant Expert opinions Expert consensus maybe false Patientís self insight Only felt symptoms affect self insight Severity indices can be organized from statistical analysis. For example, mortality rates can be regressed on patient characteristics on admission providing us with a formula for scoring patientsí prognosis on admission. Severity indices can also be organized through consensus panels of experts and clinicians. They could sit together and based on their knowledge propose a scoring procedure for identifying and grading patients with low to high severity of illness. A third approach is to ask the patient. Questions regarding health status of the patient or functional abilities of the patient can be taken to represent the severity of illness of the patient. Severity indices can be organized from statistical analysis. For example, mortality rates can be regressed on patient characteristics on admission providing us with a formula for scoring patientsí prognosis on admission. Severity indices can also be organized through consensus panels of experts and clinicians. They could sit together and based on their knowledge propose a scoring procedure for identifying and grading patients with low to high severity of illness. A third approach is to ask the patient. Questions regarding health status of the patient or functional abilities of the patient can be taken to represent the severity of illness of the patient.

6. Sources of Data ICD9 codes Readily available Maybe gamed Reflect whole visit Key clinical findings Requires chart review Maybe masked by treatment Reflects condition on admission There are at least two sources for data for describing patientsí condition on admission. We could rely on ICD9 diagnostic codes. These codes are readily available in administrative databases, but maybe gamed in order to increase reimbursement and typically reflect the entire visit and thus if the patient falls in a hospital the diagnosis of fractured hip may be added to the patientís condition even though it is clearly a sign of poor care. Patientsí conditions are also described by key clinical findings recorded in the medical record. The advantage is that one can focus on patient conditions on admission. The disadvantages are that access to these data requires time consuming and expensive chart review and clinical markers may be masked by medication use. So if the patientís blood pressure is not maintained well on first hours of admission, the patient may go through shock, even though the patient has received poor care, the case may be classified as severely ill because the blood pressure fell below 90. In short the reliance on these measures may mask the patientsí true nature.There are at least two sources for data for describing patientsí condition on admission. We could rely on ICD9 diagnostic codes. These codes are readily available in administrative databases, but maybe gamed in order to increase reimbursement and typically reflect the entire visit and thus if the patient falls in a hospital the diagnosis of fractured hip may be added to the patientís condition even though it is clearly a sign of poor care. Patientsí conditions are also described by key clinical findings recorded in the medical record. The advantage is that one can focus on patient conditions on admission. The disadvantages are that access to these data requires time consuming and expensive chart review and clinical markers may be masked by medication use. So if the patientís blood pressure is not maintained well on first hours of admission, the patient may go through shock, even though the patient has received poor care, the case may be classified as severely ill because the blood pressure fell below 90. In short the reliance on these measures may mask the patientsí true nature.

7. Disease Staging Stage 1 includes conditions with no complications and minimal risk for the patient. Stage 2 includes problems which are contained in one organ or system. Stage 3 includes problems in multiple sites and general systemic problems. Stage 4 is death This index is designed for a broad range of patients and is not limited to specific illness. Disease staging assumes that diseases are first localized and later spread to other parts of the body systems. As a disease advances to higher numbered stages, it is associated with increased risk for the patient. The Computerized Disease Staging has four stages which can be subdivided into additional categories: Stage 1 includes conditions with no complications and minimal risk for the patient. Stage 2 includes problems which are contained in one organ or system. Stage 3 includes problems in multiple sites and general systemic problems. Stage 4 is death. As frequently implemented, Computerized Disease Staging assigns severity stages based on diagnoses codes available through billing information. These codes are often based on the International Classification of Diseases, version 9, clinical modification (ICD-9-CM). The same codes are used by the Medicare reimbursement system to pay hospitals and physicians. Therefore, these codes are in wide use. During hospitalizations five to ten codes can be assigned to the patient. The first code is known as the principal diagnosis. The remaining codes are complications or other co-morbidity that were treated during the hospital stay. The severity scale produced by Computerized Disease Staging is ordinal, meaning that it preserves the order but not the magnitude of the differences between severity indices. Thus a severity score of 3 is worse than a score of 2 but not 1.5 times worst than 2. This index is designed for a broad range of patients and is not limited to specific illness. Disease staging assumes that diseases are first localized and later spread to other parts of the body systems. As a disease advances to higher numbered stages, it is associated with increased risk for the patient. The Computerized Disease Staging has four stages which can be subdivided into additional categories: Stage 1 includes conditions with no complications and minimal risk for the patient. Stage 2 includes problems which are contained in one organ or system. Stage 3 includes problems in multiple sites and general systemic problems. Stage 4 is death. As frequently implemented, Computerized Disease Staging assigns severity stages based on diagnoses codes available through billing information. These codes are often based on the International Classification of Diseases, version 9, clinical modification (ICD-9-CM). The same codes are used by the Medicare reimbursement system to pay hospitals and physicians. Therefore, these codes are in wide use. During hospitalizations five to ten codes can be assigned to the patient. The first code is known as the principal diagnosis. The remaining codes are complications or other co-morbidity that were treated during the hospital stay. The severity scale produced by Computerized Disease Staging is ordinal, meaning that it preserves the order but not the magnitude of the differences between severity indices. Thus a severity score of 3 is worse than a score of 2 but not 1.5 times worst than 2.

8. APACHE Deviations from norm on 12 physiological variables like heart rate, blood oxygen level, or respiratory rate. Age of the patient. Chronic illness include coma. This index was originally designed for critically ill adult patients but was later proposed for use by patients outside critical care units. The APACHE score is the sum of three components. These are: Deviations from norm on 12 physiological variables like heart rate, blood oxygen level, or respiratory rate. Age of the patient. Chronic illness include coma. APACHE is usually measured during the first 24 hours of hospital admission. The most abnormal values during this period are recorded and scored. APACHE produces an interval scale, where the score of 10 is twice as bad as 5. This index was originally designed for critically ill adult patients but was later proposed for use by patients outside critical care units. The APACHE score is the sum of three components. These are: Deviations from norm on 12 physiological variables like heart rate, blood oxygen level, or respiratory rate. Age of the patient. Chronic illness include coma. APACHE is usually measured during the first 24 hours of hospital admission. The most abnormal values during this period are recorded and scored. APACHE produces an interval scale, where the score of 10 is twice as bad as 5.

9. Medisgroup At level 0, there are no clinical findings. At level 1, there are minimal abnormal findings. At level 2, there are either acute findings or findings with an unclear potential for organ failure. At level 3, there are clinical findings with high potential for imminent organ failure. At level 4, organ failure is indicated. This index is designed for a broad range of patients and is not limited to specific illnesses. It scores have five levels: 0 through 4: At level 0, there are no clinical findings. At level 1, there are minimal abnormal findings. At level 2, there are either acute findings or findings with an unclear potential for organ failure. At level 3, there are clinical findings with high potential for imminent organ failure. At level 4, organ failure is indicated. The Medisgroup scoring does not follow specific mathematical rules like APACHE, where the scores of abnormal findings are added. Instead, Medisgroup relies on artificial intelligence if-then rules to score combination of clinical findings. These types of if then rules create a scoring system that has a lot more flexibility. Medisgroup relies on key clinical findings during the first 24 hours of admission of a person to the hospital. Key clinical findings may be specific laboratory findings or it could be clinical observations. It produces an ordinal severity scale. This index is designed for a broad range of patients and is not limited to specific illnesses. It scores have five levels: 0 through 4: At level 0, there are no clinical findings. At level 1, there are minimal abnormal findings. At level 2, there are either acute findings or findings with an unclear potential for organ failure. At level 3, there are clinical findings with high potential for imminent organ failure. At level 4, organ failure is indicated. The Medisgroup scoring does not follow specific mathematical rules like APACHE, where the scores of abnormal findings are added. Instead, Medisgroup relies on artificial intelligence if-then rules to score combination of clinical findings. These types of if then rules create a scoring system that has a lot more flexibility. Medisgroup relies on key clinical findings during the first 24 hours of admission of a person to the hospital. Key clinical findings may be specific laboratory findings or it could be clinical observations. It produces an ordinal severity scale.

10. Computerized Severity Index ICD9 codes Key clinical findings This index is designed for a broad range of patients and is not limited to specific illnesses. It relies on both ICD-9-CM diagnosis codes and key clinical findings. It produces an ordinal severity index, where for example a patient with a score of 4 is worst than a patient with a score of 2 but not twice as ill. This index scores range from 0 to 4. It begins with the patient's principal diagnosis and uses physiological markers to adjust the diagnosis. The internal working of this index are not publicly availableThis index is designed for a broad range of patients and is not limited to specific illnesses. It relies on both ICD-9-CM diagnosis codes and key clinical findings. It produces an ordinal severity index, where for example a patient with a score of 4 is worst than a patient with a score of 2 but not twice as ill. This index scores range from 0 to 4. It begins with the patient's principal diagnosis and uses physiological markers to adjust the diagnosis. The internal working of this index are not publicly available

11. Best Approach Ease of use Cost of gathering information Accuracy Face validity The indices differ in the source of information used to measure severity of illness. Some rely on physiological markers, others on ICD-9-CM codes. Both could be affected by treatment. Physiological markers are affected by treatment on route to the hospital and during the hospital stay. For example, medications may show a normal blood pressure for a patient that minutes before admission was in shock in the ambulance. Similarly, because ICD-9-CM codes are based on the treated diagnosis through out the hospital stay, they may be affected by complications that arise due to poor care. For example, when a patient falls and breaks her hip then this is added to her severity score while clearly the fall was not the condition the patient came in with. Because both sources of data could be affected by treatment, whenever possible both should be used. Diagnoses also reflect a summary of what the clinician taking care of the patient considered most likely reason for admission. Clinical markers, in contrast, are at best a reconstruction of what the documented information implies. Since in busy clinical practice, much of the information is not documented, the use of key clinical findings may not be as good as diagnoses codes. The indices differ in the source of information used to measure severity of illness. Some rely on physiological markers, others on ICD-9-CM codes. Both could be affected by treatment. Physiological markers are affected by treatment on route to the hospital and during the hospital stay. For example, medications may show a normal blood pressure for a patient that minutes before admission was in shock in the ambulance. Similarly, because ICD-9-CM codes are based on the treated diagnosis through out the hospital stay, they may be affected by complications that arise due to poor care. For example, when a patient falls and breaks her hip then this is added to her severity score while clearly the fall was not the condition the patient came in with. Because both sources of data could be affected by treatment, whenever possible both should be used. Diagnoses also reflect a summary of what the clinician taking care of the patient considered most likely reason for admission. Clinical markers, in contrast, are at best a reconstruction of what the documented information implies. Since in busy clinical practice, much of the information is not documented, the use of key clinical findings may not be as good as diagnoses codes.

12. Differences Among Indices Differences in scores produced Severity indices also differ in the type of scores they produce, some produce ordinal scales others produce interval scales. Interval severity scores are most helpful for bench marking, as these numbers can be averaged and used in various control charts. In contrast, ordinal scales must be transferred to interval scales before use in control charts. Differences in scores produced Severity indices also differ in the type of scores they produce, some produce ordinal scales others produce interval scales. Interval severity scores are most helpful for bench marking, as these numbers can be averaged and used in various control charts. In contrast, ordinal scales must be transferred to interval scales before use in control charts.

13. Accuracy of Predictions The accuracy of the various severity indices in predicting mortality from Myocardial Infarction in Hospitals in New Orleans area in 1985 was as follows: Patient Management Categories, 81% APACHE 76% Medisgroup 79% Computerized Severity Index 77% Computerized Disease Staging 82% of time accurately predicted patients discharge status. Predicting every one will survive accurately predicted 76% of cases. These data suggest that the performance of the major commercial severity indices for patients with myocardial infarction may not be substantially better than what can be expected from predicting that all patients will survive. Whenever possible multiple severity indices should be used to improve the accuracy of predictions.††The accuracy of the various severity indices in predicting mortality from Myocardial Infarction in Hospitals in New Orleans area in 1985 was as follows: Patient Management Categories, 81% APACHE 76% Medisgroup 79% Computerized Severity Index 77% Computerized Disease Staging 82% of time accurately predicted patients discharge status. Predicting every one will survive accurately predicted 76% of cases. These data suggest that the performance of the major commercial severity indices for patients with myocardial infarction may not be substantially better than what can be expected from predicting that all patients will survive. Whenever possible multiple severity indices should be used to improve the accuracy of predictions.††

14. What to do? Rely on a combination Rely on known indicators of prognosis Rely on consensus of experts So what to do? Rely on a combination Rely on known indicators of prognosis Rely on consensus of experts So what to do? Rely on a combination Rely on known indicators of prognosis Rely on consensus of experts

15. Risk of Fall The ability to ambulate Steadiness of gait Presence of certain chronic medical conditions Number of medications Mental status History of falls Here is an example. If I had to develop a severity index for risk of falls, I will pull together a group of experts and ask them to identify conditions that will increase the risk. The I would examine to see if these factors predict patients frequency of falls by following patients over time. I would use logistic regression to establish a relationship between these variables and whether the patient falls. Then I can use this scoring procedure to rate risk of new patients that come to my unit.Here is an example. If I had to develop a severity index for risk of falls, I will pull together a group of experts and ask them to identify conditions that will increase the risk. The I would examine to see if these factors predict patients frequency of falls by following patients over time. I would use logistic regression to establish a relationship between these variables and whether the patient falls. Then I can use this scoring procedure to rate risk of new patients that come to my unit.

16. Analyze Data At different time periods the risk for fall can be calculated for each patient Now we can analyze whether the number of falls exceeds what could be expected from patient conditions See example data at following address http://gunston.gmu.edu/708/RiskPChart.htm Suppose we collected data At different time periods the risk for fall was calculated for each patient. Now we want to analyze whether the number of falls in this nursing home exceed what could be expected from patient conditions. Suppose we collected data At different time periods the risk for fall was calculated for each patient. Now we want to analyze whether the number of falls in this nursing home exceed what could be expected from patient conditions.

17. Take Home Lesson It is best to use multiple methods of measuring severity

  • Login