1 / 21

A Review of Chief Complaint Classifiers for Syndromic Surveillance

University of Pittsburgh. Dept. of Biomedical Informatics. A Review of Chief Complaint Classifiers for Syndromic Surveillance. Wendy W. Chapman, PhD. Syndromic Surveillance from Chief Complaints. Triage clerk. Chief complaint Cough/fever. CC Classifier. Syndromic Category

felcia
Download Presentation

A Review of Chief Complaint Classifiers for Syndromic Surveillance

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. University of Pittsburgh Dept. of Biomedical Informatics A Review of Chief Complaint Classifiers for Syndromic Surveillance Wendy W. Chapman, PhD

  2. Syndromic Surveillance from Chief Complaints Triage clerk Chief complaint Cough/fever CCClassifier • Syndromic • Category • Respiratory • GI • Neurological • Rash • Hemorrhagic • Botulinic • Constitutional Alarms Epidemiology Team

  3. Overview • Chief complaint classifiers • How do they work? • How are they different? • Evaluation of chief complaint classification • How do we evaluate their performance? • How well do they work?

  4. Chief Complaint Classifiers Quantify the similarities and differences of automated chief complaint classifiers • Distributed survey to developers of classifiers • Characterized chief complaint classifiers • Syndromes they output • Methods for mapping free-text to syndromes • Type of pre-processing techniques Chief Complaint Classifiers Evaluation Future Work

  5. ESSENCE CC Classifier Biosense EARS Mayo Clinic Vocabulary Server Ontology-enhanced BioPortal CC Classifier QUESST ME Classifier NC DetectSyndrome Case Report CC-MCSVM NGram CC Classifier CoCo SyCo MPLUS NYC Syndromic Macros CC-EDS Chief Complaint Classifiers14 Survey Respondents

  6. Majority of classifiers map to Respiratory Gastrointestinal Neurological Most systems map to Hemorrhagic Rash Botulinic Some systems map to Constitutional (6) Fever (4) Shock/Coma (1) Influenza-like Illness (4) Infectious Derm (1) Trauma (1) Local Lesion (3) Lymphadenitis (2) Sudden illness/death (2) Injury (1) Specific infection (1) Syndromes The Classifiers Output

  7. Difference Between Similar Syndromes Many syndromes represent a more specific presentation of a general syndrome • Single symptom • diarrhea • Severity • severe GI • Anatomic location • upper respiratory, lower GI • Likelihood of being caused by a bioterrorist agent or being infectious • Meningoencephalitis • Febrile syndrome (febrile rash) • Not infectious • Chronic respiratory (asthma and COPD)

  8. How Do the Classifiers Work?

  9. Symptoms or Syndromes • Classify directly to syndromes “SOB” = Respiratory • Classify to symptoms then to syndromes “SOB” = Dyspnea = Respiratory • Can generate new syndromes on the fly

  10. Multiple Problems • Assigns a singlesyndrome to a string “short of breath/vomiting” = Respiratory • Assigns multiplesyndromes to a string “short of breath/vomiting” = Respiratory, GI

  11. Synonyms Short of breath  dyspnea Coughing  cough Coughs  cough Abbreviations ha  headache abd  abdominal gx  ground transportation Acronyms n/v  nausea/vomiting sob  shortness of breath Truncations diar  diarrhea poss  possible Concatenations blurredvision  burred vision flus sxs  flu symptoms Misspellings & typographic errors nausa  nausea diahrea  diarrhea Preprocessing Substantial word variation in chief complaints

  12. Preprocessing • General cleanup • replace symbols with spaces, lowercase, etc. • Synonym replacement • Truncation replacement • Abbreviation replacement • Spell-checking • Removal of certain words • Map words in string to concepts in a standardized vocabulary (e.g., UMLS) • Split multiple complaints

  13. BioPortal EMT-P (Travers) NC Detect

  14. Feature Identification Case Identification Outbreak Detection Does the NLP application accurately identify features? Do the features represent a patient’s actual state? Can we detect outbreaks from the identified features? Yes Yes No No Stop Stop Three Stages of Evaluationfor Biosurveillance Chief Complaint Classifiers Evaluation Future Work

  15. What Have we Learned from Evaluations of Classifiers? • Classifiers are good at feature detection • “n/v/d”  Gastrointestinal • Classified chief complaints show high specificity and moderate sensitivity in case detection • 10% - 70% sensitivity • Classified chief complaints can be used to detect *some* outbreaks more accurately and earlier than traditional methods

  16. Preprocessing does not improve case identification performance* • Exception: splitting chief complaints into multiple problems for classifiers that assign single syndromes • We have no idea which classifiers perform best • Partly depends on preference of mapping methods • No standardized syndromic definitions for comparing classifiers • classifiers with different output are difficult to compare * One study of two preprocessors and two classifiers

  17. Future Directions • Chief complaints are • Ubiquitous • Timely • Simple to process • Chief complaints are not • Always accurate • Complete • We need to monitor multiple data sources as they come available • Pre-diagnostic (over-the-counter sales) • Diagnostic • ED Reports • Chest radiograph reports Chief Complaint Classifiers Evaluation Future Work

  18. Thank You

More Related