slide1 l.
Skip this Video
Loading SlideShow in 5 Seconds..
Medical FactNet PowerPoint Presentation
Download Presentation
Medical FactNet

Loading in 2 Seconds...

play fullscreen
1 / 64

Medical FactNet - PowerPoint PPT Presentation

  • Uploaded on

VR. Medical FactNet. Barry Smith University at Buffalo and IFOMIS, Leipzig Christiane Fellbaum Princeton University. A consumer health medical information system must be able to map between expert and non-expert medical vocabulary

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
Download Presentation

PowerPoint Slideshow about 'Medical FactNet' - tana

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
medical factnet

Medical FactNet

Barry Smith

University at Buffalo and IFOMIS, Leipzig

Christiane Fellbaum

Princeton University

A consumer health medical information system must be able to map between expert and non-expert medical vocabulary
  • GOAL: A unified medical language system for non-expert medical vocabulary
a new methodology for the construction and validation of information resources for consumer health
A New Methodology for the Construction and Validation of Information Resources for Consumer Health
mwn specific aims
  • to extend and validate WordNet 2.0’s medical coverage in light of recent advances in medical terminology research
  • focusing initially on the English-language single word expressions used and understood by non-experts
  • provision of a mapping to MeSH
  • use as interlingua for MWNs in other languages
wordnet miller fellbaum
WordNet (Miller, Fellbaum)
  • Large lexical database; ubiquitous tool of NLP
  • coverage comparable to collegiate dictionary, over 130,000 word forms
  • 40 wordnets in different languages
  • WordNet: rich medical coverage, but pooly validated and poor formal architecture
  • How create a validated Medical WordNet (MWN)?
building blocks of wordnet synsets concepts in medical terminology
Building blocks of WordNet = ‘synsets’ = ‘concepts’ in medical terminology
  • terms in same synset = they are interchangeable in some sentential contexts without altering truth-value:
  • {car, automobile}, {shut, close}
  • synsets linked via small number of binary relations:
  • is-a
  • part-of
  • verb entailments: (walk-limp, forget-know).
strengths of wordnet 2 0
Strengths of WordNet 2.0
  • Open source
  • Very broad coverage
  • Is-a / part-of architecture
  • Tool for automatic sense disambiguation
13 senses for feel is a verb
13 senses for feel is a verb
  • experience – She felt resentful
  • find – I feel that he doesn't like me
  • feel – She felt small and insignificant;
  • feel – We felt the effects of inflation
  • feel – The sheets feel soft
  • grope –He felt for his wallet
  • finger – Feel this soft cloth!
  • explore – He felt his way around the dark room)
  • feel – It feels nice to be home again
  • feel – He felt the girl in the movie theater)
medical senses of feel
Medical senses of ‘feel’
  • palpate – examine a body part by palpation:
  • The nurse palpated the patient's stomach;The runner felt her pulse.
  • sense – perceive by a physical sensation, e.g. coming from the skin or muscles:
  • He felt the wind;She felt an object brushing her arm;He felt his flesh crawl;She felt the heat when she got out of the car; He feels pain when he puts pressure on his knee.
  • feel – seem with respect to a given sensation:
  • My cold is gone – I feel fine today;She felt tired after the long hike.
  • many word units are monosemic (clinician, stethoscope)
  • most common words are polysemic
  • lexicon of the order of 1,500 word units
  • with some 3,000 distinct word senses.
  • tested by incorporation in NLP applications used for purposes of information retrieval, machine translation, question-answer systems, text summarization
medical factnet mfn
Medical FactNet (MFN)
  • a large corpus of natural-language sentences providing medically validated contexts for MWN terms.
  • pilot corpus: 40,000 sentences
  • full MFN (for common diseases): ~250,000 sentences
  • accredited as intelligible by non-experts
  • and as true by experts
medical beliefnet mbn
Medical BeliefNet (MBN)
  • = totality of sentences about medical phenomena to which non-experts assent
  • comes for free, given our methodology for creating MFN
sources for mfn
Sources for MFN
  • WordNet glosses and arcs
  • Online health information services targeted to consumers
  • NetDoctor, MEDLINEplus
  • (factsheets on common diseases)
constructing mbn and mfn

Medical BeliefNet

Medical FactNet

Constructing MBN and MFN
  • sources (WordNet, MEDLINEplus …)
  • filtering for intelligibility by non-experts
  • pool of natural language sentences
  • filtering for non-expert assent filtering for validation by experts


mfn specific aims
  • To create a pilot open-source corpus of sentences about medical phenomena in the English language
  • restricted to natural language
  • grammatically complete
  • logically and syntactically simple sentences
  • rated as understandable by non-expert human subjects in controlled questionnaire-based experiments
mfn specific aims18
  • sentences must be self-contained in the sense that they make no reference to any prior context and do not contain any proper names or expressions like it or he or then that need to be interpreted with respect to other sentences.
constructing mfn
Constructing MFN
  • Sentences in MFN must receive high marks for correctness on being assessed by medical experts.
  • MFN designed to constitute a representative fraction of the true beliefs about medical phenomena which are intelligible to non-expert English-speakers.
constructing mbn
Constructing MBN
  • Sentences in MBN must receive high marks for assent on being assessed by non-experts.
  • MBN designed to constitute a representative fraction of the beliefs about medical phenomena (both true and false beliefs) distributed through the population of English speakers.
compiling mfn and mbn in tandem
Compiling MFN and MBN in tandem
  • will allow systematic assessment of the disparity between lay beliefs and vocabulary as concerns medical phenomena and the corresponding expert medical knowledge.
  • will allow us to establish automatically for any given sub-population which areas its beliefs about medical phenomena differ most significantly from validated medical knowledge
uses of mfn
  • used for quality assurance of MWN
  • support the population of MWN by yielding new families of words and word senses
  • medical education
  • consumer health information
  • (in conjunction with MBN) allow new sorts of experiments in the linguistics, psychology and anthropology of consumer health
evaluation of mfn
Evaluation of MFN
  • measure the benefits it brings when incorporated into an existing on-line consumer health portal based on term-search technology.
  • test whether exploiting the resources of MFN can lead to improved results in the retrieval of expert information
why study non expert medical language
Why study non-expert medical language?
  • mismatch between expert and non-expert language
  • taxonomies reflecting popular lexicalizations have small coverage (and shallow hierarchies) relative to technical vocabularies
  • no popular terms linking infectious disease and mumps
  • popular medical terms (flu) often fuzzier than technical terms
  • extension of non-expert term used also by experts sometimes smaller, sometimes larger
  • hypothesis: with few exceptions the focal meanings coincide in their extensions
mismatches in doctor patient communication
Mismatches in Doctor-Patient Communication
  • Practical skills of physician in acquiring and conveying relevant and reliable information by using non-expert language tailored to individual patient
  • The physician, too, is a human being, thus ex officio a member of the wider community of non-experts
  •  continues to use non-expert language for everyday purposes
Question: My seven-year-old son developed a rash today … a friend of mine had her 10-day-old baby at my home last evening before we were aware of the illness. … I have read that chickenpox is contagious up to two days prior to the actual rash. Is there cause for concern at this point?
  • Answer Text: Chickenpox is the common name for varicella infection. ...
  • You are correct in that a person with chickenpox can be contagious for 48 hours before the first vesicle is seen. ...
  • Of concern, though, is the fact that newborns are at higher risk of complications of varicella, including pneumonia. ...
  • There is a very effective means to prevent infection after exposure. A form of antibody to varicella called varicella-zoster immune globulin (VZIG) can be given up to 48 hours after exposure and still prevent disease. ...
lexical mismatches
Lexical mismatches
  • rooted in legal concerns?
  • both primary care physician and online information system, must respond primarily with generic, or case- or context-independent, information
  • most requests relate to specific and episodic phenomena (occurrences of pain, fever, reactions to drugs, etc.).
  • Hence focus of MFN on generic sentences = context-independent statements about causality, about types of persons or diseases or about typical or possible courses of a disease.
  • designed to map the generic medical information which non-experts are able to understand
corpus and fact based approaches to information retrieval
Corpus- and fact-based approaches to information retrieval
  • Automatic sense-disambiguation one of the hardest challenges facing linguistic science today.
  • meanings of highly polysemous terms cannot be discriminated without consideration of their contexts.
  • People do this without apparent difficulties
  • New NLP methodologies to harness computers to manipulate large text corpora
  • Train automatic systems on large numbers of semantically annotated sentences, exploit standard pattern-recognition and statistical techniques for purposes of disambiguation.
example framenet
Example: FrameNet
  • 500 Frames, each with a plurality of Frame Elements
  • Medical Frames:
  • Addiction, Birth, Biological Urge, Body Mark, Cure, Death, Health Response, Medical Conditions, Medical Instruments, Medical Professional, and Medical Specialtiesand Observable Body Parts.
frame cure
Frame: Cure
  • Frame Elements:
  • alleviate. v, alleviation. n, curable. a, curative. a, curative. n, cure. n, cure. v, ease. v, heal. v, healer. n, incurable. a, palliate. v, palliation. n, palliative. a, palliative. n, rehabilitate. v, rehabilitation. n, rehabilitative. a, remedy. n, resuscitate. v, therapeutic. a, therapist. n, therapy. n, treat. v, treatment. n.
example penn proposition bank
Example: Penn Proposition Bank
  • designed as a corpus of coherent texts. The intention is to train an automatic system to ‘learn’ the contexts for words and their context-specific meanings.
  • corpus characterized by a specific logical (function-argument-based) architecture.
both framenet and proposition bank
Both FrameNet and Proposition Bank
  • have poor medical coverage
  • Both focus on word usage in general, rather than on domain-specific contexts.
  • Neither concerned with the questions of factuality or validation of statements
example cyc knowledge base
Example: CYC knowledge base
  • collection of hundreds of thousands of statements mostly about the external world:
  • The earth is round
  • Mountains are one kind of landform
  • Albany is the capital of New York
  • parcelled into micro-theories
in contrast to cyc
In contrast to CYC,
  • MFN focuses on one single (albeit very large) domain
  • MFN stores English sentences (CYC is language non-specific);
  • MFN discriminates folk beliefs and expert knowledge (MFN designed to consistent with the body of established science;
  • MFN is publicly available.
medical coverage of wordnet 2 0
Medical coverage of WordNet 2.0
  • combines expert and non-expert vocabulary, both current and medieval:
  • suppuration#2 {pus, purulence, suppuration, ichor, sanies, festering – (a fluid product of inflammation)}
wordnet 2 0 labels 504 word forms medicine
WordNet 2.0 labels 504 word-forms ‘medicine’
  • infection#1 {(the pathological state resulting from the invasion of the body by pathogenic microorganisms)}
  • infection#3 {(the invasion of the body by pathogenic microorganisms and their multiplication which can lead to tissue damage and disease)}
  • infection#4 {infection, contagion, transmission – (an incident in which an infectious disease is transmitted)}
  • maturation#2 {growth, growing, maturation, development, ontogeny, ontogenesis – ((biology) the process of an individual organism growing organically; a purely biological unfolding of events involved in an organism changing gradually from a simple to a more complex level; he proposed an indicator of osseous development in children)}
  • maturation#3 {festering, suppuration, maturation – (the formation of morbific matter in an abscess or a vesicle and the discharge of pus)}
medically relevant errors in wordnet 2 0
Medically relevant errors in WordNet 2.0
  • snore-sleep linked via verb entailment: “if someone snores, then he necessarily also sleeps.”
  • In medicine: quite possible to snore while awake, since snoring implies the respiratory induced vibration of glottal tissues as associated not only (and most usually) with sleep but also with relaxation or obesity.
  • Methodology for constructing MFN will provide us with a systematic means to detect such errors.
  • Constructing MBN will give us the resources to do justice to the reason why such cases were included in the first place:
  • People can only snore when they are asleep and similar sentences belong precisely to the folk beliefs about medicine which MBN will document
extracting sentences from online consumer health information sources
Extracting sentences from online consumer health information sources
  • In one experiment sentences were derived by researchers in medical informatics from factsheets on Airborne allergens in NIAID’s Health Information Publications and on Hay fever and perennial allergic rhinitis in the UK NetDoctor’s Diseases Encyclopedia.
output sentences
Output sentences
  • use simple syntax and draw on natural-language terms used in original sources
  • Sentences containing anaphora, instructions, warnings, … are replaced by complete statements constructed via simple syntactic modifications.
  • 1644 sentences produced (= 20 person hours of effort)500 sentences were subjected to a preliminary evaluation by pairs of medical students (on a score of 1-5 …)
  • 58% were rated by with a score of 2 x 5
  • but: measures for inter-rater agreement too low for these results to be statistically significant.
validation methods

Medical BeliefNet

Medical FactNet

Validation methods
  • sources
  • A: filtering for intelligibility by non-experts
  • pool
  • B: filtering for non-expert assent C: filtering for validation by
  • experts
validation methods44
Validation methods
  • sources
  • filtering for intelligibility by non-experts
  • pool
  • filtering for non-expert assent filtering for validation by experts
  • 400 x 250 statements will be rated for understandability by two participants, making for a total of 200,000 ratings in response to the question:
  • on a scale from 1-5, would you describe this sentence as hard to understand or easy to understand?
  • Raters will be encouraged not to reflect on successive statements
  • Only those statements which receive a score of at least 4 from each of 2 subjects will pass on to the pool
validation methods46
Validation methods
  • sources
  • filtering for intelligibility by non-experts
  • pool
  • filtering for non-expert filtering for validation by experts
  • Collections of 200 statements from the pool will be rated for assent by each of 250 participants.
  • on a scale from 1-5, would you describe this sentence with the words do not agree at all … agree completely?
  • Raters will be encouraged to reflect upon their answers if necessary
  • Statements receiving a score of at least 4 from each of two raters will be stored as components of Medical BeliefNet (MBN).
validation methods48
Validation methods
  • sources
  • filtering for intelligibility by non-experts
  • pool
  • filtering for non-expert assent filtering for validation by
  • experts
  • Raters, selected from medical faculty and advanced medical students, will be subject to a pre-evaluation as follows.
  • A set of 40 sentences in the pool will be validated as true or false by the relevant specialists
  • Only those candidate participants with very high scores in matching these validations will be selected to serve as raters in the validations of sentences for MFN.
  • Rating for MFN will involve no time constraints
  • raters will be encouraged to use reference works
  • On a scale from 1-5, how strongly do you believe this statement?
  • Only sentences receiveing scores of 5 from each of two raters will be added to the MFN database.
  • Thus in relation to those sentences which receive a score of less than 5, raters will be encouraged to propose alternative statements, which will be used as new input to the non-expert phase for assessment.
training of expert raters for mfn
Training of expert raters for MFN
  • will include e.g. guidance as to the treatment of statements which relate only to what holds for the most part or in most cases.
  • people with a cold sometimes sneeze
  • could mean either: not all people with a cold sneeze, contradicting the fact that sneezing is a mandatory symptom for a cold,
  • or all people with a cold sneeze, but not all the time, which would be rated as correct.
evaluation of mwn and mfn
Evaluation of MWN and MFN
  • users of a consumer health information portal will be randomly assigned to one of four groups: 1. access to the unsupplemented portal; 2. access also to MWN, 3. access to MFN, 4. access to both MWN and MFN
  • then apply modified Saracevic Kantor method for evaluating user satisfaction with internet query services
  • (a) on a scale from 1-5, would you describe your problem as weakly defined [1] or clearly defined [5]?
  • (b) on a scale from 1-5, would you say that your use of this information will be open to many purposes [1], or for a specifically defined purpose [5]?
  • (c) on a scale from 1-5, how would you rank the amount of knowledge you possess in relation to the problem which motivated you request?
  • (d) on a scale from 1-5, how would you rank the probability that information about the problem which motivated your question will be found?
  • (e) related to satisfaction: on a scale from 1-5, how would you rate your satisfaction with the answers you obtained from this service? ([1] = not satisfied at all, [5] = completely satisfied.)
evaluation for retrieval of expert information
Evaluation for retrieval of expert information
  • We will take as baseline the performance of an existing information retrieval system on the OHSUMED corpus of documents collated from MEDLINE abstracts and compare this baseline with the performance obtained when the system is given additional access to MWN/MFN
future work
Future work
  • application of MBN/MFN methodology to evaluate the reliability of the medical knowledge of different non-expert communities
  • by preserving data pertaining to the sources of entries in MBN it will be possible to keep track of specific kinds of false beliefs as originating in specific kinds of informants. This may prove a valuable source of information in targeting specific groups for specific types of remedial medical education.
future work56
Future work
  • experiments in the tradition of E. Rosch to investigate how the domain of medical phenomena is conceptualized by non-expert human subjects ()
  • Basic level words: tomato, cabbage vs.
  • bean vs. vegetable (too general) / cherry tomato (too specific)
  • what is the basic level of lexical specification in the domain of medical phenomena?
  • what are the basic kinds in the ontology of medicine of natural-language-using subjects?
different roles of mfn and mbn
Different roles of MFN and MBN
  • MFN associated with constructing practical tools designed to assist users in coming to believe what is true
  • MBN associated with researchregarding what people believe about medical phenomena.
towards a comprehensive assay of consumer health knowledge
Towards a comprehensive assay of consumer health knowledge
  • Ultimate goal: to document in an ontologically coherent fashion the entirety of the medical knowledge that is capable of being understood by average adult consumers of healthcare services in the United States today.
just as english wordnet
Just as English WordNet
  • serves as an interlingual index between wordnets in different languages,
  • so MWN and MFN can function as an inter-ontology index between different expert factnets prepared for different parts of technical biomedical knowledge
use of wordnet in medical informatics
Use of WordNet in medical informatics
  • e.g. as tool for simplifying information extraction from the corpus of MEDLINE abstracts:
  • by replacing verbs with corresponding synsets and so reducing the number of relations that need to be taken account of in the analysis of texts