Loading in 2 Seconds...
Loading in 2 Seconds...
Identifying sets and classes: taxonomies as finding aids Alex Haig NHS Education for Scotland 29th September 2005
What is a Taxonomy? • From the Greek taxis and nomos, (division and law) • “Division into ordered groups or categories” • Taxonomic schemas can be developed to order almost anything
Carl Linnaeus (1701-1784) • Swedish botanist • 12 volume Systema Naturae • Domain; Kingdom; Phylum (animals) or Division (plants); Class; Order; Family; Genus; Species
What is a Taxonomy? • Retain characteristics of classification, but not always based on standards used in library settings • “Taxonomies and vocabularies are structured collections of terms that can serve as values for the meta-data elements.”IMS
Ironically Ambiguous … • Controlled Vocabulary: (usually) enumerative list of all given terms/values in a subject area • Taxonomy: top-down hierarchical arrangement that does not necessarily define components • Thesaurus: defines components as well as associative relationships; bound by international standards • Ontology: conceptual relationships (self-evident)
Controlled Vocabulary • (usually) enumerative list of all given terms/values in a subject area • Arbovirus Infections • Bronchiolitis • Encephalitis • Eye Infections • Fatigue Syndrome • Hepatitis • Meningitis • Pneumonia • RNA Virus Infections • Sexually Transmitted Diseases • Skin Diseases • Tumor Virus Infections
Taxonomy • top-down hierarchical arrangement that does not necessarily define components • Viral Diseases • Hepatitis • Hepatitis A • Hepatitis B • Hepatitis C • Chronic Hepatitis C • Hepatitis D • Hepatitis E
Thesaurus • Chronic Fatigue Syndrome: A syndrome characterized by persistent or recurrent fatigue, diffuse musculoskeletal pain, sleep disturbances, and subjective cognitive impairment of 6 months duration or longer. Symptoms are not caused by ongoing exertion; are not relieved by rest; and result in a substantial reduction of previous levels of occupational, educational, social, or personal activities See Related: FIBROMYALGIA Used For: chronic fatigue syndrome encephalomyelitis, myalgic infectious mononucleosis-like syndrome, chronic postviral fatigue syndrome chronic fatigue disorder chronic fatigue and immune dysfunction syndrome chronic fatigue-fibromyalgia syndrome fatigue syndrome, postviral myalgic encephalomyelitis royal free disease
Ontology • Conceptual relationships (self-evident) • Much more powerful way of describing an entire domain in a variety of methods • Metaphysical origins with the nature and relations of being • Viral diseases by: aetiology (cause), prognosis, diagnosis, protein regulation, affect
“I can call spirits from the vast deep.”“Why, so can I, or so can any man; but will they come when you do call for them?”Shakespeare. Henry VI Part 1 3 i
Best Evidence Medical Education • the dissemination of information which allows medical teachers, institutions and all concerned with medical education to make decisions on the basis of the best evidence available • the production of appropriate systematic reviews of medical education which reflect the best evidence available and meet the needs of the user, • the creation of a culture of best evidence medical education amongst individual teachers, institutions and national bodies.
Searching for evidence in medical education …… the need for a taxonomy … Get a Measure of the Problem.
Association for Medical Education in Europe Evidence retrieval in medical education: obstructions and opportunities. Berlin, 2001.
Methods - topic • BEME pilot and consequent review groups (Barcelona/Tel Aviv) • Feedback in Assessment
Methods - software • software used - Ovid [CGI version 7.8] • permits design of rigorous strategies • consistency
Methods - databases selected • Medline • Embase • ERIC most relevant to medical education
Methods - journal selected • limited time and resources • required a title that was most comprehensively indexed • Academic Medicine 1996 - present (2001)
Methods - strategies • Three levels of strategy: • standard (most users; limited search syntax) • enhanced (some use of search syntax) • expert (full use of search syntax) • syntax includes: free-text, controlled vocabulary, term explosions, phrase lists, subheadings, sub strings, filters, proximity operators, etc...
Methods - handsearching • “...refers to the planned searching of a journal page by page (i.e. by hand), including editorials, letters, etc., to identify all relevant items.” • time consuming and meticulous • produces the “gold standard” by which search efficiency can be measured
Sensitivity • Sensitivity (recall) - percentage of “gold standard” • Sensitivity = total retrieved by search total of the hand-search Gold Standard = 46
GS=46 Medline Sensitivity (n) Basic 0% 0 Enhanced 10.7% 5 Expert 19.6% 9 Embase Sensitivity (n) Basic 4.3% 2 Enhanced 10.7% 5 Expert 15.2% 7 ERIC Sensitivity (n) Basic 0% 0 Enhanced 4.3% 2 Expert 6.5% 3
Specificity • Specificity (precision) - positive predictive value • Specificity = relevant records identified total retrieved by search
Medline Specificity (n) Basic 0% 0 Enhanced 31.3% 16 Expert 32.1% 28 Embase Specificity (n) Basic 40% 5 Enhanced 33% 15 Expert 30.4% 23 ERIC Specificity (n) Basic 0% 0 Enhanced 40% 5 Expert 37.5% 8
A Note of Caution • Academic Medicine is a journal that specialises in medical education : • more likely to be indexed for context • journal presents information for better retrieval • Other journals will fare worse • Other specificity scores for BEME pilots (not limited to one journal) ranged from 6 to 34%, with feedback in assessment at 17.8%
Reasons for shockingly poor performance • Incomplete coverage of journals • No indexed database for medical education • Existing controlled vocabularies are inadequate for medical education
Medical Education Taxonomy/Thesaurus Research Organisation • Initial meeting in May 2002 • Group originally coalesced around special interest group discussing the subject area
Other Driving Factors • GMC-driven reforms in 1990s highlight need for life-long learning and professionalisation of university teaching • Consequently the literature expands (teachers, managers, researchers and students), with the expansion of medical education itself
NHS Education for Scotland • University of Edinburgh • University of Newcastle • Royal College of Physicians (London) • University of Birmingham • Hull/York Medical School • University College London
Phases of Construction • Analysis / planning • Design /development/ evaluation • Implementation • Maintenance
Funding • Applied to LTSN01 for small grant funding • £4000 • Travel, communication and dissemination
Prospective Applications • Primary - entities and processes directly used in a medical educational setting: VLEs and frameworks such as Scottish Doctor & GMC’s Tomorrow’s Doctor • Secondary - entities and processes involved with reporting and analysis: description, abstraction and synthesis of data; audit; evaluation • Tertiary – applied to philosophical and ontological studies and activities surrounding medical education.
Stakeholder Contexts:education/e-learning • Increased dependence on electronically supported activities and contexts • RLOs • If there are to be efficiency gains there needs to be robust semantic and symbolic representation of entities, activities, knowledge and competencies
Stakeholder Contexts: research • BEME example: groups often spread across the world and divided by language and culture • Taxonomy would aid: formulation of research question; evidence retrieval, data abstraction, data synthesis, publication and evaluation of results • Both Primary and Secondary contexts
Review of Existing Schemas • Structure: natural language (not), enumerative lists (e.g. glossary), semantically mono-dimension (taxonomy or ontology), semantic poly-dimension (thesaurus) • Purpose: descriptive or indexing • Identity: non-controlled or controlled • Assche et al. IMS Global Learning Consortia
Review of Existing Schemas • BET: modelled on ERIC, health context at high level • MeSH: global use; North American bias, freely maintained on behalf of users • EMTREE: less educational depth than MeSH • SNOMED: clinical specificity • EET: generalised educational thesaurus
Review of Existing Schemas • IIME Glossary: Institute for International Medical Education in 2000 • Scoped not relational terms (enumerative structure) • Many terms appear elsewhere • Ninewells Thesaurus • For a print catalogue in early 1980s • Never piloted/evaluated • Many terms appear elsewhere
Why not use existing schemas? • Nothing robust enough to support UK-specific contexts • No comprehensive educational terms for undergraduate, postgraduate and CPD phases • No comprehensive medical context • Yet BET and MESH were most appropriate and stable
Pragmatic Approach • Create a set of bridging terms and definitions between MeSH and BET, but only where terms are absent or require new definitions or extensions • System must be dynamic • Contexts and cultures shift and evolve • Ongoing service not one-off product • Long-term viability means appropriate rules and procedures
METRO Phase One: Processes • Submission of seed terms • Discussion and agreement of terms • Initial scope notes from debate • Voting and resolution of scoped terms and revision • Publishing of terms
Collaborative Work Environment • CWE based on VLE at UoE Medical School • Commentary • Voting on terms; adding terms • Added and imbedded links and material • Forum • Simultaneous cross searching of MeSH, BET and METRO
Phase One (Terms) • 4 months, 180 terms, considered by 16 METRO members • CWE enabled extremely valuable discussion • Many terms too general/not specific, or synonymous with BET or MESH (dropped)
Phase One: Final Workshop • Typology of terms • Generic education (BET) • Generic medicine (MeSH) • Role • Process • Teaching • Learning • Assessment • Periodicity • Design • Artefact