identifying sets and classes taxonomies as finding aids n.
Skip this Video
Loading SlideShow in 5 Seconds..
Identifying sets and classes: taxonomies as finding aids PowerPoint Presentation
Download Presentation
Identifying sets and classes: taxonomies as finding aids

Loading in 2 Seconds...

play fullscreen
1 / 82

Identifying sets and classes: taxonomies as finding aids - PowerPoint PPT Presentation

Download Presentation
Identifying sets and classes: taxonomies as finding aids
An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript

  1. Identifying sets and classes: taxonomies as finding aids Alex Haig NHS Education for Scotland 29th September 2005

  2. A Case Study: medical education

  3. What is a Taxonomy? • From the Greek taxis and nomos, (division and law) • “Division into ordered groups or categories” • Taxonomic schemas can be developed to order almost anything

  4. Carl Linnaeus (1701-1784) • Swedish botanist • 12 volume Systema Naturae • Domain; Kingdom; Phylum (animals) or Division (plants); Class; Order; Family; Genus; Species

  5. What is a Taxonomy? • Retain characteristics of classification, but not always based on standards used in library settings • “Taxonomies and vocabularies are structured collections of terms that can serve as values for the meta-data elements.”IMS

  6. Ironically Ambiguous … • Controlled Vocabulary: (usually) enumerative list of all given terms/values in a subject area • Taxonomy: top-down hierarchical arrangement that does not necessarily define components • Thesaurus: defines components as well as associative relationships; bound by international standards • Ontology: conceptual relationships (self-evident)

  7. Controlled Vocabulary • (usually) enumerative list of all given terms/values in a subject area • Arbovirus Infections • Bronchiolitis • Encephalitis • Eye Infections • Fatigue Syndrome • Hepatitis • Meningitis • Pneumonia • RNA Virus Infections • Sexually Transmitted Diseases • Skin Diseases • Tumor Virus Infections

  8. Taxonomy • top-down hierarchical arrangement that does not necessarily define components • Viral Diseases • Hepatitis • Hepatitis A • Hepatitis B • Hepatitis C • Chronic Hepatitis C • Hepatitis D • Hepatitis E

  9. Thesaurus • Chronic Fatigue Syndrome: A syndrome characterized by persistent or recurrent fatigue, diffuse musculoskeletal pain, sleep disturbances, and subjective cognitive impairment of 6 months duration or longer. Symptoms are not caused by ongoing exertion; are not relieved by rest; and result in a substantial reduction of previous levels of occupational, educational, social, or personal activities See Related: FIBROMYALGIA Used For: chronic fatigue syndrome encephalomyelitis, myalgic infectious mononucleosis-like syndrome, chronic postviral fatigue syndrome chronic fatigue disorder chronic fatigue and immune dysfunction syndrome chronic fatigue-fibromyalgia syndrome fatigue syndrome, postviral myalgic encephalomyelitis royal free disease

  10. Ontology • Conceptual relationships (self-evident) • Much more powerful way of describing an entire domain in a variety of methods • Metaphysical origins with the nature and relations of being • Viral diseases by: aetiology (cause), prognosis, diagnosis, protein regulation, affect

  11. Why Use or Create a Taxonomy?

  12. “I can call spirits from the vast deep.”“Why, so can I, or so can any man; but will they come when you do call for them?”Shakespeare. Henry VI Part 1 3 i

  13. Where are taxonomies used?

  14. A Case Study: medical education(discipline wide effort)

  15. Best Evidence Medical Education • the dissemination of information which allows medical teachers, institutions and all concerned with medical education to make decisions on the basis of the best evidence available • the production of appropriate systematic reviews of medical education which reflect the best evidence available and meet the needs of the user, • the creation of a culture of best evidence medical education amongst individual teachers, institutions and national bodies.

  16. Searching for evidence in medical education …… the need for a taxonomy … Get a Measure of the Problem.

  17. Association for Medical Education in Europe Evidence retrieval in medical education: obstructions and opportunities. Berlin, 2001.

  18. Methods - topic • BEME pilot and consequent review groups (Barcelona/Tel Aviv) • Feedback in Assessment

  19. Methods - software • software used - Ovid [CGI version 7.8] • permits design of rigorous strategies • consistency

  20. Methods - databases selected • Medline • Embase • ERIC  most relevant to medical education

  21. Methods - journal selected • limited time and resources • required a title that was most comprehensively indexed • Academic Medicine 1996 - present (2001)

  22. Methods - strategies • Three levels of strategy: • standard (most users; limited search syntax) • enhanced (some use of search syntax) • expert (full use of search syntax) • syntax includes: free-text, controlled vocabulary, term explosions, phrase lists, subheadings, sub strings, filters, proximity operators, etc...

  23. Methods - handsearching • “...refers to the planned searching of a journal page by page (i.e. by hand), including editorials, letters, etc., to identify all relevant items.” • time consuming and meticulous • produces the “gold standard” by which search efficiency can be measured

  24. Sensitivity • Sensitivity (recall) - percentage of “gold standard” • Sensitivity = total retrieved by search total of the hand-search Gold Standard = 46

  25. GS=46 Medline Sensitivity (n) Basic 0% 0 Enhanced 10.7% 5 Expert 19.6% 9 Embase Sensitivity (n) Basic 4.3% 2 Enhanced 10.7% 5 Expert 15.2% 7 ERIC Sensitivity (n) Basic 0% 0 Enhanced 4.3% 2 Expert 6.5% 3

  26. Specificity • Specificity (precision) - positive predictive value • Specificity = relevant records identified total retrieved by search

  27. Medline Specificity (n) Basic 0% 0 Enhanced 31.3% 16 Expert 32.1% 28 Embase Specificity (n) Basic 40% 5 Enhanced 33% 15 Expert 30.4% 23 ERIC Specificity (n) Basic 0% 0 Enhanced 40% 5 Expert 37.5% 8

  28. A Note of Caution • Academic Medicine is a journal that specialises in medical education : • more likely to be indexed for context • journal presents information for better retrieval • Other journals will fare worse • Other specificity scores for BEME pilots (not limited to one journal) ranged from 6 to 34%, with feedback in assessment at 17.8%

  29. Reasons for shockingly poor performance • Incomplete coverage of journals • No indexed database for medical education • Existing controlled vocabularies are inadequate for medical education

  30. Medical Education Taxonomy/Thesaurus Research Organisation • Initial meeting in May 2002 • Group originally coalesced around special interest group discussing the subject area

  31. Other Driving Factors • GMC-driven reforms in 1990s highlight need for life-long learning and professionalisation of university teaching • Consequently the literature expands (teachers, managers, researchers and students), with the expansion of medical education itself

  32. NHS Education for Scotland • University of Edinburgh • University of Newcastle • Royal College of Physicians (London) • University of Birmingham • Hull/York Medical School • University College London

  33. Phases of Construction • Analysis / planning • Design /development/ evaluation • Implementation • Maintenance

  34. A Diverse Group enriches the entire effort

  35. Funding • Applied to LTSN01 for small grant funding • £4000 • Travel, communication and dissemination

  36. METRO 1 - Scoping

  37. Prospective Applications • Primary - entities and processes directly used in a medical educational setting: VLEs and frameworks such as Scottish Doctor & GMC’s Tomorrow’s Doctor • Secondary - entities and processes involved with reporting and analysis: description, abstraction and synthesis of data; audit; evaluation • Tertiary – applied to philosophical and ontological studies and activities surrounding medical education.

  38. Stakeholder Contexts:education/e-learning • Increased dependence on electronically supported activities and contexts • RLOs • If there are to be efficiency gains there needs to be robust semantic and symbolic representation of entities, activities, knowledge and competencies

  39. Stakeholder Contexts: research • BEME example: groups often spread across the world and divided by language and culture • Taxonomy would aid: formulation of research question; evidence retrieval, data abstraction, data synthesis, publication and evaluation of results • Both Primary and Secondary contexts

  40. Review of Existing Schemas • Structure: natural language (not), enumerative lists (e.g. glossary), semantically mono-dimension (taxonomy or ontology), semantic poly-dimension (thesaurus) • Purpose: descriptive or indexing • Identity: non-controlled or controlled • Assche et al. IMS Global Learning Consortia

  41. Review of Existing Schemas • BET: modelled on ERIC, health context at high level • MeSH: global use; North American bias, freely maintained on behalf of users • EMTREE: less educational depth than MeSH • SNOMED: clinical specificity • EET: generalised educational thesaurus

  42. Review of Existing Schemas • IIME Glossary: Institute for International Medical Education in 2000 • Scoped not relational terms (enumerative structure) • Many terms appear elsewhere • Ninewells Thesaurus • For a print catalogue in early 1980s • Never piloted/evaluated • Many terms appear elsewhere

  43. Why not use existing schemas? • Nothing robust enough to support UK-specific contexts • No comprehensive educational terms for undergraduate, postgraduate and CPD phases • No comprehensive medical context • Yet BET and MESH were most appropriate and stable

  44. Pragmatic Approach • Create a set of bridging terms and definitions between MeSH and BET, but only where terms are absent or require new definitions or extensions • System must be dynamic • Contexts and cultures shift and evolve • Ongoing service not one-off product • Long-term viability means appropriate rules and procedures

  45. METRO Phase One: Processes • Submission of seed terms • Discussion and agreement of terms • Initial scope notes from debate • Voting and resolution of scoped terms and revision • Publishing of terms

  46. Collaborative Work Environment • CWE based on VLE at UoE Medical School • Commentary • Voting on terms; adding terms • Added and imbedded links and material • Forum • Simultaneous cross searching of MeSH, BET and METRO

  47. Phase One (Terms) • 4 months, 180 terms, considered by 16 METRO members • CWE enabled extremely valuable discussion • Many terms too general/not specific, or synonymous with BET or MESH (dropped)

  48. Phase One: Final Workshop • Typology of terms • Generic education (BET) • Generic medicine (MeSH) • Role • Process • Teaching • Learning • Assessment • Periodicity • Design • Artefact