1 / 82

Identifying sets and classes: taxonomies as finding aids

Identifying sets and classes: taxonomies as finding aids . Alex Haig NHS Education for Scotland 29 th September 2005. A Case Study: medical education. What is a Taxonomy?. From the Greek taxis and nomos , (division and law) “Division into ordered groups or categories”

vilmaris
Download Presentation

Identifying sets and classes: taxonomies as finding aids

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Identifying sets and classes: taxonomies as finding aids Alex Haig NHS Education for Scotland 29th September 2005

  2. A Case Study: medical education

  3. What is a Taxonomy? • From the Greek taxis and nomos, (division and law) • “Division into ordered groups or categories” • Taxonomic schemas can be developed to order almost anything

  4. Carl Linnaeus (1701-1784) • Swedish botanist • 12 volume Systema Naturae • Domain; Kingdom; Phylum (animals) or Division (plants); Class; Order; Family; Genus; Species

  5. What is a Taxonomy? • Retain characteristics of classification, but not always based on standards used in library settings • “Taxonomies and vocabularies are structured collections of terms that can serve as values for the meta-data elements.”IMS

  6. Ironically Ambiguous … • Controlled Vocabulary: (usually) enumerative list of all given terms/values in a subject area • Taxonomy: top-down hierarchical arrangement that does not necessarily define components • Thesaurus: defines components as well as associative relationships; bound by international standards • Ontology: conceptual relationships (self-evident)

  7. Controlled Vocabulary • (usually) enumerative list of all given terms/values in a subject area • Arbovirus Infections • Bronchiolitis • Encephalitis • Eye Infections • Fatigue Syndrome • Hepatitis • Meningitis • Pneumonia • RNA Virus Infections • Sexually Transmitted Diseases • Skin Diseases • Tumor Virus Infections

  8. Taxonomy • top-down hierarchical arrangement that does not necessarily define components • Viral Diseases • Hepatitis • Hepatitis A • Hepatitis B • Hepatitis C • Chronic Hepatitis C • Hepatitis D • Hepatitis E

  9. Thesaurus • Chronic Fatigue Syndrome: A syndrome characterized by persistent or recurrent fatigue, diffuse musculoskeletal pain, sleep disturbances, and subjective cognitive impairment of 6 months duration or longer. Symptoms are not caused by ongoing exertion; are not relieved by rest; and result in a substantial reduction of previous levels of occupational, educational, social, or personal activities See Related: FIBROMYALGIA Used For: chronic fatigue syndrome encephalomyelitis, myalgic infectious mononucleosis-like syndrome, chronic postviral fatigue syndrome chronic fatigue disorder chronic fatigue and immune dysfunction syndrome chronic fatigue-fibromyalgia syndrome fatigue syndrome, postviral myalgic encephalomyelitis royal free disease

  10. Ontology • Conceptual relationships (self-evident) • Much more powerful way of describing an entire domain in a variety of methods • Metaphysical origins with the nature and relations of being • Viral diseases by: aetiology (cause), prognosis, diagnosis, protein regulation, affect

  11. Why Use or Create a Taxonomy?

  12. “I can call spirits from the vast deep.”“Why, so can I, or so can any man; but will they come when you do call for them?”Shakespeare. Henry VI Part 1 3 i

  13. Where are taxonomies used?

  14. A Case Study: medical education(discipline wide effort)

  15. Best Evidence Medical Education • the dissemination of information which allows medical teachers, institutions and all concerned with medical education to make decisions on the basis of the best evidence available • the production of appropriate systematic reviews of medical education which reflect the best evidence available and meet the needs of the user, • the creation of a culture of best evidence medical education amongst individual teachers, institutions and national bodies.

  16. Searching for evidence in medical education …… the need for a taxonomy … Get a Measure of the Problem.

  17. Association for Medical Education in Europe Evidence retrieval in medical education: obstructions and opportunities. Berlin, 2001.

  18. Methods - topic • BEME pilot and consequent review groups (Barcelona/Tel Aviv) • Feedback in Assessment

  19. Methods - software • software used - Ovid [CGI version 7.8] • permits design of rigorous strategies • consistency

  20. Methods - databases selected • Medline • Embase • ERIC  most relevant to medical education

  21. Methods - journal selected • limited time and resources • required a title that was most comprehensively indexed • Academic Medicine 1996 - present (2001)

  22. Methods - strategies • Three levels of strategy: • standard (most users; limited search syntax) • enhanced (some use of search syntax) • expert (full use of search syntax) • syntax includes: free-text, controlled vocabulary, term explosions, phrase lists, subheadings, sub strings, filters, proximity operators, etc...

  23. Methods - handsearching • “...refers to the planned searching of a journal page by page (i.e. by hand), including editorials, letters, etc., to identify all relevant items.” • time consuming and meticulous • produces the “gold standard” by which search efficiency can be measured

  24. Sensitivity • Sensitivity (recall) - percentage of “gold standard” • Sensitivity = total retrieved by search total of the hand-search Gold Standard = 46

  25. GS=46 Medline Sensitivity (n) Basic 0% 0 Enhanced 10.7% 5 Expert 19.6% 9 Embase Sensitivity (n) Basic 4.3% 2 Enhanced 10.7% 5 Expert 15.2% 7 ERIC Sensitivity (n) Basic 0% 0 Enhanced 4.3% 2 Expert 6.5% 3

  26. Specificity • Specificity (precision) - positive predictive value • Specificity = relevant records identified total retrieved by search

  27. Medline Specificity (n) Basic 0% 0 Enhanced 31.3% 16 Expert 32.1% 28 Embase Specificity (n) Basic 40% 5 Enhanced 33% 15 Expert 30.4% 23 ERIC Specificity (n) Basic 0% 0 Enhanced 40% 5 Expert 37.5% 8

  28. A Note of Caution • Academic Medicine is a journal that specialises in medical education : • more likely to be indexed for context • journal presents information for better retrieval • Other journals will fare worse • Other specificity scores for BEME pilots (not limited to one journal) ranged from 6 to 34%, with feedback in assessment at 17.8%

  29. Reasons for shockingly poor performance • Incomplete coverage of journals • No indexed database for medical education • Existing controlled vocabularies are inadequate for medical education

  30. Medical Education Taxonomy/Thesaurus Research Organisation • Initial meeting in May 2002 • Group originally coalesced around special interest group discussing the subject area

  31. Other Driving Factors • GMC-driven reforms in 1990s highlight need for life-long learning and professionalisation of university teaching • Consequently the literature expands (teachers, managers, researchers and students), with the expansion of medical education itself

  32. NHS Education for Scotland • University of Edinburgh • University of Newcastle • Royal College of Physicians (London) • University of Birmingham • Hull/York Medical School • University College London

  33. Phases of Construction • Analysis / planning • Design /development/ evaluation • Implementation • Maintenance

  34. A Diverse Group enriches the entire effort

  35. Funding • Applied to LTSN01 for small grant funding • £4000 • Travel, communication and dissemination

  36. METRO 1 - Scoping

  37. Prospective Applications • Primary - entities and processes directly used in a medical educational setting: VLEs and frameworks such as Scottish Doctor & GMC’s Tomorrow’s Doctor • Secondary - entities and processes involved with reporting and analysis: description, abstraction and synthesis of data; audit; evaluation • Tertiary – applied to philosophical and ontological studies and activities surrounding medical education.

  38. Stakeholder Contexts:education/e-learning • Increased dependence on electronically supported activities and contexts • RLOs • If there are to be efficiency gains there needs to be robust semantic and symbolic representation of entities, activities, knowledge and competencies

  39. Stakeholder Contexts: research • BEME example: groups often spread across the world and divided by language and culture • Taxonomy would aid: formulation of research question; evidence retrieval, data abstraction, data synthesis, publication and evaluation of results • Both Primary and Secondary contexts

  40. Review of Existing Schemas • Structure: natural language (not), enumerative lists (e.g. glossary), semantically mono-dimension (taxonomy or ontology), semantic poly-dimension (thesaurus) • Purpose: descriptive or indexing • Identity: non-controlled or controlled • Assche et al. IMS Global Learning Consortia

  41. Review of Existing Schemas • BET: modelled on ERIC, health context at high level • MeSH: global use; North American bias, freely maintained on behalf of users • EMTREE: less educational depth than MeSH • SNOMED: clinical specificity • EET: generalised educational thesaurus

  42. Review of Existing Schemas • IIME Glossary: Institute for International Medical Education in 2000 • Scoped not relational terms (enumerative structure) • Many terms appear elsewhere • Ninewells Thesaurus • For a print catalogue in early 1980s • Never piloted/evaluated • Many terms appear elsewhere

  43. Why not use existing schemas? • Nothing robust enough to support UK-specific contexts • No comprehensive educational terms for undergraduate, postgraduate and CPD phases • No comprehensive medical context • Yet BET and MESH were most appropriate and stable

  44. Pragmatic Approach • Create a set of bridging terms and definitions between MeSH and BET, but only where terms are absent or require new definitions or extensions • System must be dynamic • Contexts and cultures shift and evolve • Ongoing service not one-off product • Long-term viability means appropriate rules and procedures

  45. METRO Phase One: Processes • Submission of seed terms • Discussion and agreement of terms • Initial scope notes from debate • Voting and resolution of scoped terms and revision • Publishing of terms

  46. Collaborative Work Environment • CWE based on VLE at UoE Medical School • Commentary • Voting on terms; adding terms • Added and imbedded links and material • Forum • Simultaneous cross searching of MeSH, BET and METRO

  47. Phase One (Terms) • 4 months, 180 terms, considered by 16 METRO members • CWE enabled extremely valuable discussion • Many terms too general/not specific, or synonymous with BET or MESH (dropped)

  48. Phase One: Final Workshop • Typology of terms • Generic education (BET) • Generic medicine (MeSH) • Role • Process • Teaching • Learning • Assessment • Periodicity • Design • Artefact

More Related