Building a Terminological Database from Heterogeneous Definitional Sources

1. Building a Terminological Database from Heterogeneous Definitional Sources Smaranda Muresan, Peter T. Davis Samuel D. Popper, Judith L. Klavans Columbia University May 21, 2003

2. Why Terminology is Important? • Each agency and each department might have different ways to define the same concept • Working with multiple databases requires understanding the data across multiple agencies and domains

3. What’s an Employee? An appointed officer or employee of USDA including special Government employees (collaborators, consultants and panel members). The term excludes independent contractors. An individual who is engaged or compensated by a railroad or by a contractor to a railroad, who is authorized by a railroad to use its wireless communications in connection with railroad operations. US Department of Agriculture Federal Railroad Administration A person who works for wages or salary in the service of an employer. The term "employee" does not include a director, trustee, or officer. Mine Safety and Health Administration US SEC

4. Desiderata for Terminological Resources • Capture the ongoing evolution of language • Provide consistency, ease of sharing and integration across agencies.

5. Architecture Building the terminological DB Collection Use Heterogeneous Definitional Corpus SemanticAnalysis Database Building GetGloss ParseGloss TerminologicalDatabase Definder database query dynamic sources relations among concepts and their attributes fast access, flexibility, sharing

6. Collection • Motivation • Definitions are rich in terminological knowledge • On-line dictionaries are static and generally incomplete • Need to capture the evolution of language • Solution • GetGloss – identification and extraction of glossaries • Definder - extraction of definitions from online free text Acquisition of Heterogeneous Definitional Corpus GetGloss Definder

7. Motivation Need to identify relationships among concepts e.g. synonyms, hypernyms, cross-reference Need to store this conceptual information for easy and fast access and integration Building the Terminological DB Gasoline: See Motor Gasoline (Finished). Motor Gasoline (Finished): A complex mixture of relatively volatile hydrocarbons with or without small quantities of additives, blended to form a fuel suitable for use in spark-ignition engines. Motor gasoline, as defined in ASTM Specification D 4814 or Federal Specification VV-G-1690C, is characterized as having a boiling range of 122 to 158 degrees Fahrenheit at the 10 percent recovery point to 365 to 374 degrees Fahrenheit at the 90 percent recovery point. "Motor Gasoline" includes conventional gasoline; all types of oxygenated gasoline, including gasohol; and reformulated gasoline, but excludes aviation gasoline. Term: Motor Gasoline (Finished) Source: (source (agency "EIA") (resource "Gasoline Glossary") (url …) Paren-modifier: Finished Full Definition: A complex mixture... for use in …. Core Definition: A complex mixture ... for use in spark-ignition engines Genus Phrase: A complex mixture of relatively volatile hydrocarbons Head Genus Word: hydrocarbons Properties: UsedIn: spark-ignition engines Excludes-Includes: includes conventional gasoline includes gasohol excludes aviation gasoline... • Solution • Transform definitional text into conceptual data • ParseGloss – partial semantic analysis of definitions to identify relations between concepts • Store data into a relational database Semantic Analysis Database Building ParseGloss TerminologicalDatabase Definitional Corpus

8. Database Use • Motivation • Enable the user to access the richness of terminological knowledge • Assure easy and fast access to data • Enable data sharing and integration across agencies • Enable dynamic update of data SQL query for inflammation 1. Redness, swelling, heat and pain resulting from injury to tissue (parts of the body underneath the skin). Also known as swelling. 2. A characteristic reaction of tissues to disease or injury; it is marked by four signs: swelling, redness, heat, and pain. 3. The reaction of tissue to injury . 4. A response to irritation , infection , or injury , resulting in pain , redness , and swelling . … • Solution • Query module for the relational database (SQL) TerminologicalDatabase

9. Putting It All Together Building the terminological DB Collection User Heterogeneous Definitional Corpus SemanticAnalysis Database Building GetGloss ParseGloss TerminologicalDatabase Definder database query dynamic sources relations among concepts and their attributes fast access, flexibility, sharing

10. GetGloss – Automatic Glossary Extraction • DGRC project • Given a URL find the glossary file • Challenges: • glossaries can constitute small parts of a web page, being embedded inside • there is no standard HTML tag formatting for marking <term,definition> pairs • a web page can contain <term,information> pairs, where information is not a definition.

11. True Positive

12. False Positive

13. Algorithm • Two step algorithm • Identification Component • Find candidate glossaries • Keyword + Rule-based algorithm (6 rules) • “glossary”, “dictionary” in HTML tags • Terms in alphabetical order … • Classification Component • Filter out false positives • Rule-based method (9 rules) • e.g filter if term is a Named Entity (e.g California) • Statistical method using SVM

14. Evaluating the Identification Component • 10,000+ pages from 5 different sites • 1,000 page sample: no glossaries (n=13) • 286,579 page sample from 268 domains • P=53% • Estimating recall is hard • Precision and Recall both very sensitive to perturbations (p=0 vs. p=53%) Klavans et. al (dg.o 2002)

15. Evaluating the Classification Components • GetGloss Categorizer assigns a score to each candidate based on a linear combination of weighted features • Corpus: 2400 glossary candidates • Test: 300 randomly chosen, manually categorized glossary candidates

16. Classification Component Performance 0 if Score < -100 1 if -100 ≤ Score < -50 2 if -50 ≤ Score < 0 3 if 0 ≤ Score < 50 4 if 50 ≤ Score < 100 5 if 100 ≤ Score

17. Definder- Automatic Extraction of Definitions from Text

18. Definder- Automatic Extraction of Definitions from Text

19. Definder • Part of NSF funded digital library project • Medical domain • Extract definitions from consumer oriented medical text • Corpus • Medical articles written by doctors for lay audience • Different genre (articles, manual chapters)

20. Algorithm • Shallow parsing • Simple definitions (e.g NP-NP pairs for synonyms) • Candidate complex definitions • Full parsing (Charniak’00 parser) • appositions, relative clauses, complex definitional sentences

21. Definition Patterns and Examples • Simple definitions NP ( NP ) moving x-ray pictures ( angiograms ) hypertension ( high blood pressure ) NP -- NP -- tachycardia – racing heartbeat -- …. NP of NP ( NP ) enlargement of the heart muscle ( hypertrophy )

22. Definition Patterns and Examples • Complex definitions [S[CNP NP , [CNP CNP CNP] CNP] [VP … VP] S] Angina, the pressing chest pain most people associate with heart problems, …. [S … [CNP NP -- [CNP CNP CNP] CNP]S] … atherosclerosis – the progressive narrowing of the heart’s own arteries by cholesterol plaque buildups, which starves the heart itself for oxygen and nutrients.

23. Evaluation • Quantitative • 4 human subjects , 10 articles • 53 definitions – gold standard • DEFINDER – p=86.27%, r = 84.60% • Qualitative • Usefulness and readability (non-specialists) • Completeness and accuracy (medical specialists) Klavans and Muresan(JCDL’01) Muresan and Klavans (LREC’02)

24. Coverage of On-line Dictionaries

25. Characteristics of Automatically Built Definitional Corpus • Large amount of data • Heterogeneous • Structure • Language • Semantics • Multiple definitions of the same term Environment domain Column Ozone: ozone between the Earth's surface and outer space. Ozone levels can be described in several ways. One of the most common measures is how much ozone is in a vertical column of air. The dobson unit is a measure of column ozone. Other measures include partial pressure, number density, and concentration of ozone, and can represent either column ozone or the amount of ozone at a particular altitude. Medical domain Arrhythmia: A disturbance in the beating pattern of the heart . Ozone: (O3)A colorless gas with a pungent odor, having the molecular form of O3 , found in two layers of the atmosphere, the stratosphere (about 90% of the total atmospheric loading) and the troposphere (about 10%). Ozone is a form of oxygen found naturally in the stratosphere that provides a protective layer shielding the Earth from ultraviolet radiation's harmful health effects on humans and the environment. In the troposphere, ozone is a chemical oxidant and major component of photochemical smog. Ozone can seriously affect the human respiratory system. See atmosphere, ultraviolet radiation. <http://www.epa.gov/globalwarming/glossary.html.xml> Ozone: A form of oxygen in which atoms combine in groups of three . <1006_Oxygen_therapy> Ozone: (O3)A colorless gas with a pungent odor, having the molecular form of O3 , found in two layers of the atmosphere, the stratosphere (about 90% of the total atmospheric loading) and the troposphere (about 10%). Ozone is a form of oxygen found naturally in the stratosphere that provides a protective layer shielding the Earth from ultraviolet radiation's harmful health effects on humans and the environment. In the troposphere, ozone is a chemical oxidant and major component of photochemical smog. Ozone can seriously affect the human respiratory system. See atmosphere, ultraviolet radiation. <http://www.epa.gov/globalwarming/glossary.html.xml> Ozone: A form of oxygen in which atoms combine in groups of three . <1006_Oxygen_therapy> • Addison Disease • -a rare disease that results from a deficiency in adrenocortical hormones • - an endocrine disorder that affects about 1 in 100,000 people • - a degenerative disease that is characterized by low blood pressure and dark brown pigmentation of the skin

26. Putting It All Together Building the terminological DB Collection User Heterogeneous Definitional Corpus SemanticAnalysis Database Building GetGloss ParseGloss TerminologicalDatabase Definder database query dynamic sources relations among concepts and their attributes fast access, flexibility, sharing

27. ParseGloss – Partial Semantic Analysis • Challenges – heterogeneous collection of definitions • Focus on identifying the main semantic relations among concepts • Partial Semantic Analysis based on shallow parsing • Genus phrase and genus term • Synonyms, cross-reference • Other common relations between the defined term and the concepts inside the definition

28. Example Term: Motor Gasoline (Finished) Source: (source (agency "EIA") (resource "Gasoline Glossary") (url …) Paren-modifier: Finished Full Definition: A complex mixture... for use in …. Core Definition: A complex mixture ... for use in spark-ignition engines Genus Phrase: A complex mixture of relatively volatile hydrocarbons Head Genus Word: hydrocarbons Properties: UsedIn: spark-ignition engines Excludes-Includes: includes conventional gasoline includes gasohol excludes aviation gasoline …

29. Evaluation • Task - User based evaluation to • build a gold standard for genus phrase • to ask people to identify the most important properties inside each definition • Data • 100 term-definition pairs • 7 different glossaries • 26 subjects

30. Results • Complex evaluation • Different notions of agreement and overlap: • Head only – 64% precision • Genus phrase – 59% precision

31. Building the Database • Relational database • XML – data transfer • Statistics • ~8000 terms • ~12,500 definitions • ~2000 different sources (web pages, articles, etc.)

32. Distribution of terms with multiple definitions

33. Putting It All Together Building the terminological DB Collection User Heterogeneous Definitional Corpus SemanticAnalysis Database Building GetGloss ParseGloss TerminologicalDatabase Definder database query dynamic sources relations among concepts and their attributes fast access, flexibility, sharing

34. Query the terminological Database - Term: Audit Definition: An examination of the financial statements, accounting records, and other supporting evidence of an institution… Source: www.fdic.gov - Term: Industrial radiography Definition: means an examination of the structure of materials… Source: www.nrc.gov - Term: Medical Surveillance Definition: is the systematic examination of medical monitoring data to determine… Source: www.osha.gov

35. Conclusions • Proposed a framework for solving the heterogeneous terminology problem • Automatically building a heterogeneous collection of definitions from dynamic sources • Partial semantic analysis of the definitions to identify main semantic relations between concepts • Building a database for fast, easy access, dynamic update of data, sharing across agencies

36. Future Work • Deep domain specific semantic analysis • But, how to automatically classify the glossary entries and definitions? • Based on the classification of their source (sites, articles, etc) • When and how to merge different definitions for the same term? • Integrate this acquired terminological knowledge into the DGRC system