1 / 17

Generating Definitional Descriptions

Generating Definitional Descriptions. Kathleen McKeown Sasha Blair-Goldensohn. What is a definitional description?. Long answer response More descriptive than dictionary definitions Shorter than an encyclopedia entry. What is the Hajj?.

sailor
Download Presentation

Generating Definitional Descriptions

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Generating Definitional Descriptions Kathleen McKeown Sasha Blair-Goldensohn

  2. What is a definitional description? • Long answer response • More descriptive than dictionary definitions • Shorter than an encyclopedia entry

  3. What is the Hajj? The Hajj, or pilgrimage to Makkah [Mecca], is the central duty of Islam. More than two million Muslims are expected to take the Hajj this year. Muslims must perform the hajj at least once in their lifetime if physically and financially able. The Hajj is a milestone event in a Muslim's life. The annual hajj begins in the twelfth month of the Islamic year (which is lunar, not solar, so that hajj and Ramadan fall sometimes in summer, sometimes in winter). The Hajj is a week-long pilgrimage that begins in the 12th month of the Islamic lunar calendar. Another ceremony, which was not connected with the rites of the Ka'ba before the rise of Islam, is the Hajj, the annual pilgrimage to 'Arafat, about two miles east of Mecca, toward Mina. The hajj is one of five pillars that make up the foundation of Islam.

  4. Approach • Integration of goal-driven and data-driven strategies • Goal-driven: • Search for sentences conveying typical definitional information using predicates • Genus: ``The Hajj is a pilgrimage.’’ • Data-driven • Drawn from summarization • Redundancies determine content

  5. DefScriber • Implemented prototype • 3 predicates • Genus, species, non-specific definitional (NSD) • New method for identifying redundancies • Heuristics from summarization for fluency • Robust, coherent definitional descriptions • Preliminary, subjective evaluation

  6. Input “What is the Hajj?” N = 20 L = 6 11 Web documents, 1127 total sentences Sentence clusters, importance ordering 383 Non-specific Definitional sentences Document Retrieval Predicate Identification Data-Driven Analysis Definition Creation 9 Genus-Species Sentences The Hajj, or pilgrimage to Makkah (Mecca), is the central duty of Islam. The Hajj is a milestone event in a Muslim 's life. The hajj is one of five pillars that make up the foundation of Islam. …

  7. Predicates Others: Target Original, Target Anaphor, Target Synonym, Definitional

  8. Implementation of Predicates • Training data • 81 documents marked for predicate instances • Drawn from retrieved documents for 14 terms in broad categories • Feature-based classification • Learned features to classify sentences • Term frequency, sentence position, words • Definitional (81%), history (65%) • Pattern recognition • Based on sentence structure • Semantic typing • Genus, species

  9. S S S NP NP VP NP VP VP NP NP contains contains The Hindu Kush NP VP NP is The Hajj, or represents NP NP Pilgrimage to Makkah (Mecca), DT? TERM FormativeVb NP PP NP PP NP PP between two major plates: the Indian and Eurasian. the boundary Genus PREP Species the central duty of Islam. The original Genus-Species sentence The extracted partial syntax-tree pattern A matching sentence Example Sentence To Pattern Matches Input Sentence

  10. Data-drivenAn integration of four techniques • Centroid based (Top N) • Find information central to definitions • NSD sentences sorted in order of distance from centroid • Clustering • Clusters are now ordered by distance from centroid • Take one sentence from each cluster • Local IDF weighting • Terms “hajj”, “Mecca”, “Koran” problematic • Weight distance metrics with local IDF from input sentences • Sentence ordering • Pick sentences 2-N based on cohesion with previous sentence

  11. What is Sarin? Genus/species sentences: Sarin is one of the organophosphorus nerve gases developed during the Second World War. Sarin is an extremely toxic substance that disrupts the nervous system, overstimulating muscles and vital organs. Two of 51 clusters: • Although sarin can be made with publicly available chemicals, a sophisticated lab is needed to make sarin that is pure and long lasting. (1) Nerve agents such as Sarin have been the chemical weapons of choice since World War II. (2) Once produced, sarin presents both a storage and delivery problem. • The latest use of Sarin was in the nerve gas attack of a Tokyo subway by the Aum Shinrikyo cult in 1995. (5) Subway riders injured in Aum Shinrikyo sarin gas attack, Tokyo, March 20, 1995. (1)

  12. What is Sarin? (full response) • Sarin is one of the organophosphorus nerve gases developed during the Second World War. (10) The latest use of Sarin was in the nerve gas attack of a Tokyo subway by the Aum Shinrikyo cult in 1995. (5) A year earlier, the cult killed seven people in a sarin gas attack in the central Japanese city of Matsumoto. (1) If purer sarin had been released, particularly as an aerosol, the attack might have been much worse. (1) Under wet and humid weather conditions, Sarin degrades swiftly, but as the temperature rises up to a certain point, Sarins lethal duration increases, despite the humidity. (4) Just prior to the attack, Aum hurriedly produced a low-lethality batch of sarin. (1) Iraq began producing sarin in 1984 and admitted to possessing 790 tons of it in 1995. (8) Sarin was first produced by German scientists in 1938 ; although it was not used during the war, many have speculated that it could have given the Germans a decisive advantage. (2)

  13. Evaluation • Participated in AQUAINT-wide F02 • TopN • 52% f-measure w/recall 5 times precision • Survey based evaluation S03 • Baselines: TopN, Simple cluster, Local IDF • Defscriber: an integration of all four • Questions • Rate structure/organization • How relevant? • Breadth of coverage? • How redundant? • Rate overall understanding

  14. Evaluation Highlights • Using all features gives best results on • structure/organization • term understanding • relevance • All methods significantly less on redundancy than TopN • Local IDF top on coverage

  15. Conclusions • Uses goal-driven approach to screen definitional from large volume of non-definitional information • Uses data-driven methods for exploiting redundancy on the web through summarization • Implemented prototype that scores well in evaluation

  16. What’s Next? • More predicates • Summarization ``smoothing’’ techniques • In-depth evaluation

More Related