Vasileios Hatzivassiloglou, Kathleen R. McKeown, Dan Jurafsky, Wayne H. Ward, James H. Martin

1. Fusing Rich Information Extracted from Multiple Media and Languages to Generate Contextualized, Complex Answers Vasileios Hatzivassiloglou, Kathleen R. McKeown, Dan Jurafsky, Wayne H. Ward, James H. Martin Columbia University Stanford University University of Colorado at Boulder University of Texas at Dallas AQUAINT Phase II Six Month Workshop – October 2004

2. Phase II Vision • Provide long, detailed, and complex answers • Handle question types other than factual questions • Develop a unified, extensible framework for treating such questions AQUAINT Phase II Six Month Workshop – October 2004

3. Research Goals • Develop new unified strategy for generating and piecing together complex answers • Shallow semantic analysis annotates answer fragments, allowing answer filtering, comparison, and composition • Extend analysis to multiple languages, media, and linked questions AQUAINT Phase II Six Month Workshop – October 2004

4. Semantic Analysis • Multiple levels • Top level provides appropriate fillers for slots dependent on the question type • Events (who? when? where? completed? conditional?) • Opinions (target, holder, group, actual opinion predicate, time frame, polarity, strength) • Definitions • Biographies AQUAINT Phase II Six Month Workshop – October 2004

5. Semantic Analysis Support • Bottom level annotates text with general features that can be used to determine the higher level features • Semantic roles (from semantic parser) • Time expressions • Lexical polarity and semantic strength values AQUAINT Phase II Six Month Workshop – October 2004

6. Maximum Coverage of Information • A new approach for formalizing the problem of information selection • Input: • Set of text units (e.g., sentences) that are potentially relevant to the answer • Set of concepts that are desirable in the answer (e.g., representations of related events) • Matrix showing which text unit covers which concepts • Information weights assigned to each concept • Costs assigned to each text unit AQUAINT Phase II Six Month Workshop – October 2004

7. C1 C2 C3 C4 C5 1 1 0 1 1 T1 T2 T3 T4 0 1 0 0 1 1 0 0 1 0 1 0 1 1 1 Example • I(T1) = I(T2 & T3) AQUAINT Phase II Six Month Workshop – October 2004

8. Benefits of the approach • Formalization allows decoupling of the features (concepts) from the information selection algorithm • Problem translates to well-known complexity theory problem (maximum set cover) • Proof that under this model, this part of Q&A is NP-hard AQUAINT Phase II Six Month Workshop – October 2004

9. But there is a silver lining… • Efficient and effective greedy algorithm for Maximum Set Cover can be applied here • Solution guaranteed to cover at least (1-1/e) ≈ 64% of the information in the ideal solution • Evaluation over DUC data showed that this approach addresses redundancy effectively (see Filatova & Hatzivassiloglou, Coling 04) AQUAINT Phase II Six Month Workshop – October 2004

10. Definitional Questions • Approach: Combine data-driven and knowledge-based methods • The latter anticipate what “should” be in the definition (e.g., “X is a kind of Y”) • System improvements • Doubled predicate pattern coverage in 2004 • Increased system robustness • Included rewriting of pronominal references AQUAINT Phase II Six Month Workshop – October 2004

11. Learning Definitional Predicates • Before, we used hand-annotated examples • Now, we • bootstrap from a few known patterns (X caused Y) signaling a given relationship to • find many pairs for this relationship (attack/explosion, speeding/ticket) • use statistical data to find new such relationships without the patterns AQUAINT Phase II Six Month Workshop – October 2004

12. Extracting Definitions • First place in “question-based” DUC 2004 definitions among 22 teams Who is Sonia Gandhi? Congress President Sonia Gandhi, who married into what was once India’s most powerful political family, is the first non-Indian since independence 50 years ago to lead the Congress. After Prime Minister Rajiv Gandhi was assassinated in 1991, Gandhi was persuaded by the Congress to succeed her husband to continue leading the party as the chief, but she refused. The BJP had shrugged off the influence of the 51-year-old Sonia Gandhi when she stepped into politics early this year, dismissing her as a “foreigner.” Sonia Gandhi is now an Indian citizen. Gandhi, who is 51, met her husband when she was an 18-year old student at Cambridge in London, the first time she was away from her native Italy. AQUAINT Phase II Six Month Workshop – October 2004

13. New Work in Opinions • Localize opinion to a specific predicate; add time and opinion holder attributes • Use WordNet hypernym/hyponym relationships to propagate positive/negative polarity values at the word level • Calculate measure of semantic strength • Participated in recent opinion pilot AQUAINT Phase II Six Month Workshop – October 2004

14. New Work in Events • Tested event model (participants + connecting verb) as a possible set of information concepts • Significant improvement over a word-based approach (tf*idf) • Use clusters of related events to learn automatically which relationships are random and which are typical of an event type AQUAINT Phase II Six Month Workshop – October 2004

15. Fusing Rich Information Extracted from Multiple Media and Languages to Generate Contextualized, Complex AnswersProject Status Wayne Ward, James H. Martin, Kadri Hacioglu Sameer Pradhan, Steven Bethard,Ying Chen, Benjamin Douglas University of Colorado Dan Jurafsky Stanford University AQUAINT Phase II Six Month Workshop – October 2004

16. Initial Focus • Semantic Role Structure for QA • Approaches complementary to Columbia • Specific Work On • Opinions • Time Expressions • Events • Multi-Lingual Work • English, Chinese, Arabic tools AQUAINT Phase II Six Month Workshop – October 2004

17. Thematic Parse Accuracy PropBank Data TREC Data AQUAINT Phase II Six Month Workshop – October 2004

18. Alternate Algorithms • Dependency tree based • Potentially more robust because of simpler path structures • Different “view” from Minipar, based on rules not trained on TreeBank • Chunking • SVM chunk syntactic base phrases • Second SVM classify chunks with semantic roles AQUAINT Phase II Six Month Workshop – October 2004

19. Semantic Parsing in Chinese • Syntactic parser • SVM POS tagger • Retrained Collins parser • Chinese Treebank 2.0 • Performance: P/R = 78.9/76.4 • Semantic parser • PropBank Tags • Features: Syntactic Path, Target, Phrasal Category • Data: 1023 sentences as training set 113 sentences test set • Performance: P/R = 81.6/67.1 AQUAINT Phase II Six Month Workshop – October 2004

20. Opinion/Opinion_Holder • Joint work with Columbia • Opinion ID as supervised Machine Learning • Answer “How does X feel about Y” • Propositional opinions (prop arg of verb) • Same SVM framework as general semantic tagger • Annotated FrameNet and PropBank sentences If [ OH she] hadn’t known [O that he liked nothing about her] she might have mistaken that note in his voice for admiration AQUAINT Phase II Six Month Workshop – October 2004

21. Opinion/Opinion_Holder • Two different SVM architectures for Opinion • Single classifier walk constituent tree CxC • 2 stage: find propositions then classify op/non-op PxP • Opinion and Opinion_Holder AQUAINT Phase II Six Month Workshop – October 2004

22. Time Expressions • Recognize time expressions in English and Chinese • SVM chunking and tagging problem • Language independent representation • Participated in TERN evaluation That’s 30 percent more than [the same period [a year ago.]] AQUAINT Phase II Six Month Workshop – October 2004

23. Time Expressions AQUAINT Phase II Six Month Workshop – October 2004

24. Event Detection • Train and test on TimeBank corpus • Determine phrases describing events • Chunk EVENT expressions in TimeBank • Label with attribute • REPORTING, PERCEPTION, ASPECTUAL, I_ACTION, I_STATE, STATE, OCCURRENCE. AQUAINT Phase II Six Month Workshop – October 2004

25. Arabic Work • SVM based NLP tools for Arabic • Tokenizer • Part-Of-Speech tagger • Syntactic base phrase chunker • Trained on Arabic TreeBank AQUAINT Phase II Six Month Workshop – October 2004

26. Arabic Work AQUAINT Phase II Six Month Workshop – October 2004

27. Next 18 months • Complete opinion work • Much more focus on events • Processing audio documents • Produce word lattice with ASR • Use chunking tagger to parse word lattice • Dialog • Decomposition • Clarification • Follow-up AQUAINT Phase II Six Month Workshop – October 2004

28. Thematic Role Tagging • Assigning semantic labels to sentence elements. • Elements are arguments of some predicate or participants in some event. • [DATEIn 1901] [PATIENTPresident William McKinley] was [PREDICATE shot][AGENTby anarchist Leon Czolgosz] [LOCATIONat the Pan-American Exposition] AQUAINT Phase II Six Month Workshop – October 2004

29. Use of thematic tagging in QA • Generating novel answers involving • Opinions (believe, confirm, deny, negate) • Events (Activities with a starting and ending point involving fixed participants) • Causal questions • Query: What effect does a prism have on light? • Thematic Tagging:[RESULTWhat effect] does [CAUSE a prism] have on [THEME light]? • Now search for a RESULT that has ‘prism’ as a CAUSE. AQUAINT Phase II Six Month Workshop – October 2004