Language Technologies Institute Carnegie Mellon University

1. Language Technologies InstituteCarnegie Mellon University Eric Nyberg (PI)Jamie CallanJaime CarbonellBob Frederking John LaffertyAlon LavieTeruko Mitamura AQUAINT Phase I Kickoff December 2001

2. Research Objectives • QA as Planning • Create a general QA planning system • How should a QA system represent its chain of reasoning? • QA and Auditability • How can we improve a QA system’s ability to justify its steps? • How can we make QA systems open to machine learning? AQUAINT Phase I Kickoff December 2001

3. Research Objectives [2] • Utility-Based Information Fusion • Perceived utility is a function of many different factors • Create and tune utility metrics, e.g.: U = Argmax k [F (Rel(I,Q,T), Nov(I,T,A), Ver(S,Sup(I,S)), Div(S), Cmp(I,A)), Cst(I,A)] - relevance- novelty- veracity, support- diversity- comprehensibility- cost I: Info item, Q: Question, S: Source, T: Task context, A: Analyst AQUAINT Phase I Kickoff December 2001

4. Research Plan • Develop end-to-end system • Architecture, Planner, Repository • Individual QA Modules • Evaluation: • English queries • English, Chinese and Japanese documents AQUAINT Phase I Kickoff December 2001

5. JAVELIN Basic Architecture All objects created orretrieved are storedcentrally for reuse Planner is independentfrom the particular QAmodules being used Details of module implementation are hidden AQUAINT Phase I Kickoff December 2001

6. JAVELIN Data Flow AQUAINT Phase I Kickoff December 2001

7. Layers &Control Flow StrategicDecisionPoints AQUAINT Phase I Kickoff December 2001

8. Question Analyzer • Parse question • Assign question & answer types, base query terms • As appropriate: • Provide syntactic and semantic analysis; translation • Initiate clarification dialog with the user AQUAINT Phase I Kickoff December 2001

9. Retrieval Strategist • Automatic, content-based resource selection • Facet-based retrieval • Example: Clinton as president, Clinton as defendant, … • Map query terms into a set of query strategies AQUAINT Phase I Kickoff December 2001

10. Retrieval Tactician • Provide a uniform interface to external data sources(“wrapper” layer) • Will support relational queries • May support queries to knowledgebases (if KB is provided) AQUAINT Phase I Kickoff December 2001

11. Request Filler • A range of parsing and information extraction techniques to produce possible answer passages • Learning optimal combination(s) or approaches (time, quality) • Is it possible to automatically learn or improve filler patterns? AQUAINT Phase I Kickoff December 2001

12. Answer Generator • Strategic vs. Tactical Generation • Compare request fills to derive answer: • Merge /remove duplicates • Detect ambiguity • Reconcile answer (secondary search) • Identify possible contradictions • Apply utility measures AQUAINT Phase I Kickoff December 2001

13. Planner • Specify planning representation • Identify decision points • Represent & manage uncertainty • Model states and operations • Model justification network • Find acceptable trade-offs: • Ratio of planning to execution • Answer utility vs. available resources AQUAINT Phase I Kickoff December 2001

14. Architecture • Abstract module APIs • Several strategies for each task • Easy modification, extension of QA modules • Comparative evaluation, ablation studies, … • Common object format& data repository • Adapters for Java, C++, XML AQUAINT Phase I Kickoff December 2001

15. Questions? Contact: Eric Nyberg ehn@cs.cmu.edu AQUAINT Phase I Kickoff December 2001