1 / 24

Ontology-based Multimodal User Interface in MOQA

Ontology-based Multimodal User Interface in MOQA. AQUAINT 18-Month Workshop San Diego, California Tanya Korelsky Benoit Lavoie Ted Caldwell. Goals of MOQA Interface. To enable the analyst in ergonomic browsing of an information-rich, ontology based repository

venetia
Download Presentation

Ontology-based Multimodal User Interface in MOQA

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Ontology-based Multimodal User Interface in MOQA AQUAINT 18-Month Workshop San Diego, California Tanya Korelsky Benoit Lavoie Ted Caldwell CoGenTex, Inc.

  2. Goals of MOQA Interface • To enable the analyst in ergonomic browsing of an information-rich, ontology based repository • To facilitate justification of search results and production of session reports CoGenTex, Inc.

  3. Interface Features • Support for both natural language (NL) queries and structured queries used for validation of NL queries • Intuitive web-based multimodal interface for answers (search results) • Tables, text, maps, timeline, and social network graphs interconnected by hyperlinks • Natural language generation used in answers, answer justification, and for query validation • Implemented using XML-based technology CoGenTex, Inc.

  4. MOQA User Interface:Accomplishments since December 2002 Workshop • Implemented all features shown in mockup at December 2002 Workshop • Incorporated all the feedback given at Workshop • All the modalities in place, including integrated third party software, MapObjects from ESRI and Graph Editor Toolkit from Tom Sawyer Software. • Fully integrated with the MOQA Fact Repository and working as a live system over the Internet and at the sites CoGenTex, Inc.

  5. Benefits of Textual Summaries • Textual descriptions complement graphics and tables by • highlighting salient facts and events • clustering data in meaningful ways • Hypertext enables web-based data exploration via “drilldown” • Text plays a crucial role in • search results justification when search involves inference • session report production CoGenTex, Inc.

  6. CoGenTex’s NLG Expertise • R&D for MOQA builds on and extends capabilities developed in previous efforts: • Web-based multimodal presentations which include automatically generated summaries • Generic text plans which support portability between subject domains • Explanation of expert system reasoning CoGenTex, Inc.

  7. Query Interface • Integrates with the NL question processor, which builds Text Meaning Representations (TMRs). • Produces a corresponding structured query, to validate the NL question, and automatically paraphrases it in a disambiguated form of language. • Paraphrases are a crucial feature of the planned ontology-based query clarification and repair dialogs CoGenTex, Inc.

  8. Search Page with NL Query: CoGenTex, Inc.

  9. TMR for NL Query (REQUEST-INFO-1007 (THEME (VALUE (COMMON MEET-WITH-1007.THEME))) (INSTANCE-OF (VALUE (COMMON REQUEST-INFO))))(MEET-WITH-1007 (TIME (VALUE (COMMON DATE-1007 (< (FIND-ANCHOR-TIME *PAST-EVENT*))))) (AGENT (VALUE (COMMON HUMAN-1007))) (INSTANCE-OF (VALUE (COMMON MEET-WITH)))) (DATE-1007 (MONTH (VALUE (COMMON 1))) (YEAR (VALUE (COMMON 2003))) (INSTANCE-OF (VALUE (COMMON DATE))))(HUMAN-1007 (AGENT-OF (VALUE (COMMON MEET-WITH-1007))) (OBJECT-NAME (VALUE (COMMON "Jalal Talabani"))) (INSTANCE-OF (VALUE (COMMON HUMAN)))) CoGenTex, Inc.

  10. Underlying XML Query • Use standard XML technology (browser, parsers, etc.) • Support for : • Modularity: XML representation is viewable and exchangeable between subsystems • Automatic validation of query instances using a query class hierarchy described by XML schemas • Complex queries via use of logical expressions in XML CoGenTex, Inc.

  11. Validation of NL Query CoGenTex, Inc.

  12. Results Page Highlights • Multimodal display with table, timeline, text, map, and relationship diagram • Hyperlinks for drilldown • Textual summaries highlight patterns in results (prompts for sorting/ filtering table) CoGenTex, Inc.

  13. Query Results Page for Previous Query CoGenTex, Inc.

  14. Person Details Display CoGenTex, Inc.

  15. Expanded Relationship Graph CoGenTex, Inc.

  16. Details Page with Arabic Alias CoGenTex, Inc.

  17. Excerpt from Arabic Source CoGenTex, Inc.

  18. Source Arabic Document CoGenTex, Inc.

  19. Session History • Captures sequence of queries and results • Analyst can reorder/delete items to retain most important info • Reports can be exported to text editor CoGenTex, Inc.

  20. Session History Page CoGenTex, Inc.

  21. R&D Plans for next 6 months • Presentation of ontology-based inference for result justification • Intelligent maintenance of session history and retrieved material to facilitate report production • Presentation of search results for partially understood natural language queries • Exploration of interface portability between subject domains based on automatic generation of interface core from the subject domain ontology CoGenTex, Inc.

  22. CoGenTex, Inc.

  23. MOQA User Interface: Validation • Validation by domain experts is crucial for producing an intelligent and ergonomic question answering interface • Highly interconnected multimodality offers new views on data and methods of browsing • Feedback is crucial • for the search results justification • for session report production CoGenTex, Inc.

  24. Thank you! CoGenTex, Inc.

More Related