170 likes | 284 Views
DenK and iCat Two Projects on Cooperative Electronic Assistants (CEA’s). Robbert-Jan Beun, Rogier van Eijk & Huub Pr ü st Department of Information and Computing Sciences Utrecht University rj,rogier,huub@cs.uu.nl. DenK (Dialoogvoering en Kennisopbouw).
E N D
DenK and iCatTwo Projects on Cooperative Electronic Assistants (CEA’s) Robbert-Jan Beun, Rogier van Eijk & Huub Prüst Department of Information and Computing Sciences Utrecht University rj,rogier,huub@cs.uu.nl
DenK(Dialoogvoering en Kennisopbouw) • Collaborative project between University Tilburg, Eindhoven University and IPO • 1990 – 1998 • Fundamental principles of communication Generic architecture • Natural language interface • Knowledge representation • Object oriented animation • Result: implementation, 8 dissertations, conference proceedings, papers, reports, …
Task domain direct observation and manipulation software interface User CEA language/symbolic Triangle-paradigm A human-computer interface should correspond to our natural interaction in two ways: • Direct: • physical world • perception and action • Indirect • with humans by language • speech acts
Task domain observation & action interaction User CEA communication DenK properties • Mental representation • Context • Reasoning • Communication • Typed English text • Interaction • Evaluation (software interface) • Task domain • Electron microscope (EM)
Spotsize Beam Shift X Magnification Beam Shift Y C2-lens On/Stand by I/D μP/nP GUN mini- condensor Focus Emission Filament Intensity specimen screen Example DenK Interface A: Why is the contrast of the visible image regulated by this aperture? B: Because the diffraction image is in the OA-plane when the microscope is in HM-mode.
Mental Representation Contexts in Type Theory Contains - ontological information - private beliefs about application domain (EM) - shared beliefs about application domain (EM) - dialogue history (discourse) Beliefs, Shared Beliefs & Dialogue Dialogue ≤ Shared beliefs ≤ Beliefs Extra context: Pending stack
Type Theory • Ontological information • Contexts and sequentiality [recipe : *s] [sobanito : recipe] [vegetarian : recipe *p] [macrobiotic : recipe *p] • Reasoning • Legal extension (well-formedness) [macrobiotic(sobanito) : *p] • Propositions-as-types (beliefs) [p17 : macrobiotic(sobanito)]
structural analysis semantic interpretation pragmatic processing From Natural Language to Type Theory to Behavior Natural language grammar lexicon ULF error/help conceptual lexicon context information Type Theory Behavior
Task domain CC-interface User CEA HC-interface Evaluation Contextual interpreter NL-parser Response generator Private beliefs NL- production Shared beliefs Pending stack DenK-architecture
Task domain User CEA Context Types in DenK • Physical Context • Private Belief • Shared Belief • Dialogue • Pending Stack • Sentence
Context 1The dialogue U: Turn on the microscope! C: OK U: Which button is on? C: The blue one. U: Turn it off! • ‘it’ refers to the last introduced button
Context 2The dialogue U: Is the button next to the start-button on? C: Yes U: Turn it off! • Two objects: button_1 and button_2 • ‘it’ refers not to the last introduced button! • ‘it’ refers to the last resolved button.
Context 3The dialogue U: Is the red button on? C: Yes U1: Turn it/that/this off! U2: Turn that/this one off! U3: Turn the/that/this button off! • The object can be in different contexts • NL provides information about context search
Context 4 Beliefs of system Ontological U1: Can you restart the internet? S1: It is impossible to restart the internet. U2: Is this the engine of that car? S2: Yes. U2: Is this the key that starts it? S2: Yes. U2: Does itrun smoothly?
Context 5Goals Remove the black block!
Context 6Physical domain Remove this one!
Conclusions • Use linguistic component (not only icons and menus) • This component should have knowledge about: • the task domain (ontology, private beliefs) • the user (shared beliefs, dialogue) • general communication principles • Use context for interpretation and production of messages • The knowledge to build this component partly exists (but is distributed): • parsers (from language to mental states) • mental states in Type Theory (to model belief states) • Gricean rules for communication (in progress) • In iCat many results from DenK can be reused