1 / 54

Introduction to the TrindiKit

Introduction to the TrindiKit. ESSLLI, Helsinki 20th Aug 2001 Staffan Larsson sl@ling.gu.se. What is TrindiKit?. a toolkit for building and experimenting with dialogue move engines and systems, based on the information state approach not a dialogue system!. This lecture.

hova
Download Presentation

Introduction to the TrindiKit

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Introduction to the TrindiKit ESSLLI, Helsinki 20th Aug 2001 Staffan Larsson sl@ling.gu.se

  2. What is TrindiKit? a toolkit for building and experimenting with dialogue move engines and systems, based on the information state approach not a dialogue system!

  3. This lecture • short demo • architecture & concepts • what’s in TrindiKit? • building a system

  4. Short demo • web-based demo + simple GUI • information state view • record • dialogue • update rules • module view • active modules highlighted • asynchronous processing

  5. Architecture & concepts

  6. control DME module1 module… modulei modulej module… modulen • Total Information State • (TIS) • Information state proper (IS) • Module Interface Variables • Resource Interface Variables resource1 resource… resourcem

  7. Information State (IS) • an abstract data structure (record, DRS, set, stack etc.) • accessed by modules using conditions and operations • the Total Information State (TIS) includes • Information State proper (IS) • Module Interface variables • Resource Interface variables

  8. Dialogue Move Engine (DME) • module or group of modules responsible for • updating the IS based on observed moves • selecting moves to be performed • dialogue moves are associated with IS updates using IS update rules • there are also update rules no directly associated with any move (e.g. for reasoning and planning) • update rules: rules for updating the TIS • rule name and class • preconditon list: conditions on TIS • effect list: operations on TIS • update rules are coordinated by update algorithms

  9. Modules and resources • Modules (dialogue move engine, input, interpretation, generation, output etc.) • access the information state • no direct communication between modules • only via module interface variables in TIS • modules don’t have to know anything about other modules • increases modularity, reusability, reconfigurability • may interact with user or external processes • Resources (device interface, lexicons, domain knowledge etc.) • hooked up to the information state (TIS) • accessed by modules • defined as object of some type (e.g. ”lexicon”)

  10. What’s in TrindiKit?

  11. What does TrindiKit provide? • High-level formalism and interpreter for implementing dialogue systems • promotes transparency, reusability, plug-and-play, etc. • allows implementation and comparison of dialogue theories • hides low-level software engineering issues • GUI, WWW-demo • Ready-made modules and resources • speech • interfaces to databases, devices, etc. • reasoning, planning

  12. TrindiKit contents (1) • alibrary of datatype definitions (records, DRSs, sets, stacks etc.) • user extendible • alanguage for writing information state update rules • GUI: methods and tools for visualising the information state • debugging facilities • typechecking • logs of communication modules-TIS • etc.

  13. TrindiKit contents (2) • A language for defining update algorithms used by TrindiKit modules to coordinate update rule application • A language for defining basic control structure, to coordinate modules • A library of basic ready-made modules for input/output, interpretation, generation etc.; • A library of ready-made resources and resource interfaces, e.g. to hook up databases, domain knowledge, devices etc.

  14. Special modules and resources included with TrindiKit • OAA interface resource • enables interaction with existing software and languages other than Prolog • Speech recognition and synthesis modules • TrindiKit shells for off-the-shelf recognisers • currently only ViaVoice, but more on the way • Possible future modules: • planning and reasoning modules • multimodal input and output

  15. Asynchronous TrindiKit • Internal communication uses either • OAA (Open Agent Architecture) from SRI, or • AE (Agent Environment), a stripped-down version of OAA, implemented for TrindiKit • enables asynchronous dialogue management • e.g.: system can listen and interpret, plan the dialogue, and talk at the same time

  16. How to build a system

  17. Relation TrindiKit – dialogue system domain-specific system domain knowledge (resources) dialogue theory (IS, rules, moves etc) domain-independent DME software engineering (basic types, control flow) TRINDIKIT

  18. Building a domain-independent Dialogue Move Engine • Come up with a nice theory of dialogue • Formalise the theory, i.e. decide on • Type of information state (DRS, record, set of propositions, frame, ...) • A set of dialogue moves • Information state update rules, including rules for integrating and selecting moves • DME Module algorithm(s) and basic control algorithm • any extra datatypes (e.g. for semantics: proposition, question, etc.)

  19. Domain independence of the Dialogue Move Engine • The DME is domain independent, given a certain type of dialogue • information-seeking • instructional • negotiative • ... • Domain independence of DME is not enforced by TrindiKit, but is good practice • promotes reuse of components • forces abstraction from domain-specific details, resulting in a more general theory of dialogue

  20. Specifying Infostate type • the Total Information State contains a number of Information State Variables • IS, the Information State ”proper” • Interface Variables • used for communication between modules • Resource Variables • used for hooking up resources to the TIS, thus making them accessible from to modules • use prespecified or new datatypes

  21. sample infostate type declaration infostate_variable_of_type( is, IS ) :- IS = record( [ private : Private, shared : Shared ] ), Shared = record( [ com : set( proposition ), qud : stack( question ), lm : set( move ) ] ), Private = record( [ agenda: stack( action ), plan : stackset( action ), bel : set( proposition ), tmp : Shared ] ) ] ).

  22. resulting infostate type AGENDA : stack( Action ) PLAN : stackset( Action ) PRIVATE : BEL : set( Prop ) TMP : (same type as SHARED) COM : set( Prop ) QUD : stack( Question ) SHARED : LM: set( Move )

  23. Sample interface variable type declarations interface_variable_of_type( input, string ). interface_variable_of_type( output, string ). interface_variable_of_type( latest_speaker, speaker ). interface_variable_of_type( latest_moves, set(move) ). interface_variable_of_type( next_moves, set(move) ).

  24. Specifying a set of moves • amounts to specifying objects of type move (a reserved type) • there may be type constraints on the arguments of moves • preconditions and effects of moves • formalised in update rules, not in the move definition itself • a move may have different effects on the IS depending e.g. on who performed it

  25. sample move specifications % Social of_type( quit, move ). of_type( greet, move ). of_type( thank, move ) . % Q&A of_type( ask(Q), move ) <- of_type( Q, question ). of_type(inform(P), move ) <- of_type( P, proposition). of_type( answer(R), move ) <- of_type( R, proposition) or of_type( R, ellipsis ).

  26. Writing rules • rule = conditions + operations • if the rule is applied to the IS and its conditions are true, the operations will be applied to the IS • conditions may bind variables with scope over the rule (prolog variables, with unification and backtracking)

  27. A sample rule rule( integrateUsrAnswer, [ $/shared/lu/speaker = usr, assoc( $/shared/lu/moves, answer(R), false ), fst( $/shared/qud, Q ), $domain : relevant_answer( Q, R ), $domain : reduce(Q, R, P) ], [ set_assoc( /shared/lu/moves, answer(R),true), pop( /shared/qud ), add( /shared/com, P ) ] ).

  28. A sample rule (old syntax) • rule( integrateUsrAnswer, [ • val#rec( shared^lu^speaker, usr ), • assoc#rec( shared^lu^moves, answer( R ), false ), • fst#rec( shared^qud, Q ), • domain :: relevant_answer( Q, R ), • domain :: reduce(Q, R, P) • ], [ • set_assoc#rec( shared^lu^moves, answer(R),true), • pop#rec( shared^qud ), • add#rec( shared^com, P ) ] ).

  29. Writing rules • available conditions and operations depend on the infostate type • the infostate is declared to be of a certain (usually complex) type • datatype definitions provide • relations:Rel(Arg1, …, ArgN) • functions:Fun(Arg1, …, ArgN,Result) • operations: Op(ObjIn,Arg1, …, ArgN,ObjOut) • New datatypes may be added

  30. Writing rules: locations in TIS • objects may be specified by giving a path to a locationin the infostate; • paths are specified using selectors, which are similar to functions • $Fun2($Fun1) ~ $Sel1/Sel2 • $fst($/shared/qud) ~ $/shared/qud/fst • ”$” evaluates a path and gives the object at the location specified • example: • is/shared/com is a path, pointing to a location in the TIS • $is/shared/com is the object in that location • theiscan be left out, giving$/shared/com

  31. Writing rules: conditions (1) • conditions do not change the information state • if a condition fails, backtracking ensues • condition syntax (incomplete) • Rel(Arg1, … , ArgN), e.g. • fst($/shared/qud,Q) • Arg1:Rel(Arg2,…,ArgN), e.g. • $/shared/qud:fst(Q) • $domain:relevant_answer(Q,A) • Arg1 = Arg2 • Q = $fst($/shared/qud) • Cond1 and Cond2 • Cond1 or Cond2 • not Cond1 • forall(Cond1, Cond2) • (Argis object or prolog variable)

  32. Writing rules: conditions (2) • quantification, binding and backtracking • if an instantiation a of a variable V in a condition C is found that makes condition C true, V is bound to a • backtracking occurs until a successful instantiation of all variables in the list of conditions has been found • example list of conditions fst($/shared/qud,Q), in($/shared/com,P), $domain:relevant_answer(P,Q) • Explicit quantification Q.P. fst($/shared/qud,Q) & in($/shared/com,P) & $domain:relevant_answer(P,Q)

  33. Writing rules: operations • operations change the information state • if an operation fails, an error is reported • variable bindings survive from conditions to operations • operation syntax (incomplete) • Op(Path,Arg1,…,ArgN) • push(/shared/qud, Q) • Path : Op(Arg1, … ,ArgN) • /shared/qud : push(Q) • Store := Fun(Obj,Arg1,…,ArgN) • /private/tmp/qud := push($/shared/qud,Q)

  34. Specifying update algorithms • uses rule classes • constructs include • Rule • RuleClass • if Cond then S else T • repeat R until C • repeat R • try R • R orelse S • test C • SubAlgorithm

  35. Sample update algorithm grounding, if $latest_speaker == sys then try integrate, try database, repeat downdate_agenda, store else repeat integrate orelse accommodate orelse find_plan orelse if (empty#rec( private^agenda ) then manage_plan else downdate_agenda repeat downdate_agenda if empty($/private/agenda)) then repeat manage_plan repeat refill_agenda repeat store_nim try downdate_qud

  36. Specifying serial control algorithms • serial constructs include • Module{:Algorithm} • if Cond then S else T • repeat R until C • repeat R • try R • R orelse S • test C • SubAlgorithm

  37. Specifying concurrent control algorithms • Agent1 | Agent2 | … | AgentN • whereAgenti is • AgentName : { • import Module1 , • … • import Modulep , • Trigger1 => SerialAlgoritm1 , • … • Triggerm => SerialAlgoritmm } • triggers: • condition(C) (C is a subset of the full condition set) • init • new_data(Stream)

  38. Sample control algorithm (1) repeat ( [ select, generate, output, update, test( $program_state == run ), input, interpret, update ] )

  39. Sample control algorithm (2) input: { init => input:display_prompt, new_data(user_input) => input } | interpretation: { import interpret, condition(is_set(input)) => [ interpret, print_state ] } | dme: { import update, import select, init => [ select ], condition(not empty(latest_moves)) => [ update, if val(latest_speaker,usr) then select else [] ] } | generation: { condition(is_set(next_moves)) => generate } | output: { condition(is_set(output)) => output } )).

  40. From DME to dialogue system Build or select from existing components: • Modules, e.g. • input • interpretation • generation • output • Still domain independent • the choice of modules determines e.g. the format of the grammar and lexicon

  41. Domain-specific system Build or select from existing components: • Resources, e.g. • domain (device/database) interface • dialog-related domain knowledge, e.g. plan libraries etc. • grammars, lexicons

  42. Building resources • Resource • the resource itself; exports a set of predicates • Resource interface • defines the resource as a datatype T, i.e. in terms of relations, functions and operations • Resource interface variable • a TIS variable whose value is an object of the type T • By changing the value of the variable, resources can be switched dynamically • change laguage • change domain

  43. sample resource variable type declarations (incl. resource interface) resource_type( lexiconT ). resource_variable_of_type( lexicon, lexiconT ). of_type( lexicon_travel_english, lexiconT ). of_type( lexicon_autoroute_english, lexiconT ). of_type( lexicon_travel_svenska, lexiconT ). of_type( lexicon_cellphone_svenska, lexiconT ). resource_condition(lexiconT,input_form(Phrase,Move),Lexicon) :- Lexicon : input_form( Phrase, Move ). resource_condition(lexiconT,output_form(Phrase,Move),Lexicon):- Lexicon : output_form( Phrase, Move ).

  44. TrindiKit Features • Explicit information state datastructure makes systems more transparent • Update rules provide an intuitive way of formalising theories in a way which can be used by a system • Domain knowledge encoded in resources; • the rest of the system is domain independent • resources can be switched dynamically • Modular architecture promotes reuse

  45. Features, cont’d • Allows both serial and asynchronous systems • Interfaces to OAA • Generic WWW interface • Runs on UNIX, Windows, Linux • Needs SICStus Prolog • Version 2 .0 is available, next version coming soon (SIRIDUS) • www.ling.gu.se/projects/trindi/trindikit • Larsson & Traum: NLE Special Issue on Best Practice in Dialogue Systems Design, 2000

  46. Systems developed using TrindiKit • GoDiS – information state based on Questions Under Discussion (Larsson et al 2000) • MIDAS – DRS information state, first-order reasoning (Bos & Gabsdil, 2000) • EDIS – information state based on PTT (Matheson et al 2000) • SRI Autoroute – information state based on Conversational Game Theory (Lewin 2000); robust interpretation (Milward 2000)

  47. Post-TRINDI applications • SIRIDUS project (EU 2000-) • command and negotiative dialogues • Spanish, Swedish • GoDiS, SRI system • D’Homme (EU 2001) • Dialogues in the Home Environment • GoDiS, SRI system • Instruction Based Learning for mobile robots (U Edinburgh) • MIDAS • Tutoring Electricity (U Edinburgh) • EDIS, Beetle

  48. ???selectors • selects an object (Obj) embedded in another object (Arg) • selector(Sel,Arg,Obj,ArgWithHole,Hole). • e.g. selector(fst,stack([E|S]),E,stack([H|S]),H). • Every selector corresponds to a function function(Sel,[Arg],Object).

  49. Datatype definitions • relations • relations between objects; true or false • format: relation(Rel,Args). • Example • definition: relation(fst,[stack([E|S]),E]). • condition: fst($/shared/qud,Q)

More Related