VoiceXML: Forms, Menus, Grammars, Form Interpretation Algorithm

VoiceXML: Forms, Menus, Grammars, Form Interpretation Algorithm

Acknowledgements • Prof. Mctear, Natural Language Processing, http://www.infj.ulst.ac.uk/nlp/index.html, University of Ulster.

Overview • Forms • Form items • Form attributes • Fields • Field Attributes • Prompts • Menus • Form Interpretation Algorithm

Forms • Interpreting a form item (e.g. field) • Select and play one or more prompts • Collect user input • Throw events (e.g. noinput) • Interpret <filled> actions (procedural logic that executes when input item variables are assigned) • Form attributes • id name of the form e.g. <form id=“mainmenu”> reference to another form <goto next=#validate”> (validate is the name of a form) • scope scope of the form’s grammars e.g. <scope = “dialog”>

Form Items Input Items <field> input using speech or DTMF <record> records user’s input as audio clip <transfer> transfers user to another telephone number <object> invokes a platform-specific object e.g. voice authentication program <subdialog> invokes another dialog Control Items <block> procedural statements for prompting and computation <initial> controls initial interaction in a mixed initiative form

C: Say students, courses, or reports H: Students. C: What is the student’s name? H: John Scott. C: What is the student’s id? H: 654556. C: What information do you require? H: Address The system prompts the user for input in order to fill in the required fields. The user responds to the system’s prompts. The system accepts the responses if they match the grammars for each field. Example: A form for getting student information

Fields • The form for the previous example consists of 4 fields: choice, student_name, student_id, info • Each field has a prompt and a grammar • Each field declares a variable which usually has the value of ‘undefined’ when the field is first interpreted • Field attributes name e.g. <field name=“choice”> expr the initial value of the form item variable cond a conditional expression e.g. to activate a field only when another field has been filled type e.g. <field name=“student_id” type=“digits”>

Example studentsystem1a.vxml Headers (see next slide) <form id="mainmenu" scope="dialog"> <field name="choice" modal="false"> <prompt version="1.0"> Welcome … To begin say one of the following: students, courses, reports </prompt> <grammar type="application/srgs+xml" root="choice" version="1.0" mode="voice"> <rule id="choice" scope="public"> <one-of> <item> students </item> <item> courses </item> <item> reports </item> </one-of> </rule> </grammar> </field> </form> </vxml>

Headers IBM Voice Toolkit: <?xml version="1.0" encoding="iso-8859-1"?> <!DOCTYPE vxml PUBLIC "-//W3C//DTD VOICEXML 2.0//EN" "vxml20-1115.dtd"> <vxml version="2.0" xml:lang="en-GB" xmlns="http://www.w3.org/2001/vxml"> <meta name="GENERATOR" content="Voice Toolkit for WebSphere Studio" /> Bevocal: <?xml version="1.0" ?> <!DOCTYPE vxml PUBLIC "-//BeVocal Inc//VoiceXML 2.0//EN" "http://cafe.bevocal.com/libraries/dtd/vxml2-0-bevocal.dtd"> <vxml version="2.0" xmlns="http://www.w3.org/2001/vxml“ xml:lang="en-US">

Tutorial Exercises • Exercise 1: Load and run the file ‘studentsystem1a.vxml’. The system will prompt you to say one of: students, courses, details. • Exercise 2: Create a file called ‘studentdetails.vxml’. The file should have fields to elicit a student's name and the name of a course. You will need to construct grammars in each field containing a few names of students and courses.

<prompt> (1) • Bargein • If an implementation platform supports bargein, the application author can specify whether a user can interrupt, or "bargein" on, a prompt using speech or DTMF input • To disable “bargein”:<prompt bargein = “false> • Timeout • Specifies the interval of silence between prompt and user input e.g.<prompt timeout = “30s”> • Default timeout value in Voice Toolkit is 7s. • Timeout can be specified for individual prompts or set as a property at a higher level e.g. in the root document

<prompt> (2) • Count • Used for prompts that change with each attempt to elicit input e.g. to provide more help<prompt count=“1”> what is the student’s id? </prompt> <prompt count=“2”> please say the student’s id. It consists of 6 digits </prompt> • Conditional prompt • Specifies that a prompt is only to be spoken if the condition is true <prompt cond=“now < 12"> Good morning </prompt> <prompt cond=“now >= 19"> Good evening </prompt>

Menus • A <menu> is a shorthand version of a form with a single field that prompts the user to make a choice and then transitions to another place (e.g. another dialog) based on that choice • Menu attributes id e.g. <menu id=“mainmenu”> scope grammar scope of the menu dtmf when set to true, first nine choices are given the implicit values ‘1’, ‘2’, etc (unless explicitly specified) accept default “exact” – defines the exact phrase to be recognised “approximate” – allows approximate recognition

C: Welcome to the Student System main menu Please choose one of the following options: You can say student details, course details, or records H: Student details. This example uses a menu to present a choice to the user. The user selects one of the options and the application moves to the next dialogue state. Menu example

examplemenu1.vxml <?xml version="1.0" encoding="iso-8859-1"?> <!DOCTYPE vxml PUBLIC "-//W3C//DTD VOICEXML 2.0//EN" "vxml20-1115.dtd"> <vxml xmlns="http://www.w3.org/2001/vxml" xml:lang="en-GB" version="2.0"> <menu id="mainmenu" scope="dialog" accept="exact" dtmf="false"> <prompt version="1.0"> To begin say one of the following: </prompt> <choice next=“studentdetails.vxml"> students </choice> <choice next=“coursedetails.vxml"> courses </choice>  </menu> </vxml>

Where to go to Grammar for this choice Choice • Can specify a speech grammar – may be automatically generated • Can specify a DTMF grammar • Can use contents to form the <enumerate> prompt string • Specifies an event to be thrown or URI to go to when the choice is selected <choice next=“reports.vxml"> reports </choice>

DTMF in Menus • DTMF only <menu> <property name=“selected” value=“dtmf” /> <prompt> For reports press 1, etc </prompt> <choice dtmf=“1” next=“reports.vxml” /> </menu> • DTMF or speech <menu> <prompt> For reports press 1 or say ‘reports’ </prompt> <choice dtmf = "1" next=“reports.vxml">reports </choice> </menu>

comment <filled> element Executable code when fields are filled Conditionals Requires documents to be specified State transitions and branching <form> <!– ‘choice’ field as in ‘studentsystem1a.vxml --> <filled><if cond="choice == 'students'"><goto next="studentdetails.vxml"/><elseif cond="choice == 'courses'"/><goto next="coursedetails.vxml"/><else/> <goto next="reports.vxml"/></if></filled> </form>

Tutorial Exercises • Exercise 3: Save the file ‘studentsystem1a.vxml as studentsystem1b.vxml. Insert the additional code after the end of the field </field> and before the end of the form </form>. • Note: To make this example work, it will be necessary to create dummy files for the options for the transitions specified in the <filled> element. • Exercise 4: Load and run the file ‘examplemenu1.vxml’

Form Interpretation Algorithm • Verification – verifying that the recognition result corresponds to what the user said • boolean builtin grammar • Variables • referencing variables • declaring variables • The Form Interpretation Algorithm (FIA)

viewdetails.vxml <form id="student_details" scope="dialog"> <!– field to collect student_name --> <field name="confirm" type="boolean" modal="false"> <prompt version="1.0"> you want details on <value expr="student_name"/> </prompt> <filled> <if cond="confirm"> <prompt version="1.0"> looking up details on <value expr="student_name"/> </prompt> <else/> let's try again <clear namelist="student_name confirm"/> </if> </filled> </field> </form>

Referencing a variable you want details on <value expr="student_name"/> • “student_name“ is the name of a field • the expression retrieves the value of the field, i.e. the value that was recognised by the system • this value is not necessarily the value that the user spoke e.g. if the input was misrecognised

More on Variables <var> - declare a variable and possibly assign an initial value to it <block><var name="status"/> <var name="amount" expr = "13"/>  </block> <assign> – assign a value to an existing variable <block> <var name="status"/> <assign name = "status" expr = "completed"/> <var name="amount" expr = "13"/> <assign name = "amount" expr = "amount + 1"/> </block>

Variables: More examples • <var name = "revised_balance" expr = "initial_balance + use_fee"/> • <var name = "days_of_the_week" expr="new Array()"/> • <var name = "days_of_the_weekend" expr = "new Array ('Saturday', 'Sunday') " />

Tutorial Exercises • Exercise 4: Load and run the file ‘viewdetails.vxml’. • In response to the prompt 'Please say the student name', you can say one of: John, David, Rosemary, Jennifer. • In response to the prompt asking for confirmation, you can say either 'yes' or 'no'. Try both options. • Exercise 5: Adding verification to studentdetails.vxml • Using the file viewdetails.vxml as a basis, save the file ‘studentdetails.vxml’ as ‘studentdetails_confirm.vxml’. Amend the file so that the system confirms the values elicited (studentname, coursename). If the user confirms, then the application terminates, otherwise it goes back to elicit the values again.

Form Interpretation Algorithm • The form interpretation algorithm (FIA) determines the order of execution in a VoiceXML form or menu • The FIA cycles through all of the fields in the form, asking the caller to supply values for each unfilled field. • The FIA provides the default sequence of processing fields within a form; and thus hides the procedural aspects of form processing the from the dialog designer. • Sometimes the fields of a form are “reset.” In this case, the FIA will select the reset field during the next cycle so all fields are filled eventually.

Amount Source Confirm Amount Source Destination Destination Date Date Form Interpretation Algorithm (FIA) Each field has a prompt and a grammar (for user input) The field variable is initially set to ‘undefined’ When a field has been completed the field variable is set to ‘true’. The Form Interpretation Algorithm skips fields that are ‘true’ If a field value is disconfirmed the variable is re-set to ‘undefined’

Confirm Amount Source Destination Date FIA (continued) Amount and Destination were disconfirmed Their values are re-set to ‘undefined’ The FIA visits the ‘Amount’ and ‘Destination’ fields again until their values are set to ‘true’

FIA: More detail • Each variable (form-level <var> elements and form item variables) is initialised to ‘undefined’ • The first form item whose guard condition is false (undefined) is chosen to be visited. • If no guard condition is false, then the FIA does an implicit <exit> operation. • <block> element • form item variable set to true (cannot be visited again) • content evaluated • no input is collected

undefined undefined undefined undefined undefined undefined true true true true undefined true FIA in operation Name disconfirmed Field Name Course Level Date Before Mid

More detail Note that the FIA selects one item on each iteration e.g. given the 4 fields above without any conditions on the fields: Iteration By-passed item Selected item 1 None name 2 name course 3 name course level 4 name course level date 5 course level date name 6 name course level date none: FIA exits

FIA example <filled> <if cond = "confirm == 'no'" ><prompt> Let's try again </prompt> <clear namelist = “student_name confirm"/>  <else/> <prompt> Let's continue</prompt> </if> </filled> <field name="confirm" > <grammar type="application/srgs+xml" root = "yes_no“ version=“1.0”> <rule id = "yes_no" scope = "public"> <one-of> <item>yes</item> <item>no</item> </one-of> </rule> </grammar> <prompt> You want details on <value expr=“student_name"/> </prompt> </field>

The <filled> element • Defines how to process input-form-item variables after they have values • e.g. to check that the input is correct or to verify it with the user (example: arrival time not earlier than departure time) • If the checking involves only the value of a single field, the <filled> element is positioned within the <field> element. • If the check involves more than a single input-form-item variable, the <filled> element is placed within the <form> element.

Example: Validating the input <form id=“get_details"> <!– The system has collected values for ‘source’ and ‘destination’ and checks that these are different --> <filled> <if cond = “source == destination"> <prompt> the destination and source accounts must be different </prompt><clear namelist = “source destination"/></if> </filled> </form>

<if> <if cond = "confirm == 'no'" > <prompt> let's try again </prompt> <clear namelist = “student_name confirm"/> <else/> <prompt> Let's continue</prompt> </if> • “confirm” is the name of a field • The condition checks if the value of “confirm” is ‘no’ • <clear> re-sets variables to ‘undefined’ • namelist specifies which variables are to be re-set to ‘undefined’, otherwise all variables within the current form are re-set

Tutorial Exercise Exercise 5 1. Save the file ‘viewdetails.vxml’ as ‘newviewdetails.vxml’ 2. Delete the line <clear namelist = "student_name confirm" /> and see what happens. 3. Insert the line but delete the value ‘confirm’. 4. Try again, this time with ‘confirm’ included and ‘student_name’ deleted.

Exercise 6: Asking which item to confirm • Scenario: • User has supplied values for studentname and coursename • System verifies and user says ‘no’ • System asks: which do you want to change ‘the student name or the course name?’ <filled> <if cond="change == ‘student name'"> <clear namelist=“student_name confirm"/> <elseif cond="change == ‘course name'" /> <!– etc. --> </if> </filled> <field name="change"> <prompt> which item do you wish to change: the student name or the course name? </prompt> <!– insert grammar for each of the 2 parameters -->

<field name=“confirm” > <prompt> so you want to details on <value expr=“student_name"/> taking <value expr=“course_name"/> </prompt> <!– grammar for ‘yes/no’ --> <filled> <if cond="confirm==‘no’"> <goto nextitem = "change" /> <else /> <assign name="change" expr="true" /> </if> </filled> </field> <field name="change"> <prompt> which item do you wish to change: the student name or the course name? </prompt> <!– insert grammar for each of the 2 parameters --> The line <assign name=“change” expr = “true” /> makes the value of “change” be ‘true’ – this ensures that the field will not be visited by the FIA.

VoiceXML: Forms, Menus, Grammars, Form Interpretation Algorithm