A system for generating teaching initiatives in a computer-aided language learning dialogue

1. A system for generating teaching initiatives in a computer-aided language learning dialogue Alistair Knott University of Otago New Zealand Nanda Slabbers University of Twente Netherlands June 9, 2005 Nancy

2. Overview • Introduction • Existing work • Goals • Formal goals • Substantive goal • Te Kaitito • Authoring mode • Student mode • Conclusion • Questions

3. Content Selection Sentence Planning Syntactic Realisation Introduction • A dialogue system has to create two different kinds of utterances: • Responses: not much content selection • Initiatives: content selection is very important • It is common to analysethe task of naturallanguage generationas a pipeline:

4. Existing Work • Existing systems which focus on generating initiatives: • Used to accomplish a certain task. • The difference is that a language learning dialogue doesn’t have to look like a tutorial dialogue. • Existing systems which focus on language learning: • Most initiatives come from the student. • In such systems it is hard to learn by adapting utterances made by the teacher.

5. Goals of the Generated Initiatives • Content selection is the key process in generating initiatives. • The process of content selection is typically defined in relation to two goals: • Formal goal: creating a coherent text. • Substantive goal: achieving a certain effect on the hearer (in our case teaching the student a number of syntactic rules).

6. Formal Goal - Possible Dialogue Acts • Dialogue coherence cannot be defined at the level of dialogue acts. • Therefore the system will only generate initiatives of the following dialogue acts: • Assertions • Questions: • Genuine questions • Teaching questions

7. Formal Goal – Topic Continuity • We prefer initiatives which maintain the current topic of the dialogue • Topic of utterance: set of individuals and predicates it introduces • Topic of dialogue: combination of topics of last utterances • Topic continuity: overlap between topic of dialogue and topic of possible initiative

8. Formal Goal – Mix of Dialogue Acts • First we want to decide which dialogue act will be taken: • Normally we decide this based on the topic continuity criterion. • If there are initiatives which score equally in the topic continuity criterion, we prefer those which belong to a dialogue act which has not been recently used.

9. Formal Goal – Other Criteria • After we have decided which dialogue act will be taken, we have to compare the possible initiatives belonging to that dialogue act. • To achieve this we introduce three dialogue act specific criteria: • Assertions: strategy • Genuine questions: order of creation • Teaching questions: complexity

10. Substantive Goal – Syntax Criterion • Every utterance is parsed and syntactic representations are created. • Using these representations a set of target syntactic rules can be created for each lesson. • Each lesson teaches a different set of rules, beginning with simple rules and progressing to more complex rules. • Using these rules initiatives which involve target rules can be preferred to other initiatives.

11. Formal and Substantive Goals - Initiative Generation Algorithm • Our algorithm for generating initiatives has four steps: • Identify a set of possible topics. • Generate a set of candidate initiatives. • Create a shortlist of initiatives by scoring and ranking them based on the criteria. • Assign a syntactic score and choose the initiative with the highest total score.

12. Te Kaitito • Te Kaitito is a bilingual dialogue system meant to teach the Maori language, but we will demonstrate the English version. • The system supports two modes: • authoring mode: to author new lessons. • student mode: to practice the lesson and learn the target rules.

13. Authoring Mode • In authoring mode the user is assumed to be a teacher. • The teacher can author a character by: • Telling facts about itself: “You are John” • Entering question generation rules: “How is a person?” • At the end the facts and questions are stored in two separate knowledge bases. • The set of target syntactic rules is created automatically.

14. Authoring Mode – Example Dialogue A How is a person? TK Okay A What is a person’s name? TK Okay A Where is a person from? TK Okay A Your name is John TK Okay A You are from Dunedin TK Okay A You are fine TK Okay

15. Student Mode – Start of the Lesson • Start of the dialogue: • Empty common ground • Load knowledge bases and set of rules • During the dialogue: • Dialogue consists of pairs of forward-looking and backward-looking dialogue acts. • At transition relevant points the initiative is passed to the student, but he or she can concede the initiative to the system.

16. Student Mode – Example Dialogue S <enter> TK Where are you from? S I am from Dunedin TK Okay S <enter> TK I am also from Dunedin S <enter> TK Let’s see if you remember: Who am I? S You are John TK Correct, well done! S <enter> TK How are you? S I am fine TK Okay S How are you? TK I am fine S <enter> TK What is your name? S My name is Ann TK Okay S Who are you TK I am John

17. Student mode – End of the Lesson • The previous dialogue continues until the system has evidence that all target rules have been assimilated by the student: • Each target syntactic rule has an assimilation score that is updated after an utterance. • When the score reaches a certain boundary value the associated rule has been assimilated. • When all rules reach the boundary value, the lesson ends successfully. • When the student does not learn the rule, even when a correct application of the rule is shown, the lesson may end unsuccessfully.

18. Conclusion • Te Kaitito’s teaching dialogues are useful to practice conversational skills. • The system only has a simple grammar, but since novice students also have a limited grammar, this is satisfactory. • Adding initiatives is also beneficial: • When the student is lost, the dialogue can be continued by simply pressing the Enter key. • The initiatives provide models of well-formed sentences, so the student can modify these and try them out.

19. Questions Are there any questions?