1 / 17

Diane Litman Computer Science Department Learning Research & Development Center

Spoken dialogue for intelligent tutoring systems: Responding to not only what students say, but how they say it. Diane Litman Computer Science Department Learning Research & Development Center Intelligent Systems Program University of Pittsburgh. Context.

ejune
Download Presentation

Diane Litman Computer Science Department Learning Research & Development Center

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Spoken dialogue for intelligent tutoring systems: Responding to not only what students say, but how they say it Diane Litman Computer Science Department Learning Research & Development Center Intelligent Systems Program University of Pittsburgh

  2. Context Speech and Language Processing for Education Learning Language (reading, writing, speaking) Processing Language Using Language (to teach everything else) Readability Tutors Digital Dialogue Tutors/ Peers Questioning & Answering Scoring CSCL Discourse Coding Lecture Retrieval

  3. Tutorial Dialogue Systems • Why is one-on-one tutoring so effective? “...there is something about discourse and natural language (as opposed to sophisticated pedagogical strategies) that explains the effectiveness of unaccomplished human [tutors].” [Graesser, Person et al. 2001] • Currently only humans use full-fledged natural language dialogue

  4. A Potential Benefit of SpokenDialogue • Speech contains prosodic information, providing new sources of information about the student • Correct student turns, but… • SYS: How does his velocity compare to that of his keys? • Student: his velocity is constant • SYS:What vertical force is always exerted on an object near the surface of the earth? • Student:Gravity

  5. Detecting and Responding to Student States • Opportunity • Adaptive spoken dialogue system technology can improve student learning and other measures of performance • Challenges • What to detect? • How to respond? • Evaluation?

  6. What to Detect? • ITSPOKE (Intelligent Tutoring SPOKEn Dialogue System): student uncertainty & disengagement • Student confusion positively correlates with learning [Craig et al. 2004] • Student disengagement / gaming/ boredom negatively correlate with learning [Aleven et al 2004, Baker et al 2008, Lehman et al 2008] • Relationships vary with type of disengagement and student knowledge [Forbes-Riley and Litman, 2011] (conference website)

  7. How to Detect? • Manual annotation of user states that will trigger system adaptation • Naturally-occurring spoken dialogue data • Prediction via machine learning • Use speech and language processing to automatically extract features from user turns • Use extracted features and annotations to learn a model for predicting user state(s) in new data • Significant reduction of baseline error

  8. Example Features • What a user says • words (speech recognition), stems (morphology) • part-of-speech, syntactic constituents (parsing) • correctness (semantic analysis) • dialogue moves (pragmatics and discourse) • How a user says it • acoustic-prosodic analysis • Challenges • Feature choice, noisy technology, real-time computation, speaker and task dependence

  9. Extracting Pitch Features

  10. Temporal Features • Duration = end time - begin time • Tempo (speaking rate) = #syllables/duration

  11. How to Respond? • Theory-based • Data-driven

  12. Theory-Based Adaptation • Uncertainty represents one type of learning impasse, and is also associated with cognitive disequilibrium • An impasse motivates a student to take an active role in constructing a better understanding of the principle. [VanLehn et al. 2003] • A state of failed expectations causing deliberation aimed at restoring equilibrium. [Craig et al. 2004] • Hypothesis: The system should adapt to uncertainty in the same way it responds to other impasses (e.g, incorrectness)

  13. Data-Driven Adaptation: • Extraction of “dialogue bigrams” from annotated human tutoring corpora • χ2 analysis to identify dependent bigrams • After uncertain, tutorBottoms Out and avoids expansions • After certain, tutor Restates

  14. Empirical Evaluation • Experimentally manipulate tutor responses to student uncertainty and investigate impact on learning [Forbes-Riley and Litman, 2011a,b] • “Wizard of Oz” tutorial dialogue system • dynamically detecting and responding to uncertainty - over and above correctness - significantly improves learning & efficiency • Fully automated system • Similar improvements, but only for a subset of students

  15. Summing Up • Spoken Dialogue Systems for Intelligent Tutoring • Natural language dialogue is a key aspect of human one-on-one tutoring • Using presently available technology, successful conversational computer tutors are now being built • Evidence that more adaptive versions of such systems will further enhance performance

  16. Thank You! • Questions? • Further Information • http://www.cs.pitt.edu/~litman/itspoke.html

  17. Extracting Energy Features

More Related