1 / 15

Engaging Learning Groups using Social Interaction Strategies

Engaging Learning Groups using Social Interaction Strategies. Rohit Kumar, Carolyn P. Rosé Language Technologies Institute Carnegie Mellon University. Background. Conversational Agents Interactively supports Users One user in each interactive session Variety of Tasks Information Access

ofira
Download Presentation

Engaging Learning Groups using Social Interaction Strategies

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Engaging Learning Groups using Social Interaction Strategies Rohit Kumar, Carolyn P. Rosé Language Technologies Institute Carnegie Mellon University

  2. Background • Conversational Agents • Interactively supports Users • One user in each interactive session • Variety of Tasks • Information Access • Education • Therapy • Entertainment

  3. Signal in the Noise < Outline • CAs in Multi-Party Interactive Situations • Social Interaction Strategies • Design • Implementation < Technical • Basilica Architecture • Evaluation < Scientific • Design • Metrics • Results

  4. Motivation • Towards Conversational Agents for Multi-Party Interactive Situations • Challenges • Technical: Agent Implementation • Scientific: Agent Behavior / Interaction Design • Group Interaction • Agents can help Kumar et. al., 2007 • Groups ignore/abuse Agents • Lack Social Communication Skills

  5. Small Group Communication • Two Fundamental processes operate in group interactionBales, 1950 • Instrumental > Task-related vs. • Expressive > Social-emotional • Need for an Equilibrium • Interaction Process Analysis (IPA) • Unit of Analysis: Utterance / Turn • 12 Interaction Categories Shows Solidarity +ve Shows Tension Release Agrees Expressive Gives Suggestion Instrumental Gives Opinion Gives Orientation Asks for Orientation Asks for Opinion Asks for Suggestion Disagrees Shows Tension -ve Shows Antagonism

  6. Social Interaction Strategies • Eleven Social Interaction Strategies developed • Based on Three Positive Social-Emotional Interaction Categories

  7. Implementation: Using Basilica • Event-driven Architecture Kumar & Rosé, NAACL 2009 • Rich-representational Power • Network of Behavioral Components • Components are programmable • Using High-level Languages (Java) • Flexibility to address Complex Interaction Dynamics • Four Agents developed using this Architecture • Multi-Party Turn Taking • Variety of Agent roles (Tutor, Peer, Mediator, …) • Users with different roles in the same interactive session • Reusability • Components • Agents in multiple Environments

  8. Implementation: WrenchTalk Tutor ConcertChat Server • Plug-In existing Components • TutoringManager / TutoringActor > Integrating TuTalk • IntroductionsManager/IntroductionsActor > State-based DM • PlanExecutor > Plan based DM • NLU > Classifiers / Annotators ConcertChatActor ConcertChatListener MessageFilter PresenceFilter DiscourseMemory AnnotationFilter OutputCoordinator SocialController ActivityDetector ProgressDetector PlanExecutor RequestDetector T.TakingCoordinator IntroductionsManager PromptingManager TutoringManager TutoringActor IntroductionsActor PromptingActor

  9. Implementation: WrenchTalk Tutor ConcertChat Server • Two primary controllers • PlanExecutor > Executes Task-related steps • SocialController > Triggers Social Behavior ConcertChatActor ConcertChatListener MessageFilter PresenceFilter DiscourseMemory AnnotationFilter OutputCoordinator SocialController ActivityDetector ProgressDetector PlanExecutor RequestDetector T.TakingCoordinator IntroductionsManager PromptingManager TutoringManager TutoringActor IntroductionsActor PromptingActor

  10. 1d. 2b. 2b. Implementation: Social Controller • Social Behavior Triggering • Hand Crafted Rules • Features • Last executed plan step • Annotations of student turns • Dictionary Lookup • Activity Levels • Groups & Individual • Strategy: 1e. (Encourage) • Social Ratio • Ratio of Social Turnsto Task-related turns • Threshold: 20%

  11. Research Context:Wrench Lab • Freshmen Mechanical Engineering Course • Underlying concepts • Force, Moment, Stress, Yield Strength, … • Three part lab • Paperless Engineering CADCAACAM • CAA includes Collaborative Wrench Design Activity • Teams of 3 – 4 students design a Wrench • Fall 2009 Wrench Lab • 98 Students

  12. Experiment: Evaluating Effectiveness • Effectiveness of Social Behavior Vs. • Gold Standard > Human performance • Baseline > No Social Behavior • Experimental Design • Three Conditions • Between Subjects • Groups & Tutors interact using ConcertChat

  13. Experiment: Survey • 7 point Likert-scale • Items Burke, 1967

  14. Experiment: Results Ratings About Tutor • Agents with Social Behavior (Both Social & Human) rated better than No Social Behavior • Automated Social Tutors significantly friendlier than No Social Behavior (Q2) Ratings About Activity> No Significant Difference w.r.t Baseline

  15. Experiment: Analysis & Conclusion • Significant benefits of employing Social Interaction Strategies • On Perceptual Metrics • Also, on Performance Metrics • Current Implementation of Social Tutors not as good as Human tutors • Why? Possibly… • Right Amount of Social Behavior Human tutors displayed significantly more Social Behavior Human = 22.17 turns Social = 16.83 turns • Better Triggering Model implicit in Human Tutor Recent Work@ ITS 2010 Publication / Slides > http://basilica.rohitkumar.net/wiki/

More Related