1 / 26

Socially Capable Conversational Tutors can be effective in Collaborative Learning Situations

Socially Capable Conversational Tutors can be effective in Collaborative Learning Situations. Socially Capable Conversational Tutors can be effective in Collaborative Learning Situations. Rohit Kumar, Hua Ai, Jack L. Beuth, Carolyn P. Rosé. Bridges. CSCW. Multi-Party Interaction.

yasuo
Download Presentation

Socially Capable Conversational Tutors can be effective in Collaborative Learning Situations

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Socially CapableConversational Tutorscan be effective inCollaborative Learning Situations Socially CapableConversational Tutorscan be effective inCollaborative Learning Situations Rohit Kumar, Hua Ai, Jack L. Beuth, Carolyn P. Rosé

  2. Bridges CSCW Multi-PartyInteraction CollaborativeLearning TutorialDialog CMC DialogSystems Our Work CommunicationStudies Small GroupCommunication ConversationalAgents 2

  3. Outline • Problem • Solution • Evaluation • Results • Summary 3

  4. Communication Skills of Tutors Unlike individual learners, Teams of students ignore / abuse automated tutorsKumar et. al. 2007a Agents lack Social Communication Skills Need to be better communicators > Problem 4

  5. Solution

  6. Small Group Communication Two Fundamental Processes In Group Interaction Bales, 1950 (Seminal Work) 3858 Citations Instrumental (task-related) vs.Expressive (social-emotional) Need for Equilibrium Interaction Process Analysis (IPA) > Solution 6

  7. Related Work > Solution > Related Work 7

  8. Social Interaction Strategies > Solution > Design 8

  9. ConcertChatActor ConcertChatListener MessageFilter PresenceFilter DiscourseMemory AnnotationFilter OutputCoordinator ActivityDetector SocialController ProgressDetector PlanExecutor RequestDetector TurnTakingCoordinator IntroductionsManager PromptingManager TutoringManager TutoringActor IntroductionsActor PromptingActor Tutors with Social Behavior • Implemented using Basilica > Solution > Implementation 9

  10. Triggering Social Behavior 1d. 2b. 2b. • Hand Crafted Rules • Four Features • Last executed plan step • Annotations of student turns • Activity Levels • Groups & Individual • Social Ratio • Ratio of Social Turns to Task-related turns • Threshold: 20% > Solution > Implementation 10

  11. Realizations of Social Behaviors Strategy 1d.Complement > Strategy 2b. >Be Cheerful> > Solution > Examples > 1 of 3 11

  12. Realizations of Social Behaviors Strategy 1a. >Do Introductions Strategy 1e.Encourage > > Solution > Examples > 2 of 3 12

  13. Realizations of Social Behaviors Strategy 3a. >Show Attention > Solution > Examples > 3 of 3 13

  14. Evaluation

  15. Collaborative Wrench Design Lab Collaborative Design Labs Mechanical Engineering Courses Freshmen: Wrench Design Teams of 3-4 students Underlying Concepts Force, Moment, Stress, Strength, … > Evaluation > Situation 15

  16. Wrench Design Task • Design a Wrench • Ease of Use • Cost • Safety Worksheet  > Evaluation > Task

  17. Methodology Controlled Experiments Between subjects Conducted over multiple sessions 35-40 minutes per session Different students in each session Students randomly assigned to teams on the spot Team mates not seated next to each other Communicate using ConcertChat Teams randomly assigned to conditions Nearly even distribution of conditions in each session > Evaluation > Methodology 17

  18. Collaborative Wrench Design Lab > Evaluation > Situation 18

  19. Experimental Design Three Conditions Manipulation: Display of Social Behavior TASK Baseline Condition No Social Behavior SOCIAL Experimental Condition Automated Social Behavior HUMAN Gold Standard Condition Human Social Behavior Same Instructional Behavior in all Conditions > Evaluation > Conditions 19

  20. Experimental Design Human Social Behavior • Picking Prompts • Allowed to change the Prompts • - Would you like to contribute to the discussion Tim? - Any inputs Tim - Tim …any thoughts you'd like to contribute? - Tim do you want to comment on that? - what do you think Tim • No “Social Ratio” constraint > Evaluation > Conditions 20

  21. Metrics Performance: Learning Outcomes Pre & Post Tests 8 Multiple choice questions (MCQs) 3 Short Essay Questions (SEQs) Perception: Survey Burke, 1967 7-point Likert-scale 1-Strongly disagree, 7-Strongly Agree 9 items Ratings about Tutor (Agent): Likeable, Friendly, … Learning Task Ratings: Satisfaction, … > Evaluation > Metrics 21

  22. Survey > Evaluation > Metrics > Survey 22

  23. Results

  24. Learning Outcomes Significant effect of Condition F(2, 93) = 10.56, p < 0.001 Post-hoc Analysis TASK vs. SOCIAL p < 0.01, 0.71σ TASK vs. HUMAN p < 0.001, 0.93σ SOCIAL vs. HUMAN n.s. > Results > Learning Outcomes 24

  25. Survey Results HUMAN & SOCIAL conditions better than TASK conditions HUMAN condition significantly better on (Q1 - Q5, Q8) SOCIAL condition significantly beter on Q2, marginal on Q4, Q5 > Results > Survey 25

  26. Summary Significant benefits of employing Social Interaction Strategies Performance & Perception Social Tutors not as good as Human tutors Right Amount of Social Behavior Significantly more Social Behavior in Human condition Average (Human) = 22.17 Average (Social) = 16.83 Human Social Behavior Triggering Publication / Slides >http://basilica.rohitkumar.net/wiki/

More Related