Joelle Pineau Michael Montemerlo Martha Pollack * Nicholas Roy Sebastian Thrun

1. Probabilistic Control of Human Robot Interaction:Experiments with a Robotic Assistant for Nursing Homes Joelle Pineau Michael Montemerlo Martha Pollack * Nicholas Roy Sebastian Thrun Carnegie Mellon University *University of Michigan

2. Introducing Pearl – A mobile robotic assistant for elderly people and nurses ROLE: Reminding to eat, drink, take meds cameras Providing info (TV, weather) LCD mouth Monitoring Rx adherence & safety Calling for help in emergencies microphone & speakers Supporting communication touchscreen Remote health services handle bars Providing physical assistance carrying tray Management support of ADLs laser Linking caregiver and resources sonars Moving things around mobile base The Nursebot Project

3. The Nursebot project in its early days The Nursebot Project

4. Architecture High-level controller Navigation Cognitive support Communication The Nursebot Project

5. Architecture High-level controller Navigation Cognitive support Communication • Localization and map building • (Burgard et al., 1999) • People detection and tracking • (Montemerlo et al., 2002) The Nursebot Project

6. Architecture High-level controller Navigation Cognitive support Communication • Autominder system • (Pollack et al., 2002) The Nursebot Project

7. Architecture High-level controller Navigation Cognitive support Communication • Speech recognition: Sphinx system • (Ravishankar, 1996) • Speech synthesis: Festival system • (Black et al., 1999) The Nursebot Project

8. The role of the top-level controller • Établir les priorités parmi les objectifs des différents modules • Négocier entre plusieurs objectifs ayant des coûts/gains variés • Négocier entre l’acquisition d’information et la rencontre des objectifs • Passer d’une tâche à l’autre en partageant l’information sensorielle • Planifier malgré la présence d’incertitude High-level controller Navigation Cognitive support Communication • ACTION SELECTION - based on the trade-off between: • - goals from different modules; • - goals with varying costs / rewards; • - reducing uncertainty versus accomplishing goals. The Nursebot Project

9. Speech recognition with Sphinx The Nursebot Project

10. Robot control under uncertainty Belief State P(st=weather-today)=0.5 P(st=appointment-today )=0.5 Speech=“today” State weather-today USER Action={ say-weather, update-appointment, clarify-query} The Nursebot Project

11. Robot control using Partially Observable Markov Decision Processes (POMDPs) Belief state Observations Costs / Rewards P(s1) P(s2) State USER + ENVIRONMENT + WORLD Actions Problem: Which action allows the robot to maximize its reward? The Nursebot Project

12. Methods to solve POMDPs Objective: Find a policy, (b), which maximizes reward. POMDP New methods? Performance AMDP FIB QMDP UMDP MDP O(S2AT) O(S2AO ) O(S2A) O(S2AO) O(S2AB) T Complexity The Nursebot Project

13. New approach: A hierarchy of POMDPs Idea: Exploit domain knowledge to divide one POMDP into many smaller ones. Motivation: Complexity of POMDP solving grows exponentially with # of actions. Assumption: We are given POMDP M = {S,A,,b,T,O,R} and hierarchy H subtask Act abstract action ExamineHealth Navigate Move ClarifyGoal VerifyPulse VerifyMeds primitive action North South East West The Nursebot Project

14. PolCA: Planning with a hierarchy of POMDPs Step 1: Select the action set Navigate AMove = {N,S,E,W} Move ClarifyGoal West South East North ACTIONS North South East West ClarifyGoal VerifyPulse VerifyMeds The Nursebot Project

15. PolCA: Planning with a hierarchy of POMDPs Step 1: Select the action set Step 2: Minimize the state set Navigate AMove = {N,S,E,W} SMove = {X,Y} Move ClarifyGoal West South East North ACTIONS North South East West ClarifyGoal VerifyPulse VerifyMeds STATE FEATURES X-position Y-position X-goal Y-goal HealthStatus The Nursebot Project

16. PolCA: Planning with a hierarchy of POMDPs Step 1: Select the action set Step 2: Minimize the state set Step 3: Choose parameters Navigate AMove = {N,S,E,W} SMove = {X,Y} Move ClarifyGoal West South East North ACTIONS North South East West ClarifyGoal VerifyPulse VerifyMeds STATE FEATURES X-position Y-position X-goal Y-goal HealthStatus PARAMETERS {bh,Th,Oh,Rh} The Nursebot Project

17. PolCA: Planning with a hierarchy of POMDPs Step 1: Select the action set Step 2: Minimize the state set Step 3: Choose parameters Step 4: Plan task h Navigate AMove = {N,S,E,W} SMove = {X,Y} Move ClarifyGoal West South East North ACTIONS North South East West ClarifyGoal VerifyPulse VerifyMeds STATE FEATURES X-position Y-position X-goal Y-goal HealthStatus PARAMETERS {bh,Th,Oh,Rh} PLAN h The Nursebot Project

18. PolCA in the Nursebot domain • Goal: A robot is deployed in a nursing home, where it provides reminders to elderly users and accompanies them to appointments. • Domain: |S|=512, |A|=20, |O|=19 • Hierarchy: The Nursebot Project

19. Sample scenario The Nursebot Project

20. Results for dialogue system POMDP policy MDP policy 0.18 0.1 0.1 The Nursebot Project

21. Summary • We have developed a first prototype robot able to serve as a mobile nursing assistant for elderly people. • The top-level controller uses a hierarchical variant of POMDPs to select actions. • This allows it to acquire necessary information and successfully complete assigned tasks. • Probabilistic techniques have been found to be very useful to flexibly model and track individuals. The Nursebot Project

22. The Nursebot team U. of Pittsburgh - Nursing: Jacqueline Dunbar-Jacobs Sandra Engberg Judith Matthews U. of Pittsburgh - CS: Don Chiarulli Colleen McCarthy U. of Freiburg - CS: Maren Bennewitz Wolfram Burgard Dirk Schulz U. of Michigan - CS: Laura Brown Dirk Colbry Cheryl Orosz Bart Peintner Martha Pollack Sailesh Ramakrishnan Standard Robotics: Greg Baltus CMU - Robotics: Greg Armstrong Michael Montemerlo Joelle Pineau Nicholas Roy Jamie Schulte Sebastian Thrun CMU - HCI/Design: Francine Gemperle Jennifer Goetz Sarah Kiesler Aaron Powers For more details: www.cs.cmu.edu/~nursebot The Nursebot Project