Assistance for the Elderly in RoboCare

1. Project “RoboCare: A multi-agent system with fixed and robotic intelligent components” MIUR Law 449/97(yr 00) – 2003-2006 Assistance for the Elderly in RoboCare Riccardo Rasconi ISTC-CNR [PST] Institute for Cognitive Science and Technology National Research Council of Italy Planning and Scheduling Team http://pst.istc.cnr.it Workshop on Telematics and Robotics for the Quality of Life of the Elderly Joint work with Amedeo Cesta, Gabriella Cortellessa, Federico Pecora 28/09/2009

2. The RoboCare Project’s Goal • A distributed system • Software Agents • Robotic Agents • Human Agents • All cooperate to provide services for human assistance http://robocare.istc.cnr.it

3. The Different Research Aspects Involved Acceptability Issues Active Supervision Framework Human-Robot Interaction Personalized Intelligent Assistance Robust Mobile Robotic Skills Distributed H/S Infrastructure http://robocare.istc.cnr.it

4. The RoboCare Domestic Environment Posture Recognition People Localization & Tracking Robot Mobility + User Interaction PDA ADL Monitoring

5. Multiple Intelligent Systems • A number of baseservices provide the building blocks for higher-level assistive behavior • Posture recognition • Person localization • Mobile robotic platform • User interaction front-end • PDA interface • Daily activity monitor

6. The robotic platform evolution 2004 2005 2006

7. Vision Sensors: People Tracking System Luca Iocchi and G. Riccardo Leone work from Univ. Rome “La Sapienza” [Bahadori et al, Applied AI, 2007]

8. Contextual Knowledge Component: Non-Intrusive Activity Supervision behavioral requirements (in terms of daily activities) Caregiver vs.Active Supervision Framework interface compilation T-REXscheduling problem physician family members Caregiver specifications compiled into scheduling problems (a temporal constraint network) [Pecora et al, ISSEJ, 2006]

9. Representing temporal prescriptions as a schedule • Activities and their mutual temporal constraints represented as a Simple Temporal Network • Dispatched for execution and monitored • Constraint violation triggers interaction

10. Schedule Execution Monitoring • Data are continuously retrieved from the stereo cameras (more about this issue later in the talk); • Activity status is updated at each execution step; • All constraint violations are detected; time now time now time now time now time now time now time now time now time now time now time now time now time now time now behavioral pattern t maximum allowed distance

11. Using the Scheduler’s Temporal Knowledge to generate contextualized dialogues breakfast cooking lunch medicine executed executed executed t time now You should hurry up taking your after-lunch medicine!

12. Using the Scheduler’s TemporalKnowledgeto generate contextualizeddialogues breakfast cooking lunch medicine executed executed t time now You should wait a little longer before having lunch!

13. Using the Scheduler’s TemporalKnowledgeto generate contextualizeddialogues breakfast cooking lunch medicine executed NOT executed t time now Maybe you should cook yourself something warm to eat!

14. Input/Output Channels Simple I/O Engine Knowledge for Interaction The Proactive Interactor • What is behind the interaction? Talking head Speech recognition Interaction Manager A set of active services ….

15. The User Interaction Agent (1/2) Speech Recognition

16. The User Interaction Agent (2/2) • Simple synthesis of Speech Acts is performed by analyzing the information contained in the Constraint Violation DB and in the Environment Status DB Verbalizations Synthesis

17. Generating environment-coherent behavior • Coordination of multiple services is achieved by solving a Multi-Agent Coordination (MAC) problem • The MAC problem is cast as a Distributed Constraint Optimization Problem (DCOP) • The DCOP is solved by the ADOPT-N algorithm, an extension of the ADOPT (Asynchronous Distributed Optimization) algorithm for dealing with n-ary constraints

18. Agents, variables and soft constraints Through cost functions, soft constraints are used to prefer (for the monitored person) healthy states and avoid dangerous states Cost functions are modeled so as to reflect the desiderata of system behavior Detailed description in [Pecora & Cesta, Comp. Int. 2007]

19. Possible Assistant/assisted interactions On-demand interaction(Person takes initiative) Question / answering Proactive interaction(RoboCare takes initiative) Danger Warning The RoboCare Environment as a Mixed-Initiative System

20. Managing interaction in RoboCare On-demand Proactive

21. Some examples of interaction

22. Proactive Warning • Feedback from sensors is a key activator • Explanation triggered by T-REX temporal knowledge

23. Proactive Alarm for Danger • A reactive routine is activated with a precompiled plan • (go-to-place; try-interaction; call-emergency)

24. On Demand Question-Answering • Query to the temporalized knowledge in T-REX • Very simple additional internal query capabilities

25. Related work Intelligent assistants Same domain and similar technologies: Autominder [Pollack et-al, 2003], PEAT [Levinson 1997], PEARL [Pineau et al. 2003; Pollack 2005], I.L.S.A. [Haigh, Kiff, & Ho 2006] Capability integration: Similar problems addressed with CALO, CMradar, etc. … although project scale quite different!

26. Conclusions RoboCare has addressed (among others) the open challenge of integrating diversified intelligent capabilities to create a proactive monitoring assistant for everyday life in a domestic environment Highlighted in this work particular use of the internal knowledge of a constraint-based scheduler (the temporal constraint network) as well as its capability of reasoning on changes in the environment constraint violations determine when the system has to interact. The analysis and interpretation of the violation contribute to determine how to interact with the user the use of a distributed coordination algorithm to create a coherent behavior of multiple “active agents”

27. THANK YOU! QUESTIONS?