Dynamic User Task Scheduling for Mobile Robots

1. Dynamic User Task Scheduling for Mobile Robots Brian Coltin, Manuela Veloso and Rodrigo Ventura Carnegie Mellon University and InstitutoSuperior Técnico Workshop on Automated Action Planning for Autonomous Mobile Robots (PAMR) AAAI 2011 Sunday, August 7, 2011 Special thanks to JoydeepBiswas and Stephanie Rosenthal for their many contributions to the CoBot project.

2. CoBots • Multiple robots available in an office environment to general users over the web to: • Send the robot to a room • Deliver a spoken message • Pick up and deliver an object • Visit co-workers with telepresence • Fully autonomous mobile robots, which ask humans for assistance

3. Selected Related Work • Many robots teleoperated by users over the web, the earliest from 1995 [1, 2] • The robot Xavier had users ask it to go to places in a hallway over the web [3] • The RoboCup@Home competition provides competition setups for indoor autonomous service robots [4] [1] Goldberg, K.; Mascha, M.; Gentner, S.; Rothenberg, N.; Sut- ter, C.; and Wiegley, J. 1995. Desktop teleoperation via the world wide web. In Proceedings of the IEEE International Conference on Robotics and Automation, volume 1, 654– 659. [2] Taylor, K., and Trevelyan, J. 1995. Australia’s telerobot on the web. In Proceedings of the International Symposium on Industrial Robots, volume 26, 39–44. [3] Simmons, R.; Fernandez, J.; Goodwin, R.; Koenig, S.; and O’Sullivan, J. 2000. Lessons learned from xavier. IEEE Robotics Automation Magazine 7(2):33–39. [4] Visser, U., and Burkhard, H.-D.2007.RoboCup: 10 years of achievements and future challenges. AI Magazine 28(2):115–132.

4. CoBot Hardware Pan / Tilt / Zoom Camera Touch Screen Tablet Kinect RGB-D Sensor Planar LIDAR Fast Omnidirectional Base Both robots are designed and built by Mike Licitra, a scaled up version of the CMDragons small size soccer robots, also built by him. CoBot1 CoBot2

5. CoBot’s Autonomous Capabilities • Robust localization with Corrective Gradient Refinement Algorithm [1] • Fully autonomous navigation • Robust obstacle avoidance • Elevator traversal with human help [1] Corrective Gradient Refinement for Mobile Robot Localization, JoydeepBiswas, Brian Coltin and Manuela Veloso. To appear in Proceedings of IROS, September 2011.

7. Challenges of Deployment in an Office Environment • Need an overall architecturefor requesting, scheduling and executing user tasks (User to Mobile Robots Architecture) • Need an easy to use and intuitive web interface for users to request robots • Need to schedule requested tasks efficiently and effectively in an online fashion on multiple robots • Need to execute scheduled tasks

8. Outline • Introduction • Users to Mobile Robots Architecture • Web interface / Example • Scheduling Algorithm • Preliminary Results • Conclusion

9. Users to Mobile Robots Architecture (Robot Side) • Robot manager takes requests from server, reports back state • Behavior interaction planner executes tasks, goes up and down elevators • Onboard user interface interacts with users • Localization, navigation, motion and other lower-level modules interface with sensors and actuators

10. Users to Mobile Robots Architecture (Remote Server Side) • Execution manager sends tasks to and manages state of each robot • Knowledge base stores robot states and scheduled tasks • Web interface allows users to schedule and manage tasks, and shows the robots’ states • Scheduling agent assigns time and robot to tasks requested by users, stores in knowledge base

11. An Example • At 6:35 PM, Alice logs in at the CoBot web site, and requests a robot to transport a bottle of water from room 7705 to 7005 as soon as possible • The other time options are at a specific time or within a time interval • Alice can pick a specific robot

12. An Example (part 2) • The web interface passes the request to the scheduler, which plans to execute it in three minutes, after the current task at 6:38 PM. • When the current task completes, CoBot heads to room 7705 (using the elevator if necessary), and announces its destination aloud. • Upon arrival, CoBot says “Please place a battle of water on me to deliver.”, displays this on its screen, and waits for the user to press a button.

13. An Example (part 3) • Once the item has been placed, CoBot heads to room 7005 to make the delivery. • Upon arrival, it says “I have brought a bottle of water from room 7705. Please press ‘Done’ once you have taken the item.” and displays the message and a ‘Done’ button. • When the user presses ‘Done’, the robot begins its next task.

14. Monitor / Cancel Tasks

15. Telepresence Brian Coltin, JoydeepBiswas, Dean Pomerleau and Manuela Veloso. Effective Semi-autonomous Telepresence. Proc. of RoboCup Symposium, 2011.

16. Outline • Introduction • Users to Mobile Robots Architecture • Web interface / Example • Scheduling Algorithm • Preliminary Results • Conclusion

17. The Scheduling Problem • Given a list of tasks Ti = <si, ei, di, lis, lie, Ri>, find a feasible schedule of tasks (an assignment of times ti and robots ri to task Ti) • Task must begin in time interval [si, ei] • Task begins at location lis and ends at lie • The task is estimated to take di seconds, not including time to travel to start location • The task can be performed by any robot in Ri • A data-driven distance function dist estimates time to travel between two locations • Scheduling algorithm must be fast enough to run online when users make new requests • Infeasible tasks should be rejected

18. Single-robot MIP • Formulate problem as mixed integer program • Objective: min 1 (for feasibility) or • Time constraints: obey requests • Indicator variables prei,j for if task j precedes task i • Scheduled times don’t intersect (indicator like an or) • Solving this MIP results in a feasible solution • Bound execution time, some failures are ok • Finishes in one second for 99% of cases for expected input size

19. Multi-robot Scheduling • MIP can be extended with additional indicator variables for each robot and task pair • Number of variables multiplies with number of robots, making solution time too long • Instead, we randomly assign tasks to robots • Attempt to find feasible solution for each • If no solution, repeat up to N times • If we don’t find a solution, ask user to relax constraints

20. Scheduling Results • Randomly generated problems of T tasks and R robots, requests for random intervals over four hour period, task durations between zero and ten minutes • Solved 100 problems for each value of T and R with both multi-robot MIP and randomized robot choices • Five second time limit to solve problems • Random algorithm succeeded more often and more quickly

21. Preliminary Deployment Results • So far, CoBot1 and CoBot2 have completed over 300 tasks. • They have been active for a combined total of over 52 hours and have traversed 13 km.

22. Conclusion • CoBots fulfill user tasks requested over the web robustly and reliably • Users to Mobile Robots Architecture • Multi-robot scheduling through solving MIPs • Extensive consistent and successful testing results

23. Questions?