Human Robot Interaction in Guardians

1. Human Robot Interaction in Guardians Amir M. Naghsh Sheffield Hallam University, UK Jeremi Gancet Space Applications Service N.V., Belgium Andry Tanoto Heinz Nixdorf Institute, Paderborn, Germany Jacques Penders Sheffield Hallam University, UK Chris R. Roast Sheffield Hallam University, UK Michel Ilzkovitz Space Applications Service N.V., Belgium RISE 2008 Worksop, Benicassim www.guardians-project.eu

2. Outline • Introduction and context: Guardians environment • On-the-field HRI • Remote HRI • Communication issues • HRI metrics • Conclusion and future work www.guardians-project.eu

3. 1. Guardians’ Environment / Fire-fighters operations Environmental Conditions No Visibility Reduced sense of hearing Restricted sense of touch

5. 1. Guardians’ Environment / Fire-fighters operations Fire-Fighters Operations Fire-fighters safety Teams of 2 (3,4) Progress along a wall when it is possible Frequently debrief the Entry Control Officer (ECO) Span of control (Overall structure) ADD DIAGRAM ???

6. 2. HSI Requirements The swarm notifies the humans of possible hazards (e.g. obstacles, high temperature, chemicals) The swarm stays within a relatively close range but also maintains its distance to the fire fighters to allow them freedom of action The swarm indicates unambiguously the direction to the scene of incident or backwards to the exit point. The interfaces indicating hazards should be most noticeable for the fire fighters.

7. 2. Swarm  Human Interactions Visual Visual presentation using LED / LCD screens Providing Navigational information / Hazards Audio Dialogue based interactions / Speech recognitions Not recommended

8. 2. Swarm  Human Interactions Tactile Using sense of touch for presenting information Hazards – Obstacles / Navigation information Vibrators

9. 2. Human  Swarm Interactions Passive Autonomous – form a bubble surrounding fire fighters – maintain a min/max distance Tangible Big buttons – high level tasks Movement-based Gesture recognition Audio / Visual Not recommended VPL Research, 1987 MIT, 2007

10. 3. Remote HRI: the base station Base station’s purpose • Monitoring and control of the robots and humans on the field • Offline and online support to mission planning, coordination, scheduling and execution • Data recording and replay with support to debriefing Users roles • Different types of users, with different roles • Guardians’ appliance coordinator • Operator(s) • Sensor data specialist(s) • Stakeholders in the commanding chain • Tier information channels (media...) www.guardians-project.eu

11. 3. Remote HRI: the base station Base station components Base Station Users Interfaces to VF robots, HCM, and their sensors Robots and HCM I/F MTE and MPSEM SDP HMI Clients Base Station Core MDRD www.guardians-project.eu

12. 3. Remote HRI: the base station Base station components Base Station Robots and HCM I/F MTE and MPSEM SDP HMI Clients Base Station Core MDRD => HMI Servers => Base station administration / supervision www.guardians-project.eu

13. 3. Remote HRI: the base station Base station components => Mission edition support tools => Planning and scheduling => Execution monitoring Base Station Robots and HCM I/F MTE and MPSEM SDP HMI Clients Base Station Core MDRD www.guardians-project.eu

14. 3. Remote HRI: the base station Base station components Base Station Robots and HCM I/F MTE and MPSEM SDP HMI Clients Base Station Core MDRD => Mission data recording / replay => Mission data dispatching www.guardians-project.eu

15. 3. Remote HRI: the base station Sensor Data Processing => Ground based (cpu-consuming) data processing => Data fusion and map building => Robot localization Base station components Base Station Robots and HCM I/F MTE and MPSEM SDP HMI Clients Base Station Core MDRD www.guardians-project.eu

16. 3. Remote HRI: interfaces Foreseen interfaces & approach to remote HRI: • ADD ILLUSTRATIONS HERE ??? www.guardians-project.eu

17. 4. Communication Issue Issues: • Interferences due to warehouse infrastructures • No communication infrastructure is available at the incident • Communication protocol • Communication bandwidth Short range Communication Middle range Communication

18. 4. Wireless Communication Three wireless communication standards: WLAN (802.11) High datarate, high power consumption, point to point connection For high-rate data transfer: video, audio Bluetooth Middle datarate, low power consumption, point to multi-point connection For continuous, middle-rate data transfer: audio, sensors ZigBee Low datarate, low power consumption, fast network formation time, high number of nodes For time-critical low-rate data transfer: critical data (commands, sensor) Which, When, and Where?

19. 5. Validation & Evaluation What to measure? • Situational awareness • To understand the extreme condition of the environment and robots • Cognitive/Work load • To ensure load below humans’ capability • Team performance • The outcome of teamwork between robots and humans How to measure? • Interaction debugger [Kooijmans & Al, 2006] • to collect, match, visualize, and analyze • SAGAT • to measure situational awareness • Wizard of Oz www.guardians-project.eu

20. 6. Conclusion and future work • HSI is a novel research area • We presented: • issues on human-robot swarm interaction for search and rescue mission • System architecture to support human-robot swarm interaction • We proposed • human-robot swarm interface • Metrics for validation and evaluation • Future work to provide conceptual design of human-robot swarm interaction • Interface prototyping and testing is ongoing with participation of end-users www.guardians-project.eu

21. Thank you for your attention!