Five Lessons Learned in Human-Robot Interaction

1. Five Lessons Learned in Human-Robot Interaction Patricia McDermott Alion Science and Technology Jennifer Riley, Ph.D. SA Technologies Douglas Gillan, Ph.D. North Carolina State University Laurel Allender, Ph.D., Army Research Laboratory

2. Bandwidth reduction • Span of control • Unmanned vehicle • characteristics • Level of automation • Interface designs • Collaborative • technologies • Teamwork • configurations Sense Communicate Process Helping the soldier… Present Understand RESEARCH OBJECTIVES Investigate the impact of key technological, human, and task factors on human robot interaction in a range of robotic tasks.

3. Research Overview Research Road Map NMSU NCSU SA Technologies Alion ARL Integrated HRI Research Findings Identified HRI Research Gaps Candidate Robot Interfaces Collaborative Research Experiments HRI Design Guides/ Implications

4. Top 5 Signs You have Made an HRI Error 5. The operator keeps covering the same ground over and over.

5. Field of View studies Widening the view is most beneficial If can’t widen view, 3rd person view can aid navigation Perception, Navigation, and Operating Robots Wide FOV/1st Person Perspective Narrow FOV/1st Person Perspective Narrow FOV/3rd Person Perspective Wide FOV/3rd Person Perspective Screenshots of conditions in FOV/Camera Perspective study

6. Limited Bandwidth Spatial vs. Temporal Resolution Participants selected higher spatial resolution, were faster to identify friend/foe If high spatial resolution not available, temporal resolution may be just as useful Perception, Navigation, and Operating Robots High Spatial Resolution High Temporal Resolution

7. Perception, Navigation, and Operating Robots • Lesson 5:Removing the human from the environment to be navigated disrupts the perception of visual information about the environment and the motion. The HRI interface can be designed to reduce this disruption and enhance the operator’s ability to navigate a robot through its distant environment and to perform the desired tasks. • Widen the field of view (FOV) if possible. Providing a third-person perspective also benefits navigation, the benefits are additive. • Temporal resolution may be sacrificed if the resolution is high and the robot is traveling at relatively low speeds. A high spatial resolution aids in target detection and identification.

8. Top 5 Signs You have Made an HRI Error 4. The operator keeps turning the wrong direction.

9. Collaboration Experiments Location of teammates Collaborative technologies Communication tools Team Factors and Processes View from UAV View from UGV Map with player locations

10. Team Factors and Processes Lesson 4:When robotic operators are distributed from Soldiers who need the information from robotic assets to complete a mission, the interfaces and collaborative technologies must support the communication of high quality spatial information in relation to the Soldier in the field.

11. Team Factors and Processes • The following solutions can be implemented to aid collaboration among humans with robotic tasks: • When robotics team members cannot be co-located, it is advantageous to incorporate aspects of co-location such as the ability to share imagery. • When sharing imagery, be sure to include enough reference information so that the recipient understands where the picture was taken in reference to his or her location. • Allow both team members to have full-time access to a map of the area without having to search between displays. • Allowing team members to communicate visually on a shared map can enhance the communication of highly spatial information.

12. Top 5 Signs You have Made an HRI Error 3. The operator keeps misidentifying targets.

13. Multitasking • High workload task - integrating and interpreting visual information from the different sources • Unique shape and color cues had a huge impact on performance

14. Multitasking and the Role of Automation Lesson 3:Operating a robot inherently involves multitasking and task switching. When designing automation to aid multitasking, designers must consider how social and human factors in HRI can interact to negatively counter the benefits of automated functions.

15. Multitasking and the Role of Automation • When implementing automation in robotic tasks, designers should keep in mind the following: • At natural break points in tasks or during down time, the system could prompt participants to switch tasks or robots. This would facilitate appropriate task switching strategies that produce better performance without sacrificing SA • Provide critical cues to help operators reconcile imagery from different perspectives such as aerial and ground or from two different ground perspectives. Shape and color were the most salient cues followed by texture, markings, and shadows. • Evaluate the coordination costs of two operators sharing one automated asset. The costs may outweigh the benefits, especially in an ad-hoc team.

16. Top 5 Signs You have Made an HRI Error 2. The operator does not notice a decrement in the robot status

17. SA Design Needs Lesson 2:Factors associated with human control of robotics assets can differentially impact SA. It is critical to determine the information needs of Soldiers within a context and to support the multiple facets of SA through system and interface design. In order to provide information at the right level, start with a goal-directed (cognitive) task analysis and behavioral and communications analysis.

18. SA-Related Design Needs • Support for visually demanding tasks. Examples include object recognition support, strategies or interface support for depth perception and judging size and distance, and support for parallel processing and divided attention. • Support for understanding robot localization and situated-ness. Examples include improved GPS for robots, mapping tools at the interface, interface designs that present data on orientation and robot configuration, and mission recording and playback for quick SA update. • Improved interface design for developing higher level SA. For example, conduct SA-oriented design of interfaces to facilitate integration of data at the interface and design for understanding of robot automation states and modes).

19. Top 5 Signs You have Made an HRI Error 1. Your paper only includes performance data.

20. HRI Measures • Performance Data • SA Data • Latency experiment example • Better control performance with no latency • Decreased SA errors with latency • Impacts interpretation in the context of complex tasks and goals

21. HRI Measures Lesson 1:Performance measures and SA measures should be examined together to get the “whole” story on operational effects in HRI. Implication: There are multiple methods to assess SA: Direct assessment Subjective assessment Process assessment Result: Better understanding of interrelationships between decision making performance and SA in HRI

22. Implications for Future Army • Ways in which humans and robots can work seamlessly together in naturalistic environments • Implications for display design, interaction design, team configuration, collaborative technologies, and HRI metrics • Future research • Development of system design requirements • Examination of interrelationships in HRI, i.e., spatial awareness and SA, spatial awareness and mental map development • More studies should be done on the interrelationships among human performance measures such as performance time and accuracy, SA, teamwork, and collaboration

23. SPECIAL ISSUE OF JOURNAL OF COGNITIVE ENGINEERING AND DECISION MAKING Improving Human-Robot Interaction in Complex Operational Environments: Translating Theory into Practice Topic Areas • Sensor Interpretation & Integration • Manipulation • Navigation • Planning • Multiple Robot Operations • Team Performance • Trust and Acceptance • Technological Issues • Research Issues Key Dates 15 October: notification of plan to submit 15 November: paper submission Special Issue Guest Editors Jennifer M. Riley SA Technologies jennifer@satechnologies.com Patricia L. McDermott Alion Science & Technology pmcdermott@alionscience.com Douglas J. Gillan North Carolina State University djgillan@gwced.ncsu.edu