Abdelfetah Hentout ( ahentout@cdta.dz ; ahentout@outlook ).

1. Short biography • AbdelfetahHentout (ahentout@cdta.dz; ahentout@outlook.com). • Affiliated with the “Robotized manufacturing systems” team. • Centre for Developmentof Advanced Technologies (CDTA). • Ph.D. degree in “Robotics” from the “University of Sciences and Technology HouariBoumediene”, Algeria in 2012. • Magister degree in “Industrial informatics” from the “Military Ploytechnic School”, Algeriain 2004. • Research interests: • Multi-agent systems, • control architectures, • flexible job-shop planning and scheduling, • mobile manipulation, • multi-robot systems, • optimal path planning, • telerobotic systems, • Collaborative robotics.

2. IEEE RO-MAN 2018 The 27th IEEE International Symposium on Robot and Human Interactive Communication HRI-SI2018 Human-Robot Interaction: From Service to Industry Challenges and open issues of industrial collaborative robotics A. Hentout, M. Aouache, A. Maoudj and I. Akli Centre for Development of Advanced Technologies (CDTA) BP 17, Baba Hassen, Algiers 16303, Algeria.

3. Plan Introduction Industrial collaborative robotics Types of collaborative robots Human-robot interaction in industrial cobotics Challenges and open issues of industrial cobotics Discussions and conclusions

4. Plan Introduction Industrial collaborative robotics Types of collaborative robots Human-robot interaction in industrial cobotics Challenges and open issues of industrial cobotics Discussions and conclusions

5. I. Introduction • Industrial robots are substitutes replacing or helping humans: • Fulfilling dangerous and tedious manufacturing tasks. • Very high precision. • Installed in physically separated workspaces away from humans. • Industrial robotics is rapidly evolving: • humans are not only sharing the same workspace with robots, • considering them as useful collaborators.

6. Plan Introduction Industrial collaborative robotics Types of collaborative robots Human-robot interaction in industrial cobotics Challenges and open issues of industrial cobotics Discussions and conclusions

7. II. Industrial collaborative robotics • Collaboration is “The action of working with someone (or many) to produce or create something”. • Exact purpose of collaborative robots. • A collaborative robot (i.e., a cobot) is a robot designed and built to collaborate with humans. • A robot intended to physically interact with humans in a shared workspace. • Industrial cobots are industrial cobots working side-by-side with humans as collaborators to accomplish manufacturing tasks.

8. Plan Introduction Industrial collaborative robotics Types of collaborative robots Human-robot interaction in industrial cobotics Challenges and open issues of industrial cobotics Discussions and conclusions

9. III. Types of collaborative robots • According to ISO 10218 standards, industrial cobots are classified into four types: • SafetyMonitored Stop. • Hand Guiding. • Speed and Separation Monitoring. • Power and Force Limiting.

10. III. Types of collaborative robots 1. SafetyMonitoredStop • When a robot is mostly working on its own, but occasionally a human might need to enter its workspace: • Take a heavy part that must be handled by a robot and a worker needs to do another operation while the robot is still handling it. • If the human enters the restricted area (safety zone), the robot stops: • The robot is not shut down. • The brakes are on. • Uses regular industrial robots, • Safety devices are used to detect employees at proximity, • Used for light cooperative operations, • Basically, the robot stops when the safety zone is violated.

11. III. Types of collaborative robots 2. Hand Guiding • Used for hand guiding or path teaching. • Pick-and-place applications. • Uses regular industrial robots, • Needs end-of-arm device to feel the forces that the worker is applying on the robot tool (force sensors, torque sensors), • Does not make robot collaborative in other functions/modes.

12. III. Types of collaborative robots 3. Speed and Separation Monitoring • The safety zones are gradated so that the robot produces different reactions according to the worker location within the different safety zones. • The robot environment is monitored by lasers/cameras to track the worker position. • If the human is within a certain safety zone, the robot responds with slow speeds. • The robot stops when the worker comes too close. • Uses regular industrial robots, • Needs vision system to detect the proximity of the worker, • Used for operations that require frequent worker presence.

13. III. Types of collaborative robots 4. Power and force limiting • Can work alongside humans without any additional safety devices. • The robot feels abnormal forces in its path. • Programmed to stop when it reads an overload in terms of force. • Dissipate forces in case of impact on a wide surface. • Not use regular industrial robot, • Force is limited, • Not require additional safety devices, • Used for direct collaboration with the worker for various task.

14. Plan Introduction Industrial collaborative robotics Types of collaborative robots Human-robot interaction in industrial cobotics Challenges and open issues of industrial cobotics Discussions and conclusions

15. IV. Human-robot interaction in industrial cobotics • Provide a state-of-the-art on HRI in industrial cobotics. • More than 100 papers presenting solutions dealing with this field were carefully reviewed. • Distribution of studied publications on the years from 2008 to 2017:

16. IV. Human-robot interaction in industrial cobotics • 50 research categories and sub-categories.

17. IV. Human-robot interaction in industrial cobotics

18. Plan Introduction Industrial collaborative robotics Types of collaborative robots Human-robot interaction in industrial cobotics Challenges and open issues of industrial cobotics Discussions and conclusions

19. V. Challenges and open issues • Several challenges and open issues in industrial cobotics still need to be addressed: • To develop highly advanced industrial cobotic systems. • 20 open issues and challenges have been identified: • Most of them have been posed by the authors of the studied research works.

20. V. Challenges and open issues • 1. Design of industrial cobots: • Type of robot (dual/single arm) and action(manipulation, mobility), • Perception and sensors, • Information exchange, • Form, shape and material of mechanical components. • 2. Development of software architectures: • Understanding the characteristics, advantages and disadvantages of the different approaches is fundamental to design, implement and use of cobotic software. • 3. Classification of human activities: • Industrial cobots must classify human activities that involve interaction with other objects and subjects. This may be achieved by online training/reasoning systems.

21. V. Challenges and open issues • 4. Voice recognition, comprehension and use of natural language: • Detect human presence and identify demands based on body language, hand gestures, activities... • Identify the speaker, understand sentences, connect them to physical worldand point out orders in the speech. • 5. Virtual & augmented reality: • Through VR/AR technology, humans will be able to obtain virtual objects, graphics instructions and information on the robot motion. • The robot will receive control commands from the human to help transferring virtual objects... • 6. Dexterity, movement and manipulation: • Industrial cobots must move effectively in their workspaces when dealing with unforeseen events to collaboratively handle objects (tools, products, etc.).

22. V. Challenges and open issues • 7. Multi-modal high-level interaction • HRI in cobotic systems must incorporate several methods with a high-level interface to guide human accomplishing operations. • Combining audio commands with gestures simultaneously, adding virtual objects on the real-world scene… • 8. Collision detection & avoidance: • Allows industrial cobots to avoid contact with co-workers and objects. • Use reactive control approaches to assure human safety. • Improve existing methods by inserting different pose estimation. • Use coordinated motions. • 9. Infrastructure: • Industrial cobots must perceive and interpret their workspaces. • Augment the environments with some sensors to make them communicants (smart sensors, IoT, etc.).

23. V. Challenges and open issues • 10. Transfer information between the human and the robot: • How to transfer data between human and robot using existing equipment and approved security? • 11. Reprogrammability, scalability and learning ability: • Perform prescribed tasks and learn to perform new tasks through imitation and/or demonstration. • Scalable: add new functionalities, examine new control algorithms, ... • 12. Safety: • Should be introduced while developing control architectures. • So as not to injure human collaborators while performing tasks. • 13. Optimal placement of industrial cobot: • Determined via an approach guaranteeing the detection of the human, minimizing possible collisions and improving his safety.

24. V. Challenges and open issues • 14. Building user-friendly human-robot interfaces: • Most HRI designed to control/program industrial cobots are complex and difficult to use, and require special training for the user. • Industrial cobotic systems may be used by unskilled and unqualified persons who may have disabilities. • Develop smart interfaces to simplify their use and operation. • Perform quantitative analysis of time and difficulty level in user operations before deployment. • 15. Cloud-based and fog-based control: • Analyze extended data from additional sensors (cameras, power). • Access of external algorithms permits using industrial cobots as smart tools. • Create interfaces adapted to connect cobots to IoT and accessing them from different locations.

25. V. Challenges and open issues • 16. Tasks scheduling for human-robot team: • Understand the role of humans and cobots in decision-making process. • Include the preferences and competences of humans and cobots. • 17. Robotics development frameworks: • Many SME still hesitate to adopt such systems. • Only a few cobots are currently commercially available. • Extensive development process and high costs. • Each manufacturer develops and uses its own operating systems, middleware and development tools further complicates the development of industrial cobots. • Use open source shared robotic development frameworks will remedy that, and considerably reduce time and resources required for development. • Make the development of industrial cobots easier and cheaper.

26. V. Challenges and open issues • 18. Real-time constraints: • Real-time constraints are among the main requirements of HRI in industrial cobotics. • Paramount importance to recognize human actions, detect multiple actions, anti-collisionmethods, control architecture, etc. • 19. Fault tolerance: • One of the major problems in developing industrial cobotic systems. • Based on three principles: fault detection, diagnosis and recovery. • Should incorporate re-planning abilities and dynamically adapt to needs based on integrated resources and reliability requirements. • 20. Development of new sensors and algorithms • Design of new sensing technology, fast sensor-fusion algorithms to track multiple moving targets in real time, etc.

27. Plan Introduction Industrial collaborative robotics Types of collaborative robots Human-robot interaction in industrial cobotics Challenges and open issues of industrial cobotics Discussions and conclusions

28. VI. Discussions and conclusions • HRI in industrial cobotics received much attention from academia. • Growing number of research in HRI, interfaces and solutions related to design and implement industrial cobots. • Researchers developed and are developing complex interaction systems and sophisticated interfaces for industrial cobots. • When humans need to physically interact with a robot, the standard of normal and effective performance is their experiences of daily interactions with other humans.

29. VI. Discussions and conclusions • To interact with humans and perform tasks side-by-side, traditional established safety procedures and workspaces separation between robots and humans were removed. • This represents a big hazard to all humans in the vicinity of robots. • Great number of studied works dealt with safety in cobotic systems using different technologies and approaches. • Trend in industrial cobotics is to obtain flexible systems where humans and robots can safely interact and cooperate to achieve assigned tasks.

30. Thank you …