Learning of Coordination of a Quad-Rotors Team for the Construction of Multiple Structures.

1. Learning of Coordination of a Quad-Rotors Team for the Construction of Multiple Structures. Sérgio Ronaldo Barros dos Santos. Supervisor: Cairo Lúcio Nascimento Júnior.

2. Objective • Present a solution for the learning of non-linear path tracking controllers for a quad-rotor using Reinforcement Learning. • Propose a learning method of the coordinating the movements of a quad-rotors Team to build multiple truss-like structures cooperatively. 1

3. Overview • The system is composed by: • A team with three quad-rotors; • Each quad-rotor is constituted by a decentralized low level controller and a collision avoidance algorithm implemented on-board. • A set of parts (beams and columns) of truss-like structures equipped with magnetic nodes. 2

4. Overview • A task planner based on Reinforcement Learning used to coordinate the team of quad-rotors during the building of a target structure. • A vision system used to get the real-time pose estimation of the quad-rotors, parts and also the assembly points. 3

5. First Stage: Low Level Controllers • Implement a control algorithm based on Reinforcement Learning that can adapt well to different flight conditions, such that the aircraft is able to track a defined trajectory. • It is taken into account during the training phase. • Different trajectories; • The transport of loads; • The presence of wind effects. 4

6. First Stage: Low Level Controllers 5

7. First Stage: Low Level Controllers • The controllers are derived off-line using the non-linear X-Plane model and Learning algorithm. • Immediately after the controllers are ported to an actual aircraft using the Real Time Workshop and Quarc Design . • The experimental results were submitted to the 2012 IEEE International Conference on Systems, Man, and Cybernetics. 6

8. Second Stage: Task Planner • Learning a set of optimal actions for each quad-rotor, such that the target structure can be built cooperatively. • The assembly plan will be learned off-line through a simulator. • The obtained solutions will be validated experimentally. • The learned assembly plan should be able to place parts without result in deadlock conditions. • The structure must be dynamically stable during the assembly. 7

9. Second Stage: Task Planner • To avoid collisions, the quad-rotors should satisfy a minimum distance condition among them while execute their maneuvers. • The system must allow several quad-rotors to pick up parts simultaneously from a supply bins. 8

10. Second Stage: Task Planner 9

11. Second Stage: Task Planner • Possible optimization conditions. • Minimize the total cost of construction; • - Minimize the time of construction of the structure, taking into account the limitations of the quad-rotor actuators; • - Maximize the operation time of the quad-rotors through the minimization of energy consumption. 10

12. Thank You ! 11