Evolutionary Algorithms applied to Trajectory Planning in Robotics

1. Evolutionary Algorithms applied to Trajectory Planning in Robotics Dragos Arotaritei "Gr. T. Popa" University of Medicine and Pharmacy, Iasi, Romania dragos_aro@yahoo.com

2. Outline Introduction to Evolutionary Algorithms Optimization and Search Problems in Robotics � overview Pareto front and multiobjective optimization Problems Trajectory Planning in Robotics and Evolutionary Algorithms Conclusions

3. Introduction Evolutionary algorithms are meta-heuristic search inspired from natural selection and survival of the fittest in the biological world. Genetic Algorithms (GA) are search techniques used in approximate solution for optimization and search problems using mechanisms inspired by evolutionary biology: selection, crossover, mutation, inheritance. Optimality is defined in terms of efficiency of the algorithm in searching for the shortest possible path from the initial to the final configuration of the end-effector of the robot arm, efficiency in avoiding possible obstacles in the path. Optimization problems: Single objective optimization problems Multi-Objective optimization Problems

4. Search Problems (Path search) Optimal multi-robot coordination Multi-task optimization Optimal motion planning of robot arms (Trajectory planning of manipulators ) Motion optimization (optimization of controller parameters - morphology in different control schemas) PID (PI) Fuzzy Neural Hybrid (neuro-fuzzy) Path planning and tracking (mobile robots) Optimal motion planning of robot arms Trajectory planning of manipulators Vision � computational optimization

5. Evolutionary Algorithms - Related techniques: Ant colony optimization (ACO) Particle swarm optimization Differential evolution Memetic algorithm (MA) Simulated annealing Stochastic optimization Tabu search Reactive search optimization (RSO) Harmony search (HS) Genetic programming (GP) Artificial Immune Systems (AIS) Bacteriological Algorithms (BA)

6. Components of one evolutionary algorithm: the encoding scheme fitness function evolutionary operators Encoding scheme transform the �real world problem� into evolutionary algorithm � chromosome is the basic part Fitness function is a measure of success of one individual from population and it is computed based on decoding of the chromosome The evolutionary operators acts on efficiency and convergence of the algorithm.

7. The encoding Scheme The bit string representation of solutions has dominated GA research Holland (1992) gave a theoretical justification for using binary encoding Encoding is the mapping of real or integer values to bit string (�0� and �1�) Linear mapping Logarithm mapping Other The Range of variables and the bit representation 5 bits representation and [0 1] range of variable give a precision 25-1 = 0.032. (0.245 ? 10110)

8. GA-terminology The genotype is the genetic construction of a cell, an organism, or an individual An allele one member gene (in GA, e.g. �0� and �1� in location k from a chromosome) A phenotype is any observable characteristic of an organism a chromosome (genome) is the representation of one set of parameters which define a proposed solution to the problem that the genetic algorithm is trying to solve One individual is one member of the population Elitism

9. GA-operators Selection Roulette Tournament Stochastic sampling Rank based selection Boltzmann selection Nonlinnear ranking selection Crossover One point Multiple points Mutation

10. Genotype length Fixed length genotype Variable-length genotype Population Fixed population Variable population Species inside population

11. Drawbacks of GA time-consuming when dealing with a large population premature convergence Dealing with multiple objective problems Coding schema

12. Crossover (reproduction ) Single point Two points "cut and splice" - results change in length of the children strings

14. Mutation mutation is a genetic operator used to maintain genetic diversity from one generation of a population of chromosomes

15. General GA Schema

16. GA and Trajectory Planning GA techniques for robot arm to identify the optimal trajectory based on minimum joint torque requirements (P. Garg and M. Kumar, 2002) path planning method based on a GA while adopting the direct kinematics and the inverse dynamics (Pires and Machado, 2000) point-topoint trajectory planning of flexible redundant robot manipulator (FRM) in joint space (S. G. Yue et al., 2002) point-to-point trajectory planning for a 3-link (redundant) robot arm, objective function is to minimizing traveling time and space (Kazem, Mahdi, 2008)

17. Optimal path generation of robot manipulators Control Schema Robotic arm � kinematic model Controller type Objective function - optimal path Optimization algorithm (method) GA use smooth operators and avoids sharp jumps in the parameter values.

18. Adaptive Control Schema � Track Control error function between outputs of a real system and mathematical model What we optimize? Which parameters must be optimized? How many objectives (single �objective or multiobjective)? Collision free? (How to model collision in GA?)

19. Three join Manipulator A three-joint robotic manipulator system has three inputs and three outputs. The inputs are the torques applied to the joints and the outputs are the velocities of the joints No ripples

20. For n-DOF we will have n inputs ui, i=1�n, (ui ? ?i) Controller PID (PI) Neural network (multilayer perceptron, recurrent NN, RBF based NN) Fuzzy Neuro-Fuzzy (hybrid)

21. NN: We must to adapt the weights and eventually the bias The chromosome: Adapt the weights

22. Fuzzy Logic Aggregation of rules defuzzification free-of-obstacles workspace (Mucientes, et. al, 2007) wall-following behavior in a mobile robot

23. Learning of fuzzy rule-based controllers Find a rule for the system Step 1: evaluate population; Step 2: eliminate bad rules and fill up population; Step 3: scale the fitness values; Step 4: repeat NI iterations for Step 4 to Step 9 Step 5: select the individuals of the population; Step 6: crossover and mutate the individuals; Step 7: evaluate population; Step 8: eliminate bad rules and fill up population; Step 9: scale the fitness values. Step 10: Add the best rule to the final rule set. Step 11: Penalize the selected rule. Step 12: If the stop conditions are not fulfilled go to Step 1

24. The chromosome encode the rules: Sn is constant in this application but it can be also variable to be optimized wall-following behavior of the robot the robot is exploring an unknown area moving between two points in a map Requirements maintain a suitable distance from the wall that is being followed to move at a high velocity whenever the layout of the environment is permitting avoid sharp movements (progressive turns and changes in velocity)

25. The requirements are �encoded� in Universes of discourse and precisions of the variables right-hand distance (RD) the distances quotient (DQ), based on left-hand distance Orientation linear velocity of the robot (LV) Linear acceleration Angular velocity Path of the robot (simulated environments)

27. Fixed points: the desired Cartesian path Pt is given the problem is to find the set of joint paths P? in order to minimize the cumulative error between desire and real path during trajectory Pk is the kinematic model Free end points case

28. fitness function (minimization) Global fitness: Linear function of individual objectives Fot � excessive driving (sum of all maximum torques), fq � the total joint traveling distance of the manipulator, fc - total Cartesian trajectory length, tT - total consumed time for robot motion Penalty function Population initialization (probability distribution) Random uniform Gaussian

29. Drug delivery using microrobots (Tao, et. al, 2005) (GA)�based area coverage approach for robot path planning. Drawbacks of most currently available drug delivery methods are that the drug target area, delivery amount, and release speed are hard to be precisely controlled. It is very difficult or impossible to eliminate side effects. Open issues actively control the delivery process Access to appropriate areas that cannot be reached using traditional devices Current Issues On-line path planning (solve unexpected obstacles problem) Optimal path planning (efficiency, path planning)

30. microcontroller is used to guide the robot movement GA-based approach uses fine grid cell decomposition for area coverage Because the robot will move cell by cell, the start point of chromosomes has to be changed dynamically whenever the robot reaches the center of a cell The end point of a chromosome is not fixed and needs to be determined by applying GA operators. The robots may move from the center of a cell to its 8 adjacent cells along 8 directions. some obstacles are unknown before drug delivery (the robot discover these obstacles during the motion)

31. Expandable chromosomes Deleting the path Crossover operator

32. New mutation operators Travel further Delete Reverse delete Stretch Shortcut The algorithm keep mind the visited nodes Extension to operational research?

33. Other applications using evolutionary algorithms Autonomous mobile robot navigation - Path planning using ant colony optimization and fuzzy cost function evaluation (Garcia, et. al, 2009). Legged Robots and Evolutionary Design Optimal path and gait generations (Pratihar, Debb, and Gosh, 2002) � 0/1 absence or presence of rule six-legged robot collision-free coordination of multiple robots (Peng and Akela, 2005)

34. Pareto front The single objective optimisation problem (SOP) conduct to a minimization (or maximization) of one cost function, less or more complex, that is a single objective is taken into account. Conversely, the multi-objective optimization problem takes into account two or more objective that has to be minimized (or maximized) simultaneously. Some objectives can be in competition, so a simultaneous minimization is not possible, but only a trade-off among them. Some time, the number of objectives can be high, like 16 objectives or more that make the multi-objective optimization problem (MOP) and interesting and challenging area of research

35. The multiobjective optimization problem could be generally formulated as minimization of vector objectives Jt(x) subject to a number of constraints and bounds:

36. In the case of competing objectives a trade-off is involved such a problem usually has no unique solution. Instead, we can admit a set of solutions, equally valid non-dominated as a set of alternative solutions known as Pareto-optimal set In what follows we assume without loss of generality that all the function objectives must be minimized. If we have a maximization case fi we simply minimize the function -fi. For any two points that are usually named candidate solutions V1,V2??, V1 dominates V2 in the Pareto sense (P-dominance) if and only if the following condition hold

37. The Pareto set is the set of PO (Pareto-Optimal) solution in design domain and the Pareto Front (PF) is the set of PO solutions in the objective domain. The most popular way to solving the MOP (Multi Objective Optimization Problem) is to reduce the minimization problem to a scalar form by aggregating the objectives in weighted sum, with the sum of weights constant: The weighted sum method has a serious drawback, the method usually fail in the case of nonconvex PF.

39. GA can provide an elegant solution for tradeoff among different minimization of cost function for each variable versus total cost or other variable. Non-convex solutions �Immigrants�, possible solution for jump from local minima. Dealing with many variables (e.g. 16 variables)

40. Pareto optimal multi-robot coordination with acceleration constraints (Jung and Ghrist, 2008) collection of robots sharing a common environment each robot constrained to move on a roadmap in its configuration space each robot wishes to travel to a goal while optimizing elapsed time considering vector-valued (Pareto) optima all illegal or collision sets are removed.

41. Conclusions GA is not a universal panacea to optimization problems. Coding the problem into a genotype is the most important challenge! The best selection schema of individuals for crossover operator is difficult to be chosen apriori (tournament selection seems to be more promising) A number of parameters are determined empirically: Size of population pc and pm even usually values inspired from biology are given Other parameters in hybrid or more sophisticated GA

42. One of the most important element in the design of a decoder-based evolutionary algorithm is its genotypic representation. The genotype-decoder pair must exhibit efficiency, locality, and heritability to enable effective evolutionary search locality, and heritability: small changes in genotypes should correspond to small changes in the solutions they represent, and solutions generated by crossover should combine features of their parents