Evolving a rule system controller for automatic driving in a car racing competition

1. Evolving a rule system controller for automatic driving in a carracing competition Diego Perez, Yago Saez Member, IEEE, Gustavo Recio, Pedro Isasi Presented by Evgenia Dubrovsky

2. Overview • Introduction • Objectives • Controller Design • Results • Future work

3. Introduction • In 1970, the visionary Robert E. Fenton predicted how automatic vehicle guidance would evolve. • The system would be implemented in three stages. • The installation and use of various driver aids so that the driver would be a more effective decision maker and improve the performance of the driver-vehicle system. • Involve the gradual introduction of various subsystems for partial automatic control. • The transition to complete automatic vehicle control.

4. Introduction • Today, we are in the second stage: • We can buy cars with • electronic aided brake systems (ABS) • adaptive cruise controls which maintain a set distance from the car ahead, automatically accelerating or decelerating, and even applying the brakes. • All these new technical advances are a good starting point for approximating the third stage: to complete automatic vehicle control.

5. Introduction • Annual competition is organized by the DARPA for driverless cars which evaluates all the improvements. • However, researchers have a long way to go until these vehicles capable of driving in a fully automated way are actually available. • Most of the works were engineered in real prototypes, this involves two main problems: • The costs of buying and modifying a car. • The time needed for each test. • To overcome these constraints use car simulators.

6. Introduction • A. Evolutionary computation techniques applied to the automated driving. • Research how these techniques can help in automatic driving. • Some work was collected related to evolutionary computation applied to automated driving. • These works fall into two different categories: • Autonomous driving. • Vehicle features optimization.

7. IntroductionA.1. Autonomous driving • This work uses modules which combine high level task goals with low-level sensor constraints. • Those modules are dependent on a large number of parameters. • PBIL is proposed for automatically setting each module’s parameters. • The evaluation function takes into account different aspects, such as serious crashes, collisions, etc.

8. IntroductionA.1. Autonomous driving • For the simulation, the system uses a program called SHIVA which reproduces a microsimulation of vehicles moving and interacting in a roadway. • The algorithm can influence the vehicles’ motion sending simulated commands: • Steering • Accelerate • Brake

9. IntroductionA.1. Autonomous driving • PBIL was compared to GA in the same framework: • At the end of experiments the vehicles automatically entered the test track, completed one lap and a half and finally took the exit. • Although they were not capable of avoiding collisions with other vehicles. • These experiments showed the potential for intelligence behavior in the tactical driving.

10. IntroductionA.1. Autonomous driving • An approach was proposed for automated evolutionary design of driving agents. • GAs can help in designing an agent able to remotely operate a racing car. • The agent perceives the environment from a camera (position, velocity, approach angle, distance, etc). • The agent sends commands to the remote control car (forward, reverse, left, straight or right). • The analysis established that on long runs the agent’s operated car was 5% slower than the human operated one.

11. IntroductionA.2. Vehicle parameter optimization • The study of the suspension system has been an interesting topic for researchers. • Because it contributes to the car’s handling and braking for good active safety and driving pleasure. • Also keeps vehicle occupants comfortable and reasonably well isolated from road noise, bumps, and vibrations.

12. IntroductionA.2. Vehicle parameter optimization • A recent work showed how to optimize 66 variables from a Formula one racecar with a GA. • The parameters affect to: • Suspension system • Engine revolutions limits and gear ratios • Aerodynamics • Brake modelling • These variables were tested in Formula One ’99-’02. • A racing simulator released in 2003 with an advanced real time physics engine.

13. Overview • Introduction • Objectives • Controller Design • Results • Future work

14. Objectives A. TORCS Simulator • TORCS allows the competitors more flexibility to develop the controllers. • Advantages: • Has a high level of realism in graphics and physics. • Provides a large quantity of vehicles, tracks and controllers. • Easy to develop a controller and to integrate it within the simulator. • Disadvantage: • Memory leak each time the race is restarted.

15. Objectives B.Controllers, Sensors and Effectors • Total: 17 sensors and 5 effectors. • The sensors included information such as the angle of the car, the position, speed, location of opponents, etc. • The effectors used to drive the vehicle are the steering wheel value, both pedals (throttle and brake) and gearing change. • Five controllers were presented to the competition and three more were provided by the organizers.

16. ObjectivesC. Controller evaluation • The evaluation of the controllers results from their performance on three different test circuits which are previously unknown by the competitors. • The tournament is divided into two stages: • Every controller is thrown alone in each circuit. • All controllers are tested together measuring the arrival order to evaluate the drivers. • In both cases, the points assigned to each controller follows the Formula One pointscoring system (10 points for the first, 8 for the second, 6 till 1).

17. Overview • Introduction • Objectives • Controller Design • Results • Future work

18. Controller Design A. Input Data • Angle • The angle between the car direction and the direction of the track axis. • -π to π. • Discretization to range: [0, 4] • Where 0 means small angles.

19. Controller Design A. Input Data • Track Position • The distance between the car and the track axis. • -1 to 1. • Discretization to range: [0, 1] • Where 0 means centered in the track.

20. Controller Design A. Input Data • An important feature of the design is the usage of symmetry for the first two sensors. • A value of ‘−a’ and ‘a’ from any of those sensors will mean the same discretized value. • The objective of this approach: • To reduce the search space. • To avoid the algorithm learning how to face similar situations twice.

21. Controller Design A. Input Data • Speed • The speed of the car. • 0 to … (km/h). • Discretization to range: [0, 3] • Where 0 means lower speed that higher values.

22. Controller Design A. Input Data • Track • Discretization to range: [0, 2] • 0 means that the track edge has been detected between 20 and 100 meters. • 1 means that the track edge is up to 20 meters. • 2 means that no track edge is seen.

23. Controller Design B. Effectors • Throttle and brake • Discretization to range: [0, 1] • Both pedals have been codified in a common output, in order to avoid non-sense values. • For instance, a full gas value in both pedals at the same time would produce uncertain (and useless) outcomes.

24. Controller Design B. Effectors • Steering • Discretization to range: [-1, 1] • Gear • Gear changes did not take part in the learning process. • Is defined as follows: • x is the revolutions per minute of the engine. • If x > 6,000, then gear value is increased by one. • Else if x < 3,000, then current gear is decreased. • Otherwise, the gear does not change.

25. Controller Design C. Rules • The discretization applied over the input data allows to create a set of 120 rules. • The conditional part is composed of the above four sensors and the actions (acceleration, braking and steering). • This set of initial rules are the basis of the base individual.

26. Controller Design D. Base individual • Find the rule set that allows the vehicle to end the lap, centered on the track. • Minimize angle to track axis. • Symmetry: Could have been the problem?

27. Controller Design E. Evolutionary Rule System • Rule GA Individual • Rules from base individual compose the GA individual. • N rules depend on the track of base individual (N is between 10 and 20). • Include sensors and effectors.

28. Controller Design E. Evolutionary Rule System • Rule GA Step • Create a new rule by genetic operators. • Substitute the new one for the most similar in the individual. • If fitness decreases the rule stays. If not, recover the previous one. • Linear combination of lap time (40%) and damage (60%).

29. Overview • Introduction • Objectives • Controller Design • Results • Future work

30. Results • The controller developed for the competition (called DIEGO) was tested in three circuits and it obtained acceptable results in the first two.

31. Results • The usage of symmetry produced a side effect: the car drives in a smooth zig-zag trajectory centered on the track. • Because a small steering value can center the vehicle on the track, but not necessarily drive it parallel to the track axis. • The controller behaved in this way, only on specific circuits: the oval ones. • The circuits have banked bends which make zig-zag movement completely uncontrollable.

32. Results • This problem affected dramatically to the behavior of this driver in the third circuit of the test bed, as can be seen in the results.

33. Results • In the second stage, the races performed again over these circuits gave the last set of points assigned to the drivers. • These results are very similar to the results obtained in first two circuits on stage 1. • However, the results from third circuit on stage 1 were bad, so finishing in higher positions became impossible.

34. Overview • Introduction • Objectives • Controller Design • Results • Future work

35. Future work • The main objective now is to improve this controller in order to obtain better results for this new call.

36. Future work • Discretization • Use some methodology that will help us to determine the target ranges. • Symmetry • Next controller will introduce a system without symmetry, or a limited symmetry, to avoid the zig-zag movement of the vehicle.

37. Future work • Evaluation in more than one track • TORCS simulator does not easily allow restarting the race in a circuit different than the one used for starting the execution. • Use more than one circuit in the learning process collecting more efficient rules for the controller. • Opponent analysis • The vehicles hit them in the back without evading them. • Obtain a competitive advantage in the race phase.

38. The End Any Questions ?

39. The End Thank you ;)