Optimized Sensory-motor Couplings plus Strategy Extensions for the TORCS Car Racing Challenge

1. Optimized Sensory-motor Couplings plus Strategy Extensions for the TORCS Car Racing Challenge COBOSTAR Cognitive BodySpaces for Torcs based Adaptive Racing

2. Outline: • TORCS Competition Setup • COBOSTAR Design • Parameter Optimization • Strategy Modifications

3. TORCS Competition Setup

4. TORCS Competition Setup: Only local information / Main Idea • no complete track Information available

5. TORCS Competition Setup: Available Sensors • Angle Sensor:current angle between the car direction and the track axis. • Speed Sensors:speed in both axes. • 19 Range Sensors:free track space in front of the car. • 36 opponent Sensors: notice opponents around the car. • Additional Sensors: current engine speed, current gear, rotation and speed for all wheels, damage of the car, etc.

6. TORCS Competition Setup: Car Control • Gas and Break Pedals • Gear shifting • Steering

7. COBOSTAR Design

8. COBOSTAR Design: 1. On Track Strategy • Distance and direction of the longest range sensor 2. Off Track Strategy • Angle to track axis • Relative position to track center

9. COBOSTAR Design: On Track Strategy Using: Calculate: Target steering angle: Target speed: Finally:

10. COBOSTAR Design: on Track Strategy :Gas or Break How to Break: How to Steer:

11. COBOSTAR Design: off Track Strategy what changes? • Distance sensors to track borders are unavailable. • Steering becomes more difficult. • Wheel slippage is much stronger.

12. COBOSTAR Design: off Track Strategy Using: Calculate: Target steering angle: Target speed:

13. COBOSTAR Design: off Track Strategy Anti Slip Regulation: Stuck behavior: If nothing else works, Switch to reverse gear and stay in this mode until angle to track axis is halved or until stuck again

14. Parameter Optimization

15. Parameter Optimization Optimization Algorithm: • CMA – Covariance Matrix Adaptation • Smart(er) search for the best solution. Takes in account the dependencies between the parameters Evolution Process: • All parameters were optimized on various TORCS tracks. • All sets of parameters were compared on all tracks. • The most general parameter set was chosen.

16. Parameter Optimization – On Road Fitness Function: 1 / (distance raced + 1)

17. Parameter Optimization – On Road Interesting findings • The most general set, wins only in four of the tracks (not that general). • Second set with most wins, is only rated 6th – different behavioral strategies suite for different track types. • Best strategy for a track isn’t always the strategy that was optimized on that track – local optimum. • Differences between worst and best performance on each track – hard to get a general strategy

18. Parameter Optimization – On Road Blast from the past: • some strategies control this formula with P1 and some with P2.

19. Parameter Optimization – Off Road Why not the same as on-road? • Not the same Fitness function. If the car controller is good, then the car will not reach “off-road” Solution • Crash Strategy: every 300 meters causes the car to go off the road. • Now the same fitness function can be used.

20. Strategy Modifications

21. Strategy Modifications – Gear Shifting Shift up: • Each time the engine reaches 9500 RPM Shift down: • Each time the engine drops bellow: • 3300, 6200, 7000, 7300, 7700 • For gears: 2, 3, 4, 5, 6 respectively

22. Strategy Modifications – Large Track The problem: • The target angle is interpolated between the maximal distance sensor and its neighbors, causing the to drive on slightly wavy trajectory. Solution: • Measuring the track width at the beginning of the race. If it exceeded a hand-set threshold, some steering factors were set by hand instead of the evolved ones.

23. Strategy Modifications – 2nd Lap Switching Strategies: • Analyze the general properties of the track and switch to more suitable strategy. Crash Point: • Remembering crash points from the first lap, in the next lap the car would go into “passive mode”.

24. QUESTIONS ?