Behavior design

1. Behavior design CS 395: Behavior-based systems Ian Horswill

2. Previously • Programs  Policies • Determine action from world state rather than from internal program counter • Building simple control loops from feedback • Compare measured state and desired state • Action is f(state error) • Linear feedback is when f is multiplication by a constant (or constant matrix)

3. Outline • Behaviors and naïve behavior-based systems • Simulating a frog • Motor schemas and potential fields

4. Composing policies from behaviors • Behavior = policy + trigger • Naïve behavior-based system • Run lots of behaviors in parallel • Mutually exclusive triggers • Use policy of currently triggered behavior

5. Common types of behaviors • Orientation behaviors • Tropisms • Followers • Avoidance behaviors • Ballistic actions (fixed action patterns)

6. GRL code for behaviors • Behavior data type • Activation level • The triggerBoolean or numerical • Motor-vector • Implementation-dependent • drive-base modified • Uses the policy of the leftmost currently active behavior Constructor:(behavior activation-levelmotor-vector) behaviorAccessors:(activation-level behavior) behavior’s trigger (motor-vector behavior) output of behavior’s policy (drive-base behavior behavior …)

7. Outline • Behaviors and naïve behavior-based systems • Simulating a frog • Motor schemas and potential fields

8. Frog feeding behaviors • Three separate neural circuits • Orientation behavior • Triggered by black dot moving on a white background (i.e. fly) • Turns body toward prey • Pouncing • Moves body within range of prey • Eating • Ballistic motion of the tongue

9. Simplified robot frog • Motor vector • Rotation/translation control • Eat? fires the tongue • Tongue is not steerable • (we won’t simulate hopping) • Sensors • Detect flies using motion on the retina • Doesn’t work when frog is moving! • Vestibula (inner ear) • Measures rotation like a gyroscope (frog-motor-vector motion eat?) control info for “base” fly-motion direction of fly, if visible doesn’t work when frog moves see-prey? true, if fly is visible rotational-velocity  current rotation speed of frog

10. Simulated feeding behaviors • Eat • Fires the tongue when facing prey • Face prey • Steers toward prey when visible • Inactive when already facing prey • (no pouncing) (drive-base eat face-prey)

11. The eat behavior (define-signal facing-prey? (< (abs fly-motion) facing-prey-tolerance)))(define-signal sit-still (rt-vector 0 0)) (define-signal eat (behavior (and see-prey? facing-prey?) (frog-motor-vector sit-still #t))) Activation level (trigger):Wait until prey in sights Control policy: fire tongue while standing still

12. The face-prey behavior (define-signal facing-prey? (< (abs fly-motion) facing-prey-tolerance)))(define-signal sit-still (rt-vector 0 0)) (define-signal face-prey (behavior (and see-prey?(not facing-prey?)) (frog-motor-vector (rt-vector (* fly-motion face-prey-gain) 0) #f))) P-controller

13. Problem • Fly-motion sensor detects relativemotion of frog and fly • Stops responding as soon as the frog moves • Need to remember the fly’s position somehow (define-signal face-prey (behavior (and see-prey? (not facing-prey?)) (frog-motor-vector (rt-vector (* fly-motion face-prey-gain) 0) #f)))

14. Solution: dead reckoning (define-signal frog-orientation (integrate rotational-velocity)) (define-signal prey-orientation (latch (+ frog-orientation fly-motion) (zero? rotational-velocity))) (define-signal prey-heading (subtract-angle-degrees frog-orientation prey-orientation)) (define-signal face-prey (behavior (and see-prey? (not facing-prey?)) (frog-motor-vector (rt-vector (* prey-heading face-prey-gain) 0) #f)))

15. Outline • Behaviors and naïve behavior-based systems • Simulating a frog • Motor schemas and potential fields

16. Designing behaviors using potential fields Avoidance behavior • Suppose: • Your state is your position in space • Your action is a velocity vector • General technique for constructing policies: • Assign every point in space x a “value” V(x) • Use the gradient V(x) as your policy • Analogy to physics • V(x) is like a force field • V(x) is its potential Approach behavior (tropism)

17. Advantages ofpotential field methods • Simple to compute • Can combine policies by adding value functions (same as adding gradients)

18. “Motor schemas” • Like potential fields but we program directly in terms of the vectors – no value function • One schema (vector field) per behavior • Sum outputs of behaviors

19. Example: sonar-based obstacle avoidance • First, computeobstacle positions • 16 equally spaced sonars • So sonar n is at orientation 2n/16 • So obstacle is at:r(cos 2n/16, sin 2n/16) • where r is the distancereading for the sonar (define-signal sonar-count 16) (define-signal (sonar-direction s-num) (/ (* 2 pi s-num) sonar-count))(define-signal (unit-vector direction) (xy-vector (cos direction) (sin direction))) (define-signal (obstacle-vector s-num reading) (* (unit-vector (sonar-direction s-num)) reading)) positive rotation X-axis and direction of motion

20. Vectors in GRL • Like arrays in C • Constructors • (vector a b c)  [a b c] • (make-vector k elt)  [elt elt … elt] k times • (index-generator i)  [0 1 2 … i] • Accessors • (vector-ref v i)  i’th element of v • (vector-length v)  number of elements in v

21. Implicit mapping in GRL • Scheme and C++ STL let you map a function over a list: • (map f (list a b c d)) (list (f a) (f b) (f c) (f d)) • GRL automatically maps functions over groups and vectors: • (f (vector a b c d)) (vector (f a) (f b) (f c) (f d)) • (* 2 (vector 1 2 1 3)) (vector 2 4 2 6) • (* (rt-vector a b) (rt-vector c d)) (rt-vector (* a c) (* b d)) • Exceptions: length, vector-ref, etc.

22. Example: sonar-based obstacle avoidance • Each obstacle (i.e. each sonar reading) generates a force • Opposite in direction to the obstacle • Magnitude is inverse square of distance • Like electrostaticforce (define-signal (obstacle-force ob-vector) (- (/ ob-vector (cube (magnitude ob-vector))))) (define-signal obstacle-vectors (obstacle-vector (index-generator sonar-count))) (define-signal obstacle-forces (obstacle-force obstacle-vectors)) (define-signal total-force (vector-reduce + 0 obstacle-forces))

23. Following the xy-vector • Problem • Policy generates Cartesian (xy) velocities • Base controlled with polar (rotate/translate) velocities • Most xy vectors can’t be executed directly by the base • Need to transform Cartesian policy into a polar policy • Lots of techniques possible. Here are two. ;; CMU policy (Stone et al.)(define-signal (follow-xy vec) (rt-vector (y-of vec) (x-of vec)) ;; GATech policy (Arkin et al.) (define-signal (follow-xy vec) (if (< (abs (y-of vec)) threshold) (rt-vector 0 speed) (rt-vector (* (y-of vec) r-gain) 0)))

24. Local minima • Different forces from different objects can exactly cancel, leaving the robot stuck • Work-arounds • Unwedging • Noise • Add random vectors to motion • Avoid-recent behavior • Remember recent locations • Generate repulsive force away from them • Path planning • Run a search algorithm to find a path through freespace