Biological Foundations of the Reactive Paradigm

1. Biological Foundations of the Reactive Paradigm • Describe the three levels in a Computational Theory. • Explain in one or two sentences each of the following terms: reflexes, taxes, fixed-action patterns, schema, affordance. • Be able to write pseudo-code of an animal’s behaviors in terms of innate releasing mechanisms, identifying the releasers for the behavior. • Given a description of an animal's sensing abilities, its task, and environment, identify an affordance for each behavior. • Given a description of an animal's sensing abilities, its task, and environment, define a set of behaviors using schema theory to accomplish the task. Review Why? -comp. theory IRM Perception -Summary Chapter 3: Biological Foundations

2. Robots In the Hierarchical Paradigm Chapter 3: Biological Foundations

3. Timeline of Influences Braitenberg’s Vehicles Arkin’s Schemas Neisser Arbib’s Schemas J.J. Gibson Payton Brook’s Insects Middlestat Marr’s Computational Theory Tinbergen & Lorenz & von Frisch 1970 1980 1990 Chapter 3: Biological Foundations

4. Level 1: What is the phenomena we’re trying to represent? Level 2: How it be represented as a process with inputs/outputs? for (i=nCol.. Level 3: How is it implemented? Marr’s Computational Theory Chapter 3: Biological Foundations

5. Level 1: What is the phenomena we’re trying to represent? Level 1: Existence Proof Goal: how to make line drawings of objects? people can do this by age 10, computers should Chapter 3: Biological Foundations

6. Level 2: How it be represented as a process with inputs/outputs? for (i=nCol.. Level 2: Inputs, Outputs, Transforms drawing light lines (edges) drawing light retina (gradient) Chapter 3: Biological Foundations

7. -1 0 +1 0 -2 +2 +1 +2 +1 -1 0 +1 0 0 0 -1 -2 -1 Sobel Edge Detector in computer vision Level 3: How is it implemented? Level 3: Implementation - - - - - + - - - Center Surround Cell in retinal ganglion Chapter 3: Biological Foundations

8. Class Discussion • Give three examples of how biology has influenced modern technology? • ex. Wright Brothers- control flaps on airplane wings Chapter 3: Biological Foundations

9. Behavior Definition (graphical) BEHAVIOR Pattern of Motor Actions Sensory Input Chapter 3: Biological Foundations

10. Types of Behaviors • Reflexive • stimulus-response, often abbreviated S-R • Reactive • learned or “muscle memory” • Conscious • deliberately stringing together WARNING Overloaded terms: Roboticists often use “reactive behavior” to mean purely reflexive, And refer to reactive behaviors as “skills” Chapter 3: Biological Foundations

11. Reflexive behaviors • Reflexes - lasts as long as the stimulus only, • Taxes - moves in a particular direction (tropotaxis in baby turtles, chemotaxis in ants), • Fixed-action patterns - continues for a longer duration than the stimulus. Chapter 3: Biological Foundations

12. Ethology: Study of Animal Behaviors Nobel 1973 in physiology or medicine • von Frisch • Lorenz • Tinbergen INNATE RELEASING MECHANISMS www.nobel.se Chapter 3: Biological Foundations

13. Ways to acquire a behavior • To be born with a behavior (innate) • To be born with a sequence of innate behaviors (sequence of innate behaviors) • To be born with behaviors that need some initialization (innate with memory) • To learn a set of behaviors Chapter 3: Biological Foundations

14. Arctic terns live in Arctic (black, white, gray environment, some grass) but adults have a red spot on beak When hungry, baby pecks at parent’s beak, who regurgitates food for baby to eat How does it know its parent? It doesn’t, it just goes for the largest red spot in its field of view (e.g., ethology grad student with construction paper) Only red thing should be an adult tern Closer = large red Arctic Terns Chapter 3: Biological Foundations

15. behavior template BEHAVIOR Pattern of Motor Actions Sensory Input Chapter 3: Biological Foundations

16. “the feeding behavior” Feeding BEHAVIOR RED PECK AT RED Pattern of Motor Actions Sensory Input Chapter 3: Biological Foundations

17. Releaser present? N /dev/null Y the releaser template Sensory input and/or internal state Chapter 3: Biological Foundations

18. Example IRM in pseudo-code(assume synchronous processing, update occurs every N ms or “1 step”) enum Releaser={PRESENT, NOT_PRESENT};Releaser predator;while (TRUE){ predator = sensePredators(); //perception of releaser if (predator == PRESENT) //releaser flee(); //behavior} Perception of predator “persists” even if not directly in sight leading to a Fixed-Pattern Action style of behavior Chapter 3: Biological Foundations

19. “the feeding releaser” RED & HUNGRY sensory input Releaser internal state present? N /dev/null Y Feeding BEHAVIOR RED PECK AT RED Chapter 3: Biological Foundations

20. Compound Releaser enum Releaser={PRESENT, NOT_PRESENT};Releaser food;while (TRUE){ food = senseFood(); //perception of releaser Part 1 hungry = checkState(); //perception of internal state if (food == PRESENT && hungry==PRESENT) //releaser feed();} Must be hungry AND sense food to feed Chapter 3: Biological Foundations

21. Implicit Sequence enum Releaser={PRESENT, NOT_PRESENT};Releaser food, hungry, nursed;while (TRUE) { food = sense(); hungry = checkStateHunger(); child = checkStateChild(); if (hungry==PRESENT) searchForFood(); //sets food = PRESENT when done if (hungry==PRESENT && food==PRESENT) feed(); // sets hungry = NOT_PRESENT when done if (hungry== NOT_PRESENT && parent==PRESENT) nurse(); // set nursed = PRESENT when done if (nursed ==PRESENT) sleep();} Why is this implicit, not explicit? If asynchronous, could have conflicts Chapter 3: Biological Foundations

22. Releaser present? N /dev/null Y Innate Releasing Mechanisms Sensory input and/or internal state BEHAVIOR Pattern of Motor Actions Sensory Input Chapter 3: Biological Foundations

23. Example: Hide Behavior • shows • taxis (oriented relative to light, wall, niche) • fixed action pattern (persisted after light was off) • reflexive (stimulus, response) • implicit sequencing • use of internal state Chapter 3: Biological Foundations

24. Example: Cockroach Hide • light goes on, the cockroach turns and runs • when it gets to a wall, it follows it • when it finds a hiding place (thigmotrophic), goes in and faces outward • waits until not scared, then comes out • even if the lights are turned back off earlier Chapter 3: Biological Foundations

25. Reflexive Behaviors S-R • light goes on, the cockroach turns and runs • when it gets to a wall, it follows it • when it finds a hiding place (thigmotrophic), goes in and faces outward • waits until not scared, then comes out • even if the lights are turned back off earlier Chapter 3: Biological Foundations

26. Fixed Pattern Actions • light goes on, the cockroach turns and runs • when it gets to a wall, it follows it • when it finds a hiding place (thigmotrophic), goes in and faces outward • waits until not scared, then comes out • even if the lights are turned back off earlier Chapter 3: Biological Foundations

27. Exhibits Taxis • light goes on, the cockroach turns and runs • when it gets to a wall, it follows it • when it finds a hiding place (thigmotrophic), goes in and faces outward • waits until not scared, then comes out • even if the lights are turned back off earlier to light to wall to niche Chapter 3: Biological Foundations

28. Class Exercise • Draw flowchart of how this works • light goes on, the cockroach turns and runs • when it gets to a wall, it follows it • when it finds a hiding place (thigmotrophic), goes in and faces outward • waits until not scared, then comes out • even if the lights are turned back off earlier Chapter 3: Biological Foundations

29. Break into Behaviors Flee • light goes on, the cockroach turns and runs • when it gets to a wall, it follows it • when it finds a hiding place (thigmotrophic), goes in and faces outward • waits until not scared, then comes out Follow- wall hide Chapter 3: Biological Foundations

30. LIGHT LIGHT present? present? N N Find Releasers Y Flee • light goes on, the cockroach turns and runs • when it gets to a wall, it follows it • when it finds a hiding place (thigmotrophic), goes in and faces outward • waits until not scared, then comes out Follow- wall SCARED & SURROUNDED Ooops, need internal state: Scared present? N hide Chapter 3: Biological Foundations

31. LIGHT present? N Internal State Set Y Flee • light goes on, the cockroach turns and runs • when it gets to a wall, it follows it • when it finds a hiding place (thigmotrophic), goes in and faces outward • waits until not scared, then comes out SCARED BLOCKED & SCARED present? N Follow- wall SCARED & SURROUNDED present? N hide Chapter 3: Biological Foundations

32. LIGHT present? N Action steer 180, drive forward Y Flee • light goes on, the cockroach turns and runs • when it gets to a wall, it follows it • when it finds a hiding place (thigmotrophic), goes in and faces outward • waits until not scared, then comes out SCARED BLOCKED & SCARED present? N steer =F(dist to wall) drive forward const. Follow- wall SCARED & SURROUNDED present? steer =F(dist to wall) drive forward const. stop N hide Chapter 3: Biological Foundations

33. LIGHT present? N IR Sensory Input steer 180, drive forward Y encoders Flee • light goes on, the cockroach turns and runs • when it gets to a wall, it follows it • when it finds a hiding place (thigmotrophic), goes in and faces outward • waits until not scared, then comes out SCARED BLOCKED & SCARED IR present? N steer =F(dist to wall) drive forward const. IR Follow- wall SCARED & SURROUNDED IR present? steer =F(dist to wall) drive forward const. stop N IR hide Chapter 3: Biological Foundations

34. LIGHT present? N IR How Do You Link Them? steer 360, drive forward Y encoders Flee • light goes on, the cockroach turns and runs • when it gets to a wall, it follows it • when it finds a hiding place (thigmotrophic), goes in and faces outward • waits until not scared, then comes out SCARED BLOCKED & SCARED IR present? N steer =F(dist to wall) drive forward const. IR Follow- wall SCARED & SURROUNDED IR present? steer =F(dist to wall) drive forward const. stop N IR hide Chapter 3: Biological Foundations

35. Analogy:IRMs work on THREADS,not sequential processing! • Very simple modules • Nice building blocks since not directly linked • If one module (part of brain) fails, what happens? Chapter 3: Biological Foundations

36. What happens when there’s a conflict from concurrent behaviors? • Equilibrium • Feeding squirrels: feed or flee - hesitate in-between • Dominance • Sleepy, hungry: either sleep or eat • Cancellation • Sticklebacks : defend or attack - build a nest instead ? Chapter 3: Biological Foundations

37. Inhibition while (TRUE) { predator = sensePredator(); //has a time delayif (predator==PRESENT) //as long as predator persists flee();else { food = senseFood(); hungry = checkStateHunger(); ... }} Could also be done as an interrupt Chapter 3: Biological Foundations

38. World Acts & Modifies World Samples, Finds Potential Actions Directs what to look for Cognitive Activity Perception of Environment Perception • Two uses of perception (can be the same percept) • Release a behavior • Guide a behavior • Action-oriented perception (Neisser) • Planning is not needed to act • Perception is selective Chapter 3: Biological Foundations

39. Gibson’s Ecological Approach • Acting and sensing co-evolved as agent survived in a particular environment. The environment affords the agent what it needs to survive. • The perception needed to release or guide the “right action” is directly in the environment, not inferred or memorized • Ex. Red on Artic Terns== food • Ex. Sound of filling container==full • Percepts are called affordances or said to be obtained through direct perception Chapter 3: Biological Foundations

40. Gibsonian Affordances • How do you know you’re going fast in a car? Or in a space movie? • How do animals know when to mate? • How do mosquitoes know to bite in the most tender areas? • What should you do when you think you’re being stalked by a mountain lion? • What’s your favorite fishing lure? Chapter 3: Biological Foundations

41. Sittability Chapter 3: Biological Foundations

42. But does this really hold for everything? • “my car” versus “your car”? • Difference is • where I parked it (memory) • Semantic meaning (cars aren’t generic like nuts to a squirrel) • Neisser’s Two Systems • Direct Perception: older, behavioral • Recognition: evolved later, deliberative Chapter 3: Biological Foundations

43. Review Questions • What are the levels of a computational theory? • Existence proof, inputs-outputs-transformations, implementation • What is a behavior? • A behavior is a mapping of sensory inputs to a pattern of motor actions • Is sequencing normally implicit or explicit in IRM? • implicit • What is an affordance? • A potentiality in the environment for an action Chapter 3: Biological Foundations

44. Schema Theory schema- is used in cognitive science and ethology to refer to a particular organized way of perceiving cognitively and responding to a complex situation or set of stimuli • is generic, equivalent to an object in OOP • schema specific knowledge (local data) • procedural knowledge (methods) • schema instantation is specific to a situation, equivalent to an instance in OOP • a behavior is a schema, consists of • perceptual schema • motor schema Chapter 3: Biological Foundations

45. Behavioral Schema alternative PS, MS sequencing logic for reactive skills (judgment value function) action, intensity percept, gain Perceptual Schema (PS) Motor Schema (MS) Reflexive behaviors usually just have “methods”, not data Chapter 3: Biological Foundations

46. Ex. Fly Snapping Behavior IRM Releaser: small moving dark blob present? N /dev/null Y snap, 100% x,y,z, 100% track(blob) snap(blob) Chapter 3: Biological Foundations

47. Schema Instantiation (SI) Releaser: small moving dark blob present? N /dev/null Y snap, 100% x,y,z, 100% track(blob) snap(blob) Chapter 3: Biological Foundations

48. releaser Perceptual Schema Library Motor Schema Library 1. track(blob) snap(blob) present? 2. track(blob) snap(blob) N track(blob) snap(blob) behavior schema 3. x,y,z, 100% track(blob) snap(blob) Schema/Schema Instantiation Chapter 3: Biological Foundations

49. Advantages • modular • can assemble new behaviors from existing schemas • learning by experimentation • can substitute alternatives • reroute nerves Chapter 3: Biological Foundations

50. Snap at vector sum (middle) Instantiation for each eye Releaser: small moving dark blob present? Left eye N /dev/null Y snap, 100% x,y,z, 100% track(blob) snap(blob) Releaser: small moving dark blob present? N /dev/null Right eye Y snap, 100% x,y,z, 100% track(blob) snap(blob) Chapter 3: Biological Foundations