Control Behaviors of Mobile Robots

1. Control Behaviors of Mobile Robots Architectures and Behaviors With special interest in hexapods and soccer

2. Pets -- expensive and not Sony Aibo $1500 NEC R100 $ a lot iCybie $160 Tekno $40 Poo-chi $30

3. Pets -- expensive and not Sony Aibo $1500 NEC R100 $ a lot iCybie $160 Tekno $40 Poo-chi $30 • voice recog. • 18 motors • wireless LAN • color camera • walks, gets up • voice recog. • 6 sonars et al. • wireless LAN • 2 color cameras • rolls around • voice recog. • 16 motors • wireless LAN • walks… ? • sound recog. • 2 motors • light sensor • walks • sound recog. • 1 motor • pivots

4. dogfight

5. You can find this material in Norvig’ book, chapter 2 2.1 Perception and Action • Perception • Action • Boolean Algebra • Classes and Forms of Boolean Functions 2.2 Representing and Implementing Action Functions • Production Systems • Networks • The Subsumption Architecture

6. Other material covered • Control models • the sense-think-act control cycle • model-based controllers • Reactive robotics • reactive controllers • behaviour-based controllers • Other approaches • other reactive controllers • hybrid controllers • learning robots

7. Control of Hexapod Robots • 1. Goals and overview • 2. Brooks’ Subsumption • 3. Biologically Inspired • 4. Logically Inspired • 5. Neural and Immunological Networks • 6. Hardware issues

8. Legged Locomotion From the ONR proposal • Make use of simple nervous systems involved in sensorimotor performance to specify the circuits involved in adaptive legged locomotionin invertebrates. • The objectives of this program are to develop: • neural models that reproduce the experimentally derived characteristics of the neuronal circuitry underlying legged locomotion • new controller architectures for autonomous legged robots: • which exhibit stable locomotion, • and which respond adaptively to unknown terrain and obstacles.

9. Legged Locomotion • The research issues include: • biomechanics of dynamically stable locomotion, • specification of local leg control circuits, • including central pattern generators, • the neural mechanisms of adaptation to context and task, • implementation of neural models as real time controllers on prototype hexapod robots

10. M2 is a 3D bipedal walking robot which is currently being developed in the MIT Leg Laboratory. • The robot has 12 active degrees of freedom: 3 in each hip, 1 in each knee, and 2 in each ankle. • It will be used to investigate • Various walking algorithms • Motion description and control techniques, particularly Virtual Model Control • Force control actuation techniques, particularly Series Elastic Actuation • Automatic learning techniques. • The goals of M2 are the following: • Walk fast (1.0 meters/second) • Walk efficiently • Be reliable (work 9 out of 10 attempts) • Have a large margin of stability and be robust to small disturbances (reasonable pushes) • Be confident looking • Become a "robotic workhorse" - • a robot which can be reliably used to perform experiments without breaking • Get it right the first time in design and schedule Example of walker

11. Biologically Inspired

12. Past Control Architectures for Hexapods • 1. Marc Donner, 1986, decomposition of task, locality of control, special language • 2. Rodney Brooks, MIT, 1989 - Subsumption Architecture • 3. Yuasa and Ito, Nagoya, Japan, 1990 - decentralized autonomous system based on generators. Biologically inspired. • 4. Randall Beer and John Gallagher, 1992 - genetic algorithm to develop a neural net to control a simulated cockroach. • 5. Graham Spencer, 1992 - genetic programming and a priori knowledge, not real robot.

13. Past Control Architectures for Hexapods • 6. Gary Parker, 1992, Indiana Cyclic Genetic Algorithm with a priori knowledge, simulation. • 7. Salama, Hingston, University of Western Australia, 1995, GA develops NN. Stiquito. • 8. Berns, Cordes, Karlsruhe, 1995 - NN, real robot, small. • 9. Ishiguro, Kuboskiki, Ichikawa and Uchikawa, Nagoya, Japan, 1996-1998 - Immune Networks. Simulation. • 10. Gallagher, Beer, Espenschied, Quinn, Case, 1996. Dynamical neural network. Real robot. • 11. Parker, Braun and Cyliax, 1997. CGA for Stiquito Robot. • 12. Ilg, Muhlfriedel and Berns, Karlsruhe, Germany, 1997, Self-Organizing Neural Net

14. Past Control Architectures for Hexapods • 13. Eng, Robertson and Blackman, Monash, Australia, 1997-1999 - Layered Learning System architecture, agents, real robot. • 14. Barnes, Wardle, University of Salford, UK, 1999 - reactionary system with central pattern generators • 15. Ahuja, Nelson, Delcomyn, Hart, University of Illinois, 1997 - 2000 (?) - Biologically Inspired • 16. Reinforcement Learning • 17. Levy, PSU, 2001, Generators controlled by Constructive Induction, robot walks but not completed. • 18. Zielinska, Singapur, Poland, 2001 - current - big, network of parallel processors and controllers, sensors.

15. Conclusion on past research on hexapod control • Based on my limited literature search: • 1. The total of walking robots in the world estimated to be 200, not counting toys and kits • 2. Not much recent work, recently more on bio-inspired • 3. Even less work on real robots • 4. Especially on controlling large hexapods • 5. Few papers on sensors and sensors integration • 6. Relatively few papers from USA • 7. Not much practical evaluation and description of experiments.

16. Past research on mobile robot controllers • The “brains” behind a mobile, autonomous robot (using the “lite” definition of autonomous) • Even though the controller is sometimes not physically on the robot • Three schools: • The Old School:Sense-Plan-Act (Shakey at Stanford) • The Brooks School: Subsumption • The Modern School: Three-Layer • New Approaches: • Bio-Inspired • Probabilistic

17. Overview: Robot Control Schemes Representative examples Representative examples • RAPs, Firby `87 • AuRA, GeorgiaTech, Arkin `87 • Atlantis, JPL/NASA, Gat `92 • Saphira, SRI, Saffiotti et al, `93 • TCA, CMU, Simmons `94 • AMOR, Pirjanian et al. `94 • ALLIANCE, Parker `94 • DAMN, CMU, Rosenblatt & Payton 96 • 3T, NASA, Kortenkamp et al `96 Representative examples • Subsumption Arch, MIT, Brooks `86 • Motor schemas, GeorgiaTech, Arkin `87 • Activation nets, MIT, Maes `89 • Dynamical sys., Schoner `92 • MOBC, Pirjanian `97 • Stanford Cart, Moravec 1977 • NASREM, NASA/NIST, Albus et al 1987 • VINAV, Christensen et al 1992

18. Control Problems for an autonomous robot • Rapidly changing boundary conditions • Real-time response • Information collected over noisy channels

19. Perception and Action • Stimulus-response (S-R) agents • Machines that have no internal state and that simply react to immediate stimuli in their environments • Based on motor response to rather simple functions of immediate sensory inputs • Examples: • Machina speculatrix, • Braitenberg machine

20. Stimulus Response Agents

21. Mapping Perception into Action “obstacle” 0 1 0 1 1 0 1 1 Feature detectors SENSORS ACTION FUNCTION MOTORS “wall”

22. Example: Robot navigator Clips N 2 E 1 0 0 1 2

23. A Robot in a Two-Dimensional Grid World (1) • Environment • Enclosed by boundaries • Contains unmovable objects • No tight spaces • Spaces between objects and boundaries that are only one cell wide • Task • Go to a cell adjacent to a boundary or object and then follow that boundary along its perimeter

24. A Robot in a Two-Dimensional Grid World (2) • Sensory inputs: • Robot movements • north moves the robot one cell up in the cellular grid • eastmoves the robot one cell to the right • south moves the robot one cell down • west moves the robot one cell to the left • Division of processes • Perception processing and action computation

25. A Robot in a Two-Dimensional Grid World (3) • Perceptual processing • produces feature vectorX • numeric features: real numbers • categorical features: categories • Action computation • selects an action based on feature vector

26. A Robot in a Two-Dimensional Grid World (4) • The split between perception and action is arbitrary • The split is made in such a way that the same features would be used repeatedly in a variety of tasks to be performed • The computation of features from sensory signals can be regarded as often used library routines • needed by many different action functions • The next problems • (1) converting raw sensory data into a feature vector • (2) specifying an action function

27. Perception example • For the robot task, there are four binary-valued features of the sensory values that are useful for computing an appropriate action • Perceptual processing might occasionally give erroneous, ambiguous, or incomplete information about the robot’s environment • Such errors might evoke inappropriate actions • For robots with more complex sensors and tasks, designing appropriate perceptual processing can be challenging

28. Action example • Specifying a function that selects the appropriate boundary-following action • None of the features has value 1, the robot can move inany direction until it encounters a boundary

29. Boolean Algebra • Boolean algebra is a convenient notation for representing Boolean functions • Rules for Boolean algebra • Commutative • Associative • DeMorgan’s law • Distributive law

30. Classes and Forms of Boolean Functions • Aconjunction of literals or amonomial: • The conjunction itself is called a term • Bound of the number of monomials of size k or less: • A clause or a disjunction of literals: • Terms and clauses are duals of each other • Disjunctive normal form (DNF):disjunctionof terms • K-term DNF: disjunction of k terms • Conjunctive normal form (CNF):conjunctionof clauses • K-clause CNF: the size of its largest clause is k

31. Outline on Reactive Agents • How to design stimulus-response (reactive) agents? • Abstracting sensory values into features • Modes of action • Representation of mapping from sensors to actions • Examples • Layered reactive agents (subsumption architecture)

32. Properties of Reactive Agents • No internal state: current value of sensors determines response • Speed: Reactive agents respond quickly to a new stimulus • Sensor values are mapped into features using boolean(state-free) functions • Actions are computed using an action function

33. Feature Selection • What are good features? • Easy to compute from sensor values • Useful in multiple tasks • Need not be perfect • Help improve task performance • Examples from biology: • Insect detector for a frog • Sun direction and earth’s magnetic field for navigating birds • Imprintingin birds

34. OBELIX: A Robot SENSORS: 8 ultrasonic sonar sensors, 4 long IR’s, 8 short IR’s, 1 very-short-range IR

35. Feature Detectors on OBELIX • “Stuck”: motor current > threshold • “Bumped”: very-short-range IR on • “Intersection”: Long side IR off • “Corridor”: Two long side IR’s on • “Box-detectors”: boolean combinations of FAR and NEAR sonar bits

36. Action Functions Three approaches are the most popular: • 1. Production Systems • List of condition-action rules • Parallelizable for fast matching • 2. (artificial, neural) Networks • Threshold-logic circuits • Noise immunity, learnability • 3. Subsumption architectures • Layered set of mappings

37. Production Systems

38. Production Systems Condition1 Action 1 Action 2 Condition2 Example: RIR = Long right IR LIR = Long left IR

39. Representing and Implementing Action Functions: Production systems (1) • Production system comprises an ordered list of rules called production rules or productions • , where is the condition part and is the action part • Production system consists of a list of such rules • Condition part • Can be any binary-valued function of the features • Often a monomial • Action part • Primitive action, a call to another productive system, or a set of actions to be executed simultaneously

40. Production Systems (2) • Production system representation for the boundary following routine • An example of a durative systems-system that never ends • Teleo-reactive (T-R) programs • Each properly executed action in the ordering works toward achieving a condition higher in the list • Usually easy to write, given an overall goal for an agent • Quite robust: actions proceed inexorably toward the goal • Can have parameters that are bound when the programs are called • Can call other T-R programs and themselves recursively

41. Discussion Issues for robot soccer architectures • Origination • Low Cost vs. Big Model • Elegancy vs. Efficiency

42. Basic Architecture for Soccer Using software simulator as a basic development methodology in soccer robot research Production rules Neural Nets Subsumption

43. Production system based agent architecture for a soccer robot

44. Switch Competitive behaviors Switching and voting, subsumption

45. Teleo-reactive ProgramsNilsson Variant of Production Systems to handle goals and durative behaviors. 1st Rule specifies the goal: goal Nil A rule is placed at a point where it achieves the condition of some higher level rule. The condition “1” means it is always true.

46. Example of a teleo-reactive (TR) Program Exercise: write the wall-follower example as a TR program

47. Black Board Architecture KS communicate through Black Board Knowledge specialist

48. Example: RoboCup98 rule hierarchy Play Normal Attack left Run to wing

49. Labyrinth, Robot Environments and Object-Based Simulation

50. Simulation Simulation is the mimicking of a real system within a computer. A computer model is created to represent part of a real-world environment. The computer model is then executed to investigate real-world behavior as predicted by the computer model.