Common Sensing Techniques for Reactive Robots (12-11-7)

1. Common Sensing Techniques for Reactive Robots (12-11-7) Sungmin Lee (이성민) Division of Electronic Engineering, Chonbuk National University Intelligent Systems & Robotics Lab. http://robotics.jbnu.ac.kr

2. Chapter objectives • Describe difference between active and passive sensors • Describe the types of behavioral sensor fusion • Define each of the following terms in one or two sentences: proprioception, exteroception, exproprioception,proximity sensor, logical sensor, false positive, false negative, hue, saturation, computer vision • Describe the problems of specular reflection, cross talk, foreshortening, and if given a 2D line drawing of surfaces, illustrate where each of these problems would be likely to occur • If given a small interleaved RGB image and a range of color values for a region, be able to 1) threshold on color and 2) construct a color histogram

3. Contents • Logical sensors • Behavioral Sensor Fusion • Attributes of a sensor • Sensor Categories • Computer vision • Case study • Summary

4. Motivation • Sensing is tightly coupled with acting in reactive systems, so need to know about sensors • What sensors are out there? • Ultrasonics, cameras are traditional favorites • Sick laser ranger is gaining fast in popularity • How would you describe them (attributes)? • How would you decide which one to pick and use for an application?

5. Logical Sensors • A unit of sensing or module (supplies a particular percept). • It consists of the signal processing and the software processing. • Can be easily implemented as a perceptual schema. • Different sensors/perceptual schemas can produce the same percept - motor schema doesn’t care! • Behavior can pick what’s available • Example: ring of IRs, ring of sonars • If sensor fails, then another can be substituted without deliberation or explicit modeling • Conflicts in allocation can be solved by using logical sensors (deliberation is required to assign)

6. Active vs. Passive (Example) • Active sensors • - Sensor emits some form of energy and then measures the impact as a way of understanding the environment • - Ex. Ultrasonics, laser • Passive sensors • - Sensor receives energy already in the environment • - Ex. Camera • Passive consume less energy, but often signal-noise problems • Active often have restricted environments Thermal sensor Stereo Camera pair Laser ranger Sonars Bump sensor

7. Behavioral Sensor Fusion Sensor fusion is a broad term used for any process that combines information from multiple sensors into a single percept. In some cases multiple sensors are used when a particular sensor is too imprecise or noisy to give reliable data. Adding a second sensor can give another “vote” for the percept. When a sensor leads the robot to believe that a percept is present, but it is not, the error is called a false positive. The robot has made a positive identification of percept, but it was false. Likewise, an error where the robot misses a percept is known as a false negative. False positive False negative

8. Sensing Model 11 Sensor/Transducer Behavior Action

9. Sensing in Reactive Paradigm Behavior Behavior Behavior Each behavior has its own dedicated sensing. One behavior literally does not know what another behavior is doing or perceiving.

10. Behavioral Sensor Fusion: -sensor fission Motor Schemas Perceptual Schemas This sensor fission in part as a take off on the connotations of the word “fusion” in nuclear physics. In nuclear fusion, energy is created by forcing atoms and particles together, while in fission, energy is creating by separating atoms and particles.

11. Behavioral Sensor Fusion: -action-oriented sensor fusion Motor Schema Perceptual Schema This type of sensor fusion is called action-oriented sensor fusion to emphasize that the sensor data is being transformed into a behavior-specific representation in order to support a particular action, not for constructing a world model.

12. Behavioral Sensor Fusion: -sensor fashion Motor Schema Perceptual Schema Sensor fashion, an alliterative name intended to imply the robot was changing sensors with changing circumstances just as people change styles of clothes with the seasons.

13. Designing a Sensor Suite-Attributes of a sensor • Field of view, range : does it cover the “right” area • Accuracy & repeatability : how well does it work? • Responsiveness in target domain : how well does it work for this domain? • Power consumption : may suck the batteries dry too fast • Reliability : can be a bit flakey, vulnerable • Size : always a concern! • Computational Complexity : can you process it fast enough? • Interpretation Reliability : do you believe what it’s saying?

14. Designing a Sensor Suite-Attributes of a sensor suite Should be considered for the entire sensing suite : • Simplicity • Modularity • Redundancy - physical redundancy (there are several instances of physically identical sensors on the robot.) • logical redundancy (another sensor using a different sensing modality can produce the same percept or releaser.) - fault tolerance

15. Sensor Categories • Proprioceptive • Inertial Navigation System(INS) • Global Positioning System(GPS) • Exteroceptive • Proximity • Range • Contact • Computer Vision

16. Proprioceptive Sensors(1)-Inertial navigation system (INS) MQ-9 Reaper Measure movements electronically through miniature accelerometers INS can provide accurate dead reckoning to 0.1 percent of the distance traveled. However, this technology is unsuitable for mobile robots for several reasons.(cost, Size, etc)

17. Proprioceptive Sensors(2)-Global Positioning System (GPS) GPS systems work by receiving signals from satellites orbiting the Earth. GPS is not complete solutions to the dead reckoning problem in mobile robots. GPS does not work indoors(environmental limit)

18. Proximity Sensors(1)-Sonar or ultrasonic • Sonar refers to any system for using sound to measure range. (use a sonar for underwater vehicles ). • Ground vehicles commonly use sonar with an ultrasonic frequency. • Ultrasonic sensors generate high frequency sound waves and evaluate the echo which is received back by the sensor. Sensors calculate the time interval between sending the signal and receiving the echo to determine the distance to an object. • Ultrasonic is possibly the most common sensor on commercial robots operating Polaroid ultrasonic transducer

19. Proximity Sensors(1)- Three problems with sonar range readings specular reflection foreshortening cross-talk chairs, tables– legs, edges too thin for resolution

20. Proximity Sensors(1)- sonar maps lab hallway Maps produced by a mobile robot using sonars in: a.) a lab and b.) a hallway. (The black line is the path of the robot.)

21. Proximity Sensors(1)- Attributes of ultrasonic • Power consumption • High • Reliability • Lots of problems • Size • Size of a Half dollar, board is similar size and can be creatively packaged • Computational Complexity • Low; doesn’t give much information • Interpretation Reliability • poor

22. Proximity Sensors(1)- Ultrasonic Summary • Physics : active sensor, works on time of flight • Advantages : range, inexpensive ($30 US), small • Disadvantages : specular reflection, crosstalk, foreshortening, high power consumption, low resolution

23. Proximity Sensors(2)- Infrared ray (IR) They emit near-infrared energy and measure whether any significant amount of the IR light is returned. These often fail in practice because the light emitted is often “washed out” by bright ambient lighting or is absorbed by dark materials (i.e., the environment has too much noise). Sharp GP2Y0A21YK

24. Proximity Sensors(3)- Bump and feeler sensors Popular class of robotic sensing is tactile, or touch, done with bump and feeler sensors. The sensitivity of a bump sensor can be adjusted for different contact pressures Roomba 500 Bump

25. Computer Vision- Definition Computer vision refers to processing data from any modality which uses the electromagnetic spectrum which produces an image. face recognition

26. Computer Vision- Attributes • Physics : light reflecting off of surfaces, respond to wavelenght • Field of view, range : depends on lens; lens typical have a different VFOV and HFOV (vertical, horizontal) • Accuracy & repeatability : good • Responsiveness in target domain : depends on lighting source, inherent constrast between objects of interest • Power consumption : low • Reliability : good • Size : can be miniaturized • Interpretation Reliability : good

27. Computer Vision- CCD cameras • A charge-coupled device (CCD) is a device for the movement of electrical charge, usually from within the device to an area where the charge can be manipulated, for example conversion into a digital value. • CCD sensors typically produce less NOISE. • CCD sensors typically are more light-sensitive. • CMOS sensors use far less power. • CMOS sensor cost less to produce.

28. Computer Vision- Color planes • RGB (red, green, blue) is the NTSC output • Poor color constancy in “real world” • H,S,I (hue, saturation, intensity) has theoretical color constancy • But not with conversion from RGB to HSI • Alternatives SCT (Spherical Coordinate Transform) • That color space was designed to transform RGB data to a color space that more closely duplicates the response of the human eye. Original image RGB HSI SCT

29. Computer Vision- Common Vision Algorithms • For reactive applications: • Color segmentation • Imprint on a color region, then follow it • (or remember it) • Color histogramming • Imprint on a region with a distribution of color, then follow it (or remember it)

30. Range from Vision-Stereo camera pairs • Using two cameras to extract range data is often referred to as range from stereo, stereo disparity, binocular vision, or just plain “stereo.” One way to extract depth is to try to superimpose a camera over each eye. • Each camera finds the same point in each image, turns itself to center that point in the image, then measures the relative angle. The cameras are known as the stereo pair. Ways of extracting depth from a pair of cameras

31. Range from Vision-Light stripers Light striping, light stripers or structured light detectors work by projecting a colored line (or stripe), grid, or pattern of dots on the environment. Then a regular vision camera observes how the pattern is distorted in the image.

32. Range from Vision-Laser ranging(Sick) • Accuracy & repeatability - Excellent results • Responsiveness in target domain • Power consumption • - High; reduce battery run time by half • Reliability - good • Size - A bit large • Computational Complexity • Not bad until try to “stack up” • Interpretation Reliability • Much better than any other ranger flat surface an obstacle a negative obstacle SICK PLS100

33. Range from Vision-Laser Ranger Summary • 180o plane • Advantages: high accuracy, coverage • Disadvantages: 2D, resistant to miniaturization, cost ($13,000 US) NASA/CMU Nomad robot (Carnegie Mellon University )

34. Case Study : Hors d’Oeuvres, Anyone? (Borg Shark and Puffer Fish) Digital thermometer: “Face” temperature check Camera pair (redundant): Face color Laser range: Count treat removal If blocked, Puffed up Sensor fusion: Reduced False positives, False negatives From 27.5% to 0% Sonars: Avoid obstacles,

35. State diagrams for the Borg Shark shaft encoders sonar thermal map: evidence grid at waypoint OR food removed vision serving food: finding faces, counting treat removal waypoint navigation: move to goal, avoid laser range time limit serving exceeded sonar food depleted full tray awaiting refill: finding faces vision thermal

36. Summary • Design of a sensor suite requires careful consideration • Almost all robots will have proprioception, but exteroception needs to be closely matched to the task and the environment • Most common exteroceptive sensors on mobile robots are: • Ultrasonics • Computer vision • Laser range • Color vision can be hard, almost all vision is computationally expensive unless focus on affordances • Borg shark and Puffer fish with color plus heat • Polly and texture

37. Thank you