Sensors

1. Sensors Where am I!? Third topic in proposed class. Advanced but yet does not rely on having seen previous lectures.Third topic in proposed class. Advanced but yet does not rely on having seen previous lectures.

2. Emphasize physical quantity, not state. I REPEAT NO STATE!Emphasize physical quantity, not state. I REPEAT NO STATE!

3. Before sensors? Early Robots did not have sensors They were unable to adapt or cope with new situations Example: An industrial grabbing robot would grab as it programmed no matter what was there. Early robots were sad creatures, incapable of coping with “life” in the world.Early robots were sad creatures, incapable of coping with “life” in the world.

4. Early Unimation Robot General electricGeneral electric

5. Animals can adapt because they have sensors Sensors enable us to gather information about the environment. Animals have eyes, ears, skin, etc. Most creatures process sensory stimuli in a central location. Some have a distributed control network. Example: Octopi Examples of human and animal sensors: eyes, ears, skin, nose, etc. In humans processed in the brain. Octopi have large neuron concentrations in their tentacles to allow for quick and independent reaction which gives an advantage. In a later class we discuss how these models can be applied to robotics.Examples of human and animal sensors: eyes, ears, skin, nose, etc. In humans processed in the brain. Octopi have large neuron concentrations in their tentacles to allow for quick and independent reaction which gives an advantage. In a later class we discuss how these models can be applied to robotics.

6. Types of Sensory Information Proprioceptive Exteroceptive

7. Proprioceptive Information about the state of one’s own body Examples: Direction your head is facing Position of your limbs Position of your (robot’s) wheels

8. Exteroceptive Information about the state of the external world relative to your (or the robot’s) frame of reference Examples: Light level Temperature Distance to objects Sound levels

9. Mention that sensors require power, a connection, and a place to be processed much like in the animal world.Mention that sensors require power, a connection, and a place to be processed much like in the animal world.

10. Sensor Examples NXT sensors include: touch/bump, ultrasonic, light, microphone, accelerometer, compass.NXT sensors include: touch/bump, ultrasonic, light, microphone, accelerometer, compass.

11. Sensor Processing For a sensor to provide useful information some processing must occur.For a sensor to provide useful information some processing must occur.

12. Signal To Symbol Problem Sensors do not provide state, just a measurement of a physical quantity (a signal). Symbols are an abstraction relating to exteroception and proprioception. Translating sensory information into symbols is a difficult fundamental problem in Robotics. (more on this difficulty later). Examples of physical quantity: sound, light, temp, etc. Examples of symbols: cat, wall, daytime, etc.Examples of physical quantity: sound, light, temp, etc. Examples of symbols: cat, wall, daytime, etc.

13. Why do it? It is easier to program actions based on symbols Robot sees Cat Robot takes action against the Cat To Recognize the Symbol (Cat) certain signals must be processed Ultrasonic sensor detects obstacle Sound sensor registers sound Temperature sensor detects a change It is far easier to program at the level of using symbols than at the level of specific signal levels. It allows for information abstraction and non sensor specific representations such as logical sensors (more on them later). It is far easier to program at the level of using symbols than at the level of specific signal levels. It allows for information abstraction and non sensor specific representations such as logical sensors (more on them later).

14. Sensor Fusion Combining multiple sensors to get better information is Sensor Fusion. Not simple: Sensors have noise and inaccuracy. Combining sensors increases noise and inaccuracy. Sensors are combined for redundancy, to complement each other, and for coordination.

15. Types of Fusion Redundant (competing) : Physical Logical Complementary: Disjoint sensory inputs read simultaneously Coordinated: Disjoint sensory inputs read in sequence Physical redundancy is having several sensors of the same kind, such as an array of ultrasonic sensors. Logical redundancy is having multiple sensors that return the same type of information: laser range finder vs ultrasonic range finder. Complementary Example: A search and rescue robot seeing heat and detecting motion to id a survivor. Coordinated example: A predator hears something, then focuses on the scene to find signs of prey such as motion/scent/etc.Physical redundancy is having several sensors of the same kind, such as an array of ultrasonic sensors. Logical redundancy is having multiple sensors that return the same type of information: laser range finder vs ultrasonic range finder. Complementary Example: A search and rescue robot seeing heat and detecting motion to id a survivor. Coordinated example: A predator hears something, then focuses on the scene to find signs of prey such as motion/scent/etc.

16. Logical Sensors A logical sensor is an abstraction of sensors. Enables the perception of a symbol by processing sensor signal(s) Contains all available methods for detecting that symbol Example: Cat Sensor Processes sensor inputs (primary and/or alternates) Uses algorithm to conclude Cat detected or not Logical sensors are an abstraction and not a sensor specific representation.Logical sensors are an abstraction and not a sensor specific representation.

17. Passive Sensors Measure some quantity but do not interact Examples: Touch Sensors Passive Shaft Encoders Sound Sensor Light Sensor (some) And many more… Passive shaft encoders use a switch that gets depressed every so often to determine rotation.Passive shaft encoders use a switch that gets depressed every so often to determine rotation.

18. Active Sensors Composed of Emitters and Detectors Output a signal and measure the reaction Examples: Active Shaft Encoders Light sensors (some) Ultrasonic Laser range finders X-rays

19. Sensor Complexity Sensors can be divided based upon the level of processing required Simple Complex Sensors can also be divided by their level of activity in the world Passive Active A sensor belonging to the simple or complex class does not determine if the sensor is passive or active.A sensor belonging to the simple or complex class does not determine if the sensor is passive or active.

20. Diagram showing the progression of sensor processing complexity.Diagram showing the progression of sensor processing complexity.

21. Simple Sensors Do not require a great deal of processing to be useful to the robot Example: Switches/touch Light sensor Shaft Encoders Switch/bump: Uses a circuit to determine if the sensor is activated or not. Light Sensor: Photoresistive material changes the resistance in a circuit determining light level. Shaft encoders: Determine the number of rotations of a shaft. Can be both active and passive depending on how designed.Switch/bump: Uses a circuit to determine if the sensor is activated or not. Light Sensor: Photoresistive material changes the resistance in a circuit determining light level. Shaft encoders: Determine the number of rotations of a shaft. Can be both active and passive depending on how designed.

22. Switches Simple sensors that work by using a circuit. 2 values: pressed and not pressed relate to open or closed circuit

23. Light Sensors Photocells Reflective Optosensors Infra Red Sensors Break Beam Sensors

24. Shaft Encoders Measure the angular rotation of an axle/shaft. Made either using light sensors or switches. Examples: odometer, speedometer.

25. Complex Sensors Require a greater amount of processing than Simple Sensors. Examples: Ultrasonic Lasers Camera (Robot Vision) Ultrasonic range finder: emits an ultrasound and then waits for a reflection and calculates the distance based on the speed of sound. Laser Range finder: work similarly to ultrasonic, except using light. More accurate, and can work over greater distances ie to the moon. Camera: Most complex as images are grids of many bits. Whole series of courses devoted to image processing. Ultrasonic range finder: emits an ultrasound and then waits for a reflection and calculates the distance based on the speed of sound. Laser Range finder: work similarly to ultrasonic, except using light. More accurate, and can work over greater distances ie to the moon. Camera: Most complex as images are grids of many bits. Whole series of courses devoted to image processing.

26. Ultrasonic Sensors Active sensors that measure distance by using pulses of ultrasound and calculating the time it takes for a reflection to occur. Uses the time of flight principle Distance = (time x speed of sound) / 2

27. Laser Range Finders Active sensors that use lasers to calculate distance. Most use phase-shift analysis instead of time of flight to determine distance. Phase shift involves sending out a continuous light wave and measuring the change in phase. Time of flight can be used for lasers, but speed of light makes this expensive.

28. Cameras Complex sensors that allow the robot to process snapshots of the environment. Limited by range of distances that can be focused (Depth of field). Return images composed of pixels. Can be grayscale or color. 2 or more cameras allows for stereo vision. Send streams of images over time. Vision processing computationally intensive.

29. Robot Vision Techniques Robot vision is very demanding so there are some techniques to speed it up. Use color to recognize objects Use color and motion aka blob tracking Use sensor fusion and combine faster/less complex sensors with cameras Use environmental knowledge to your robots advantage.

30. Signal to Symbol revisited This problem is difficult because: Sensor noise and errors Sensor Limitations Hallucinations Imperfect Information After describing the various sensor types it is now important to mention what can go wrong. Sensor noise is a dirty signal being returned to the sensor. An example would be electromagnetic interference from a nearby device altering the sensors readings. Sensor errors can be either predictable or unpredictable. If the error is predictable such as a distance reading being off a certain amount every time it can be accounted for when programming. If it is an unpredictable error then other measures such as sensor fusion must be used to overcome it. Different sensors have different parameters they work within. For example the nxt range finder can sense objects within 8 feet, but will not be able to detect something 15 feet away. When a sensor returns a false positive when nothing is there, or a false negative when something is there it is said to be experiencing a sensor hallucination. Ultrasonic sensors are prone to this because of specular reflection. Glass obstacles tend to reflect sound very well meaning that a signal may take much longer to return to the sensor causing the robot to believe that an obstacle is not present. For example there was a robot developed by IBM which used ultrasonics for navigation. It performed well enough to be deemed ready for presentation, and was allowed to direct itself from the lab through the hall to the foyer. In the lab and hall it performed as expected, but when it reached the foyer it went amok. It was eventually determined that the large amount of glass (used to enhance the beauty of the lobby/foyer) caused sensor hallucinations and led the robot to think things weren’t where they were. Nobody has perfect knowledge of the world, and it is impossible for the robot to. All possible occurrences cannot be programmed for in advance.After describing the various sensor types it is now important to mention what can go wrong. Sensor noise is a dirty signal being returned to the sensor. An example would be electromagnetic interference from a nearby device altering the sensors readings. Sensor errors can be either predictable or unpredictable. If the error is predictable such as a distance reading being off a certain amount every time it can be accounted for when programming. If it is an unpredictable error then other measures such as sensor fusion must be used to overcome it. Different sensors have different parameters they work within. For example the nxt range finder can sense objects within 8 feet, but will not be able to detect something 15 feet away. When a sensor returns a false positive when nothing is there, or a false negative when something is there it is said to be experiencing a sensor hallucination. Ultrasonic sensors are prone to this because of specular reflection. Glass obstacles tend to reflect sound very well meaning that a signal may take much longer to return to the sensor causing the robot to believe that an obstacle is not present. For example there was a robot developed by IBM which used ultrasonics for navigation. It performed well enough to be deemed ready for presentation, and was allowed to direct itself from the lab through the hall to the foyer. In the lab and hall it performed as expected, but when it reached the foyer it went amok. It was eventually determined that the large amount of glass (used to enhance the beauty of the lobby/foyer) caused sensor hallucinations and led the robot to think things weren’t where they were. Nobody has perfect knowledge of the world, and it is impossible for the robot to. All possible occurrences cannot be programmed for in advance.

31. Sensor Errors 2 categories of sensor error Predictable Unpredictable Predictable sensor errors can be accounted for with programming/design Unpredictable sensor errors are more difficult Can create false positives and false negatives

32. Sensor Noise Outside signals interfere with accurate sensor reading. Examples Ambient light level washing out led from light sensor Changing ambient sound levels Electromagnetic interference

33. Dealing with Sensor Noise Calibrate sensors often Take multiple readings and process Take the average of the readings or discard any extreme outliers. Sensor fusion with complementary sensors

34. Sensor Limitations Sensors exist in the physical world so they obviously have physical limitations. Examples: Camera resolution and depth of field Ultrasonic range Touch sensor sensitivity

35. Specular Reflection Light and sound waves reflect off surfaces. Very little of the emitted light is returned to the sensor. Very smooth surfaces reflect little to none of the wave back sending it in all directions. Waves can take many bounces before they return leading to incorrect readings and hallucinations.

36. Imperfect Information It is impossible to have perfect information about the world All possibilities can not be designed/programmed for Example: The kidnapped robot problem

37. Picking Sensors Some basic questions help when picking sensors: What are the goals of the robot? What is the environment like that the robot will operate in? What are the physical limitations of the robot?

38. NXT Sensors We’ve talked about them before… Touch Light Color Sound Compass Accelerometer Ultrasonic (sonar)

39. NXT Touch Sensor It is a switch Function by completing or breaking a circuit Simplest passive sensor Returns 1 if pressed, 0 if not pressed.

40. #pragma config(Sensor, S1, touchSensor, sensorTouch) task main() { //loop while the touchsensor's value is 0 while(SensorValue(touchSensor) == 0) { motor[motorA] = 100; motor[motorB] = 100; } motor[motorA] = -75; motor[motorB] = -75; wait1Msec(1000); }

41. NXT Light Sensor Passive and active simple sensor Returns a percentage rating 0 to 100. 100 is bright, 0 is dark. Composed of a phototransistor and a LED. When LED is on it is an active sensor, when it is off it is passive. Can detect some color changes with LED on via the change in reflected light.

42. #pragma config(Sensor, S1, lightSensor, sensorLightActive) //*!!Code automatically generated by 'ROBOTC' configuration wizard !!*//task main() { wait1Msec(100); while(SensorValue(lightSensor) < 45) { motor[motorA] = 100; motor[motorB] = 100; } motor[motorA] = 0; motor[motorB] = 0; }

43. Hightechnic Color Sensor An active somewhat-complex sensor Contains a microcontroller Uses 3 different LED lights to illuminate the target surface and measures the intensity of each color reflected by the surface (as described by its manufacturer). The sensor then calculates the color number and returns it to the program.

44. Color Sensor Number Assignments

45. NXT Sound Sensor Passive simple sensor Uses a microphone to detect sound Measures sound intensity in a percentage (0-100) between 0 and 90 dB. Samples between 20 and 30 Hz…not fast enough to recognize human speech.

46. #pragma config(Sensor, S1, soundSensor, sensorSoundDB) //*!!Code automatically generated by 'ROBOTC' configuration wizard !!*// task main() { wait1Msec(1000); while(SensorValue(soundsensor) <= 50) { motor[motorA] = 75; motor[motorB] = 75; } }

47. Hightechnic Compass Sensor Passive somewhat-complex sensor Has a microcontroller Contains a digital compass that measures the earth’s magnetic field and returns a heading angle to the program from 0 to 359 degrees. Can be influenced by local magnetic phenomenon, so it must be mounted away from the NXT and its batteries.

48. #pragma config(Sensor, S1, compassSensor, sensorI2CHiTechnicCompass) //*!!Code automatically generated by 'ROBOTC' configuration wizard !!*// task main(){ if(SensorValue(compassSensor)==0){ //Due north motor[motorA]=100; motor[motorB]=100; } }

49. Hightechnic Accelerometer Sensor Passive somewhat-complex sensor Has its own microcontroller Contains a 3 axes accelerometer that measures acceleration on 3 axes: x,y, and z. Acceleration is measured between -2 and 2 g with scaling of about 200 counts per g. This is refreshed 100 times a second. Can be used to measure tilt in 3 dimensions.

50. Accelerometer Axes

51. Gyro Vs Accelerometer An acceleration sensor like that included in our sets cannot easily be used for balancing like a gyro can. This is because the accelerometer is sensitive to the vibration of the motors (and is sensitive to acceleration in all directions) and that cannot be easily distinguished from gravity. Whereas the Gyro sensor detects angular acceleration perpendicular to a particular axis.

52. NXT Ultrasonic Sensor The most complex sensor we have in class. Active complex sensor Contains a microcontroller Works like a bat – emits a high frequency sound and detects the reflections. Measures the time to receive the reflections to determine the distance within 2-3 cm accuracy.

53. NXT Ultrasonic Sensor Range returned is between 0 to 255 cm. Accuracy depends on temperature and to a lesser extent humidity. Certain surfaces (like glass) do not return an accurate reading. Other ultrasonic sensors can interfere (crosstalk).

54. #pragma config(Sensor, S1, sonarSensor, sensorSONAR) //*!!Code automatically generated by 'ROBOTC' configuration wizard !!*// task main() { do { motor[motorA] = 75; motor[motorB] = 75; } while(SensorValue(sonarSensor) > 20); }

55. Lab time!