1 / 46

Let’s think about future smart sensors !

Let’s think about future smart sensors !. Chong-Min Kyung KAIST. Contents. Introduction Overview of smart sensors Low-power smart sensor design Hierarchical adaptive event detection Adaptive storing scheme in black-box systems Task partitioning in multiple sensors

konala
Download Presentation

Let’s think about future smart sensors !

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Let’s think about future smart sensors! Chong-Min Kyung KAIST

  2. Contents • Introduction • Overview of smart sensors • Low-power smart sensor design • Hierarchical adaptive event detection • Adaptive storing scheme in black-box systems • Task partitioning in multiple sensors • 3D IC platform for smart sensors • Conclusions

  3. Introduction : world of mass, energy • E = mc2 • How is information, intelligence to be involved with mass and energy? • So far the world of [m, E] has been separate from that of [I]. • cf) School of EE, CS in the univ, society. • How much mass and energy is needed to store, process and transmit a bit of information?

  4. Putting spirit into the body/machine • [mechanical moving] Bones and muscles • [energy supply thru feeding and recirculation] Blood vessels, digestive organs, … • But, what is more needed beyond mass and energy? • How to put spirits into the machine so that it behaves right and appropriate? • The beginning is smart sensing.

  5. After smart phone, where is the next market? • Let us look around ; Find out what makes us most urgent, desperate. • Natural and man-made disasters, breakdown of social infrastructures • How to predict them 1 minute earlier • How to better secure the blackbox [airplane crash, ship/buildings attacked, bridges fall, …] • Diseases; epidemic, fatal and/or acute, e.g., cancer • How to detect them in early stage • Being connected ; mobile and ubiquitous • How to connect and avail the node processing power to the real world [sensor/actuator] and to the wireless network

  6. So, Where are the Emerging Markets? • Basically safety and convenience. • Demand on safety will grow at a faster rate. [Applications] • Health/Bio • Security/Safety • Surveillance • Environment/social infra monitor • Vehicular • Mobile

  7. Categories of smart sensors • Purpose of deployment ; • Health [from disease] • Security/Safety [from violence] • Protection of/from nature [from abuse/disaster] • Locations of deployment ; • Ubiquitous • Vehicular • Mobile • Wearable • Implantable

  8. STTA (sense, think, talk, act) of smart sensor in the ‘Health’ case • Health (example) • Sensing : Monitor/Capture • Thinking : Diagnostics • Talk : communication between nodes or human interaction for making globally optimal decision • Actuator : Therapy(Drug Delivery)/Surgery

  9. Being ‘smart’ in smart sensor means • Making right decision [contents] [contexts] • At the right time (timeliness) • At the muddy/bloody frontier(point of action), not in the air-conditioned room • With little energy consumed and little error(distortion)

  10. In smart sensor, • Question is how to compromise between Energy investment and Distortion (Error) produced • Decision needs to be made on • ‘Sleep’ or ‘work’, and in what level, if awake. • Degree of data processing • Which one to let work, if multi-sensor case • Where to dispose the data (flash or air through antenna), and how (modulation scheme, beam forming or not, redundancy) ? • Trade-off between error rate and energy consumption

  11. Being ‘smart’ in smart sensor means • Having many Enabling Gadgets • Communicator [Power amp & RF antenna] • Actuator [scissors for sampling, micro surgery, and lab-on-a-chip for drug delivery] • Accessories [lighting] • Systematic Integration is a MUST for quality support (coming from scale of economy) • Hardware platform • Software platform

  12. Overview of SS

  13. Sensors are already Everywhere!

  14. Vision/Image sensor • Chosen as our research because • Most dependable source of information • Occupies largest bandwidth

  15. World Sensor Market

  16. What is “Smart Sensor”? • Definition of “Sensor” • A device that measures or detects a real-world condition • Definition of “Smart Sensor” • A sensor that includes information processing unit Transmitter Camera Event detector Video Encoder Storage < Fundamental components of smart cameras >

  17. How It Works For Smart Life: Smart Surveillance System

  18. How It Works For Smart Life: Health Care

  19. Applications Memory Black-box Memory Surveillance Memory Memory Memory RF transceiver Memory Interposer Video Processor Processor Medical Ubiquitous

  20. Lifetime matters! • Energy breakdown Q. How can we realize “battery-powered sensor system with battery lifetime > 1 year ”?

  21. Overview of Low-Power Smart Sensor Design • Energy-efficient event detection and criticality evaluation • Adaptive encoding considering P-R-D relationship • Selective transmission • Beam forming • Camera clustering

  22. Event Detection • Hierarchical event detection • To minimize misjudgment and sampled data rateof WSN • Control of video encoding configurations • Based on our power-rate-distortion model of the target video encoder

  23. Event Model • Event criticality • The degree of significance of an event • Multiple event criticality levels • Low, medium and high

  24. Hierarchical Event Detection • Three event detection methods • 1)Thresholding-based (SAD) [Hengstler07] [He04] [Feng05] [Hampapur05] • 2) Edge-based [Kim01] • 3) Pattern recognition-based [Oren97] [Papageorgiou98] Thresholding-based Edge-based Pattern recognition-based

  25. Hierarchical Event Detection • Comparison of accuracy and complexity • Configurations • Event criticality is evaluated based on the existence of human body in the scene.

  26. Hierarchical Event Detection • Select energy-optimal event detection method, or switch among them according to the situation [input statistics, battery/channel situation] Thresholding Edge Pattern recognition

  27. Power-Rate-Distortion Analysis • Conventional rate-distortion model • Trade-off between rate and distortion • Need for power-rate-distortion (P-R-D) model • Achievable rate and/or distortion can vary according to the power consumption of video encoding.

  28. Complexity-Distortion Analysis • Video encoding configurations • 480 combinations: 2 x 3 x 2 x 4 x 5 x 2 • Motion estimation methods (2) • Motion estimation search range (3) • Number of reference frames (2) • Sub-pixel motion estimation method (4) • MB partition size (5) • Deblocking filter (2)

  29. Complexity-Distortion Analysis • Power-distortion (P-D) model • Distortion-minimal encoding configurations among 480 combinations • foreman CIF: bpp (bit per pixel) = 0.10

  30. Hierarchical Adaptive Event Detection • Event detector • Coarse-to-fine detection • Events: Encoded / No events : Discarded

  31. Hierarchical Adaptive Event Detection • Energy-aware detection • Hierarchical detection • Low-to-high complexity mode • Adaptive detection • Control configuration parameters for each mode

  32. Hierarchical Adaptive Event Detection • Event detection strategy • Minimize the whole system energy while satisfying required accuracy

  33. Hierarchical Adaptive Event Detection • Event detection strategy • Adaptive control based on event statistics

  34. System-wide Low-Power Techniques • Battery-powered event-driven operation • Limited memory space Event Capture • How can we maximize the lifetime of smart sensor with limited resources? • Encoding bit-rate control • Adaptive transmission for mobile sensor node

  35. Power-Minimal Bit-Rate Allocation • Power consumed by video encoder, flash, and wireless RF depends on encoding bit-rate. ※ We can find the power-minimal encoding bitrate, i.e., encoding configuration.

  36. Rate Control for Balancing Lifetime • Two resource lifetime • Memory lifetime (Fmemory) • Until memory-full • Battery lifetime (Fbattery) • Until battery exhaustion • System lifetime (Fcamera) Fcamera = min(Fmemory, Fbattery) In case that all encoded data is stored into the memory, Fmemory = Fbattery We can find the system lifetime-maximal bit-rate (Rbal) .

  37. Decision of Threshold Distance for Mobile Sensor Node • In the memory-critical system • Storing all encoded data into memory is impossible. • “Some data needs to be transmitted to base station” • Fcamera = Fbattery(until battery exhaustion) • Threshold distance (dth) • Power consumed by transmission has the exponential dependence on distance. • Decide when to transmit the encoded data to base station. • dth > dcurrent=> transmission • dth < dcurrent => storage

  38. Lifetime-Maximal Threshold Distance • We can find the lifetime-maximal threshold distance, i.e., minimal distance satisfying the memory constraint. Transmission Storage

  39. Other schemes for energy reduction : Multi-Camera System • Multi-camera system (small sensor network) • Two key points • Image quality • System lifetime (Single camera node)

  40. Multi-Camera System • Find capture and store node pair for co-optimizing image quality and battery lifetime Q: 0.6 Q: 0.4 Q: 0.8 Capture! Q: Image quality Full transmit data Battery empty Full Q: 0.5 Memory Capacity empty Store!

  41. Strategy for Lifetime Maximization • Evenly utilize resources (battery, memory) of cameras belonging to the system System lifetime Numberof nodes

  42. Power Amplifier and Antenna : another frontier for E-D co-optimization • Beam Forming with antenna array desired camera

  43. Another Frontier : 3D Integration Image Sensor Memory Image Sensor Memory Memory Memory Memory RF transceiver Memory Interposer Video Processor Video Processor 3D integration for smart camera system Advantages of using 3D integration: 1. Heterogeneous integration 2. Smaller footprint 3. Less power 4. Improved performance 5. Lower cost 3D integration using TSV

  44. (Video) Processor + Memory Issues: CPU always waits for data from memory. Memory takes a large part of chip area. Benefits due to: Wire length reduction More interconnections using TSV Memory Memory On-chip memory 3D-stacked memory with TSV Memory CPU BW = 100GB/s – 1TB/s CPU Off-chip memory Memory CPU BW > 1TB/s BW < 100GB/s

  45. Image Sensor with TSV Image sensor wafer level packaging with TSV Traditional camera module >2004 2005 2006 2007 2008 Micron XinTec Samsung Oki Toshiba 64% size down Camera modulewith TSV Aptina SMT Source: IEK/ITRI(2008)

  46. Conclusion • Now is the time for engaging I-tech with E/m-tech by blowing spirit into the machine. • This is now possible and very much needed with nano technology and growing needs of safety. • For a more reliable and safer world, EE engineers definitely need to be more involved with the real society. • Only then our future is bright.

More Related