Networks of Tiny Devices Embedded in the Physical World

1. Networks of Tiny Devices Embedded in the Physical World David Culler Computer Science Division U.C. Berkeley www.cs.berkeley.edu/~culler Intel Research Berkeley

2. Processing Storage Sensing Actuation Communication I SD Q SD baseband PLL filters mixer LNA Technology Push • Complete network embedded systems going microscopic Power TinyOS IPAM

3. Disaster Management Circulatory Net Habitat Monitoring Condition-based maintenance Application Pull • Complete NW embedded systems going microscopic • Huge space of new applications Monitoring & Managing Spaces Ubiquitous computing TinyOS IPAM

4. Bridging the Technology-Application Gap • Power-aware, communication-centric node architecture • Tiny Operating System for Range of Highly-Constrained Application-specific environments • Network Architecture for vast, self-organized collections • Programming Environments for aggregate applications in a noisy world • Distributed Middleware Services (time, trigger, routing, allocation) • Techniques for Fine-grain distributed control • Demonstration Applications TinyOS IPAM

5. Critical issues • Highly constrained devices • power, storage, bandwidth, energy, visibility • primitive I/O hierarchy • Observation and action inherently distributed • many small nodes coordinate and cooperate on overall task • The structure of the SYSTEM changes • Devices ARE the infrastructure • ad hoc, self-organized network of sensors • Highly dynamic • passive vigilance most of the time • concurrency-intensive bursts • highly correlated behavior • variation in connectivity over time • failure is common TinyOS IPAM

6. A de facto platform for EmNets • Developed a series of wireless sensor devices • TinyOS concurrency framework • Messaging Model • Networking stacks (RF and Serial) • Multihop routing • Several Key components • sensing, logging, data filters, broadcast • Simulation tools • DARPA NEST OEP USC robomote TinyOS IPAM

7. UCB NEST SensorWeb Blackout Glaser structures CBE BFD BRWC UCLA USC, ISI Rutgers winlab Intel Bosch Crossbow U Wash Rutgers UIUC NCSA U Virginia Ohio State UCSD Dartmouth MIT UT Austin, ASU, Iowa Accenture and many more Many Research Groups using it TinyOS IPAM

8. The MICA architecture • Atmel ATMEGA103 • 4 Mhz 8-bit CPU • 128KB Instruction Memory • 4KB RAM • 4 Mbit flash (AT45DB041B) • SPI interface • 1-4 uj/bit r/w • RFM TR1000 radio • 50 kb/s – ASK • Focused hardware acceleration • Network programming • Rich Expansion connector • i2c, SPI, GIO, 1-wire • Analog compare + interrupts • TinyOS tool chain • sub microsecond RF synchronization primitive 51-Pin I/O Expansion Connector Digital I/O 8 Analog I/O 8 Programming Lines Atmega103 Microcontroller DS2401 Unique ID Coprocessor Transmission Power Control Hardware Accelerators SPI Bus TR 1000 Radio Transceiver 4Mbit External Flash Power Regulation MAX1678 (3V) TinyOS IPAM 2xAA form factor

9. Mica PINS Tone Intr PHOTO SOUNDER Mic Signal MICROPHONE TEMP Y Axis Gain Adjustment X Axis MAGNETOMETER ACCELEROMETER ADC Signals (ADC1-ADC6) On/Off Control I2C Bus Interrupt Rich Sensor board Microphone Sounder Magnetometer 1.25 in Temperature Sensor Light Sensor 2.25 in TinyOS IPAM Accelerometer

10. More Sensors and Actuators • Motor-Servo boardinterfaces any combination of two motors, servos, and solenoids to a toy car platform • whisker board for obstacle detection • digital accelerometer (ADXL202) board for crude odometry • GPS Board • Weatherboard • light, temp, humidity, barometric pressure, occupancy (thermopile) TinyOS IPAM

11. Acadia National Park Mt. Desert Island, ME Great Duck Island Nature Conservancy Getting Ready for Outdoors TinyOS IPAM

12. A Operating System for Tiny Devices? • Traditional approaches • command processing loop (wait request, act, respond) • monolithic event processing • bring full thread/socket posix regime to platform • Alternative • provide framework for concurrency and modularity • never poll, never block • interleaving flows, events, energy management => allow appropriate abstractions to emerge TinyOS IPAM

13. msg_rec(type, data) msg_send_done) Tiny OS Concepts • Scheduler + Graph of Components • constrained two-level scheduling model: threads + events • Component: • Commands, • Event Handlers • Frame (storage) • Tasks (concurrency) • Constrained Storage Model • frame per component, shared stack, no heap • Very lean multithreading • Efficient Layering Events Commands send_msg(addr, type, data) power(mode) init Messaging Component internal thread Internal State TX_packet(buf) Power(mode) TX_packet_done (success) init RX_packet_done (buffer) TinyOS IPAM

14. Application = Graph of Components Route map router sensor appln application Active Messages Serial Packet Radio Packet packet Temp photo SW Example: ad hoc, multi-hop routing of photo sensor readings HW UART Radio byte ADC byte 3450 B code 226 B data clocks RFM bit Graph of cooperating state machines on shared stack TinyOS IPAM

15. Radio Packet packet Radio byte byte RFM bit TOS Execution Model • commands request action • ack/nack at every boundary • call cmd or post task • events notify occurrence • HW intrpt at lowest level • may signal events • call cmds • post tasks • Tasks provide logical concurrency • preempted by events • Migration of HW/SW boundary data processing application comp message-event driven active message event-driven packet-pump crc event-driven byte-pump encode/decode event-driven bit-pump TinyOS IPAM

16. Dynamics of Events and Threads bit event => end of byte => end of packet => end of msg send thread posted to start send next message bit event filtered at byte layer radio takes clock events to detect recv TinyOS IPAM

17. Maintaining Scheduling Agility • Need logical concurrency at many levels of the graph • While meeting hard timing constraints • sample the radio in every bit window • Retain event-driven structure throughout application • Tasks extend processing outside event window • All operations are non-blocking • lock-free scheduling queue TinyOS IPAM

18. Demonstration applications • 29 Palms • Cory Hall network • ½ million packets over 3 weeks • Surge network and environment display • 800 node ad hoc network • CBE • Glaser Shakes • Granlibakken retreat watcher • Robomote • Group response => continued application focus • more real and long lived • more dynamics • extract architecture and create framework TinyOS IPAM

19. Example TinyOS study • UAV drops 10 nodes along road, • hot-water pipe insulation for package • Nodes self-configure into linear network • Synchronize (to 1/32 s) • Calibrate magnetometers • Each detects passing vehicle • Share filtered sensor data with 5 neighbors • Each calculates estimated direction & velocity • Share results • As plane passes by, • joins network • upload as much of missing dataset as possible from each node when in range • 7.5 KB of code! • While servicing the radio in SW every 50 us! TinyOS IPAM

20. Structural performance due to multi-directional ground motions (Glaser & CalTech) Mote infrastructure . Mote Layout 14 5 ` 15 15 13 6 12 9 11 8 Comparison of Results Wiring for traditional structural instrumentation + truckload of equipment TinyOS IPAM

21. Energy Monitoring/Mgmt System • 50 nodes on 4th floor • 5 level ad hoc net • 30 sec sampling • 250K samples to database over 6 weeks TinyOS IPAM

22. 20-ton chiller GW GW GW MYSQL Energy Monitoring Network Arch sensor net control net 802-11 telegraph PC PC modbus scada term UCB power monitor net Browser TinyOS IPAM

23. Meeting Social Network TinyOS IPAM

24. Wealth of Research Challenges • Large numbers of highly constrained (energy & capability), connected devices • able to be casually deployed in infrastructure (existing or in design) • imperfect operation and reliability • operating in aggregate • New family of issues across all the layers application service prog / data model network mgmt / diag / debug algorithm / theory system architecture technology TinyOS IPAM

25. Application Controller RF Transceiver Node Communication Architecture Classic Protocol Processor Direct Device Control Hybrid Accelerator TinyOS IPAM

26. Novel Protocol Examples • Low-power Listening • Really Tight Application-level Time Synchronization • Localization • Wake-up • MACs • Self-organization TinyOS IPAM

27. Low-Power Listening • Costs about as much to listen as to xmit, even when nothing is received • Must turn radio off when there is nothing to hear. • Time-slotting and rendezvous cost BW and Latency • Can turn radio on/of in <1 bit • Small sub-msg recv sampling • Trade small, infrequent tx cost for freq. Rx savings TinyOS IPAM

28. Exposing Time Synchronization Up • Many applications require correlated data sampling • Distributed time sync accuracy bounded by ½ the variance in RTT. • Successful radio transmission requires sub-bit synchronization • Provide accurate timestamping with msg delivery • Jitter < 0.1us (propagation) + 0.25 us (edge capture accuracy) + 0.625 us (clock synch) TinyOS IPAM

29. Error Error Noise Noise Localization • Many applications need to derive physical placement from observations • Spatial sampling, proximity, context-awareness • Radio is another sensor • Sample baseband to estimate distance • Need a lot of statistical data • Calibration and multiple-observations are key • Acoustic time-of-flight alternative • Requires good time synchronization TinyOS IPAM

30. Probability Y Distance X Node A Node B Statistical Approach TinyOS IPAM

31. Integrated Architecture • Chip Area ~5 mm2 • AVR core with protocol Accelerators .5 mm2 • 16 Kbytes on-chip ram 4 mm2 • ADC • 800-1GHz FSK transceiver, -90dBm receive sens’y .5 mm2 • Expected sleep current = 1 uW • lifetime on a single AA = 400+ years • Expected active (processing current) • Processor @ 4 Mhz < 1 mW • Radio: 1mW power consumption, 100Kbps • ADC: 20 pJ/sample 10 Ksamps/second = .2 uW. TinyOS IPAM

32. Networking • Hands-on Experience with Large Networks of Tiny Network sensors • intense constraints, freedom of abstraction • Re-explore entire range of networking issues • encoding, framing, error handling • media access control, transmission rate control • discovery, multihop routing • broadcast, multicast, aggregation • active network capsule (reprogramming) • localization, time synchronization • security, network-wide protection • density independent wake-up and proximity est. • Fundamentally new aspects in each TinyOS IPAM

33. The Nodes are the Infrastucture • Simple Epidemic Algorithm Schema if (new mcast) then take local action retransmit modified request • Examples: Network wakeup, command propagation • Build spanning tree • record parent • Naturally adapts to available connectivity • Minimal state and protocol overhead => surprising complexity in this simple mechanism TinyOS IPAM

34. Network Discovery: Radio Cells TinyOS IPAM

35. Network Discovery TinyOS IPAM

36. Controlled Empirical Study • Experimental Setup • 13x13 grid of nodes • separation 2ft • flat open surface • Identical length antennas, pointing vertically upwards. • Fresh batteries on all nodes • Identical orientation of all nodes • The region was clean of external noise sources. • Range of signal strength settings • Log many runs TinyOS IPAM

37. Example “epidemic” tree formation TinyOS IPAM

38. Final Tree TinyOS IPAM

39. Power Laws ? • Most nodes have very small degree (ave = .92) • Some have degree = 15% of the population • Few large clusters account for most of the edges TinyOS IPAM

40. Open Territory => Many Children • Example: Level 1 TinyOS IPAM

41. Open Territory => Many Children • Example: Level 2 – variation in distance TinyOS IPAM

42. Open Territory => Many Children • Example: Level 3 – long links TinyOS IPAM

43. Understanding Connectivity • 16 transmit power settings • For each transmit power setting, each node transmits 20 packets. • Receivers log successfully received packets. • Nodes transmit one after the other in a token-ring fashion • No collisions. • Define “range”: radius where 75% of enclosed nodes receive 75% of packets • Often good nodes at a distance probability of reception from center node vs xmit strength TinyOS IPAM

44. Importance of Asymmetric Links • Asymmetric Link: • >65% successful reception in one direction • <25% successful reception in the other direction • 10%-25% of links are asymmetric • Many long links are asymmetric • in large field it is likely that someone far away can hear you • what does this mean for protocol design? TinyOS IPAM

45. Collisions are primary factor • Nodes out of range may have overlapping cells • hidden terminal effect • Collisions => these nodes hear neither ‘parent’ • become stragglers • As the tree propagates • folds back on itself • rebounds from the edge • picking up these stragglers. • Seen in many experiments • Mathematically complex because behavior is not independent beyond singe cell TinyOS IPAM

46. Stragglers • significant fraction of links point ‘backwards’ TinyOS IPAM

47. Minimal lessons learned • Don’t think about wireless networks as bunch of circles of radius r • connectivity is a probability distribution • falls off with distance, but not as simple fading law • shape varies with time and context • With large, dense arrays the low-probability events are common • Must strike a balance in exploiting structure and adapting to observed behavior • Want simple local rules that have predictable, robust global behavior TinyOS IPAM

48. More typical routing for sensor nets • Current applications dominated by data acquisition • route from many nodes to nearest gateway • aggregate from many nodes • routing determined by simple local rules • Nodes listen to data transfers from neighbors • carries hop-count info • monitors link goodness of potential ‘parents’ • dynamically selects best node is lesser hop count • includes hysterisis and continuous rediscovery • gateways emit null data with 0-hop • Much to understand about how such algorithms manage major change TinyOS IPAM

49. Self-propagating Programs? • TinyOS components support class of applns. • Tiny virtual machine adds layer of interpretation for specific coordination • Primitives for sensing and communication • Small capsules (24 bytes) • Propagate themselves through network TinyOS IPAM

50. Multihop Bandwidth Management • Should self-organize into fair, dynamic multihop net • Hidden nodes between each pair of “levels” • CSMA is not enough • P[msg-to-base] drops with each hop • Investment in packet increases with distance • need to optimize for low-power fairness! • RTS/CTS costly (power & BW) • Local rate control to approx. fairness • Priority to forwarding, adjust own data rate • Additive increase, multiplicative decrease • Listen for retransmission as ack TinyOS IPAM