CS294-1 Deeply Embedded Networks TinyOS Sept 4, 2003

1. CS294-1 Deeply Embedded NetworksTinyOSSept 4, 2003 David Culler Fall 2003 University of California, Berkeley

2. sensors actuators storage network Characteristics of Network Sensors • Small physical size and low power consumption • Concurrency-intensive operation • multiple flows, not wait-command-respond • Limited Physical Parallelism and Controller Hierarchy • primitive direct-to-device interface • Asynchronous and synchronous devices • Diversity in Design and Usage • application specific, not general purpose • huge device variation => efficient modularity => migration across HW/SW boundary • Robust Operation • numerous, unattended, critical => narrow interfaces cs294-1 f03

3. Classical RTOS approaches • Responsiveness • Provide some form of user-specified interrupt handler • User threads in kernel, user-level interrupts • Controlled Scheduling • Static set of tasks with specified deadlines & constraints • Generate overall schedule • Doesn’t deal with unpredictable events (comm) • Threads + synchronization operations • Sophisticated scheduler to coerce into meeting constraints • Priorities, earliest deadline first, rate monotonic • Priority inversion, load shedding, live lock, deadlock • Sophisticated mutex and signal operations • Communication among parallel entities • Shared (global) variables: ultimate unstructured programming • Mail boxes (msg passing) • dynamic allocation, storage reuse, parsing • external communication considered harmful • Fold in as RPC • Requires multiple (sparse) stacks • Preemption or yield cs294-1 f03

4. Alternative Starting Points • Event-driven models • Easy to schedule handfuls of small, roughly uniform things • State transitions (but what storage and comm model?) • Usually results in brittle monolithic dispatch structures • Structured event-driven models • Logical chunks of computation and state that service events via execution of internal threads • Threaded Abstract machine • Developed as compilation target of inherently parallel languages • vast dynamic parallelism • Hide long-latency operations • Simple two-level scheduling hierarchy • Dynamic tree of code- block activations with internal inlets and threads • Active Messages • Both parties in communication know format of the message • Fine-grain dispatch and consume without parsing • Concurrent Data-structures • Non-blocking, lock-free (Herlihy) cs294-1 f03

5. TinyOS design goals • Simple framework for resource constrained concurrency • Single stack • Flexible hardware/software and system boundary • Expressive enough to build sophisticated, application specific system structures • Avoid arbitrary constraints on optimization • Communication is integral to execution • Asynchrony is first class • Promote robustness • Modular • Static allocation • Explicit success/fail at all interfaces • Reuse • Ease of interpositioning cs294-1 f03

6. msg_rec(type, data) msg_send_done) Tiny OS Concepts • Scheduler + Graph of Components • constrained two-level scheduling model: tasks + events • Component: • Commands • Event Handlers • Frame (storage) • Tasks (concurrency) • Constrained Storage Model • frame per component, shared stack, no heap • Very lean multithreading • Efficient Layering • Events can signal events Events Commands send_msg(addr, type, data) power(mode) init Messaging Component internal thread Internal State TX_packet(buf) Power(mode) TX_packet_done (success) init RX_packet_done (buffer) cs294-1 f03

7. Application = Graph of Components Route map router sensor appln application Active Messages Serial Packet Radio Packet packet Temp photo SW Example: ad hoc, multi-hop routing of photo sensor readings HW UART Radio byte ADC byte 3450 B code 226 B data clocks RFM bit Graph of cooperating state machines on shared stack Execution driven by interrupts cs294-1 f03

8. Radio Packet packet Radio byte byte RFM bit TOS Execution Model • commands request action • ack/nack at every boundary • call cmd or post task • events notify occurrence • HW intrpt at lowest level • may signal events • call cmds • post tasks • Tasks provide logical concurrency • preempted by events • Migration of HW/SW boundary data processing application comp message-event driven active message event-driven packet-pump crc event-driven byte-pump encode/decode event-driven bit-pump cs294-1 f03

9. Dynamics of Events and Threads bit event => end of byte => end of packet => end of msg send thread posted to start send next message bit event filtered at byte layer radio takes clock events to detect recv cs294-1 f03

10. Programming TinyOS - nesC • TinyOS 1.x is written in an extension of C, called nesC • Applications are too! • just additional components composed with the OS components • Provides syntax for TinyOS concurrency and storage model • commands, events, tasks • local frame variables • Rich Compositional Support • separation of definition and linkage • robustness through narrow interfaces and reuse • interpositioning • Whole system analysis and optimization cs294-1 f03

11. Event-Driven Sensor Access Pattern command result_t StdControl.start() { return call Timer.start(TIMER_REPEAT, 200); } event result_t Timer.fired() { return call sensor.getData(); } event result_t sensor.dataReady(uint16_t data) { display(data) return SUCCESS; } • clock event handler initiates data collection • sensor signals data ready event • data event handler calls output command • device sleeps or handles other activity while waiting • conservative send/ack at component boundary SENSE LED Photo Timer cs294-1 f03

12. TinyOS Commands and Events { ... status = call CmdName(args) ... } command CmdName(args) { ... return status; } event EvtName)(args) { ... return status; } { ... status = signal EvtName(args) ... } cs294-1 f03

13. Split-phase abstraction of HW • Command synchronously initiates action • Device operates concurrently • Signals event(s) in response • ADC • Clock • Send (UART, Radio, …) • Recv – depending on model • Coprocessor • Higher level (SW) processes don’t wait or poll • Allows automated power management • Higher level components behave the same way • Tasks provide internal concurrency where there is no explicit hardware concurrency • Components (even subtrees) replaced by HW and vice versa cs294-1 f03

14. Tasks events commands Interrupts Hardware TinyOS Execution Contexts • Events generated by interrupts preempt tasks • Tasks do not preempt tasks • Both essential process state transitions cs294-1 f03

15. Storage Model • Local storage associated with each component (or instance of) • Internally managed • Only objects passed by reference are message buffers cs294-1 f03

16. Data sharing • Passed as arguments to command or event handler • Don’t make intra-node communication heavy-weight • If queuing is appropriate, implement it • Send queue • Receive queue • Intermediate queue • Bounded depth, overflow is explicit • Most components implement 1-deep queues at the interface • Don’t force marshalling/unmarshalling through generic queue mechanism • If you want shared state, created an explicit component with interfaces to it. cs294-1 f03

17. TASKS • provide concurrency internal to a component • longer running operations • are preempted by events • able to perform operations beyond event context • may call commands • may signal events • not preempted by tasks { ... post TskName(); ... } task void TskName { ... } cs294-1 f03

18. Typical application use of tasks • event driven data acquisition • schedule task to do computational portion event result_t sensor.dataReady(uint16_t data) { putdata(data); post processData(); return SUCCESS; } task void processData() { int16_t i, sum=0; for (i=0; i ‹ maxdata; i++) sum += (rdata[i] ›› 7); display(sum ›› shiftdata); } • 128 Hz sampling rate • simple FIR filter • dynamic software tuning for centering the magnetometer signal (1208 bytes) • digital control of analog, not DSP • ADC (196 bytes) cs294-1 f03

19. Tasks in low-level operation • transmit packet • send command schedules task to calculate CRC • task initiated byte-level data pump • events keep the pump flowing • receive packet • receive event schedules task to check CRC • task signals packet ready if OK • byte-level tx/rx • task scheduled to encode/decode each complete byte • must take less time that byte data transfer • i2c component • i2c bus has long suspensive operations • tasks used to create split-phase interface • events can procede during bus transactions • Timer • Post task in-critical section, signal event when current task complete cs294-1 f03 Make SW look like HW

20. Example: Radio Byte Operation • Pipelines transmission – transmits single byte while encoding next byte • Trades 1 byte of buffering for easy deadline • Separates high level latencies from low level real-time requirements • Encoding Task must complete before byte transmission completes • Decode must complete before next byte arrives … Encode Task Byte 1 Byte 2 Byte 3 Byte 4 Bit transmission start Byte 1 Byte 2 Byte 3 RFM Bits Hardware accelerators in MICA eliminate bit pumps cs294-1 f03

21. Task Scheduling • Currently simple fifo scheduler • Bounded number of pending tasks • When idle, shuts down node (except clock) • Uses non-blocking task queue data structure • Simple event-driven structure + control over complete application/system graph • instead of complex task priorities and IPC cs294-1 f03

22. Structured Events vs Multi-tasking • Storage • Control Paradigm • Always block/yield – rely on thread switching • Never block – rely on event signaling • Communication & Coordination among potentially parallel activities • Threads: global variables/mailboxes, mutex, signaling • Preemptive – handle many potential races • Non-premptive • All interactions protected by costs system synch ops • Events: signaling • Scheduling: • Complex threads require sophisticating scheduling • Collections of simple events ?? cs294-1 f03

23. Communication • Essentially just like a call • Receive is inherently asynchronous • Don’t introduce potentially unbounded storage allocation • Avoid copies and gather/scatter (mbuf problem) cs294-1 f03

24. Tiny Active Messages • Sending • Declare buffer storage in a frame • Request Transmission • Name a handler • Handle Completion signal • Receiving • Declare a handler • Firing a handler • automatic • behaves like any other event • Buffer management • strict ownership exchange • tx: done event => reuse • rx: must rtn a buffer cs294-1 f03

25. bool pending; struct TOS_Msg buf; command result_t IntOutput.output(uint16_t value) { IntMsg *message = (IntMsg *)buf.data; if (!pending) { pending = TRUE; message->val = value; message->src = TOS_LOCAL_ADDRESS; if (call Send.send(TOS_BCAST_ADDR, sizeof(IntMsg), &buf)) return SUCCESS; pending = FALSE; } return FAIL; } destination length Sending a message • Refuses to accept command if buffer is still full or network refuses to accept send command • User component provide structured msg storage cs294-1 f03

26. Send done event event result_t IntOutput.sendDone(TOS_MsgPtr msg, result_t success) { if (pending && msg == &buf) { pending = FALSE; signal IntOutput.outputComplete(success); } return SUCCESS; } } • Send done event fans out to all potential senders • Originator determined by match • free buffer on success, retry or fail on failure • Others use the event to schedule pending communication cs294-1 f03

27. Receive Event • Active message automatically dispatched to associated handler • knows the format, no run-time parsing • performs action on message event • Must return free buffer to the system • typically the incoming buffer if processing complete event TOS_MsgPtr ReceiveIntMsg.receive(TOS_MsgPtr m) { IntMsg *message = (IntMsg *)m->data; call IntOutput.output(message->val); return m; } cs294-1 f03

28. Maintaining Scheduling Agility • Need logical concurrency at many levels of the graph • While meeting hard timing constraints • sample the radio in every bit window • Retain event-driven structure throughout application • Tasks extend processing outside event window • All operations are non-blocking cs294-1 f03

29. CRCfilter RadioTiming ChannelMon A Complete Application SenseToRfm generic comm IntToRfm AMStandard RadioPacket UARTPacket packet noCRCPacket photo Timer MicaHighSpeedRadioM phototemp SecDedEncode SW byte SPIByteFIFO RandomLFSR HW ADC UART ClockC bit SlavePin Mica2 stack Interpostioning ChipCon cs294-1 f03

30. Composition • A component specifies a set of interfaces by which it is connected to other components • provides a set of interfaces to others • uses a set of interfaces provided by others • Interfaces are bi-directional • include commands and events • Interface methods are the external namespace of the component provides StdControl Timer provides interface StdControl; interface Timer: uses interface Clock Timer Component Clock uses cs294-1 f03

31. Components • Modules • provide code that implements one or more interfaces and internal behavior • Configurations • link together components to yield new component • Interface • logically related set of commands and events Clock.nc interface Clock { command result_t setRate(char interval, char scale); event result_t fire(); } StdControl.nc interface StdControl { command result_t init(); command result_t start(); command result_t stop(); } cs294-1 f03

32. IntToRfm Photo ClockC Example top level configuration configuration SenseToRfm { // this module does not provide any interface } implementation { components Main, SenseToInt, IntToRfm, ClockC, Photo as Sensor; Main.StdControl -> SenseToInt; Main.StdControl -> IntToRfm; SenseToInt.Clock -> ClockC; SenseToInt.ADC -> Sensor; SenseToInt.ADCControl -> Sensor; SenseToInt.IntOutput -> IntToRfm; } Main StdControl SenseToInt ADCControl ADC IntOutput Clock cs294-1 f03

33. Nested configuration StdControl IntOutput includes IntMsg; configuration IntToRfm { provides { interface IntOutput; interface StdControl; } } implementation { components IntToRfmM, GenericComm as Comm; IntOutput = IntToRfmM; StdControl = IntToRfmM; IntToRfmM.Send -> Comm.SendMsg[AM_INTMSG]; IntToRfmM.SubControl -> Comm; } IntToRfmM SendMsg[AM_INTMSG]; SubControl GenericComm cs294-1 f03

34. IntToRfm Module command result_t StdControl.start() { return call SubControl.start(); } command result_t StdControl.stop() { return call SubControl.stop(); } command result_t IntOutput.output(uint16_t value) { ... if (call Send.send(TOS_BCAST_ADDR, sizeof(IntMsg), &data) return SUCCESS; ... } event result_t Send.sendDone(TOS_MsgPtr msg, result_t success) { ... } } includes IntMsg; module IntToRfmM { uses { interface StdControl as SubControl; interface SendMsg as Send; } provides { interface IntOutput; interface StdControl; } } implementation { bool pending; struct TOS_Msg data; command result_t StdControl.init() { pending = FALSE; return call SubControl.init(); } cs294-1 f03

35. A Multihop Routing Example cs294-1 f03

36. Given the framework, what’s the system? • Core Subsystems • Simple Display (LEDS) • Identity • Timer • Bus interfaces (i2c, SPI, UART, 1-wire) • Data Acquisition • Link-level Communication • Power management • Non-volatile storage • Higher Level Subsystems • Network-level communication • Broadcast, Multihop Routing • Time Synchronization, Ranging, Localization • Network Programming • Neighborhood Management • Catalog, Config, Query Processing, Virtual Machine cs294-1 f03

37. Timer • Clock abstraction over HW mechanism • 3-4 registers of various sizes • Operations to set scale, phase, limit • One-shot, periodic • Signals critical event • One physical clock must operate in minimum energy state with rest of the device power off • Timer provides collection of logical clocks • Clean units • One-shot or periodic • Manage Underlyng Clock Resources • Mapping to physical clock • Minimum spacing and accuracy • Signals non-critical event cs294-1 f03

38. Data Acquistion • Many different kinds of sensor connections • Analog sensor into ADC • Digital Sensors (i2c) • External ADCs • Get Command to Logical Sensor • Data-ready signal • Power management • Quality of sampling interval (jitter) depends on timer and sensor substack • Sample rate limits? cs294-1 f03

39. Communication • Parameterized Active Message Abstraction • Multiple Media • Radio • RFM, Mica-RFM, ChipCon • UART • i2C • Media routing based on destination address • Side Attributes • Link-level ack • Timestamp • Signal-strength • Security can be interposed (TinySec) • Inherently Bursty and Unpredictable • Unless higher level protocols make it predictable cs294-1 f03

40. Composable Power Management • Scheduler • On idle drops into preset sleep state till interrupt • Proc: Active: 5 mA, Idle: 2 mA, Sleep: 5 uA • Radio, Sensor, Co-Processors • All Components implement Std_Control Interface • Power mgmt of non-processor resources • Start: bring subsystem to operational level, inform pwr_mgmt • Stop: bring subsystem to sleep, inform pwr_mgmt • Power-management Component • Establishes sleep state should scheduler idle • Knows what is shut down, potential sleep duration • Timer • Pulls system out of deep sleep • What happens after depends on what event handler does • Applications compose policy cs294-1 f03

41. Example Pwr Mgmt Policies • Great Duck Island • Every 5 mins: sample, send, low_power_route • Low-power listen • Transmit include wake preamble (x) • Receive powers off radio (~x) – if sufficiently long • Application just adjusts x • Generic Sensor Kit: epochs • Periodic: Active phase / Sleep Phase • Active • Sample, process query, route, aggregate • Sleep • Power down for set time cs294-1 f03

42. Dynamics • All sleep till clock expires • Interrupts proc to HW event • Critical handling, signal clock event • Timer Handles Clock Event • If merely expired as part of longer sleep, system returns to deep sleep • If events are associated with event, they are signalled • Post tasks • Call Start, … cs294-1 f03

43. Typical Operational Mode • Major External Events • Trigger collection of small processing steps (tasks and events) • May have interval of hard real time sampling • Radio • Sensor • Interleaved with moderate amount of processing in small chunks at various levels • Periods of sleep • Interspersed with timer mgmt cs294-1 f03

44. Where to classical RTOS issues arise? • Low latency events • Two-level scheduling, short critical events • Lowest layer of components complete critical section before signalling logical event • Closely aligned low-latency events? • Radio/uart/timer happen to hit at same time • Periodic, pseudo-periodic • Timer component design • Static scheduling • Rate monotonic? • Scheduler? • Networking is bursty. Network sheduling? • Deadline? • Data queuing • Load shedding? • At load duty cycle? • Provability? cs294-1 f03

45. Supporting HW evolution • Distribution broken into • apps: top-level applications • lib: shared application components • system: hardware independent system components • platform: hardware dependent system components • includes HPLs and hardware.h • Component design so HW and SW look the same • example: temp component • may abstract particular channel of ADC on the microcontroller • may be a SW i2C protocol to a sensor board with digital sensor or ADC • HW/SW boundary can move up and down with minimal changes cs294-1 f03

46. Scalable Simulation Environment • target platform: TOSSIM • whole application compiled for host native instruction set • event-driven execution mapped into event-driven simulator machinery • storage model mapped to thousands of virtual nodes • radio model and environmental model plugged in • bit-level fidelity • Sockets = basestation • Complete application • including GUI cs294-1 f03

47. Issues • Timer Architecture far more fundamental than anticipated • Simple applications built around clock • One component set period, all took multiples of it. • Poor composition. • Application Oblivious Timer too imprecise • More complex applications have 1 high precision timer • maybe 2 • sometimes high rate • Besides radio • Must be serviced by hardware periodic counter • Other general uses should have a clean way to obtain intervals in multiples of primary tick in clean manner • Offset in phase to avoid collision • Coordinate multiple low-jitter subsystems • UART and Radio cs294-1 f03

48. Multiple levels of tasks • Easy to implement multiple, static priorities upon single stack • Lowest level events handler immediately • “System” tasks to realize concurrency within components • Must still be quick to keep flow throughout system smooth • Grains of sand, not bricks • “Background” tasks • Run as long as they want, preempted by system tasks • Build next to it, but not on top of it. cs294-1 f03

49. Co-processors for fidelity • Several designs have subsystems with dedicated tinyOS processors • Ranging • MotoControl cs294-1 f03

50. Ranging Board • Lessons from sounder, UCLA ultrasound, VU ranging • Dedicated Atmega 8, like motor-control board • Dual proc TinyOS with UART link • TX • Command Processor • Generate 25 KHz 10v signal (range) • RX • Triggered by radio chirp • Analog switch • Starts Atmega 8 timer • Analog compare amplified recv with digital pot threshold. • Completes timer • Signals time-stamped event 25 KHz US Transceiver tone Atmel 8 digital pot UART AM channel Main TinyOS Mote cs294-1 f03