Implementation and Research Issues in Query Processing for Wireless Sensor Networks

1. Implementation and Research Issues in Query Processing for Wireless Sensor Networks Wei Hong Intel Research, Berkeley whong@intel-research.net Sam Madden MIT madden@csail.mit.edu MDM Tutorial, January 19th 2004

2. Motivation • Sensor networks (aka sensor webs, emnets) are here • Several widely deployed HW/SW platforms • Low power radio, small processor, RAM/Flash • Variety of (novel) applications: scientific, industrial, commercial • Great platform for mobile + ubicomp experimentation • Real, hard research problems to be solved • Networking, systems, languages, databases • We will summarize: • The state of the art • Our experiences building TinyDB • Current and future research directions Berkeley Mote

3. Earthquake monitoring in shake-test sites. Vehicle detection: sensors along a road, collect data about passing vehicles. • Traditional monitoring apparatus. Sensor Network Apps Habitat Monitoring: Storm petrels on Great Duck Island, microclimates on James Reserve. Just the tip of the iceberg -- more tomorrow!

4. Declarative Queries • Programming Apps is Hard • Limited power budget • Lossy, low bandwidth communication • Require long-lived, zero admin deployments • Distributed Algorithms • Limited tools, debugging interfaces • Queries abstract away much of the complexity • Burden on the database developers • Users get: • Safe, optimizable programs • Freedom to think about apps instead of details

5. TinyDB: Prototype declarativequery processor • Platform: Berkeley Motes + TinyOS • Continuous variant of SQL : TinySQL • Power and data-acquisition based in-network optimization framework • Extensible interface for aggregates, new types of sensors

6. Agenda • Part 1 : Sensor Networks (50 Minutes) • TinyOS • NesC • Short Break • Part 2: TinyDB (1 Hour) • Data Model and Query Language • Software Architecture • Long Break + Hands On • Part 3: Sensor Network Database Research Directions (1 Hour, 10 Minutes)

7. Part 1 • Sensornet Background • Motes + Mote Hardware • TinyOS • Programming Model + NesC • TinyOS Architecture • Major Software Subsystems • Networking Services

8. A Brief History of Sensornets • People have used sensors for a long time • Recent CS History: • (1998) Pottie + Kaiser: Radio based networks of sensors • (1998) Pister et al: Smart Dust • Initial focus on optical communication • By 1999, radio based networks, COTS Dust, “Motes” • (1999) Estrin + Govindan • Ad-hoc networks of sensors • (2000) Culler/Hill et al: TinyOS + Motes • (2002) Hill / Dust: SPEC, mm^3 scale computing • UCLA / USC / Berkeley Continue to Lead Research • Many other players now • TinyOS/Motes as most common platform • Emerging commercial space: • Crossbow, Ember, Dust, Sensicast, Moteiv, Intel

9. Why Now? • Commoditization of radio hardware • Cellular and cordless phones, wireless communication • (some radio pictures, etc.) • Low cost -> many/tiny -> new applications! • Real application for ad-hoc network research from the late 90’s • Coming together of EE + CS communities

10. Motes 4Mhz, 8 bit Atmel RISC uProc 40 kbit Radio 4 K RAM, 128 K Program Flash, 512 K Data Flash AA battery pack Based on TinyOS Mica Mote Mica2Dot

11. History of Motes • Initial research goal wasn’t hardware • Has since become more of a priority with emerging hardware needs, e.g.: • Power consumption • (Ultrasonic) ranging + localization • MIT Cricket, NEST Project • Connectivity with diverse sensors • UCLA sensor board • Even so, now on the 5th generation of devices • Costs down to ~$50/node (Moteiv, Dust) • Greatly improved radio quality • Multitude of interfaces: USB, Ethernet, CF, etc. • Variety of form factors, packages

12. Motes vs. Traditional Computing • Lossy, Adhoc Radio Communication • Sensing Hardware • Severe Power Constraints

13. From Ganesan, et al. “Complex Behavior at Scale.” UCLA/CSD-TR 02-0013 Radio Communication • Low Bandwidth Shared Radio Channel • ~40kBits on motes • Much less in practice • Encoding, Contention for Media Access (MAC) • Very lossy: 30% base loss rate • Argues against TCP-like end-to-end retransmission • And for link-layer retries • Generally, not well behaved

14. Types of Sensors • Sensors attach via daughtercard • Weather • Temperature • Light x 2 (high intensity PAR, low intensity, full spectrum) • Air Pressure • Humidity • Vibration • 2 or 3 axis accelerometers • Tracking • Microphone (for ranging and acoustic signatures) • Magnetometer • GPS

15. Power Consumption and Lifetime • Power typically supplied by a small battery • 1000-2000 mAH • 1 mAH = 1 milliamp current for 1 hour • Typically at optimum voltage, current drain rates • Power = Watts (W) = Amps (A) * Volts (V) • Energy = Joules (J) = W * time • Lifetime, power consumption varies by application • Processor: 5mA active, 1 mA idle, 5 uA sleeping • Radio: 5 mA listen, 10 mA xmit/receive, ~20mS / packet • Sensors: 1 uA -> 100’s mA, 1 uS -> 1 S / sample

16. Energy Usage in A Typical Data Collection Scenario • Each mote collects 1 sample of (light,humidity) data every 10 seconds, forwards it • Each mote can “hear” 10 other motes • Process: • Wake up, collect samples (~ 1 second) • Listen to radio for messages to forward (~1 second) • Forward data

17. Sensors: Slow, Power Hungry, Noisy

18. Programming Sensornets: TinyOS • Component Based Programming Model • Suite of software components • Timers, clocks, clock synchronization • Single and multi-hop networking • Power management • Non-volatile storage management

19. Programming Philosophy • Component Based • “Wiring” to components together via interfaces, configurations • Split-Phased • Nothing blocks, ever. • Instead, completion events are signaled. • Highly Concurrent • Single thread of “tasks”, posted and scheduled FIFO • Events “fired” asynchronously in response to interrupts.

20. NesC • C-like programming language with component model support • Compiles into GCC-compatible C • 3 types of files: • Interfaces • Set of function prototypes; no implementations or variables • Modules • Provide (implement) zero or more interfaces • Require zero or more interfaces • May define module variables, scoped to functions in module • Configurations • Wire (connect) modules according to requires/provides relationship

21. Component Example: Leds module LedsC { provides interface Leds; } implementation { uint8_t ledsOn; enum { RED_BIT = 1, GREEN_BIT = 2, YELLOW_BIT = 4 }; …. async command result_t Leds.redOn() { dbg(DBG_LED, "LEDS: Red on.\n"); atomic { TOSH_CLR_RED_LED_PIN(); ledsOn |= RED_BIT; } return SUCCESS; } …. }

22. Configuration Example configuration CntToLedsAndRfm { } implementation { components Main, Counter, IntToLeds, IntToRfm, TimerC; Main.StdControl -> Counter.StdControl; Main.StdControl -> IntToLeds.StdControl; Main.StdControl -> IntToRfm.StdControl; Main.StdControl -> TimerC.StdControl; Counter.Timer -> TimerC.Timer[unique("Timer")]; IntToLeds <- Counter.IntOutput; Counter.IntOutput -> IntToRfm; }

23. Split Phase Example module IntToRfmM { … } implementation { … command result_t IntOutput.output (uint16_t value) { IntMsg *message = (IntMsg *)data.data; if (!pending) { pending = TRUE; message->val = value; atomic { message->src = TOS_LOCAL_ADDRESS; } if (call Send.send(TOS_BCAST_ADDR, sizeof(IntMsg), &data)) return SUCCESS; pending = FALSE; } return FAIL; } event result_t Send.sendDone (TOS_MsgPtr msg, result_t success) { if (pending && msg == &data) { pending = FALSE; signal IntOutput.outputComplete (success); } return SUCCESS; } } }

24. Major Components • Timers: Clock, TimerC, LogicalTime • Networking: Send, GenericComm, AMStandard, lib/Route • Power Management: HPLPowerManagement • Storage Management: EEPROM, MatchBox

25. Timers • Clock: Basic abstraction over hardware timers; periodic events, single frequency. • LogicalTime: Fire an event some number of H:M:S:ms in the future. • TimerC: Multiplex multiple periodic timers on top of LogicalTime.

26. Radio Stack IntMsg *message = (IntMsg *)data.data; … message->val = value; atomic { message->src = TOS_LOCAL_ADDRESS; } call Send.send(TOS_BCAST_ADDR, sizeof(IntMsg), &data)) • Interfaces: • Send • Broadcast, or to a specific ID • split phase • Receive • asynchronous signal • Implementations: • AMStandard • Application specific messages • Id-based dispatch • GenericComm • AMStandard + Serial IO • Lib/Route • Mulithop event TOS_MsgPtr ReceiveIntMsg. receive(TOS_MsgPtr m) { IntMsg *message = (IntMsg *)m->data; call IntOutput.output(message->val); return m; } Wiring to equate IntMsg to ReceiveIntMsg

27. A B B R:{…} R:{…} B B C B B B B R:{…} D R:{…} B Node D NeighQual B .75 C .66 E .45 F .82 Node C NeighQual A .5 B .44 D .53 F .35 R:{…} B B B F E B Multihop Networking • Standard implementation “tree based routing” Problems: Parent Selection Asymmetric Links Adaptation vs. Stability

28. Geographic Routing • Any-to-any routing via geographic coordinates • See “GPSR”, MOBICOM 2000, Karp + Kung. • Requires coordinate system* • Requires endpont coordinates • Hard to route around local minima (“holes”) B A *Could be virtual, as in Rao et al “Geographic Routing Without Coordinate Information.” MOBICOM 2003

29. Power Management • HPLPowerManagement • TinyOS sleeps processor when possible • Observes the radio, sensor, and timer state • Application managed, for the most part • App. must turn off subsystems when not in use • Helper utility: ServiceScheduler • Peridically calls the “start” and “stop” methods of an app • More on power management in TinyDB later • Approach works because: • single application • no interactivity requirements

30. Non-Volatile Storage • EEPROM • 512K off chip, 32K on chip • Writes at disk speeds, reads at RAM speeds • Interface : random access, read/write 256 byte pages • Maximum throughput ~10Kbytes / second • MatchBox Filing System • Provides a Unix-like file I/O interface • Single, flat directory • Only one file being read/written at a time

31. TinyOS: Getting Started • The TinyOS home page: • http://webs.cs.berkeley.edu/tinyos • Start with the tutorials! • The CVS repository • http://sf.net/projects/tinyos • The NesC Project Page • http://sf.net/projects/nescc • Crossbow motes (hardware): • http://www.xbow.com • Intel Imote • www.intel.com/research/exploratory/motes.htm.

32. Part 2 The Design and Implementation of TinyDB

33. Part 2 Outline • TinyDB Overview • Data Model and Query Language • TinyDB Java API and Scripting • Demo with TinyDB GUI • TinyDB Internals • Extending TinyDB • TinyDB Status and Roadmap

34. Sensor Network TinyDB Revisited SELECT MAX(mag) FROM sensors WHERE mag > thresh SAMPLE PERIOD 64ms • High level abstraction: • Data centric programming • Interact with sensor network as a whole • Extensible framework • Under the hood: • Intelligent query processing: query optimization, power efficient execution • Fault Mitigation: automatically introduce redundancy, avoid problem areas App Query, Trigger Data TinyDB

35. Feature Overview • Declarative SQL-like query interface • Metadata catalog management • Multiple concurrent queries • Network monitoring (via queries) • In-network, distributed query processing • Extensible framework for attributes, commands and aggregates • In-network, persistent storage

36. Architecture TinyDB GUI JDBC TinyDB Client API DBMS PC side 0 Mote side 0 TinyDB query processor 2 1 3 8 4 5 6 Sensor network 7

37. Data Model • Entire sensor network as one single, infinitely-long logical table: sensors • Columns consist of all the attributes defined in the network • Typical attributes: • Sensor readings • Meta-data: node id, location, etc. • Internal states: routing tree parent, timestamp, queue length, etc. • Nodes return NULL for unknown attributes • On server, all attributes are defined in catalog.xml • Discussion: other alternative data models?

38. Query Language (TinySQL) SELECT <aggregates>, <attributes> [FROM {sensors | <buffer>}] [WHERE <predicates>] [GROUP BY <exprs>] [SAMPLE PERIOD <const> | ONCE] [INTO <buffer>] [TRIGGER ACTION <command>]

39. Comparison with SQL • Single table in FROM clause • Only conjunctive comparison predicates in WHERE and HAVING • No subqueries • No column alias in SELECT clause • Arithmetic expressions limited to column op constant • Only fundamental difference: SAMPLE PERIOD clause

40. TinySQL Examples SELECT nodeid, nestNo, light FROM sensors WHERE light > 400 EPOCH DURATION 1s “Find the sensors in bright nests.” Sensors 1

41. 2 SELECT AVG(sound) FROM sensors EPOCH DURATION 10s • SELECT region, CNT(occupied) AVG(sound) • FROM sensors • GROUP BY region • HAVINGAVG(sound) > 200 • EPOCH DURATION 10s 3 Regions w/ AVG(sound) > 200 TinySQL Examples (cont.) “Count the number occupied nests in each loud region of the island.”

42. Event-based Queries • ON event SELECT … • Run query only when interesting events happens • Event examples • Button pushed • Message arrival • Bird enters nest • Analogous to triggers but events are user-defined

43. Query over Stored Data • Named buffers in Flash memory • Store query results in buffers • Query over named buffers • Analogous to materialized views • Example: • CREATE BUFFER name SIZE x (field1 type1, field2 type2, …) • SELECT a1, a2 FROM sensors SAMPLE PERIOD d INTO name • SELECT field1, field2, … FROM name SAMPLE PERIOD d

44. Using the Java API • SensorQueryer • translateQuery() converts TinySQL string into TinyDBQuery object • Static query optimization • TinyDBNetwork • sendQuery() injects query into network • abortQuery() stops a running query • addResultListener() adds a ResultListener that is invoked for every QueryResult received • removeResultListener() • QueryResult • A complete result tuple, or • A partial aggregate result, call mergeQueryResult() to combine partial results • Key difference from JDBC: push vs. pull

45. Writing Scripts with TinyDB • TinyDB’s text interface • java net.tinyos.tinydb.TinyDBMain –run “select …” • Query results printed out to the console • All motes get reset each time new query is posed • Handy for writing scripts with shell, perl, etc.

46. Using the GUI Tools • Demo time

47. SELECT AVG(temp) WHERE light > 400 T:1, AVG: 225 T:2, AVG: 250 Queries Results Aggavg(temp) Name: temp Time to sample: 50 uS Cost to sample: 90 uJ Calibration Table: 3 Units: Deg. F Error: ± 5 Deg F Get f: getTempFunc()… got(‘temp’) get (‘temp’) Tables Samples getTempFunc(…) Inside TinyDB Multihop Network Query Processor ~10,000 Lines Embedded C Code ~5,000 Lines (PC-Side) Java ~3200 Bytes RAM (w/ 768 byte heap) ~58 kB compiled code (3x larger than 2nd largest TinyOS Program) Filterlight > 400 Schema TinyOS TinyDB

48. Q:SELECT … A Q Q R:{…} R:{…} Q B C Q Q Q Q R:{…} D R:{…} Q R:{…} Q Q Q F E Q Tree-based Routing • Tree-based routing • Used in: • Query delivery • Data collection • In-network aggregation • Relationship to indexing?

49. Power Management Approach Coarse-grained app-controlled communication scheduling Epoch (10s -100s of seconds) Mote ID 1 … zzz … … zzz … 2 3 4 5 time 2-4s Waking Period

50. Time Synchronization • All messages include a 5 byte time stamp indicating system time in ms • Synchronize (e.g. set system time to timestamp) with • Any message from parent • Any new query message (even if not from parent) • Punt on multiple queries • Timestamps written just after preamble is xmitted • All nodes agree that the waking period begins when (system time % epoch dur = 0) • And lasts for WAKING_PERIOD ms • Adjustment of clock happens by changing duration of sleep cycle, not wake cycle.