ENS Platforms for Environmental Observatories CENS Seminar August 26, 2005

1. ENS Platforms for Environmental Observatories CENS Seminar August 26, 2005 Dustin McIntire Bernie Yip Kei Ho Hing Prof. Bill Kaiser

2. Seminar Overview • Environmental Observatories • What are they? • What makes them different? • Example Applications • Embedded Networked Sensing (ENS) Application Classes • A New ENS Platform for Environmental Observatories • Hardware Architecture • Software Architecture • Spring 2005 EE209S Test Bed and Current Status • Test bed design and project description • Example EE209S project solution • Recent developments and improvements • Future Work • A new design flow • Iterative development cycle description • Collaborations

3. Environmental Observatories What are they? • Multi-user / Multi-application systems • Support more than one user and/or application possibly at the same time • May be in remote or difficult to reach locations • e.g. James Reserve, Medea Creek, Merced River, White Mountains, Costa Rica • May have long dormancy periods with intermittent periods of high activity • could be user initiated or environmentally stimulated • Contain high performance sensing equipment • High fidelity sensors often require large power draw for low distortion and/or high gain • Relatively high instantaneous peak power consumption • Large data payloads possible (imagers may be MB to GB of data) • Mote / Microserver architecture currently most popular for this application space

4. Environmental Observatories: Application Examples • Micrometeorology • Cold Air Drainage investigations at James Reserve • Regular, triggered, and model-driven sampling of: • Atmospheric temperature, humidity, others • AMARSS Program at James Reserve • Comprehensive observation of forest soil • subsurface and surface phenomena for understanding of plant growth • Regular, triggered, and model-driven sampling of: • Micrometeorology, gas analysis, soil properties • Imaging (root growth) • NAMOS (Profs. David Caron and Gaurav Sukhatme at USC) • Biological phenomena in aquatic environments deployed at JR. • Regular, triggered, and model-driven sampling of biologicals • Relies on complex fluorometer instrument • Requires management of energy and data processing support • Plant Phenology • High precision actuated imager tracking plant growth • Regular, triggered, or model-driven imaging • NIMS • Increasing reliance on complex instruments for water quality, visible imaging, thermal infrared imaging, and other sensors

5. ENS Applications in Observatories • Always vigilant sensing with support for in-network processing • Favors mote architecture • Large processing demands may be offloaded into network • Intermittent vigilance with scheduled processing • Current microserver solutions are an example • Require complete platform shutdown to reduce energy usage • Always vigilant with scheduled demand for platform resources • Compatible with many observational problems in the environment • Work may be deferred until a later scheduled time • Always vigilant with intermittent, on demand processing • Can be applied to many problems from environmental to security • Similar to human awareness levels • Processing demand according to • Sensor event triggers • Model-driven scheduling

6. A New ENS Platform Approach

7. ENS Platform Design Requirements • Long term low energy operation for extended lifetime • Support for high performance sensors and actuators on demand • Utilize open source projects as much as possible • operating systems • libraries • tools • Provide broadband networking including: • user authentication • secure communications

8. ENS Platform Design Requirements • Permit multiple users with simultaneous platform access • Provide exclusive access to specific resources when required • Provide accurate accounting and quota capability for constrained platform resources (memory, energy, bandwidth).

9. ENS Platform Approach • Current microserver systems tend to operate only at the high performance and high power end of the spectrum • From an operational efficiency perspective this is good! • Stargate vs. MICA2 Mote computation efficiency (next slide) • 802.11x vs Zigbee communication efficiency • But under utilized resources lead to lots of wasted energy • Need to utilize DPM (dynamic power management) to control the energy usage • Can leverage lots of prior work done on processor centric DPM • We require a real-time visibility into how energy in consumed at a fine grained level • Need feedback on how policy changes have affected the system both at the micro and macro levels

10. Computing Efficiency vs. Power • Low power does not always mean high efficiency • High performance CPU’s often offer high efficiency as well

11. Communication Efficiency vs. Power • Again low power does not always mean high efficiency • High performance networking devices often offer higher efficiency • Previous experiments with 802.11b and RFM TR1000 show the same results

12. Slauson + LEAP Hardware Architecture Slauson (Sensoria PXA255 Platform) LEAP (Low power Energy Aware Preprocesor) SDCard/MMC SPI I2C GPIO JTAG Ethernet Shutdown Address/Data Vsense PCMCIA1 SDRAM Flash + - Current Sensed Supply Outputs PCMCIA2 x5 Sensor Voltage Sensor Inputs x2 USB Host Controller USB Host

13. Slauson + LEAP Software Architecture Slauson LEAP PXA255 MSP430 • Modified uC/OS for MSP430 to use msp-gcc compiler • uC/OS is a small footprint RTOS with simple tasks, semaphores, etc. • MSP code debuggable from remote PC (external JTAG) or Slauson PXA (GPIO JTAG) via msp-gdbproxy uC/OS JFFS2 Flash File System Busybox Cramfs Linux 2.6.12 OS RedBoot Bootloader • All open source GPL projects • ipkg capable file system updates from local repo • Remote firmware upgradeable • Looking into OpenEmbedded or Gentoo for flash file system building

14. EE209S Project Spring 2005

15. EE209S Test Bed Motivation • Provide a general embedded systems teaching platform with little to no hardware specific knowledge necessary • Allow students to be ‘idea’ constrained rather than ‘implementation’ constrained • Reduce much of the time spent debugging basic interfaces and focus on the overall system architecture • Provide many basic building blocks to construct a complex system from simple software components • Allow greater latitude in algorithm exploration • Move from data acquisition/logging applications to event detection and classification in a controlled environment • Provide research test bed for CENS and external groups

16. EE209S Test Bed Architecture EE209S Node 1 Test Bed System Components • Linux PC • internet gateway to ENS network • cross compiler for nodes • user home directories and data storage • EE209S nodes • basic file system applications • ENS utility applications • user specific applications • Events Generator • red/green light array • controlled from PC application Internet Linux PC EE209S Node 2 EE209S Node 3 … EE209S Node N

17. EE209S Test Bed Example • Red event detected and ignored • Green event detected and classified Viewing Obstacle Event Generator Server Field of View Of Trigger Sensor Actuated Image Sensor Node

18. EE209S Project Description • Project goal is to detect and classify a finite number of known environmental events • results scored by accuracy and total system energy usage • Events defined by simple state based contexts • A context is an issue rate and speed vector tuple {slow,medium,fast} • Student teams must correctly detect and classify all events with minimum energy usage • Context duration or dwell time is varied • Students may change node behavior based upon current context state and previous detection events • Many adaptive heuristics were used • Some performed well at very low energy but missed transition events • Others were extremely accurate but higher energy solutions

19. EE209S Node Architecture Sony Camera 802.11b PCMCIA Ethernet PCMCIA Slauson LEAP Photo Diode ADC • Slauson + LEAP • provides local processing and power duty cycling • provides micro-power vigilant state • Sony SNC-RC30N Camera • 360 degree zoom/pan/tilt • power is duty cycled by LEAP • 802.11b PCMCIA card • broadband communications channel • may be duty cycled by Slauson • Silicon photo diode • micro-power detection sensor • used as a wakeup signal

20. EE209S Test Bed Software Architecture Power Mgt Scheduler I2C Messaging Tripwire Detect MSP Client App Energy Accumulation Student Algorithms Imaging Support Apps Communications Apps Slauson LEAP PXA255 MSP430 Linux 2.6 uC/OS

21. EE209S LEAP Preprocessor API • Linux side msp-client application abstracts messaging API to MSP software • Removes the complex message framing and handshaking over I2C bus • Handles bus collision and error recovery • Simple command line interface that allows single or batched (file based) command sequences to facilitate scripting • EmStar wrapper library would be a simple addition • Sample of msp-client commands: • charge – read all charge accumulator values and optionally reset • sensor – read sensor ADC values and optionally do simple processing (min,max,ave.) • readtime – read the MSP current tick value to coordinate local time values • showpower – read the currently scheduled power management commands • showtrigger – read the voltage level setting for asynchronous resource wakeups • resetalarm – reset a previously triggered alarm • power – schedule a new power management command for future activation (off,on,standby,resume) • hwreset – force a complete hardware reset of entire platform • trigger – set a new triggering voltage and/or channel for the alarm setting • settime – set a new clock time in the MSP • subscribe – subscribe a file name to be written with the time value of the next triggering event

22. EE209S Communications API • 802.11 cards employ WPA encryption for secure physical layer • Node to node communications based on SSL layer for robust, secure links • Pre-exchanged keys provide fast authentication • Key management mechanism will be discussed later • OpenSSH based scp for file transfer and ssh for IPC • Created a set of basic command line utilities: • ping_remote – detect the presence of one or more peers • copy_to_remote – send a local file to one or more peers • get_from_remote – retrieve a remote file onto the local file system • view_remote_directory – view the contents of a remote directory on one or more peer nodes • remote_command – execute a shell command on one or more remote peers

23. EE209S Imager API • JPEG image files retrieved from camera via FTP • Retrieved files are processed by local image processing libraries for image centroid and intensity. Returns (x,y) vector for centroid location. • Created a set of basic command line utilities: • camctrl – direct access to camera functions via command line utility. Camera may be panned, tilted, zoomed, and images captured • search_green – performs 360° rotation in 30° steps, images are green filtered and centroid located • search_red - performs 360° rotation in 30° steps, images are red filtered and centroid located

24. EE209S Example Experiment Data • Events: 10 {f,f} + 10 {m,m} + 10 {s,s}

25. EE209S Example Experiment Data

26. EE209S Example Experiment Data

27. EE209S Algorithm Comparisons Algorithm A Energy Performance (detection/classification accuracy %) Experiment 1 Transition Rate (context duration-1)

28. EE209S Lessons Learned • Scheduling and coordination of system resources is not optional, it’s mandatory • Each group with its own testbed is not practical for cost and space reasons • Unrestricted access for all groups led to numerous resource conflicts • Scheduling node usage via a simple web application (no enforcement) led to accidental node contentions and low resource utilization • Need an automated resource scheduler and resource policing • Manual log harvesting from nodes after experiments then merging is tedious • Need to provide a standard log format for simple offline post processing • Need to provide an automated means to terminate experiments and collect the log records • Iteration process is slow due to excessive manual intervention • Each iteration required code modifications to be pushed out to all nodes • Nodes had to be resyncronized and the experiments restarted by hand • Most of this process is redundant between iterations • Need to provide an automated means to remove old algorithms from the nodes, push new algorithms to the nodes and restart all nodes synchronously

29. Current ENS Test Bed Progress

30. ENS Testbed Progress • Added versatile resource scheduling and policing system to node test bed • Fine or course grained resources may be specified (e.g the serial port on slauson1 or all nodes in test bed) • Users or groups must be on permitted access list for requested resource • Users must request resource for specified time interval or be put on a waiting list to be allowed access • Users must check-in at start of requested time interval to eliminate no-shows. The waiting list is used if no check-in is received • Added automated algorithm deployment and mechanism • At end of allocated time interval the previous experiment can be terminated and processes killed. The previous user is optionally removed from nodes after saving the results directory • The active user is deployed and his processes started from a specified startup file • The active user may kill, deploy, and restart via single interface

31. Device Management Software Architecture

32. Device Management Software Architecture System administration interface • Direct control of the system through access/permissions files • Run-time adding / removing devices, users, groups, and schedules • Adjustable system resource allocation, e.g. reserved period length, user reservation priorities User interface • Sign-in in advance to reserve parts of the system for exclusive access • Check-in when ready to use. Priority is given to the order of the sign-in users, then waitlist users, and then drop-in users • User check-out prior to allocated time allows anyone to check-in

33. User Home Directory Structure install source

34. User Home Directory Structure • modules - contains all the development system binaries • checkContent - check what is inside remote directory • checkProcess - check what processes are running by the current user • editCode - help .c and .sh file editing • getFiles/sendFiles - transfer file from/to slausons • getSysytem... - help maintain directory structure as shown in the graph • key-regen - regenerate ssh keys for slausons + put key in authorized_key • compile - cross compile source files for target architecture • push/pull - put/get all files in the 'install' directory to their corresponding slausons • utilities – contains basic hardware access utilities for the target nodes described previously. Includes scripts to install user boot scripts and start user binaries. • backup - contains tar files of a users home directory on each slauson node. They are created whenever the “pull” module is executed. The name of the tar files are followed by a date index. • source - contains source code and/or scripts developed by a user for each node. The compile utility in the modules directory will compile the source code in this directory and place the binary files into the user’s bin directory for each node. • install - contains any file that the user wants to be present on each of the remote slausons. It includes binaries located in “slausonx/bin” that is compiled using compile module from the source directory.

35. ENS Test Bed Near-Term Additions • Imaging obstructions • Add known and unknown obstructions between the event generator and the node imagers • Color discrimination • Add light color discrimination utilities to node software • Increase complexity for event descriptions • Current event context is a simple 9 state variable {ss,sm,sf,…ff} • May be expanded to include more complex events (e.g color, direction, number of illuminated lights, etc) • Unknown node ordering • Known fixed ordering may be replaced with unknown ordering • Would require a learning phase prior to running experiment • Increased test bed dimensionality • Linear array can be changed to 2D or 3D version • Could add a mobile event or node

36. The Future – Bridging the Design Process Gap

37. The Design Process Gap • Currently a system designer is unable to accurately account for assignment of energy and other resource usage to individual sensors, processes, and applications • Simple in early systems with small numbers of simple sensors • Soon we will face ENS systems with multiple multimodal sensors • Resource and Performance Profiler • Critical need for resource profiling tools (OS profiling tools starting to appear e.g. LTT, KProbes) • Enables an iterative design cycle and facilitates a continued improvement process • Primary objective is energy efficiency • Other objective functions also possible for other resource types (e.g. maximize off time for battery relaxation, minimize the peak to average power) • Facilitates budgeting and scheduling of shared resources

38. Design Process Overview Test Bed System Control Physical Devices - System Execution Visualization and Analysis Coding and Scripting 1 File distribution 2 System initialization 3 System termination 4 Log harvesting ENS Algorithm Design Event Generation in Matlab Event Generator Control Read test vectors and set the event generator lights (red/green patterns) Algorithm Refinement

39. Design Process Elements • Test bed system control provides functions to: • Transfer files from/to remote nodes • Maintain system directory structure • System startup (e.g. time synchronize and execute from entry) and termination (e.g. killing user’s processes) • Log information harvesting • Event generation in Matlab: • User creates an abstract event type and saves to an event type file • Matlab assists in generating complex models for event sequence (e.g. HMM, stochastic process) • Index.txt contains list of event execution times. It is read and executed by the event generator control program.

40. Conclusions • Environmental observatories are an increasingly important application of ENS systems • Environmental observatories have special needs not addressed in traditional ENS design • Specialized platforms and software are needed to address these shortcomings • Increased system visibility during the algorithm design process is needed to improve results and speed deployment • Iterative design process increases search of solution space and quantitative comparisons of algorithm alternatives

41. Collaborations • ENS-box Concept from Winter 2004 CENS Retreat • Jeff Tseng and Richard Pon • Design based largely on LEAP platform from EE209S • PCB starting fabrication process • Acoustic Localization • Lew Girod • Slauson PXA255 platform with acoustic sampling and 802.11 WLAN via PCMCIA • Linux 2.6 kernel and OpenEmbedded for Stargate • Martin and Naim • Create an open source end-to-end distribution for microserver class platforms

42. Special Thanks • Thank you to Bernie and Kei for their work on the project as well as their help collecting data and creating slides

43. Extra Slides

44. Environmental Observatories: Application Examples (Detailed) • Micrometeorology • Example, Cold Air Drainage investigations at James Reserve • Regular, triggered, and model-driven sampling • AMARSS Program at James Reserve • Comprehensive observation of forest soil • subsurface and surface phenomena for understanding of plant growth • Regular, triggered, and model-driven sampling • micrometeorology and gas analysis (air temperature, humidity, solar radiation) • soil properties (soil moisture, CO2, nitrate concentration ) • Imaging (root growth – mini-rhizotron) • Atmospheric gas analysis • NAMOS • Investigation of biological phenomena in aquatic environments deployed at JR. (Profs. David Caron and Gaurav Sukhatme at USC) • Relies on complex fluorometer instrument • Requires management of energy and data processing support • Plant Phenology • High precision actuated imager tracking plant growth • Image capture on regular, triggered, or model-driven schedule • NIMS • Rapidly deployable arrays of coordinated fixed and mobile nodes • Increasing reliance on complex instruments for water quality, visible imaging, thermal infrared imaging, and other sensors • Increasing importance of low energy to reduce reliance on energy storage for rapidly deployable systems

45. Model Driven Scheduling • Light sensing • Model derived from past diurnal and seasonal behavior determine when light sampling will occur • Plant phenology • Weather conditions drive model that indicates when additional imaging is required

46. Environmental Observatories • New Architectures • New investigations drive requirements for capable instruments • Sensor device periperals are advancing • Node must support intelligent peripheral • Node Applications • Node • Micropower Microserver • Application Categories • Regular, deterministic schedule • Preprocessor manages sensor assets • Processor applied only on demand (data storage or data compression and transport) • Sensor Data Trigger • Multisensor trigger algorithm hosted on preprocessor • Processor applied on demand to adapt to changing environment context • Optional dedicated preprocessor • Model driven • Multisensor model algorithm hosted on preprocessor • Processor applied on demand to adapt to changing environment context • Optional dedicated preprocessor • EE209 Field exp