390 likes | 561 Views
DyMND: Robust Adaptive Coordination in Dynamic Meshes of Networked Devices NEST PI Meeting June 29, 2004. Prasanta Bose Advanced Technology Center Lockheed Martin Space Systems Company Palo Alto, CA. Outline. Robust DyMND Engineering DyMNDSim and DyMNDES: The first generation
E N D
DyMND:Robust Adaptive Coordinationin Dynamic Meshes of Networked DevicesNEST PI MeetingJune 29, 2004 Prasanta Bose Advanced Technology Center Lockheed Martin Space Systems Company Palo Alto, CA
Outline • Robust DyMND Engineering • DyMNDSim and DyMNDES: The first generation • Lessons learned • DyMND-EE: The next generation • Architecture • Implementation • Experiments • Adaptive Coordination in DyMND • Explicit local coordination for optimal control • Dynamic programming (reinforcement learning) • Design and validation strategy within DyMND-EE • Summary and Future Plan
Client Proxy Simulation Director Motes Radio Effectors Sensors Target models Active message handler SvcCoord TaskQ DyMNDSim server DyMNDSim and DyMNDES – The First Generation • DyMNDSim • Discrete Event Simulation • Full-featured tool for implementing sensing/radio models • Integration with Motes and Tossim • DyMNDES • Real-time injection and data retrieval • Scenario driven interaction • Script, save, and load emulation event sequences • Generation of abstract mote and target capabilities via virtual objects • 3D visualization of aggregate state
Lessons Learned • Translation of services from DyMNDSim to motes required manual recoding • DyMNDSim emulated TinyOS active messages and event/task distinction, but executed Java classes • Hardware nodes led to noisy, non-repeatable experiments • Debugging and monitoring required source-code modifications • Principled integration of simulation and emulation components required for • Accurate validation of NEST systems
DyMND-EE – The Next Generation • Goal: End-to-end validation of sensor net software • Efficient execution of embedded source code • Realistic models for radio, sensing, actuation • Mixed models of execution for staged development • Goal: Accurate validation environment • Model of execution (concurrency) faithful to TinyOS • Goal: Scale to NEST challenges (e.g. EXSCAL) • Exploit tiered structuring via aggregate simulation and interaction across aggregates
Tossim Emsim / Emstar Processor fidelity DyMNDEE DyMNDSim Ptolemy II / VisualSense Generic modeling Simulink Evolving DyMNDSim To Realize DyMND-EE Goals Based on principled composition of existing simulators • Target-level simulators • Emsim (Emstar): cross-compiled nesC in Linux user space • High-level DE/ continuous-time simulators • Ptolemy II/VisualSense: Multiple models of computation, actor-oriented programming • Simulink: Mostly continuous time, rich control/DSP libraries
Ptolemy II Features in DyMND-EE • Actor-oriented development environment • Actors implement operations on data • Directors specify model of computation (timing and order of communication between actors) • Multiple models of computation in a designed system • Example: discrete-event sensor model to track evader with continuous control law • Extensions to WirelessDirector (implicit connectivity) • ComposableChannel allows modification of each token • Channel modifiers for propagation delay, deterministic / probabilistic message loss, bit errors • Extension to create distributed director
EmStar Features in DyMND-EE • Emstar is a flexible software framework for heterogenous NesC Code simulation using the FUSD kernel module. (http://cvs.cens.ucla.edu/emstar) • Features • Executes TinyOS/NesC code via EmTOS • HIL and simulated execution • Heterogeneous node simulations • Can use real or simulated RF channel • FUSD user space devices • Non-TinyOS dependent as any device can be registered under FUSD • Provides a clean interface to abstract hardware.
DyMND-EE Concept of Operations Graphical representation of a configuration of the DyMND-EE simulation environment Data Logging to Files for Analysis DyMNDES generates simulation configuration and generates Ptolemy Models. Ptolemy Executes aggregate sensing and comm. models EmStar runs NesC code
DyMND-EE Architecture Ptolemy II • Decomposes simulation into sensing/actuation “universes” • Implemented as composite actors with directors • Radio and sensors as actors • Services as Emstar processes with proxy actors • Software services execute in Emstar (native code) or Ptolemy (prototype code) • Communicates with external processes via XML/object streams • Configurations generated from DES script using XSLT and predefined Ptolemy II actors Acoustic Universe Processors Radio Universe Node 1 Microphone Node 1 EmstarProxy Node 1 Radio Magnetometer Universe Node 2 EmstarProxy Node 2 Radio Node n Microphone Node n EmstarProxy Node n Radio Emstar Gateway Emlink Radio Channel Abstraction Radio Channel Abstraction Sensing Channel Abstractions (EmSensorServer) FUSD Emstar (Linux user space) Radio Channel Abstraction Radio Channel Abstraction Radio Channel Abstraction Node 1 Processes Node 2 Processes Node n Processes
DyMND-EE Component Collaboration Ptolemy II Acoustic Universe Processors Radio Universe • Target emits chirp • WirelessDirector transmits chirp to node k mic • Mic processes chirp • Ptolemy forwards ActiveMessage (AM) to node k EmstarProxy • Proxy routes AM to Emstar gateway • Gateway routes AM to EmSensorServer • EmSensorServer forwards AM to node k processes via FUSD • Node k process generates detection message • Node k process forwards radio AM to radio abstraction via FUSD • Radio abstraction forwards AM to Emstar gateway • Node k EmstarProxy forwards AM to node k radio • Node k radio broadcasts alert Node 1 Mic Target Chirp Node 1 EmstarProxy Node 1 Radio 1 12 2 Node k Mic Node k EmstarProxy Node k Radio 3 4 Node n Mic Node n EmstarProxy Node n Radio 5 11 Emstar Gateway 10 6 Emlink Radio Channel Abstraction Radio Channel Abstraction Sensing Channel Abstractions (EmSensorServer) 9 7 Emstar (Linux user space) Radio Channel Abstraction Radio Channel Abstraction Radio Channel Abstraction Node 1 Processes Node k Processes Node n Processes 8
Outline • Robust DyMND Engineering • DyMND-Sim and DyMNDES: The first generation • Lessons learned • DyMND-EE: The next generation • Architecture • Implementation • Staged Development Configuration • Experiments • Adaptive Coordination in DyMND • Explicit local coordination for optimal control • Dynamic programming (reinforcement learning) • Design and validation strategy within DyMND-EE • Summary and Future Plan
Implementing DyMND-EE: Extensions To Emstar • Primary Modifications • Interfaces into TinyOS scheduler for fine grained control of timing and interrupt execution • New sim-component EmLink implements radio and sensing link layer driver registration for interface into Ptolemy • Sensing components in Emstar component library are rewritten to use new ADC.nc components that register link users. No changes are required to get NesC code working as expected in DyMND-EE. • Additional components, including hypothetical components, are emulated to examine effects of sensor networks with actuation capabilities via Ptolemy environment • Extensions for non-TinyOS-based node synchronization with external simulations
Implementing DyMND-EE: Emstar-Ptolemy Link • EmLink • Encapsulates TinyOS ActiveMessages as Ptolemy tokens • EmSensorServer implements Linux user-space proxy for Ptolemy sensor/radio models • P-Star protocol defines send/receive, device read/write, synchronization, sense/actuate, and configuration messages • Communication between Ptolemy-space and user space transparent to services and actors • P-Star Protocol • Protocol between Ptolemy and EmStar • Header format includes simulation time, device ids, group ids (for spatial or logical partitioning of EmStar simulations), and various simulation options. • EmStar Java Gateway • Consumes P-Star messages and dispatches them to EmstarProxy actors • Actors can represent entire nodes, components of a node, node tasks and services.
DyMND-EE Configurations Configuration Options • Modeling the physical environment • Radio Channel models • Sensor channel models • Modeling component interactions • Service coordination • Timing & interrupts • Sensing & actuation
Matlab Minimal Staged Development in DyMND-EE • Matlab – processing • Ptolemy – distributed radio and sensing model
Staged Development in DyMND-EE (Contd.) • Introducing NesC code into the simulations
Staged Development in DyMND-EE (Contd.) • Replacing Ptolemy models with actual environment • Testing performance in real world
Staged Development in DyMND-EE (Contd.) • Transfer code to actual hardware and test like you fly
Outline • Robust DyMND Engineering • DyMND-Sim and DyMNDES: The first generation • Lessons learned • DyMND-EE: The next generation • Architecture • Implementation • Staged Development Configuration • Experiments • Adaptive Coordination in DyMND • Explicit local coordination for optimal control • Dynamic programming (reinforcement learning) • Design and validation strategy within DyMND-EE • Summary and Future Plan
DyMND-EE Experiment 1: Target Detection • Simple translation of wireless sound detection Ptolemy demo (UCB) to nesC • Simulation setup models radio and sensing channels • NesC code performs triangulation
Experiment Configurations • Experiment parameters are easily changed in Ptolemy. • Experiment parameters can be set either by the DyMNDES or by the Vergil GUI in Ptolemy • Experiment runs can be saved in Ptolemy or as a script by the DyMNDES.
Source Localization Results Limited radio range More aggregators; modified density of detectors Increased target speed; limited radio ranges Increased target speed
Issues • Insight from these simple experiments demonstrate that its easy to find the optimal simulation configuration to give data that “fits” your algorithm. Results from the perfect simulation configuration
DyMND-EE Experiment 2: Localization • Kestrel localization code • No changes necessary in NesC code • Simulation setup models radio and sensing channels. • NesC code performs localization
Comparing Fidelity of Simulation • DyMND-EE results match Kestrel’s results (TOSSIM w/custom sensor models) • Confirmed that localization produced standard deviation ~0.7-1.0% of total distance for accurate ranging
Advantages of the DyMND-EE Approach • Software reuse and rapid prototyping • Reuse/test embedded code • Implement software prototypes as Ptolemy actors or Linux user-space processes (managed memory, out-of-band communications) • Implement new sensing concepts as Ptolemy actors • Extensibility • Simulation container not restricted to single hardware model • Scalability • Functional decomposition allows tradeoffs between spatial locality (neighboring nodes) and functional locality (sensors vs. radios vs. processors) for distributed implementation • Synchronization • Ptolemy messaging bus synchronizes events and communication between services
Limitations • Limitations • Task-level simulator executes in a single thread per node. Timing and interrupts may not be handled as expected if using interrupts directly • NesC component libraries only partially ported – UART, ADC, and actuation not yet completed. • Radio and sensor models still under development and require compilation for new models • Each node runs C or NesC code
Outline • Robust DyMND Engineering • DyMND-Sim and DyMNDES: The first generation • Lessons learned • DyMND-EE: The next generation • Architecture • Implementation • Staged Development Configuration • Experiments • Adaptive Coordination in DyMND • Explicit local coordination for optimal control • Dynamic programming (reinforcement learning) • Design and validation strategy within DyMND-EE • Summary and Future Plan
Optimal control: The DyMND Approach • Management of sensor nets can be framed as an optimal control problem • Optimal policy maximizes value function: Expected (discounted) sum of future rewards • Engineer specifies rewards/penalties for good/bad events • Actions rewarded/penalized per node (no global oracle) • Objective function depends on … • Quality of sensor data (higher is better) • Power consumption (lower is better) • Detection performance (precision & recall) … over the life of the network • Large sensor nets require global optimization over (large numbers of) local state variables • Low communications overhead (packet size, power management) • Low memory overhead (!)
Optimal control: An example Intruder detection with wireless sensors • State variables (per mote) • Internal state: e.g., mote battery power, current detection assignment • Observed state: e.g., amplitude and frequency of acoustic sample • Actions (per mote) • Changing sensor sampling rates • Sending a detection message • Switching between sensing “policies” (e.g. “vigilant” / “slack”) • Rewards/penalties (per mote) • Penalty proportional to power consumption: Encourages long network life • Reward proportional to “quality” of sensor coverage Variance of observations over mote and neighbors
Sampling rate Lo Hi Junction Tree RL Coordination Service Initializing a junction tree • Identify neighbors and shared state • Build routing tree and propagate shared state Making decisions (policy improvement) • Pass messages to ascertain neighbors’ policies • Determine the optimal policy conditioned on • Neighbors’ policies • Current local value function estimate • Pass state observations in toward root, then out toward leaves Updating value estimates (policy evaluation) • New estimate is convex combination of: • Old estimate • New observation of (rewards + successor-state expected value)
Loopy value propagation • Junction-tree control imposes routing tree on communications mesh • Worst-case path length radius of network • Costly to maintain running intersection property (storage exponential in path length) for large networks • Expensive recovery from link/node loss to maintain tree • Analogy to probabilistic inference • Junction-tree RL relies on optimal-control form of Generalized Distributive Law (GDL) • Statistical-mechanics approximation to GDL assumes interaction graphs have cycles (atoms in a crystal lattice) • Junction graph relaxes global tree structure • Tree-structured coordination graph per variable • Natural coordination structure for sensor nets (immediate neighborhoods) • Introduces circular dependencies to decision-making process • Decisions are globally suboptimal, but locally optimal (and approach optimality after many iterations) • Standard JTRL: • 5 hops • 2 extra nodes • 3 hops • 1 extra node • Loopy JTRL: • 2 hops • 2 hops • 0 extra nodes! Junction-tree RL makes MDPs feasible for small sensor nets; Loopy junction-tree RL makes approximate MDPs feasible for large sensor nets!
RL in TinyOS: Design Constraints • RL implementation requires three components • Exploration strategy to balance “exploration” (visiting new states) and “exploitation” (returning to old states) • Exploration can occur in simulation/lab • Deployed system will favor exploitation • Value function approximator to identify best actions for states • Initial values biased to incorporate domain knowledge • Exact (tabular) value functions are storage-intensive • Nonlinear approximators (neural nets) are compute-intensive • Linear combinations of features are efficient • Update rule to apply changes to value function/policy Minimal computational overhead • RL service requires access to service state • Messages between nodes should be small and infrequent • Loopy value propagation avoids long message paths • Coarse-grained policy learning reduces complexity of interaction
RL in TinyOS: Implementation Options • Runtime configuration: RL-aware services register callback with RL service • Registered services provide accessors for relevant state/action or • Registered services fire events for state changes, and RL service fires events for policy changes • Similar to ServiceC/ServiceM approach • Requires retrofit of existing services • Compile-time configuration: RL service code generated with static references to ordinary nesC services • Exploits static allocation: RL service directly reads state variables, writes configuration parameters • Reuses services without modification • Violates isolation of nesC services • “Shared memory” hack risks race conditions
Summary • Developed DyMND-EE for • End-to-end validation of NEST systems • Staged development of NEST systems • Robust analysis via controlled experiments within an accurate validation environment • Experimented with DyMND-EE • Replicated experiments to validate claims (localization) • Design of adaptive mechanisms for optimal performance in the context of change • Developed algorithms • API design for implementing within DyMND-EE
Future Plan • Extensions to DyMND-EE • Include actuation processes • Interruptable tasks • Higher fidelity sensor, radio models • Experimental approach to design/evaluation of robust sensing/actuator networks • Modeling (sensor, radio, actuation) in DyMND-EE • Experiment driven design of services (parameters) for specific application contexts and dynamics • Adaptive coordination implementation