SATware: Middleware for Sentient Spaces WMSC 2007

1. SATware: Middleware for Sentient Spaces WMSC 2007 Bijit Hore, Hojjat Jafarpour, Ramesh Jain, Shengyue Ji, Daniel Massaguer, Sharad Mehrotra, Nalini Venkatasubramanian, and Utz Westermann

2. Traditional sensor network . . . Stream Stream Stream Uninterpreted streams of sensor event observations Privacy QoS Ad-hoc Application Sensing layer

3. SATware: Semantic Layer Stream Application Stream Interpreted streams of entity-related events Stream Privacy QoS Situation model Semantic layer: entities and activities Domain model Stream Stream Stream Uninterpreted streams of sensor event observations Privacy QoS Sensing layer

4. SATware: Challenges In emergency response, being able to specify quality of the service and having the system guarantee it is crucial Different levels of abstraction from user's high abstraction level to sensors' level Users care about entities and events Privacy of users (both direct and indirect users) has to be protected Different kinds of sensors: different resources, mobility capabilities, data streams produced (scalar, video), phenomena being sensed Lots of sensors some sensors produce streams with very high sample rates and sample sizes (e.g. video). In emergency response failures happen Sensors and services need to be added, removed and discovered

5. Queries are translated into Operators Queries in SATware • Queries written at the entity level • E.g., Identifywho left the coffee pot burning? • E.g., Notify when someone leaves a mess in the shared kitchen. • E.g., Notify when Sharad is in the shared area and his status is “Not Busy”. • E.g., Notify when Iosif and Sharad are both in the shared area

6. Stream processing model -Data model- A stream is an infinite discrete flow of packets: stream = list(packet) packet = <t , c> t : time stamp c: content C domain(DT) DT := int | float | char | byte | void | bool | event DT := list(DT) | tuple(DT) Event := <event_id, confidence, DT> <14:01:12.50 Jan 1st 2007, <LocationChange, 0.65, void>> Stream <14:01:12.50 Jan 1st 2007, Temperature = 50>  Stream

7. Stream processing model -Processing model- Processing based on operators OP = { fDTx…xDT | fDTx…xDT: streamx…xstream -> stream } • Code reuse • Sharing operator instances • Simplify design • Load distribution • Virtual sensors WhoBurnedCoffeex,y = O10(CoffeeBurnx, PersonIDy) Image processing-based event detection < ti , <Door Status, ci>> < ti , Jpeg image> Oi Time synchronizer < ti , (<PersonID, ci>, <Door Status, cm)]> < ti , <PersonID, ci >> Oi < ti, <Door Status, cm>> Event-based event detection < ti , <ID_Enter_y, ck >> < ti , <PersonID, ci >> Oi < ti, <Door Status, cm>> Source < ti , Jpeg image > Oi Output < ti , <PersonID, ci >> Oi

8. Sample application Remote Windshield Assessment UCI has been instrumented with a large set of sensorsthroughout all campus. The instrumentation would allow conducting a quick assessment of the status of buildings from any device that has access to Internet by accessing an online Remote Windshield Assessment application. The system would allow any user (for example, UCIPD) to register for events. Fire location: UCI#building1#floor4#(300,20)‏

9. Sample query Sensor Sensor Sensor Event detection1: Estimation of number of people in and out a certain door. ED1 SRC SRC ED2 ED1 UI SRC ED1 ED2 stream stream stream stream stream Event detection 2: Estimation of number of people in a room. Sensor stream stream Sensor SRC Source agent: Gets JPEG frames from a network camera. stream SRC ED1 ED2 SRC ED1 stream stream UI stream stream Source agent: User interface. Query: Notify when there is a change of distribution of people in a building

10. <Query results> SATware architecture Application <Query>SATQL SATQL <Query>SATLite High-level query language SATLite Deployment (topology) description SATDeployer Deployment optimization SATRuntime SATRuntime Distributed Reflexive Mobile-agent based runtime Sensor infrastructure Heterogeneous sensors

11. {SATQL} {SATLite} SATRuntime Sensor UI MA SATQL SATRuntime MA MA SATLite QoS QoS SATRuntime SATRuntime stream SATDeployer stream Processing node QoS QoS SATRepository Sensor Processing node MA SATware snapshot, configurations, operators repository, spatial model Event and data Storage subsystem

12. SATRuntime • Runtimes are connected instead of the agents. • Agents are programmed in a topology-agnostic manner. • SATRuntime provides mobility support and message-passing. • SATRepository contains • repository of operators and • state of SATware (sensors, runtime nodes, network topology, and current operator deployment). MA MA MA SATRuntime SATRuntime stream SATRepository SATware snapshot, configurations, operators repository, spatial model

13. Sample query Sensor Sensor Sensor SRC ED UI SRC ED ED2 SRC ED1 {SATQL} SATRuntime SATRuntime SATRuntime SATRuntime SATRuntime SATRuntime SATRuntime SATRuntime Sensor stream stream Sensor stream stream stream stream stream SATQL Processing node Processing node SATLite Processing node Processing node Processing node Processing node SRC ED Processing node Processing node SRC ED1 ED2 stream SATDeployer SATRuntime SATRuntime SATRuntime SATRuntime SATRuntime stream stream SATRepository stream stream Processing node Processing node Processing node Processing node SATware snapshot, configurations, operators repository, spatial model Processing node

14. SATLite SATLite is a language for describing queries //operators O1 = CreateOperator(ReadCam, camera_url); O3 = CreateOperator(DetectBurner); //data flows Image = O1(); BurnerStatus = O3(Image); BurnerOn = O6(BurnerStatus); O11(PersonId);

15. Predicate optimization “Which image contains Iosif?” Assume RFID also available YES MAYBE MAYBE RFID tag matches Iosif’s tag?” MAYBE λ2 λ3 λ4 λ1 NO Which pictures contain an object?” λ2 Which pictures contain a person?” YES λ3 MAYBE MAYBE MAYBE MAYBE λ2 λ4 λ1 λ3 Which pictures contain Iosif?” λ4 NO 1Iosif Lazaridis and Sharad Mehrotra, “Optimazation of Multi-Version Expensive Predicates”, SIGMOD 2007.

16. Problem formulation • Given • Sequence of selection predicates • λ1, λ2, …, λn • With Per-tuple cost: • c1, c2, …, cn • And MAYBE selectivity: #Maybe / #All • m1, m2, …, mn • Such that: • c1 < c2 < … < cn • m1 > m2 > … > mn • Determine the least cost execution plan for λn • λk(t) = Yes • No • Maybe 1Iosif Lazaridis and Sharad Mehrotra, “Optimazation of Multi-Version Expensive Predicates”, SIGMOD 2007.

17. λi+1 λi+1 YES λi+2 MAYBE MAYBE OGP λi+2 Evaluate OGP … λi+1 λi+2 λn … YES λi NO λn λn λn MAYBE MAYBE Group Evaluate … λi λi+1 λn NO Solutions Optimal Generalized Plan Individualized Handling Tuple Grouping 1Iosif Lazaridis and Sharad Mehrotra, “Optimazation of Multi-Version Expensive Predicates”, SIGMOD 2007.

18. SATDeployer • Translating a query plan into a deployment plan • Mapping query plan nodes into processing nodes • Minimizing query processing cost • Network cost • Computing cost • Query evaluation time • Optimizations • Operator reuse • Load balancing

19. Privacy Privacy is protected unless the user violates a specific rule Trusted node Untrusted DB 2Bijit Hore, Jehan Wickramasuriya, Sharad Mehrotra, and Nalini Venkatasubramanian, “ Privacy-Preserving Event Detection for Pervasive Spaces” (2007, UCI-ICS tech-report)

20. Privacy Abnormal event detection Aggregated behavior of human beings: N(t) = N0(t) + NE(t) Markov Modulated Poisson Normal behavior: periodicity N0(t) ~ P(N; (t))  Time-Varying Poisson (t) = 0 * dd(t) * hd(t),h(t) 1 ≤ d(t) ≤ 7 and 1 ≤h(t) ≤ 48/288/… Abnormal behavior: rare and short 0 z(t) = 0 NE(t) ~ P(N; g(t)) z(t) = 1 z(t) ~ Markov Process Unsupervised learning of model parameters via statistical methods (MCMC) 3Alexander Ihler, Jon Hutchins, and Padhraic Smyth, "Adaptive Event Detection with Time–Varying Poisson Processes", KDD 2006

21. Implementation Responsphere: 1/3 UCI campus: 2 Indoor buildings + Outdoors +200 sensors +10 types

22. Summary Sentient systems Challenges Layered architecture Privacy applications

23. Thank you http://satware.ics.uci.edu

24. Extra slides

25. Project RESCUE June 20, 2007 http://www.itr-rescue.org Funded through National Science Foundation Funding: ITR award  NSF Research Infrastructure award 

26. Mission The mission of RESCUE is to enhance the ability of emergency response organizations and the public to mitigate crises, save lives, and prevent secondary and indirect human and economic loss by radically transforming ways in which these organizations gather, process, manage, use and disseminate information during man-made and natural catastrophes.

27. Approach: Information Technologies for Improved Situational Awareness Hypothesis:Right Information to the Right Person at the Right Time can result in dramatically better response Response Effectiveness • lives & property saved • damage prevented • cascades avoided Quality of Decisions • first responders • consequence planners • public Quality & Timeliness of Information Situational Awareness • incidences • resources • victims • needs

28. Research Team • Privacy • Security • Trust • Natural Hazards Center • Social Science • Data Management • Security and Trust • Disaster Analysis • Earthquake Engineering • GIS • Civil Engineering • Data Analysis & Mining • Data Management • Middleware & Distributed Systems • Civil Engineering • Transportation Engineering • Computer Vision • Networking • Multimodal Speech • Transporation Modeling • Urban Planning • Privacy • Social Science • Transportation Science • Wireless

29. Industrial Partners

30. Government Partners

31. Social & Disaster Science context, model & understanding of process, organizational structure, needs Engineering & Transportation validation platform for role of IT research Security, Privacy& Trust Cross cutting issue at every level Information Centric Computing enhanced situational awareness Networking & Computing systems Computing, communication, & storage systems under extreme situations RESCUE Research • Social Science • context and understanding of crisis domain • Information Technology • infrastructure & tools to enhance flow of information & situational awareness • Engineering • platform for realization, real-world physical constraints that help test, and validate IT solutions

32. Risk Communication System Robust Networking Solution Policy Engine Real-time Alert System Smart Reconnaissance System Enterprise Service Bus Integrated Information Dashboard Internet-based Loss Estimation System FUTURE TESTBEDS CHAMPAIGN CAMAS GLQ METASIM Policy-driven Information Sharing Privacy Situation Awareness Robust Networking Customized Dissemination Project Structure INFORMATION ANALYSIS RESCUE Thrust Areas INFORMATION SHARING INFORMATION COLLECTION INFORMATION DISSEMINATION Integrative Artifacts Testbeds RESCUE Research Projects