A High Level Pub/Sub Layer for Open Distributed Heterogeneous Environments

1. A High Level Pub/Sub Layer for Open Distributed Heterogeneous Environments Candidate: Mario E. Antollini Thesis Supervisor: Mariano A. Cilia Universidad Nacional del Centro de la Provincia de Buenos Aires Facultad de Ciencias Exactas Tandil, Argentina - February, 2004

2. Agenda • Motivation • Problem Statement • Proposed Approach • Design & Implementation • Conclusions

3. Motivation • Today: Client/Server (Request/Reply) predominance • Trend: Convergence of technologies • WWW, sensor networks, enterprise strategies • Data exchanged across traditional borders • Information-driven • Large-scale distributed systems • Pull-based (Request/Reply) does not fit • Polling implies resource waste • Leads to network saturation or server breakdown! • New Paradigm for information-driven applications • Push-based (event-based dissemination)

4. Publish/Subscribe - Intro • Publish/Subscribe Middleware • Reflects intrinsic behavior of Information-driven Application • Loosely-coupling • Support asynchronous communication • Participants • anonymous to each other • can “dynamically” join and leave • Doesn’t require publishers and subscribersat the same time • Location Transparency • It acts as an “intermediator” among participants

5. Pub/Sub – Intro (cont.) • Independent Process Data Flow Style • Message Manager or Notification Service (JMS, Rebeca) • Clients (producers/consumers/both) • Communication links

6. Problem Statement • Even when Notification Services support loosely-coupling and anonymous participants, they • Do not provide any support for Data Exchange among heterogeneous participants • Do not support Event (message) Correlation

7. Problem Statement – Data Exchange • Data from different sources/components is represented differently • Different organizations/departments use different units and representation formats • Context information is usually left implicit and consequently it is lost when crossing component or institutional boundaries • (date) 7/11/2003 Which one is the month? • (price) 200 Currency? €?, U$S?… • Data from different apps needs to be interpreted by applications • no cultural assumption!  To process events in a semantically meaningful way, explicit information about semantics of data is required

8. Concept-based Approach • Provides a higher level of abstraction to describe the interests of publishers and subscribers • Subscribers and Publishers can specify their assumptions • Price < 100 [€] • DeliveryDate := 7/11/2003 [dd/mm/yyyy] • The notification service delivers ready-to-process events to subscribers • No further processing is needed

9. Concept-based Approach – Overview

10. Design & Implementation • MIX • Context Transformation • Mapping Different Addressing Models • Notification Service Independency

11. Model for Data Exchange - MIX • MIX Data Model • Allows the definition of concepts to describe data and metadata from a given domain • Event Representation • Concepts of the ontology • Composed by a tree-shape object structure • Subscription Representation • Boolean expressions • Include contextual information for the correct interpretation and matching of incoming events

12. Context Transformation • Adapters • Applications’ contact point with the NS • They realize mapping operations from local context to the matching one and vice versa • Context transformation occurs in case of both event producers and event consumers • Before events arrive at subscriber’s side, adapters automatically convert message's context to the one specified by subscribers (at subscription time)

13. Mapping Different Addressing Models • Data delivery problems is delegated to the underlying NS • The Concept-based approach is able to work on top of NSs that can rely on different addressing models • This flexibility requires different mapping strategies

14. Mapping Different Adressing Models (cont.)

15. NS Independency

16. NS Independency -Abstract Factory Pattern

17. Concept-based in J2EE • Concept-based through MDBs

18. Problem Statement – Event Correlation • Complex Event Detection • Involves de occurrence of two or more primitive and/or composite events • Composite events are expressed using an event algebra • Order of occurrence of events is crucial • In open distributed environments • Global time is not applicable • Total order of events cannot be guaranteed

19. Complex Event Detection Platform • Proper Interpretation of Time • Timestamp representation • Partial Order of Events • Situations of uncertain timestamp order are detected. Action taken is exposed and well defined • Events are not erroneously ordered • Transmission Delays, Failures, Order and Uncertainty • Heartbeat • Window mechanism • Consumption modes

20. Complex Event Detection Platform - EventList • Temporarily maintains event instances before composition

21. Complex Event Detection Platform – Event Compositor

22. Conclusions • Functionality of Notification Service was extended with • Higher-level layer • Common Vocabulary • Contextual information • Mapping to different addressing models • NS API: independence of underlying Notification Service • Ready-to-process notifications • Support meaningful data exchange among heterogeneous applications • Complex Event Detection • Component-container model • Easy to develop operator • Definition is restricted to operator logic • Other aspects are handled with policies • J2EE Integration

23. Future Work • Concept-based implementation • Experiment with different addressing models • Remotely accessible operation • Hot-swapping of Notification Service • Other Enhancements • Visualization tools to graphically trace flow of event instances • Trust and Security