410 likes | 504 Views
Enabling Online Analysis of Notification Services. Sebastian Salvucci. Advisor: Mariano Cilia Co-Advisor: Ricardo Orosco. Facultad de Ciencias Exactas Universidad Nacional del Centro de la Provincia de Buenos Aires. Tandil, August 2006. intro/motivation. proposed approach. architecture.
E N D
Enabling Online Analysis of Notification Services Sebastian Salvucci Advisor: Mariano Cilia Co-Advisor: Ricardo Orosco Facultad de Ciencias Exactas Universidad Nacional del Centro de la Provincia de Buenos Aires Tandil, August 2006
intro/motivation proposed approach architecture implementation conclusions/outlook
Pub/Sub Systems intro
NS characteristics intro Adressing Models • channel-based • subject/topic-based • content-based Behavior (performance influenced by…) • Matching: is the process carried out by event brokers • to check messages against subscriptions. • Routing: is in charge of forwarding delivering the messages • to the subscribers. Different algorithms (flooding, simple • routing, identity routing, covering, merging) • Other features: persistence, priorities, transactions, selectors, • caching, expiration, filters, message types, … Implementations (different features) • Research Projects: REBECA, Siena, Hermes, Jedi, Gryphon, Srcibe • Commercial Products: Tibco Rendezvous, WebSphereMQ, • SonicMQ, Fiorano • JMS Standard API specification: Open Source Implementations • (OpenJMS, JBossMQ, ActiveMQ, Joram)
Centralized/Distributed intro Centralized Distributed event broker network
Traditional Analysis Aproaches intro Tracing, log files:add extra source code finding first the right places and then modifying the code to write the relevant data into a text file. • enormous ad-hoc text files • source code abstraction level Black-box analysis: data are observed and registered at the clients, when messages are published and/or received • thick granularity does not allow • to determine behavior details.
Motivation/Goal intro Motivation • Today is difficult to analyze notification services • focalizing individuals aspects of the behavior • Few partials and proprietary solutions • Should help to improve NS behavior and performance • (research/development/tuning) Goal • Create an NS-independent analysis framework • based on observations, metrics definitions and visual • representations of them.
intro proposed approach architecture implementation conclusions/outlook
Proposed Approach proposed approach
Observations proposed approach
Observations (2) proposed approach
Following messages proposed approach
Metrics from observations proposed approach
Metrics Analysis proposed approach Online analysis metric values external visualization techniques can be configured to represent metrics in different ways Post-mortem analysis Offline analysis: replay of executions Static analysis: traditional queries, data mining, etc.
intro/motivation proposed approach architecture implementation conclusions/outlook
Proposed Architecture architecture
Considerations architecture • Visualization • Reusable and configurable visualization techniques • Metrics independence • Multi users (client) • Metrics Composition • Data in movement • Generic data transformation • High-level metrics definition • Post-mortem analysis • Global Time • Behavior Observation • Notification service independence • Influence over the NS under observation • Sending and generation mechanisms performance • Communication • Push-based communication • Many-to-many • Firewalls, security restrictions • Management • Configuration • Components orchestration
Behavior Observation Layer architecture • Adopting AOP to avoid • intrusive modification of • source code. • AOP lets separate • and modularize • observation logic.
Behavior Observation Layer (2) architecture • Enable/disable observations generation according metrics composition needs • Sending frequency: (online, offline, time intervals, fixed amount, package size) Implementation Alternatives -static weaving -dynamic weaving
Metrics Composition Layer architecture
Compositors architecture Generic transformation over data streams (configurable) metric values time interval observations Implementation alternative: Stream Process Engines
Metrics Definition: Data Warehouse Metaphor architecture • High-level and intuitive modelling (from static analysis) • Generic and abstract data transformation • Different measures from the same observations • Decoupling metric composition output from visualization • Metadata for stored data
Metrics Composition architecture Configuring compositors from metrics definition
From Data to Visualization architecture
Visualization architecture Visualization panel (web browser) Visualization components Implementation Alternatives • Applets • DHTML/AJAX • SVG • Flash • VRML/X3D
Communication architecture centralized approach pub/sub approach
Components Orchestration architecture • Test Run Definition • load generation scenario • metrics • visualization techniques • Test Run Recording • Enable observations • Deploy compositors (subs) • Init visualization components (subs) • Subscribe repository (all) • Start Load generation • Test Run Reproduction • Init visualization panel • Stored data publication
intro proposed approach architecture implementation conclusions/outlook
Initial decisisons implementation Data representation Communication • XML data representation for observations and metric values HTTP-based publish/subscribe framework Repository HTTP streaming Instead of closing the HTTP connection after fetching an HTML page, the connection remains open while fresh data is pushed to the client • Adopting eXist XML native database • (XQuery, XUpdate support) XML repository structure • Pushlets Characteristics • subject based • runs in a servlet container • very thin Java client • JavaScript client for browsers • modes stream/polling • Pushlets Limitations • scalability with mode stream • message type
Reusing Aspects implementation method signature & parameters
Compositors: Aggregator filter + primitives implementation Enhancing Filtering in Notication Services Primitives based filters Aggregator Filter Implementations • Pushlet adapters • XPath primitive • Compositors creation (from metrics) • Compositors deploy/undeploy (pub/sub)
Composing metrics: data transformation implementation
Visualization Panel implementation • Metrics and components • from test run definition
Visualization Components implementation • Visualization components description. • Mapping between metrics and components
Components Orchestration implementation Test Run Reproduction Test Run Recording
intro proposed approach architecture implementation conclusions/outlook
Conclusions conclusions The utilization of three different kinds of components to observe the behavior (aspects), compose streams of observations (metrics) and graphically represent them (visualization techniques) as well as their loosely-coupled communication through a pub/sub service offer a high flexible framework to enable online analysis of notification services.
Conclusions (2) conclusions The use of a repository to store the data produced by aspects and metrics composition supports post-mortem (offline and static) analysis using the same architecture that was developed for online analysis.
Conclusions (3) conclusions • Not intrusive modification of the source code • (by using AOP to generate observations) • Independence of notification service implementations • (by associating semantics of observations to behavior • description instead source code) • Generic and intuitive modeling of computations over data streams • Metrics decoupled from visualization • Metadata associated to stored executions • (by data cube description)
Outlook conclusions • Declarative description of observations and automatically generate • the aspects involved letting choose different AOP alternatives. • By seeking and improving the data warehouse metaphor could be possible • to associate a complete and unique multidimensional • analysis to data streams and historical data. • Graphical wizards and interfaces upon XML descriptions towards • a higher abstraction level for users, offering a complete • analysis-oriented application (bookmarks, notes, etc..) • Other domains (alternative data sources): software analysis, • business activity monitoring, applications that rely on pub/sub systems, etc..
Muchas Gracias! questions / comments / feedback