Response Time Analysis of a Middleware Demultiplexing Pattern for Network Services

1. Response Time Analysis of a Middleware Demultiplexing Pattern for Network Services Aniruddha Gokhale a.gokhale@vanderbilt.edu Asst. Professor of EECS, Vanderbilt University, Nashville, TN Jeff Gray gray@cis.uab.edu Asst. Professor of CIS Univ. of Alabama at Birmingham Birmingham, AL Swapna Gokhale ssg@engr.uconn.edu Asst. Professor of CSE, University of Connecticut, Storrs, CT Presented at IEEE Globecom 2005 Symposium on Advances in Networks and Internet St. Louis MO Nov 28-Dec 1, 2005 Work supported by collaborative grant from NSF CSR-SMA Program

2. Problem Statement: Estimating Performance Characteristics of Network Services at Design-time • Network services need support for efficient (de)-multiplexing, dispatching and routing/forwarding • .e.g., VPN Service provided by a virtual router • Provides differentiated services to customers, e.g., prioritized service • VPN setup messages must be efficiently (de) multiplexed, serviced and forwarded • Need to estimate capacity of the system at design-time • Standards middleware is increasingly being used to develop network services e.g., J2EE, .NET, CORBA, Web services • Middleware frameworks incorporate elegant patterns-based building blocks • Problem boils down to estimating performance of middleware

3. Solution Approach: Middleware Performance Analysis using Stochastic Reward Nets A2 A1 StSnpSht N2 N1 B2 B1 T_SrvSnpSht T_EndSnpSht Sn1 Sn2 S2 S1 SnpShtInProg Sr2 (a) (b) Sr1 Transition Inhibitor arc Place Immediate transition Token • Stochastic Reward Nets (SRNs) are an extension to Generalized Stochastic Petri Nets (GSPNs) which are an extension to Petri Nets. • Extend the modeling power of GSPNs by allowing: • Guard functions • Marking-dependent arc multiplicities • General transition probabilities • Reward rates at the net level • Allow model specification at a level closer to intuition. • Solved using tools such as SPNP (Stochastic Petri Net Package).

4. Goal: Performance Analysis of Reactor Pattern in VR • Customers send VPN setup messages to router • VPN setup messages manifest as events at the VR • VR must service these events (e.g., resource allocation) and honor the prioritized service, if any • Accepted messages are forwarded • Events could be dropped in overload conditions The Reactor architectural pattern allows event-driven applications to demultiplex & dispatch service requests that are delivered to an application from one or more clients. • Reactor pattern decouples the detection, demultiplexing, & dispatching of events from the handling of events • Participants include the Reactor, Event handle, Event demultiplexer, abstract and concrete event handlers

5. Modeling VR Capabilities in a Reactor • Consider VPN service for two customer classes • Reactor accepts and handles two types of input events • Differentiated services for two classes • Events are handled in prioritized order • Each event type has a separate queue to hold the incoming events. Buffer capacity for events of type one is N1 and of type two is N2. • Event arrivals are Poisson for type one and type two events with rates l1and l2, resp. • Event service time is exponential for type one and type two events with rates m1and m2, resp. Model of a single-threaded, select-based reactor implementation

6. Performance Metrics of Interest for VR (i.e., Reactor) • Throughput: • -Number of events that can be processed • -Applications such as telecommunications call processing. • Queue length: • -Queuing for the event handler queues. • -Appropriate scheduling policies for applications with real-time requirements. • Total number of events: • -Total number of events in the system. • -Scheduling decisions. • -Resource provisioning required to sustain system demands. • Probability of event loss: • -Events discarded due to lack of buffer space. • -Safety-critical systems. • -Levels of resource provisioning. • Response time: • -Time taken to service the incoming event. • -Bounded response time for real-time systems.

7. Modeling the Reactor using SRN (1/2) A2 A1 StSnpSht N2 N1 B2 B1 T_SrvSnpSht T_EndSnpSht Sn1 Sn2 S2 S1 SnpShtInProg Sr2 (a) (b) Sr1 Event arr. Drop events on overflow Service queue Prioritized service Servicing the event Service completion • Models arrivals, queuing, and prioritized service of events. • Transitions A1 and A2: Event arrivals. • Places B1 and B2: Buffer/queues. • Places S1 and S2: Service of the events. • Transitions Sr1 and Sr2: Service completions. • Inhibitor arcs: Place B1and transition A1 with multiplicity N1 (B2, A2, N2) • - Prevents firing of transition A1 when there are N1 tokens in place B1. • Inhibitor arc from place S1 to transition Sr2: • - Offers prioritized service to an event of type one over event of type two. • - Prevents firing of transition Sr2 when there is a token in place S1.

8. Modeling the Reactor using SRN (2/2) A2 A1 StSnpSht N2 N1 B2 B1 T_SrvSnpSht T_EndSnpSht Sn1 Sn2 S2 S1 SnpShtInProg Sr2 (a) (b) Sr1 • Process of taking successive snapshots • Reactor waits for new events when currently enabled events are handled • Sn1 enabled: Token in StSnpSht & Tokens in B1 & No Token in S1. • Sn2 enabled: Token in StSnpSht & Tokens in B2 & No Token in S2. • T_SrvSnpSht enabled: Token in S1 and/or S2. • T_EndSnpSht enabled: No token in S1 and S2. • Sn1 and Sn2 have same priority • T_SrvSnpSht lower priority than Sn1 and Sn2

9. N1 = N2 = 1 N1 = N2 = 5 Perf. metric #1 #2 #1 #2 Throughput 0.37/s 0.37/s 0.40/s 0.40/s Queue length 0.065 0.065 0.12 0.12 Total events 0.25 0.27 0.32 0.35 Loss probab. 0.065 0.065 .00026 .00026 VR SRN: Performance Estimates • SRN model solved using Stochastic Petri Net Package (SPNP) to obtain estimates of performance metrics. • Parameter values:l1 = 0.5/sec, l2 =0.5/sec, m1 = 2.0/sec, m2 =2.0/sec. • Two cases: N1 = N2 = 1, and N1 = N2 = 5. • Observations: • Probability of event loss is higher when the buffer space is 1 • Total number of events of type two is higher than type one. • Events of type two stay in the system longer than events of type one. • May degrade the response time of event requests for class 2 customers compared to requests from class 1 customers

10. VR SRN: Sensitivity Analysis • Analyze the sensitivity of performance metrics to variations in input parameter values. • Vary l1 from 0.5/sec to 2.0/sec. • Values of other parameters:l2 =0.5/sec, m1 = 2.0/sec, m2 =2.0/sec, N1 = N2 = 5. • Compute performance measures for each one of the input values. • Observations: • Throughput of event requests from customer class #1 increases, but rate of increase declines. • Throughput of event requests from customer class #2 remains unchanged.

11. Next Steps: Addressing Variability in Middleware Although middleware provides reusable building blocks that capture commonalities, these blocks and their compositions incur variabilities that impact performance in significant ways. • Per Building Block Variability • Incurred due to variations in implementations & configurations for a patterns-based building block • E.g., single threaded versus thread-pool based reactor implementation dimension that crosscuts the event demultiplexing strategy (e.g., select, poll, WaitForMultipleObjects • Compositional Variability • Incurred due to variations in the compositions of these building blocks • Need to address compatibility in the compositions and individual configurations • Dictated by needs of the domain • E.g., Leader-Follower makes no sense in a single threaded Reactor

12. Next Steps: Model-driven Performance Analysis of Middleware-based Network Services workload workload Applying design-time performance analysis techniques to estimate the impact of variability in middleware-based DPSS systems • Build and validate performance models for invariant parts of middleware building blocks • Weaving of variability concerns manifested in a building block into the performance models • Compose and validate performance models of building blocks mirroring the anticipated software design of DPSS systems • Estimate end-to-end performance of composed system • Iterate until design meets performance requirements Composed System Refined model of a pattern Refined model of a pattern Refined model of a pattern Invariant model of a pattern Refined model of a pattern Refined model of a pattern weave weave variability variability Refined model of a pattern Refined model of a pattern system

13. Concluding Remarks • Network services are implemented using middleware building blocks • Need to estimate performance early in development lifecycle • Stochastic Reward Nets enables scalable & intuitive performance analysis • Goal is to use model-driven generative techniques to automatically synthesize performance models for network services • Analysis for other dimensions of quality of service e.g., trustworthiness, dependability www.cse.uconn.edu/~ssg (Swapna Gokhale) www.dre.vanderbilt.edu/~gokhale (Aniruddha Gokhale) www.gray-area.org (Jeff Gray)

14. Questions?

15. EXTRAS

16. Designing & Evaluating SRNs for Network Services A2 A1 N2 StSnpSht N1 B2 B1 Sn1 T_SrvSnpSht Sn2 T_EndSnpSht S2 S1 SnpShtInProg Sr2 Sr1 (a) (b) • Initial Step • Obtain performance measures for individual patterns-based building blocks • Iterative Algorithm • Compose systems vertically and horizontally to form a DPSS system • Determine performance measures for specified workloads and service times • Alter the configurations until DPSS performance meets specifications.

17. VR SRN: Disruption Detection • Obtain an anomaly score for the Reactor based on each one of the • performance metrics for each event type. • Correlate the anomaly scores based on each event type to obtain an overall • anomaly score for the Reactor. • - Anomaly score for the Reactor used at each CE to demultiplex events from • two groups within a single organization. • Anomaly score for the Reactor in the VR used to demultiplex events from the • two organizations. • Correlate the anomaly score of the Reactor in the VR with the score of the • Reactor in CE #1 to determine service disruptions for organization #1. • Correlate the anomaly score of the Reactor in the VR with the score of the • Reactor in CE #2 to determine service disruptions for organization #2. • Source of disruption may be identified by correlating the scores at various layers.

18. Collaborative Research • Performance analysis methodology (UConn – S. Gokhale) • Develop and validate performance models for invariant characteristics of building blocks. • Compose and validate performance models for common building block compositions. • Develop model decomposition and solution strategies to alleviate state-space explosion issue. • Model-driven generative methodology (Vanderbilt – A. Gokhale) • Manually developing performance models of each block with its variations is cumbersome • Compositions of building blocks cannot be made in ad hoc, arbitrary manner • Model-driven generative tools use visual modeling languages and model interpreters to automate tedious tasks and provide “correct-by-construction” development • Aspect-oriented methodology (Univ of Alabama, Birmingham – J. Gray) • Variability in building blocks and compositions is a primary candidate for separating the concern as an aspect • Aspect weaving technology can be used to refine and enhance the models by weaving in the concerns into the performance models

19. VR SRN: Expected Behavior • VPN service has two modes of operation: normal & inclement. • Normal mode: • - Daily basis, some employees have negotiated telecommute plans and • use VPN for remote access. • Inclement mode: • - Hazardous driving conditions due to bad weather may keep people at home. • - Large number of telecommuters • - Increase in the connection set up and tear down requests. • Modes of operation can be defined at a finer level of granularity, such as • a few hours, rather than a day.

20. Perf. Metric Normal Inclement Average Event #1 Throughput 0.40/s 0.90/sec 0.4510/s Queue length 0.12 1.86 0.2940 Loss probab. 0.09 0.21 0.0291 VR SRN: Expected Behavior • Normal mode: • - l1 = 0.5/sec, l2 =0.5/sec, m1 = 2.0/sec, m2 =2.0/sec, N1 = N2 = 5 • - Probability – 0.9 • Inclement mode: • - l1 = 1.0/sec, l2 =1.0/sec, m1 = 2.0/sec, m2 =2.0/sec, N1 = N2 = 5 • - Probability – 0.1