160 likes | 340 Views
Performance Modeling in the Large: A Case Study. B.D. Theelen. Contents. Introduction Parallel Object-Oriented Specification Language (POOSL) Case Study: Internet Router Modeling the Internet Router Exhaustive Performance Analysis Performance Evaluation by Simulation Accuracy Analysis
E N D
Performance Modeling in the Large:A Case Study B.D. Theelen www.ics.ele.tue.nl/~btheelen
Contents • Introduction • Parallel Object-Oriented Specification Language (POOSL) • Case Study: Internet Router • Modeling the Internet Router • Exhaustive Performance Analysis • Performance Evaluation by Simulation • Accuracy Analysis • Conclusions www.ics.ele.tue.nl/~btheelen
System-Level Design Methods • Focus on developing executable models, which • Describe a system in the earliest phases of the design process • Abstract from many implementation details • Are relatively cheap to create, validate, test, debug and simulate • Allow the analysis of system properties before the system is actually being realized in terms of hardware and software • Correctness properties: absence of deadlock, … • Performance properties: throughput, packet-loss, jitter, … • Enable to take well-founded design decisions based on the obtained system property analysis results • Require the use of a well-defined modeling language Parallel Object-Oriented Specification Language (POOSL) www.ics.ele.tue.nl/~btheelen
POOSL • System-level modeling language for complex real-time distributed hardware/software systems • Small set of very expressive primitives • Formal (mathematically defined) semantics • Probabilistic features for performance analysis • Process part based on real-time extension of CCS • Process objects model the elementary time-related behavior • Channels enable synchronous message passing between process objects • Clusters allow for describing hierarchy • Data part based on imperative object-oriented languages • Data objects model data structures and operations on them www.ics.ele.tue.nl/~lvbokhov/poosl www.ics.ele.tue.nl/~btheelen
Alcatel 7770 RCP (Routing Core Platform) Internet Router 640 Gb/s – 10 Tb/s OC-3 – OC-192 8.5 kW $ 290.000 www.ics.ele.tue.nl/~btheelen
Overview of the Internet Router Sources Input Buffers Switch Core Output Buffers Sinks 1 1 1 1 1 1 N M Outputs Inputs 1 N N N 1 N N M Is the specified flow-control mechanismsuitable for future product variants? www.ics.ele.tue.nl/~btheelen
System Parameters N and M • Sources NxM Source Activities • Input Buffers N Input Dispatchers NxN Queue Output Handlers • Switch Core NxN Switch Activities • Output Buffers N Queue Input Handlers N Queue Output Handlers • Sinks N Sink Activities 2N2 + (M+ 4) N Activities Huge number of concurrent activities www.ics.ele.tue.nl/~btheelen
Scalability of the Model • Instantiating a process object is performed by drawing the object • Modeling each component with a separate process object is not favorable for large N and M • Initialize parameterized number of similar concurrent activities Initialize()() InitializeActivity(1)(). InitializeActivity(Identity: Integer)() par Activity(Identity)() /* Invokes the actual Activity */ and if (Identity < N) then InitializeActivity(Identity + 1)() fi rap. (Optimization: dynamic creation and termination of concurrent activities) www.ics.ele.tue.nl/~btheelen
Executable Model • Create executable model with SHESim tool • Freely available on: www.ics.ele.tue.nl/~mgeilen/shesim www.ics.ele.tue.nl/~btheelen
Validation • Is the model adequate for analyzing the performance metrics? • Validate, test and debug model using SHESim tool www.ics.ele.tue.nl/~btheelen
Formalization of Performance Properties • The flow-control mechanism should contribute to • Minimizing packet-loss • Minimizing buffer capacity • Minimizing latency • Maximizing throughput • Performance metrics to analyze • Probability of losing packets • Average and maximum buffer occupancy • Average and maximum latency • Throughput Use POOSL formalism to extend model for performance analysis www.ics.ele.tue.nl/~btheelen
UnderstandablePOOSL Model&Extensions for Performance Analysis Formal Semantics 1 Out!Packet(p) Markov Decision Process&Reward Structure 1 0.9 3 S1 S2 S3 S4 0.1 External Scheduler Discrete-Time Markov Chain & Reward Structure Performance Evaluation with POOSL Markov Chain is implicitly defined by expressive modeling language and not by hand www.ics.ele.tue.nl/~btheelen
Performance Evaluation with POOSL • Analytically: Ergodic Theorem for Markov Chains • Example: time-average buffer occupation • Simulation: Central Limit Theorem for Markov Chains • Dependable random variables • Example: packet-loss probability • Enables accuracy analysis with confidence intervals www.ics.ele.tue.nl/~btheelen
Performance Evaluation by Simulation • Fast simulation with Rotalumis tool • Freely available on: www.ics.ele.tue.nl/~lvbokhov/poosl/rotalumis • Simulation = generation of one trace through Markov chain • How long should a simulation take to obtain accurate results? • Analyze accuracy of simulation results with confidence intervals • Automatic termination of simulation www.ics.ele.tue.nl/~btheelen
Accuracy Analysis • For N=512 and M=128, a 99% confidence interval[0.001046, 0.001052] is obtained for the average latency • Estimation of average latency is 0.001049 seconds • We are 99% sure that the real average latency lies in [0.001046, 0.001052] • An upper bound for the relative error is 1% • Takes 0.0072 seconds of simulated real-time • Takes 11.9 hours of simulation time • POOSL classes for accuracy analysis of long-run averages • Long-run sample average: packet-loss probability, latency • Long-run sample variance: jitter • Long-run time average: buffer occupancy, processor utilization • Long-run time variance: buffer occupancy Rare events (losing a packet) may change the status of a metric www.ics.ele.tue.nl/~btheelen
Conclusions • POOSL allows for a compact abstract description of real-life systems to adequately investigate their performance properties • Performance analysis requires extending the original model • POOSL model implicitly defines a Markov chain • Formal semantics enables computation of performance • POOSL tools offer good validation and simulation possibilities • Confidence and accuracy of simulation results should be known • POOSL library classes for accuracy analysis • Future research • Long-run rate average (throughput) and rate variance (burstiness) • Rare events • Support for performance metric monitors www.ics.ele.tue.nl/~btheelen