1 / 14

Performance Modelling of Complex Hardware/Software Systems

Performance Modelling of Complex Hardware/Software Systems. B.D. Theelen. Overview. Introduction System-Level Design Software/Hardware Engineering Case Studies Modelling and Validation Performance Evaluation Accuracy Analysis Summary. Concepts Requirements. System-Level Design.

Download Presentation

Performance Modelling of Complex Hardware/Software Systems

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Performance Modelling of Complex Hardware/Software Systems B.D. Theelen www.ics.ele.tue.nl/~btheelen

  2. Overview • Introduction • System-Level Design • Software/Hardware Engineering • Case Studies • Modelling and Validation • Performance Evaluation • Accuracy Analysis • Summary www.ics.ele.tue.nl/~btheelen

  3. Concepts Requirements System-Level Design Design Time Component-Level Design (Automatic)System Implementation Implementation Introduction • Designing a complex system within limited time involves taking important decisions in an early phase of the design process, which may have a deep impact on the performance of the system • Early assessment of the impact of design decisions involves using system-level design methods and tools www.ics.ele.tue.nl/~btheelen

  4. Three Phases • Formulation • Informal identification and specification of system concepts and design issues • Formalisation • Formal specification of behaviour and architecture • Validation of adequacy of formal system specification • Formal specification of the properties to evaluate • Evaluation • Property analysis and design decisions www.ics.ele.tue.nl/~btheelen

  5. Software/Hardware Engineering (SHE) • Formulation using(stereotyped) UML diagrams • Performed case studies: documents with plain texts and various diagrams • Formalisation based on system-level modelling language POOSL • Intended for complex real-time distributed hardware/software systems • Small set of very expressive language primitives • Formal semantics • Defines unambiguously how to handle concurrency, time and probabilism • Evaluation • Performance evaluation founded onMarkov chain analysis • Analytic computation or estimation by simulation • Formal verification of correctness properties based on model checking • Tools • SHESim: Interactive creation and validation of POOSL models • Rotalumis: Fast execution of POOSL models for analysis www.ics.ele.tue.nl/~btheelen

  6. Case Study: Network Processor • (Deep) packet processing in telecommunication systems • Utilisation of shared busses • Identify possible bottlenecks • Comparison POOSL - SystemC www.ics.ele.tue.nl/~btheelen

  7. Case Study: Dataflow System • Part of Alcatel’s Internet Router • Prepares/processes packets for correct routing • Arbitration of memory accesses from 3 components • Evaluate alternative arbitration mechanisms www.ics.ele.tue.nl/~btheelen

  8. Sources Input Buffers Switch Core Output Buffers Sinks 1 1 1 1 1 N Outputs Inputs 1 N N N N N Case Study: Internet Router Is the specified flow-control mechanismsuitable for various product variants? www.ics.ele.tue.nl/~btheelen

  9. Executable Model • Modelling guidelines (validation) • Readability (aggregate data objects), postpone optimising for execution speed • Log considerations on the adequacy of abstractions • Modelling patterns • Point abstractions, abstraction from scheduling mechanisms, stream-based modelling, scalability of model, model refinement www.ics.ele.tue.nl/~btheelen

  10. Validation • Validation guidelines • Perform step-by-step inspections • Analyse easy-to-check properties • Investigate impact of rare events • Consider symmetries in system • Recall considerations on adequacy of abstractions www.ics.ele.tue.nl/~btheelen

  11. Understandable POOSL Model + Extensions for Performance Evaluation Formal Semantics Markov Decision Process + Reward Structure External Scheduler Discrete-Time Markov Chain + Reward Structure Property Formalisation • Two options for expressing performance metrics • Performance analysis based on external observers: temporal rewards • Reflexive performance analysis: POOSL • Extend model with additional behaviour to evaluate performance metrics • Guidelines for reflexive performance analysis • Define extensions in subclasses • Use method overriding capabilities of POOSL www.ics.ele.tue.nl/~btheelen

  12. Performance Evaluation • Analytical computation: Ergodic Theorem • Estimation by simulation: Central Limit Theorem • Example: packet-loss probability • Enables accuracy analysis with confidence intervals • Enables automatic termination of simulation when results are accurate www.ics.ele.tue.nl/~btheelen

  13. Accuracy Analysis • For Internet Router example: a 99% confidence interval [0.001046, 0.001052] is obtained for the average latency • Estimation of average latency is 0.001049 seconds • We are 99% sure that the real average latency lies in [0.001046, 0.001052] • An upper bound for the relative error is 1% • Takes 0.0072 seconds of simulated real-time • Takes 11.9 hours of simulation time • Central Limit Theorem only applicable for simple averages • Latency, packet loss probability • Complex averages and variances: algebra of confidence intervals • Defines set operations on confidence intervals • Enables accuracy analysis for range of different averages and variances • Jitter, buffer occupancy, processor utilisation, throughput, burstiness • POOSL library classes for analysing different long-run averages www.ics.ele.tue.nl/~btheelen

  14. Summary • Three large case studies in corporation with industrial partners • SHE method distinguishes three phases • Formulation based on informal UML diagrams • Formalisation with formal modelling language POOSL • Guidelines for modelling and validation • Modelling patterns • Guidelines for reflexive performance analysis • Performance evaluation based on Markov chain analysis • POOSL model implicitly defines Markov chain • Extensions for performance analysis define reward structure • Analytical computation or estimation by simulation • Techniques for performance analysis • Confidence intervals allow automatic termination of simulation • Algebra of confidence intervals for accuracy analysis of complex metrics • Library classes for accuracy analysis www.ics.ele.tue.nl/~btheelen

More Related