Run Time Performance Modeling and Measurement of Adaptive Distributed Object Application DOA 2002

1. Run Time Performance Modeling and Measurement of Adaptive Distributed Object ApplicationDOA 2002 Oct 30, 2002 John Zinky Joe Loyall Richard Shapiro BBN Technologies

2. ... Property Managers Policy Managers ... Name Services Event Services Configuration Management Status Collection Bandwidth Control QoS Adaptation Needs to be Supported at Every Layer in a Distributed Application Applications Applications Logical Method Calls Client Object Middleware Middleware QoS Adaptive Layer QoS Adaptive Layer Network Based Services Distributed Objects COTS ORB Schedulers Distributed Objects COTS ORB Schedulers Replication Protocols Group Communications Replication Protocols Group Communications Operating System Operating System Resource Managers Resource Managers Client Host Network Servant Host

3. in args OBJECT (SERVANT) OBJECT (SERVANT) in args Contract Contract CLIENT CLIENT OBJECT (SERVANT) OBJECT (SERVANT) operation() OBJ REF out args + return value IDL SKELETON IDL STUBS OBJECT ADAPTER Network ORB ORB ORB ORB IIOP IIOP IIOP IIOP Network QuO Adds Specification, Measurement, and Adaptation into the Distributed Object Model Application Developer CORBA DOC MODEL Mechanism Developer Application Developer CLIENT CLIENT operation() OBJ REF out args + return value Delegate Delegate QuO Developer SysCond SysCond SysCond QUO/CORBA DOC MODEL SysCond IDL SKELETON MECHANISM/PROPERTY MANAGER IDL STUBS OBJECT ADAPTER Mechanism Developer

4. Client Object QuO Contracts Measured QoS Delegate Expected QoS Probes Delegate Resource Status Service Correlate Probes Infer Probes Integrate Piggybacked Measurements Translate Collect ORB ORB Probes Probes Status Resource Resource Resource QoS Parameters are Measured bothIn-band and Out-of-Band

5. Key Points • QoS Adaptation can be separated from Functional behavior • The running example will be extended from simple QoS adaptation to comprehensive adaptation • The base application is unchanged, just different QoS adaptations are inserted between the client and server • Predicted QoS behavior complements Measured QoS • They can be compared to diagnose performance problems • They can be combined for more robust QoS metrics • A Model of Application is implicate in all QoS Adaptation • Model resources, application and how they are bound together • QuO Resource Status Service (RSS) can make a Run Time performance model of the application and the resources it uses

6. An Example: User Can Adapt by Changing the Amount of Data and the Level of Processing Processed UnProcessed Check Box For User Specification Big Small

7. Quo Delegate Can Change the Method Called Based on the Value of the Contract and Sysconds Application • The User can change the desired QoS, by setting the value of a SysCond • The Contract integrates the User’s desires with other information to form a Region • The Delegate uses the Region to dispatch to a behavior. In this case call a different method on the object User Sets Expectation Client Code Call Back Reference SysCond QuO Kernel Delegate Contract Proxy ORB ORB Network Control Manager QuO Kernel ORB SC SC Proxy Del. Contract Object

8. QuO Contract Regions Summarizes the User’s Desires Contract Description Language contract UserAdapt ( // Expectation from users application syscond ValueSC ValueSCImpl userEnableSmall, syscond ValueSC ValueSCImpl userEnableUnprocessed ) { region Big (not userEnableSmall) { region Processed (not userEnableUnprocessed) // Use Big Processed Pictures {} region Unprocessed (userEnableUnprocessed) // Use Big UnProcessed Pictures {} } region Small (userEnableSmall) { region Processed (not userEnableUnprocessed) // Use Small Processed Pictures {} region Unprocessed (userEnableUnprocessed) // Use Small UnProcessed Pictures {} } }; Generated Code Abundant Resources Panel From QuO GUI

9. QuO Components Are Packaged Into Reusable Bundles of “Systemic Behavior” Called Qoskets • The Qosket encapsulates a set of contracts (CDL), system condition objects (IDL), and QoS adaptive behavior (ASL) • The Qosket exposes interfaces to access QuO controls and information (specified in IDL) • The Qosket separates the functional adaptive behavior from the QoS adaptive behavior and the middleware controls from the QoS mechanisms Client Code Qosket Delegate QuO Contract Functional Behavior Adaptation QoS Apdative Behavior SysCond QoS Mechanisms Object

10. Application Specific Behavior Is Bound to the Qosket to Achieve the Desired QoS Adaptively Aspect Specification Language behavior UserAdapt () { remote_object slide::SlideShow server; void slide::SlideShow::read(in long gifNumber, out string size, out octetArray buf) { inplaceof METHODCALL { region Big { region Processed { server.readBigProcessed(gifNumber, size, buf); } region Unprocessed { server.readBig(gifNumber, size, buf); } } region Small { region Processed { server.readSmallProcessed(gifNumber, size, buf); } region Unprocessed { server.readSmall(gifNumber, size, buf); } } } } }; Delegate Behaviors BigProcessed() BigUnprocessed() SmallProcessed() SmallUnprocessed()

11. Suppose: When the Call Latency Is Too High,Get Lower Quality Picture Which Use Less Resources DownShift Contract (CDL) Delegate Behaviors (ASL) • Contracts Support State machine representation of regions • QoS Designer has to decide which behavior to use for each region Fast BigProcessed() Latency > HighThreshold Latency < LowThreshold BigUnprocessed() or SmallProcessed() Med Latency > HighThreshold Latency < LowThreshold SmallUnprocessed() Slow

12. Contract Probe Sysconds Can Measure the Latency of a Call by Observing the Internal Contract Evaluations • Observation of Contract evaluation is supported as part of the CDL language • Latency Sysconds are part of the QuO Library of Sysconds. Call Return Pre-eval Delegate Post-eval SysCond

13. The Call Latency Changes Depending on the Method Call (User Selected Region) BigProcessed() Instrumentation Settling Time SmallUnprocessed() SysCond Panel from QuO GUI Note: The Latency will also change if the underlying resources change

14. Suppose: The Contribution to the Latency can be Determined for the Network and Server Component Latency Contract (CDL) Delegate Behaviors (ASL) • Contract Region predicate compare the time spent in the Network (or Server) to a threshold • Contract can hold in a region until the statistic have converged Fast Network BigProcessed() Fast Server Slow Server BigUnprocessed() SmallProcessed() Slow Network Fast Server Slow Server SmallUnprocessed()

15. Detailed Timing Measurement can be made by adding code to update a trace record Call Trace Record • ASL is used to weave in calls to update the trace record • The trace record is sent to the server-side via an additional parameter in the interface TS1 TS2 TS3 TS4 Excpt size Client Delegate Server Delgate

16. Suppose: When there is Excess Resources Choose High Quality Picture Resource Capacity Contract (CDL) Delegate Behaviors (ASL) • Contract Region predicate compare the FittedResources capacity for the Network (or Server) to a threshold Excess Network BigProcessed() Excess Server Constrained Server BigUnprocessed() SmallProcessed() Constrained Network Excess Server Constrained Server SmallUnprocessed()

17. Contract System Conditions Hold the Processed Summary Statistics which are used by the Contract Kernel • SysConds have to deal with issues concerning startup, loss of information, invalid information. • SysConds collect only the system properties need by contracts • SysConds have a CORBA interface so that they can be queried by centralize management systems • SysConds can detect thresholds and alert its clients to changes SysCond Signal Delegate probes Summary Fits Trace Records Statistics

18. Fits are extremely sensitive to changes in the underlying process The middle graph show points from both before and after the cross traffic started Contracts support hysteresis, so that the threshold to enter a region is different then the threshold to level Cross Traffic can Change theEffective Capacity of the Network

19. Suppose: The Resource Capacity Comes from Out-of-Band Measurements Component Latency Contract (CDL) Delegate Behaviors (ASL) • Contract Region predicate compare the ExpectedResources Capacity of the Network (or Server) to a threshold • Many sources of out of band measurement must be integrated at each Client. Rich Network BigProcessed() Rich Server Constrained Server BigUnprocessed() SmallProcessed() Constrained Network Rich Server Constrained Server SmallUnprocessed()

20. Resource Status Service Integrates External Data about Remote Resources to Infer Expected QoS Expected QoS Isolation: The Quosketeer will work with a high level description of available resources Integration: Conflicting measurements will be resolved to always give the best guess. Translation: different standards for Resources MIBs will be translated into a QuO Resource Ontology Collection: interfacing details will be handled by Data Feed Resource Ontology Resource Status Service Data Scopes Formulae Model Level Resource Level Integration Level Translate Store Translate Store Translate Store Data Feeds Collect Collect Collect http Custom CORBA Configuration Remos StatusTEC (Base-Line) (Network) (Host)

21. Out-of Band Measurements are Organized into a QoS MIB Host_10.23.12.1_CPU_LoadAverage_Expected Host _10.23.12.1_Process_2232_Socket_5223 Host _10.23.12.1_Process_2232_Object_Key_tao/foo/2 IPFLOW_ 10.23.12.1_ 10.23.12.1_Capacity_Max • Keys form a hierarchy of names • Root is an entity • Leaves are QoS Properties • Data Values have multiple attributes • Value (dynamic type, such a boolean, int, double, string…) • Credibility (belief on how good the measurement is) • Units, Timestamp, collector name, etc) • Subscribe to Changes on QoS Property values • Based on Existing Standard MIBs, • WBEM/CIM, SNMP, QoSMIB,

22. Each Collection Service has its own Dissemination Infrastructure When there are multiple values, The most Creditable is used. STEC Host Probes Channel Mesh push Resource Status Service Integration Level Network Management System eg. Remos Network Routers Collection Host Translate Store Translate Store Translate Store Data Feeds poll Collect Collect Collect Host Configuration Network CORBA Web pages Host Sites Appl Test Custom http

23. Suppose: Pick the Server with the Lowest Latency to the Client • Contracts support selecting the “best” region out of a pool of regions, based on a selection criteria • Many factors need to be added to model for predicting latency between a client and server Sever Selection Contract (CDL) Delegate Behaviors (ASL) Select: Latency[i] Server[1].BigProcessed() Server[1] Server[2].BigProcessed() Server[2] Server[3].BigProcessed() Server[3]

24. RSS Data Representation Shadows the Real World with Objects and Relationships • Data Scopes represent objects with attributes and relationships • Containment relation allows a child to inherit all the attributes of its parent. • Specialized relations can be followed to inherit other attributes from peers. • Query Functions help find Scopes • Symbol represents a link to another Scope or an attribute value. • Symbols are Static, value changes are not pushed. • Symbols are used to find formulas • Formula is a calculation, which depends on the values of other Formulas. • Value Changes are Pushed to Subscribers, such as other formulas or SysConds. • Raw Data from Feeds are published as formulas Host Process Object Method Method Object Object Key Interface Calls Latency= Load/Capacity

25. Scopes Represent the Relationship between Real World Objects • Scopes are the foundation on which to find formulas • Scopes Represent different layers of resources • Queries can find scopes by following relationships. • Paths follow the containment relationship • Host(lemon)->Process(1243)->Object(773) • Symbols keep the value of scope attributes • Pointers to other scopes • Identifiers • Random facts Host Process Object Class Method Method Method Method IP Flow TCP IIOP Host Process Object Class Method Method Method Method

26. Host capacity Process Object Class Method Method Method Method Load Latency= Load/Capacity Formulas form a Publish and Subscribe Network which Traces and Transforms Data • Formulas Subscribe to values relative to the scopes for which they are attached • Formula Setup time uses scopes to find the sub-formulas. • The expensive setup happens only once. • Value Calculations are done continuously as raw data changes • Both forward chaining and backward chaining modes are supported, • Pull vs Push • Formulas are extremely Reusable • Attach them to a scope and they will calculate a value

27. Java performance depends on many factors CPU Clock speed, Number of CPUs,OS, Java VM, CPU architecture, Micro-benchmarks are run when the application process starts up. Dynamic Factors, such as load average, are also used to model the effective CPU capacity at runtime Micro-Benchmarks are used to Calibrate CPU Resources.

28. RobustnessManagerNode ManagementAgent Attack Assessment Plugin DosServlet Detect Status NodeAgentC NodeAgentB Detect Status QuO Components were Used to Implement Distributed Denial of Service Architecture Cougaar Distributed Agent Architecture GUI NodeAgentA Attack Detector Plugin AttackStatus Listener Plugin Dos Detectors Syscond QuO RSS SSL Aspect Traffic Mask Aspect

29. Metric Service Customizes the QuO RSS for the Cougaar Environment Sensor Components Node-level adaptation Metrics Servlets Adaptivity Engine SysCond (Keys) (Paths) QuO/RSS Metrics Writer Service Data Scopes Data Formulas Node/Agent Models Metrics Reader Service Integration Formulas Cougaar Data Feed STEC Data Feed Config Data Feed NMS Config Files STEC Channel STEC Channel Mesh STEC Host Probe Optional Sharing

30. QuO-Based Actuators Control Behavior Using Both Blackboard Objects and Metric Services Blackboard Attack Status Actuator Aspect Attack Status Listener Plugin SysCond QuO Delegate In-band Adaptive Behavior QuO Contract SysCond Attack Status Service Metrics Service Resource Control Services Control Aspects Measurement Aspects Resource Configuration

31. Conclusions • QuO Code generators inserts measurement probes in-band to the invocation of method calls going from the client to the server. • QuO RSS integrates Out-of-Band measurements to predicted the expected resource capacity • QuO RSS includes support for evaluating performance models at run time. • QuO supports reuse of the QoS measurement code, e.g. libraries of Sysconds, Data Scopes, Data Formulas, and Data Feeds.