An Object-level Gateway Supporting Integrated Property Quality of Service

1. DARPA QuO An Object-level Gateway SupportingIntegrated Property Quality of Service Air Force Research Lab/Rome Laboratory Rick Schantz John Zinky, David Karr, Dave Bakken, James Megquier, Joe Loyall, Partha Pal BBN Technologies ISORC ‘99 Saint Malo, France May 4, 1999 “Adapt or perish, now as ever, is nature's inexorable imperative.” – H. G. Wells

2. QuO Outline Motivation & Context Background & Overview of QuO technology The Object Gateway Component Examples of Use Design Considerations Experience to Date & Current Status

3. CINC CINC Operations Planning Group CJTF Planners J2 Missions Centers of Gravity Task refinement Forces refinement Phases refinement COA evaluation COA selection IDB Crisis assessment OPS/ INTEL WORKSTATION TARGET TARGET COA Air COA eval eval Mar Missions COAs Strategy FLTCINC Maritime Planning Center JOINT FORCE MARITIME COMPONENT CMDR JOINT FORCE AIR COMPONENT CMDR Maritime & Air Campaign Assess ment Refinement & eval XIDB Refinement & eval Task refinement Integrated Target Priorities Forces refinement Schedule refinement COA evaluation CASES TARGET TMS CASES & HPC TARGET ACPT Target data / Weaponeering Master Target List TAMPS/ COMPASS ATO JFACC Combat Ops Rear-echelon Sim Center JMCIS IDB rehearse XIDB collaborative development APS/FLEX Target Nomination List Master Attack Plan Attack Plan Status to NATO OPS/INTEL WORKSTATION Air Tasking Order (ATO) APS/FLEX RAAP SHAPE Target Nomination List Weaponeering Many Distributed Systems have Critical QoS Requirements, e.g., Performance, Dependability, and Survivability

4. Chronology QualityObjects (QuO) CRONUS/CORBUS [Simnet] • RSEXEC SBS National Software Works (NSW) OMG/CORBA C++ WWW JAVA RPC OSF-DCE OLE/DCOM Distrib SQL Distrib Objects Ethernet ARPANET MessagePassing IP/TCP NGI? TENEX/Multics MS-DOS UNIX PC 1970 1975 1980 1985 1990 1995 200?

5. State of the Art Today • No effective way to control behavior of critical applications in today’s highly networked environment • Gap between • Emerging low-level mechanisms to control some types of resources • High-level strategies appropriate for adaptable, dependable systems • Researchers generally working on one piece of the solution e.g. • Focus on one critical QoS property: real time behavior, fault tolerance ... • Focus on implications of one time epoch; (boundaries are fuzzy and getting fuzzier with improved dynamic binding) • Functional integration has taken precedence over system properties • End goal is support building integrated QoS, adaptive applications (with varying requirements on granularity of changing behavior): • adaptable: change at runtime while application/service running • reconfigurable: change at runtime while application/service halted • evolvable: change at development time

6. QuO The Quality Objects (QuO) framework supports development of distributed applications with QoS The QuO framework provides • Separation of concerns between software functional properties and QoS needs • Consistent interfaces for QoS measurement and resource management control • Standard middleware interfaces between application and QoS-provider layers • Facilities to enable application- and system-level adaptation QuO is being developed as a common approach to adaptable QoS for a number of projects focusing on different QoS dimensions: • Dependability, replication, group communication (U.Illinois, Cornell, UCSB) • Managed bandwidth, resource reservation (CMU, Columbia Univ.) • Real-time behavior (Washington Univ in St. Louis) • Security, access control, survivability, intrusion detection (TIS) • Integrating different dimensions in an application, e.g., real-time and dependability • Suitability for supporting different approaches to a dimension

7. QuO Outline Motivation & Context Background & Overview of QuO technology The Object Gateway Component Examples of Use Design Considerations Experience to Date & Current Status

8. QuO’s emphasis is on specification, measurement, control, integration, and adaptation, not enforcement • QuO is based upon distributed object computing • Prototype system is based on CORBA, but the concepts are equally applicable to DCOM, RMI • QuO provides a combination of high-level languages for specifying aspects of handling QoS • Aspect Oriented Programming • Code generators generate executable code from the high-level descriptions • QuO enables the application to control and measure QoS and adapt to changes in the level of QoS provided • If a manager or mechanism(s) exists that enforces QoS, QuO provides the means to interface to it

9. Application Alternate Implementations Contract (operating regions) QuO applications specify, control, monitor, and adapt to QoS in the system Specification of operating regions, alternate implementations, and adaptation strategies using QuO’s QDL • Multiple layers of adaptation • managers and mechanisms can adapt to changes in the system • QuO contracts provide another layer of adaptation • Client and user can also adapt System Condition Objects • System condition objects monitor QoS in the system • system condition objects recognize changes in the system and notify the contracts that observe them • QuO contracts notify client programs, users, managers, and other system condition objects through transition behavior • Mechanisms and managers control QoS in the system • a layer below QuO that provides ORB-level services, such as managed communi-cation, replication, or security • contracts and delegates interface to these services through system condition objects Replication Mgr IDS Resource Reservation Manager ORB Network Servers

10. Logical Method Call Object Application Developer Client ORB Proxy ORB Proxy Middleware Developer COTS ORB COTS ORB Network Client Network Server Simplified DOC (CORBA) Runtime Components

11. A QuO application contains additional components (from traditional DOC applications) • Contracts summarize the possible states of QoS in the system and behavior to trigger when QoS changes • Regions can be nested, representing different epochs at which QoS information becomes available, e.g., negotiated regions represent the levels of service a client expects to receive and a server expects to provide, while reality regions represent observed levels of service • Regions are defined by predicates over system condition objects • Transitions specify behavior to trigger when the active regions change • System condition objects are used to measure and control QoS • Provide interfaces to system resources, client and object expectations, mechanisms, managers, and specialized ORB functions • Changes in system condition objects observed by contracts can cause region transitions • Methods on system condition objects can be used to access QoS controls provided by resources, mechanisms, managers, and ORBs • Delegates implement the QoS specific adaptivity behavior • Upon method call/return, delegate can check the current contract state and choose behavior based upon the current state of QoS • For example, delegate can choose between alternate methods, alternate remote object bindings, perform local processing of data, or simply pass the method call or return through

12. As_expected: Measured capacity < 10 Measured capacity >= 2 Extra_resources: Measured capacity >= 10 Insufficient_resources: = Expected Region Measured capacity < 2 = Reality Region Contracts summarize system conditions into regions and define transitions between them • The contract defines nested regions representing possible states based upon measured system conditions • Predicates using system condition objects determine which regions are valid • Transitions occur when a region becomes invalid and another becomes valid • Transitions might trigger adaptation by the client, object, ORB, or system Normal: Expected capacity >= 10 Degraded: Expected capacity < 10 Expected capacity >= 2 Unusable: Expected capacity < 2 As_expected: As_expected: Measured capacity >= 10 Measured capacity < 2 Insufficient_resources: Extra_resources: Measured capacity < 10 Measured capacity >= 2

13. System Conditions Project a Value to the Application, But also Must Maintain the Value Composed Value Application Developer Simple Value Control Value Status Value Measured Value (Sensor) Control Value QoS Developer QuO Kernel CORBA Object RSVP Controller Device Status Service Mechanism Developer Specialized ORBs or Services

14. Delegates change their behavior based on their contract’s current regions Method Call SysCond Result PreMethod Measurements Delegate Contract Current Regions Dispatch Alternative Behaviors Current Regions Pass Through Block Shape PostMethod Measurements SysCond Lower Result Lower Method Call

15. The QuO Toolkit provides tools for building QuO applications • Quality Description Languages (QDL) • Support the specification of QoS contracts (CDL), delegates and their adaptive behaviors (SDL), connection, creation, and initialization of QuO application components (TBD) • QuO includes code generators that parse QDL descriptions and generates Java and C++ code for contracts, delegates, creation, and initialization • System Condition Objects, implemented as CORBA objects • QuO Runtime Kernel • Contract evaluator • Factory object which instantiates contract and system condition objects CORBA IDL Contract Description Language (CDL) Contracts Delegates QuO Runtime Code Generators Structure Description Language (SDL)

16. Contract Contract Network QuO adds QoS control and measurement into the DOC remote method call Logical Method Call Application Developer Client Object SysCond SysCond Delegate Delegate SysCond SysCond SysCond Qosketeer SysCond SysCond ORB Proxy ORB Proxy Mechanism/Property Manager Mechanism Developer Specialized ORB Specialized ORB Client Network Server

17. QuO Outline Motivation & Context Background & Overview of QuO technology The Object Gateway Component Examples of Use Design Considerations Experience to Date & Current Status

18. QuO Gateway Control IIOP IIOP IIOP Glue Client-Side ORB QuO Gateways Support Specialized Communication Protocols and Below the ORB Mechanisms • The QuO gateway enables insertion of below-the-ORB mechanisms and specialized network controls • The gateway translates IIOP messages into specialized communication protocols or network level controls • To the client-side, the QuO gateway looks like the remote ORB • To the object-side, the QuO gateway looks like the client’s ORB • The two ends of the gate-way are on the same LAN as the client/object • Currently, we have gate-ways that support Ensemble group communication, RSVP resource reservation, and IIOP over TCP/IP QuO Gateway Control Group Replication (AQuA) IIOP Glue Server-Side ORB Bandwidth Reservation (DIRM) IIOP over TCP/IP (default) WAN

19. contract Contract QuO Based Bandwidth ManagementMeasures and Controls Network Resources Object Client Logical Method Call User Expectation Application Manager Functional Functional Delegate Delegate Sensors Expected Performance Signal Bandwidth Control Rights Policies CORBA Measured Performance Measured Performance Rights Policies Signal Sensors CORBA Functional With Trace Record Functional With Trace Record CORBA Corba Skeleton Skeleton ORB Proxy ORB Proxy Network Management Status Collection Bandwidth Control Commercial ORB Commercial ORB custom SNMP SNMP Host Status Sensor Host Status Sensor Darwin bandwidth reservation IIOP Glue Resource Aware Transport IIOP Glue IIOP IIOP Resource Aware Transport Network Server Host Client Host RSVP bandwidth reservation QuO Gateway

20. An RSVP Aware Gateway can Set up and Tear Down RSVP Sessions Between Hosts. Client CORBA QoS API Object QuO Delegate and Contract SysCond QuO D & C ORB Proxy ORB Proxy QoS Socket API Bandwidth Reservation API IIOP IIOP QuO-GTWY QuO-GTWY QoSME Smarts InCharge QoSME QoSMib RT. RSVP SNMP RT. RSVP RSVP Reservation Application Management API QoSME from Columbia University will be used to control RSVP for the <X>-ORB

21. Contract Low Cost: Requested = 1 Too Low: Measured < 1 Normal: Measured > 1 Too Low: Measured < 2 Available: Requested >= 2 Normal: Measured > 2 Object Replica Group A simplified example: Improved Dependability through Replication Client application Code Remote method call Object Delegate Region? Normal Multicast

22. Groups in AQuA Replication Groups: composed of one or more identical objects Connection Groups: composed of the members of two replication groups that wish to communicate PCS Group: composed of the Proteus replicas and objects that desire to communicate with the Proteus manager Point-to-point Groups: composed of onedependability manager replica and one object factory Connection group 1 Replication group 1 Replication group 2 Object Factory 1 Point-to-Point groups • • • • • • • • • Object Factory 2 Connection group 2 Proteus manager 1 PCS group Object Factory 3 • • • • • • • • Proteus manager 2 Object Factory 4 Replication group 3 Proteus manager 3 Replication group 4 • • • Object Factory 5 • • • • Connection group 3

23. Application Process Object Gateway-Half Process Client CORBA ORB IIOP Glue Dispatcher Handler #1 Group Member Buffers Voter Maestro/Ensemble

24. Network DTEL++ and OO DTE add access control into the DOC remote method call Logical Method Call Application Developer Client Object DTE metadata ORB Proxy ORB Proxy OO domain definitions DTEL++ Security Developer type bindings for interfaces, and operations; policy distribution manager ORB ORB ORB Gateway Mechanism Developer OO DTE Client Network Server

25. The QuO Gateway Manages IIOP Connections and Interfaces to Protocols which Manage QoS • Shell is a Base to build Specialized QoS Aware Gateways • To the “Client” ORB, the QuO Gateway looks like the object • To the “Server” ORB, the QuO Gateway looks like a client • The ends of the gateway are on the same LAN as the Client/Object and may be on the same host • IIOP Glue will multiplex and demultiplex GIOP messages Client-Side Gateway half Server-Side Gateway half Control Control WAN Client Specialized Protocol Server IIOP Glue IIOP Glue IIOP IIOP

26. Control Client-Side Interface Presents a Partially Parsed GIOP Request and Call Block for the Reply • The DSI interface returns a Server_Request structure. • Additional information is derived from call context • The suspended thread is remembered in Call block • This info is bundled into a Call block and puts it on the Request_Queue for the specialized transport • When the specialized transport attaches the reply and puts it on the Reply_Queue for the DSI wrapper. Request Queue DSI Call Block Context Mechanism Specific Handler Client IIOP Glue IIOP Thread Suspend Server Request Call Block Reply Queue

27. Control Server-Side Interface Accepts a Call Blockand Creates a DII Request. • The Call Block is removed from the Request_Queue • The DII interface sends a Request structure to an object reference • The return values is bundled into the Call block and put on the Reply_Queue for the specialized transport Request Queue Call Block DII Mechanism Specific Handler Obj Ref Server IIOP Glue Thread Suspend IIOP Request Call Block Reply Queue

28. SNMP SNMP C O N V E R T Instrumentation Instrumentation C O N V E R T IIOP Glue Bandwidth Aware Transport Bandwidth Aware Transport IIOP Glue RSVP Network Bandw - Control Bandw - Control Bandwidth Reservations RSVPd RSVPd QoSME QoSME Specialized Embedded Protocols: Network Resource Reservations

29. (Leader) ... C-Rep1 C-Rep2 C-RepN ORB ORB ORB 1 1 1 2 2 2 GW GW GW 4 3 3 3 5 Group Comm 5 5 5 6 6 GW GW GW 7 7 7 ... ORB ORB ORB S-Rep1 S-Rep2 S-RepM (Leader) Specialized Embedded Protocols: Group Communications

30. QuO Outline Motivation & Context Background & Overview of QuO technology The Object Gateway Component Examples of Use Design Considerations Experience to Date & Current Status

31. Status • QuO version 1.0 released September 1998, including the QuO runtime kernel, CDL and SDL and their code generators, system condition object library, concept demonstration gateway, example applications, and documentation • Version 2.0 scheduled for end of May 1999 • improved language and contract support • a revised general gateway mechanism supporting RSVP • instrumentation support • Quorum Distributed Object individual QoS property incremental results: • DIRM bandwidth management using RSVP & CMU Beagle • AQuA dependability using Proteus and Ensemble • TAO real time ORB and schedulableevent channels • TIS ORB based DTE access control with dynamic security policy (tentative)

32. Network Group Replication (AQuA) Bandwidth Reserve (DIRM) IIOP over TCP/IP (default) QuO Architecture • Quality description languages • currently CDL (QoS contracts), SDL (adaptation strategies), simple connection language • soon to come: capabilities to describe real-time behavior (TAO’s RIDL) and security (TIS’s DTEL++), plus improved SDL and connection • QuO runtime kernel • System condition libraries, example applications, documentation • QuO gateway and instantiations for group communication, resource reservation, TCP/IP, and (soon to come) security • QuO instrumentation Contract Description Language (CDL) CORBA IDL Delegates Contracts QuO Runtime Code Generators Structure Description Language (SDL) Object Client Logical Method Call SysCond SysCond Delegate Delegate SysCond SysCond SysCond SysCond SysCond ORB Proxy ORB Proxy Mechanism/Property Manager Specialized ORB Specialized ORB QuO Gateway QuO Gateway Control Object-Side ORB Control Client-Side ORB IIOP IIOP IIOP Glue IIOP Glue WAN

33. DARPA SM Caching and Consistency Georgia Tech OO domain definitions type bindings for modules, interfaces, and operations SIGMA Trusted Information Systems SNMP SNMP ORB Gateway S OO DTE Instrumentation Instrumentation Bandwidth Aware Transport Bandwidth Aware Transport RSVP Network Bandw - Control Bandw - Control RSVP Bandwidth Reservations RSVPd RSVPd QoSME QoSME QuOIN - Distributed Object Integration Realtime Control Language QuO Based Dependability Mgmt Common IDL Compiler Application Logic Specialized Realtime ORB Expectations Callbacks Application Proteus Dependability Manager Advisor Object Delegate SystemConditions Contract QuO Protocol Coordinator IIOP / Proteus / Ensemble Interface AQuA Gateway AQuA Gateway Managed Network with Ensemble Object Replica Object Replica Object Replica Object Replica Object Replica Object Factory Object Factory Object Factory ... Network Management Remos Collector Darwin Control Obj OO Security Beagle SNMP SNMP Darwin Bandwidth Reservation Remos HostLoad Cached Data & Impl Consistency Protocols Remos HostLoad Network QuO Based Bandwidth Mgmt