CoSMIC: A Model-Driven Middleware for Provisioning Large-scale Distributed Real-time and Embedded Systems

1. CoSMIC: A Model Driven Middleware for Provisioning Large-scale Distributed Real-time and Embedded Systems Dr. Aniruddha Gokhale a.gokhale@vanderbilt.edu www.dre.vanderbilt.edu/~gokhale Assistant Professor (EECS & ISIS) Vanderbilt University Nashville, TN 37203 Work supported by AFRL contract# F33615-03-C-4112 for DARPA PCES Program

2. DRE Applications Middleware Services distributed system Middleware Operating Sys & Protocols Hardware & Networks Research Synopsis Model Driven Approach for Distributed Real-time & Embedded Middleware • Develop, validate, & help to standardize technologies that: • Model • Analyze • Synthesize & • Provision • multiple layers of middleware for distributed real-time and embedded (DRE) systems that require simultaneous control of multiple quality of service properties end-to-end <CONFIGURATION_PASS> <HOME> <…> <COMPONENT> <ID> <…></ID> <EVENT_SUPPLIER> <…events this component supplies…> </EVENT_SUPPLIER> </COMPONENT> </HOME> </CONFIGURATION_PASS>

3. Distributed Real-time & Embedded Systems The Future The Past • Network-centric & large-scale • Dynamic context • Stringent simultaneous quality of service (QoS) demands • Part of larger systems • Resourceconstrained • Stringent simultaneous quality of service (QoS) demands • Part of larger systems • Resourceconstrained

4. BSE IOM BSE BSE IOM IOM IOM IOM IOM IOM BSE BSE BSE IOM IOM IOM DRE Applications DRE Applications IOM IOM IOM BSE BSE BSE IOM Middleware Services Middleware Services IOM IOM IOM IOM Middleware Middleware Operating Systems Operating Systems Hardware & Networks Hardware & Networks Historical Overview of DRE Systems Development Technology Problems DRE systems have historically tended to be: Stovepiped Proprietary Brittle & non-adaptive Expensive Vulnerable • Middleware has effectively factored out many reusable mechanisms & services from what was traditionally DRE application responsibility • Middleware is no longer primary DRE system performance bottleneck • Historically, mission-critical apps were built directly atop hardware • Tedious • Error-prone • Costly over lifecycles

5. DRE Applications Middleware Services Middleware Operating Sys & Protocols Hardware & Networks Layers of Middleware for DRE Systems • Middleware characteristics • Uniform abstraction over h/w & OS e.g., JVM, ACE • Distribution capabilities like location transparency, data marshaling e.g., CORBA, J2EE, webservices • Commonly used higher level services e.g., Naming, Event • Services specific to domains e.g., Bold Stroke (Avionics), AMW (telecom)

6. DRE Applications Middleware Services Middleware Operating Sys & Protocols Hardware & Networks R&D Contributions to Middleware for DRE Systems • Network Element Software Management, Network contact center service • patent pending • Fault tolerant CORBA technologies & standardization effort • HiPC 00, DOA 00, J. Cluster Computing 03 • The TAO high-performance, Real-time CORBA ORB • Influenced Real-time CORBA standard • ACM Sigcomm96, IEEE Journal of Selected Areas in Communication 99, IEEE Transactions on Computing 97, Journal of Real-time systems 99, IEEE Globecomm 96/97, IEEE Comm 97 • Automatic test suite generation for protocol conformance testing • ASU Master’s thesis

7. … … … Middleware Bus Container Replication Security Persistence Notification Emergence of Component Middleware • Component middleware gaining importance (CCM, J2EE, .NET) • Components encapsulate application core logic • Components possess • Event sinks & sources • Connection points e.g., receptacles • Interfaces e.g., facets • attributes • Containers provide execution environment for components with common operating requirements • Containers communicate via a middleware bus

8. MIDDLEWARE ARCHS CORBA Apps J2EE Apps .NET Apps DRE Applications CORBA Services J2EE Services .NET Services Middleware Services CORBA J2EE .NET Solaris VxWorks Middleware Win2K Linux LynxOS Operating Sys & Protocols Hardware & Networks DRE Systems: The Challenges Ahead • There is a limit to how much application functionality can be factored into broadly reusable COTS middleware • Middleware has become extremely complicated to use, configure, & provision statically & dynamically • There are now multiple middleware technologies to choose from

9. … … … … Middleware Bus Container Container Replication Security Persistence Notification Our Solution: Model-Driven Middleware for DRE Systems Key Benefits • Preserves DRE application functional & systemic QoS properties as high level models • Domain-specific languages & analysis/synthesis tools transform models to customize underlying multi-layered middleware platforms • Leverages & shapes standards for wider applicability <CONFIGURATION_PASS> <HOME> <…> <COMPONENT> <ID> <…></ID> <EVENT_SUPPLIER> <…events this component supplies…> </EVENT_SUPPLIER> </COMPONENT> </HOME> </CONFIGURATION_PASS> Related Work: • MIC, Vanderbilt (Sztipanovits, Karsai, et al) • Ptolemy, UC Berkeley (Lee et al) • Cadena, KSU (John Hatcliff et al) • Quality Connector, LMCO (Joe Cross et. al)

10. MDA-Middleware Integration • Configuring and deploying application services end-to-end • Composing components into component servers • Configuring application component containers • Synthesizing application component implementations • Synthesizing dynamic QoS provisioning and adaptation logic • Synthesizing middleware-specific configurations • Synthesizing middleware implementations

11. Our Target Middleware: CIAO CORBA Component Model Focus on infrastructure support for composition of the following aspects • CIDL compiler to synthesize component descriptor metadata & stubs/skeletons • RT event channel integration with CIAO containers • Assembly & deployment framework • Collaboration with Washington University RT Event Channel RT Event Channel www.dre.vanderbilt.edu/CIAO Component Integrated ACE ORB (CIAO)

12. Boeing Bold Stroke: Our Research Vehicle • Avionics Product Line Component Model • DRE system with 3,000+ domain-specific software components, 3-5 million lines of C++ code • 100+ developers • Mission-control software for Boeing military aircraft, e.g., F-18 E/F, Harrier, UCAV • Leverages the ACE+TAO middleware • Used as Avionics Open Experimental Platform (OEP) for DARPA/IXO PCES & MoBIES programs • Moving towards using CIAO CCM

13. Application components (consumer) • Push-based event model • Pull-based data model • This common usage simplifies assembly complexity • Application components (supplier) • ACE+TAO real-time CORBA distribution middleware • Hardware organization Bold Stroke Architectural Elements

14. Bold Stroke Illustrative Example Adapting to changing operating conditions Determining the right assembly and deployment Determining the right assignment of priorities Determining the right concurrency strategy Basic Modal Single Process Product Scenario (2 operational modes)

15. … … … Container Middleware Bus Replication Security Persistence Notification Research Thrusts of CoSMIC • Applying MDA to address • the end-to-end deployment aspect of DRE applications • the component container configuration aspect • the middleware configuration aspect • the dynamic QoS provisioning & adaptation aspect • Our tool suite is called CoSMIC • CoSMIC = Component Synthesis using Model Integrated Computing

16. Challenge 1: Component Assembly & Deployment • Application components are assembled and then deployed in a way that provides optimum resource utilization & delivers required QoS to the application • e.g., Bold Stroke scenarios involve assembling & deploying hundreds of components • Assembly & deployment can be scripted by using XML descriptors & deployment tools CONTEXT

17. Challenge1: Component Assembly & Deployment XML file in excess of 3,000 lines for medium sized scenarios PROBLEMS Existing practices involve handcrafting the XML descriptors <!– Associate components with impls --><componentfiles> <componentfile id=“RateGenerator"> <fileinarchive name=“HouseRateGen.csd"/> </componentfile> <componentfile id=“HiResGPS"> <fileinarchive name=“aGPS.csd"/> </componentfile> <componentfile id=“cockpitDisplay"> <fileinarchive name=“navDisplay-if.csd"/> </componentfile> </componentfiles> Partitioning, Distribution and Deployment done in ad hoc manner Modifications in assembly requires modifying XML file

18. Challenge 1: Component Assembly & Deployment • ESML developed by Dr. Gabor Karsai et. al for DARPA/IXO MoBIES program • CIDL compiler developed by our group at ISIS SOLUTION • A domain-specific Component Descriptor Modeling Language (CDML) • Currently leverages ESML for synthesis of assembly descriptors • ESML allows modeling component behavior & interactions in Bold Stroke • Analyze component requirements and synthesize deployment scripts • Synthesize component glue code to interact with environment

19. Synthesize custom strategies e.g., lazy or eager Challenge 1: Component Assembly & Deployment Next Steps Synthesize assembly descriptor metadata Develop a component descriptor modeling language (CDML) Synthesize platform-specific metadata Determine appropriate assembly & deployment

20. High priority Medium priority Low priority Challenge 2: Configuring Container Policies CONTEXT • Components execute in containers that decouple runtime configuration from the component implementation e.g., • Priority propagation models • Threading models • Security, replication, persistence policies • Internal buffer sizes • e.g., Boldstroke components run in a multithreaded environment with different end-to-end priorities • Usually specified by the deployer using XML-based metadata

21. server object management middleware Challenge 2: Configuring Container Policies PROBLEMS • Existing techniques for metadata configurations rely on ad hoc manual configurations e.g., CORBA server-side programming Determine various middleware policies for server objects e.g., security, lifetime, replication Determine thread pool sizes; how are they shared; number of lanes and their priorities; if borrowing is enabled Determine right buffer sizes Determine end-to-end priority propagation model to use Determine the server object management policies Ensure semantic compatibility among chosen configurations • This “glue code” is traditionally handcrafted

22. Challenge 2: Configuring Container Policies SOLUTION • Develop a domain-specific Container Policy Modeling Language (CPML) • Current version restricted to container configuration glue code generation in the CORBA environment • Container policies are still manually chosen

23. High priority server object management middleware Challenge 2: Configuring Container Policies Next Steps • Extend CPML to capture application QoS requirements in a platform independent form • Design model transformers to desired platforms • Develop tools that will determine the right values for various platform-specific parameters

24. Stub Skeleton 20 10 5 1 20 10 5 1 M/WBus I/O Subsystem Challenge 3: Configuring Middleware End-to-End CONTEXT • Middleware must be configured with the appropriate systemic metadata end-to-end • e.g., in Bold Stroke example, appropriate priority banded connections must be set between application services

25. Stub Skeleton 20 10 5 1 20 10 5 1 M/W Bus I/O Subsystem Challenge 3: Configuring Middleware End-to-End PROBLEMS Determine right marshaling optimizations Determine right demux strategy Determine right concurrency strategy Configuring subset of underlying transports Determine right connection mgmt policy • Highly flexible middleware tend to provide numerous configuration knobs that can be configured to deliver required systemic properties to applications • Existing techniques of metadata configurations rely on ad hoc manual selection of configuration parameters

26. Challenge 3: Configuring Middleware End-to-End SOLUTION • Developed a domain-specific modeling language for TAO/CIAO called Options Configuration Modeling Language (OCML) using GME • User provides a model of desired options & their values e.g., • Middleware bus resources • Concurrency & connection management strategies • Constraint checker flags incompatible options • Synthesizes XML descriptors for middleware configuration

27. High priority Challenge 3: Configuring Middleware End-to-End Next Steps • Extend OCML to capture application QoS requirements in a platform independent form • Design model transformers to synthesize platforms-specific configuration models • Design tools that will determine the right values for various platform-specific configuration parameters

28. Appln Server Appln Server Static configuration Challenge 4: End-to-end QoS Provisioning & Enforcement CONTEXT e.g., BBN QuO, UIUC Quarterware, Lancaster OpenORB Made feasible using adaptive & reflective middleware Need to provide runtime QoS adaptation along functional path

29. Challenge 4: End-to-end QoS Provisioning & Enforcement Glue code depends on the adaptive/reflective middleware used PROBLEMS Need to add instrumentation code to collect QoS metadata Instrumentation code manually handcrafted Need to retrofit middleware to collect and distribute desired QoS metadata Most existing middleware not designed to collect and distribute QoS metadata

30. Applications Interceptor Middleware Domain-Specific Services Common Services Distribution Middleware Infrastructure Middleware Operating System Local Resource Manage- ment Endsystem Language-specific QoS aspects generator Challenge 4: End-to-end QoS Provisioning & Enforcement SOLUTION QoS Metadata model Middleware Models Integrated Models Program TransformationTool Collaboration with Dr. Jeff Gray (UAB)

31. Applications Interceptor Middleware Domain-Specific Services Common Services Distribution Middleware Infrastructure Middleware Operating System Local Resource Manage- ment Endsystem Language-specific QoS aspects generator Challenge 4: End-to-end QoS Provisioning & Enforcement Next Steps QoS Metadata model Domain specific modeling language for middleware models Middleware Models Domain specific modeling language for QoS metadata Develop model weaver Integrated Models Program TransformationTool Develop generators

32. Research Impact & Future Work Advanced MDA • Current progress stems from years of iteration, refinement, & successful use Shape the standards e.g., OMG’s Model Driven Architecture (MDA) Model integrated middleware CoSMIC RT/CCM ACE/TAO CORBA Component Model (CCM) • Future Research Directions • High confidence, geographically distributed DRE systems • Grid applications • Large enterprise systems • Focus on platform-independent models Component Models (EJB) Real-time (RT) CORBA CORBA & DCOM DCE Micro-kernels RPC ARPAnet 1970 2000 2005 2010 Year

33. Concluding Remarks Model, analyze, synthesize, and provisionmiddleware technologies at multiple layers for distributed real-time and embedded systems that require simultaneous control of multiple quality of service properties end-to-end Model Driven Approach to: • Configure and deploy DRE applications end-to-end • Configure application component containers • Synthesize middleware-specific configurations • Synthesize dynamic QoS provisioning and adaptation logic www.dre.vanderbilt.edu/~gokhale/cosmic.html

34. Vanderbilt DRE DOC Group Capabilities Configurable Communication Systems Mission Critical DoD Systems Industrial Process Control Medical Imaging Systems CoSMIC Model Driven Architecture CIAO CORBA Component Middleware Patterns & Pattern Languages TAO CORBA Distribution Middleware ACE Host Infrastructure Middleware • Faculty: Dr. Douglas C. Schmidt • Research Scientist: Dr. Aniruddha Gokhale • Research Engineers: Bala Natarajan, Jeff Parsons, Boris Kolpakov, Tao Lu • Grad/UGrad Students: B. Krishnakumar, George Edwards, Emre Turkay, • Arvind Krishna, J. Balasubramaniam, Gan Geng

35. Downloading the Middleware & Tools • Beta and Stable release can be accessed from http://www.dre.vanderbilt.edu/Download.html • http://www.dre.vanderbilt.edu/cosmic

36. Additional Information • Patterns & frameworks for concurrent & networked objects • www.posa.uci.edu • www.ace.uci.edu • ACE & TAO open-source middleware • www.cs.wustl.edu/~schmidt/ACE.html • www.cs.wustl.edu/~schmidt/TAO.html • DRE research papers • www.cs.wustl.edu/~schmidt/research.html • DRE tutorials • www.cs.wustl.edu/~schmidt/tutorials.html • ACE books • www.cs.wustl.edu/~schmidt/ACE/