250 likes | 358 Views
RepoMan A Component Repository Manager for Enterprise DRE Systems. Stoyan Paunov & Douglas C. Schmidt Vanderbilt University Institute for Software Integrated Systems (ISIS). Context: Service – Oriented Architectures.
E N D
RepoManA Component Repository Manager for Enterprise DRE Systems Stoyan Paunov & Douglas C. Schmidt Vanderbilt University Institute for Software Integrated Systems (ISIS)
Context: Service–Oriented Architectures • Historically, distributed real-time & embedded (DRE) systems were built directly atop OS & protocols Applications Operating System & Communication Protocols Hardware Devices
Domain-Specific Services Service-Oriented Architecture Middleware Common Services Distribution Middleware Infrastructure Middleware Context: Service–Oriented Architectures • Historically, distributed real-time & embedded (DRE) systems were built directly atop OS & protocols Applications • Traditional methods of development have been replaced by middleware layers to reuse architectures & code for enterprise DRE systems • Viewed externally as Service-Oriented Architecture (SOA) Middleware Operating System & Communication Protocols Hardware Devices
Domain-Specific Services Multi-layer Resource Manager (MLRM) Common Services Distribution Middleware Infrastructure Middleware Context: Service–Oriented Architectures • Historically, distributed real-time & embedded (DRE) systems were built directly atop OS & protocols Applications • Traditional methods of development have been replaced by middleware layers to reuse architectures & code for enterprise DRE systems • Viewed externally as Service-Oriented Architecture (SOA) Middleware • e.g., DARPA Adaptive & Reflective Management System (ARMS) program’sMulti-layer Resource Manager (MLRM) • MLRM leverages standards-based SOA middleware to manage resources for shipboard computing environments Operating System & Communication Protocols Hardware Devices dtsn.darpa.mil/ixodarpatech/ixo_FeatureDetail.asp?id=6
ARMS Multi–Layer Resource Manager (MLRM) Applications Domain-Specific Services • ARMS MLRM architecture includes • Top domain layer containing components that interact with the ship mission manager • Middle resource pool layer is an abstraction for a set of computer nodes managed by a pool manager • Bottom resource layer manages the actual resource computing components, i.e., CPUs, memory, networks, etc. www.cs.wustl.edu/~schmidt/PDF/JSS-2006.pdf
Applications composed of loosely-coupled, course-grained distributed components Large number of deployment & configuration artifacts Need to shield deployment & configuration logic from heterogeneous hardware/software environments Continuous system evolution Enable the dynamic redeployment & reconfiguration of application & middleware components Domain-Specific Services Multi-layer Resource Manager (MLRM) Common Services Distribution Middleware Infrastructure Middleware Problem: New Complexities Applications Operating System & Communication Protocols Hardware Devices Need to provide the right implementation under the right circumstances
… … … Component A v1 Component B v1 Component B v2 Solution: Standardized Deployment & Configuration (D&C) • Keeps track of software implementation artifacts & their configuration metadata in heterogeneous environments • Need an administrative entity which • Facilitates online upgrades, reconfiguration, & redeployment of components RepoMan: an implementation of OMG CCM Repository Manager specification tailored to the needs of enterprise DRE systems
… … Component A v1 Component B v1 CORBA R E P O M A N Filesystem HTTP Overview of RepoMan • Organizes various configurations of component packages • Knows how to parse the XML metadata • Keeps track of component interfaces & their relationships • Facilitates the dynamic update of components at run-time Metadata XML Descriptors Implementation Artifacts bundled via ZIP compression have *.CPK extension
ADMIN Client Client Client Client CORBA Object HTTP Server … File System Component RepoMan Design Challenges Efficiency Concurrency Synchronization Consistency Portability • The CCM specification defines the interface, but it does not prescribe any design & implementation details • We were faced with a number of challenges which we discuss next
File System The Triangle of Mystery CORBA Object HTTP Server Challenge 1: RepoMan & HTTP Server Architecture • Context • The RepoMan architecture consists of a CORBA object & an HTTP Server • Need to implement the CORBA interface specified by the CCM Specification • Need to allow retrieval of implementation artifacts via HTTP • Problem • Shared memory • Lack of portability • Inflexible • Extend the CORBA server & client • Limited to CORBA • Unnecessary effort
ADMIN I,C,Q,D Clients R LEGEND I = install C = configure Q = query R = retrieve D = delete CORBA Object HTTP Server File System Solution: Loose Coupling Between CORBA Object & HTTP Server • The CORBA object supports the standard set of operations enabling • manipulation & querying of data in repository • retrieval & updating of configuration information • A collocated HTTP server enables retrieval of implementation artifacts • No explicit knowledge of each other • Do not share internal state • Share a common filesystem • JAWS vs. Apache http://httpd.apache.org/ http://www.dre.vanderbilt.edu/JAWS/
Challenge 2: PackageConfiguration Complexity • Context • PackageConfiguration – one of the most complex elements in CCM • Location of implementation artifacts not known at package creation • Problem • Monolithic components • 8 levels of hierarchy • Assembly-based components • 11 levels of hierarchy • Programming nightmare • Need to update the locations of the implementation artifacts in the PackageConfiguration
Structure Solution: Use Visitor pattern to Cope with Accidental complexity of PackageConfiguration Intent centralize operations on an object structure so that they can vary independently but still behave polymorphically Applicability • class relationships of objects in the structure rarely change, but the operations on them change often • algorithms keep state that’s updated during traversal Logic for handling each element has a well defined scope
Read Package Store Local uncompress Read descriptors Populate Challenge 3: Cost of Data Movement & XML Parsing HTTP Retrieve Un-ZIP • Context • Manipulating contents of component packages • Problem • Data movement costs: • Uncompressing • Writing to disk • Parsing with XERCES-C • Storing in equivalent C++ classes • XML parsing • PackageConfiguration parsing involves on the order of tens of files & is therefore very slow Disk Parse XML C++ data Structure
*.EPC • Extract the package contents only once at installation • Wasteful on hard disk usage, but avoiding a bulk of CPU & I/O processing myPackage.cpk *.cpk */descriptors/* */implementations/* • Externalizing via CORBA CDR to minimize XML parsing • Leveraging the cost of externalizing over lifetime of PackageConfiguration • Humanly visible speed-up of metadata retrieval operations PackageConfiguration Solution: Minimizing Data Movement & XML Parsing to Improve CPU & I/O Usage File System CORBA Object Install
… … Component Component Challenge 4: Organizing & Managing Data Efficiently Install Delete • Context • Need to store local copies of packages • Problem • Ensure consistency • Access, traversal & clean-up • No standard filesystem API • Need for portability precludes a tightly coupled approach • Third-party libraries place limitations on the user community File System
myPackage.cpk Solution: Use Package Name & Structure • Use the fact that installation name uniqueness in enforces to device the local directory structure • Mimic the package structure on the local filesystem • Use layout of package to guide you through the clean-up process Install File System myPackage.cpk myPackage.epc myPackage/descriptors/* myPackage/implementations/*
Challenge 5: Scalability & Synchronization Client CORBA Object • Context • Need to provide high scalability & efficiency • Need to preserve consistency • Concurrency & synchronization • Problems • Concurrency strategy choice • Single-threaded model • Thread-per-request model • Can exhaust system resources & degrade performance • Thread pool • Synchronization inherently incurs overhead Thread-per-Request Client CORBA Object Thread Pool
Solution: Use a Synchronized Variable-size Threadpool to Handle Incoming Requests • Scalability • Thread pool implemented with the Leader-Followers pattern to handle requests • Hash maps to keep internal state O(1) best case access • Synchronization • Synchronize only the hash maps, not the entire functions • Much more efficient than the Monitor pattern
ADMIN Client Client Client Client CORBA Object HTTP Server … File System Component Lessons Learned • Building enterprise DRE systems requires a component repository to keep track of software implementations & their configuration metadata • Thereby enabling the automated (re)deployment & (re)configuration of the system • Applying patterns helps ensure good design • Patterns applied to RepoMan included • Visitor, Null Object, Leader-Followers, & Memento in the CORBA object • Strategy, Bridge, Service Configurator, Iterator, & Singleton in HTTP server • Amortizing cost over the lifetime helps improving performance
ADMIN Client Client Client Client CORBA Object HTTP Server … File System Component Concluding Remarks • RepoMan strikes an effective balance between flexibility and efficiency. It keeps client code simple and supports system (re)deployment and (re)configuration and dynamic updates. • RepoMan is available as part of the Component Integrated ACE ORB (CIAO) at www.dre.vanderbilt.edu/CIAO/ • For more information on JAWS check: http://www.dre.vanderbilt.edu/JAWS/
Installation installPackage () createPackage () Deletion deletePackage () Retrieval findPackageByName () findPackageByUUID () Querying getAllNames () getAllTypes () findNamesByType () Retrieving implementations Package • Validate Package content • Install IA • Associate package w/ IN • Associate CI w/ IN CORBA Object HTTP Server HDD HTTP PC = PackageConfiguration IA = Implementation Artifact CI = Component Interface IN = installation name Functionality of RepoMan Clients
Installation installPackage () createPackage () Deletion deletePackage () Retrieval findPackageByName () findPackageByUUID () Querying getAllNames () getAllTypes () findNamesByType () Retrieving implementations Find IN Return PC CORBA Object HTTP Server HDD PC = PackageConfiguration IA = Implementation Artifact CI = Component Interface IN = installation name Functionality of RepoMan Clients
Installation installPackage () createPackage () Deletion deletePackage () Retrieval findPackageByName () findPackageByUUID () Querying getAllNames () getAllTypes () findNamesByType () Retrieving implementations IA.dll http://.../IA.dll CORBA Object HTTP Server HDD PC = PackageConfiguration IA = Implementation Artifact CI = Component Interface IN = installation name Functionality of RepoMan Clients