Context Specific Middleware Specialization Techniques for Optimizing

Context Specific Middleware Specialization Techniques for Optimizing Product-line Architectures http://www.dre.vanderbilt.edu/~arvindk/MW-Spl.pdf Arvind S. Krishna arvindk@dre.vanderbilt.edu Institute for Software Integrated Systems Vanderbilt University Nashville, Tennessee

Presentation Road Map • Technology Context • Research Challenges • Middleware Specialization Techniques • Feature Oriented Customizer (FOCUS) for Middleware • Applying FOCUS to PLAs & Middleware – A Case Study • Concluding Remarks & Future Work

Air Frame GPS AP HUD Nav Nav Nav Nav HUD HUD HUD Air Frame AP AP AP FLIR FLIR Air Frame IFF GPS IFF IFF IFF Cyclic Exec Cyclic Exec Cyclic Exec Cyclic Exec UCAV F/A-18 A/V-8B F-15 Consequence: Small HW/SW changes have big (negative) impact on DRE system QoS & maintenance Motivation Air Frame FLIR FLIR GPS GPS • Legacy distributed real-time & embedded (DRE) systems have historically been: • Stovepiped • Proprietary • Brittle & non-adaptive • Expensive • Vulnerable

F/A 18 product variant A/V 8-B product variant UCAV product variant F-15 product variant Product-line architecture Distribution Middleware Host Infrastructure Middleware OS & Network Protocols Hardware (CPU, Memory, I/O) Motivation Air Frame FLIR AP HUD GPS Nav IFF Domain-specific Services Common Middleware Services • Middleware factors out many reusable general-purpose & domain-specific services from traditional DRE application responsibility • Essential for product-line architectures (PLAs)

F/A 18 product variant A/V 8-B product variant UCAV product variant F-15 product variant Product-line architecture Distribution Middleware Host Infrastructure Middleware OS & Network Protocols Hardware (CPU, Memory, I/O) Motivation Air Frame FLIR AP HUD GPS Nav IFF Domain-specific Services Common Middleware Services • Middleware factors out many reusable general-purpose & domain-specific services from traditional DRE application responsibility • Essential for product-line architectures (PLAs) • However, standards-based, general-purpose, layered middleware is not yet adequate for the most demanding & mission-critical PLA based DRE systems

F/A 18 product variant A/V 8-B product variant UCAV product variant F-15 product variant Product-line architecture Customized Middleware OS & Network Protocols Hardware (CPU, Memory, I/O) Motivation Air Frame FLIR AP HUD GPS Nav IFF • Middleware factors out many reusable general-purpose & domain-specific services from traditional DRE application responsibility • Essential for product-line architectures (PLAs) • However, standards-based, general-purpose, layers middleware is not yet adequate for the most demanding & mission-critical PLA based DRE systems My research optimizes middleware for PLA-based DRE systems

Research Challenges Overview of Research Challenges • Resolving the tension between • Generality Middlewareis designed to be independent of particular application requirements • Specificity PLAs are driven by the functional & QoS requirements for each product variant Product-line Variant Standards-based, General-purpose, Layered Middleware Architecture Customized Middleware Stack

Overview of Research Challenges Middleware Functionality Specialization • Specifications impose certain middleware functionality • Need to work on machines with different endianess; Not required for homogenous PLAs • How can specification imposed generality be specialized to improve QoS? Specialization Automation • Mature middleware implementations for PLAs, e.g., ACE+TAO (www.dre.vanderbilt.edu) have over million lines of code; hand crafting specializations for production middleware not feasible Framework Specialization • Middleware architecture define “hooks” • Plug protocol implementations; PLAs however require one alternative • How can middleware frameworks be specialized to remove hooks? Deployment Specialization • Middleware implementations run on different hardware/OS/compiler platforms; PLAs are deployed on specific platforms • How can we leverage platform characteristics to improve QoS?

Overview of FOCUS Middleware Instrumentation Phase • Middleware developers • Capture specialization transformations via FOCUS specialization language (FSL) • Annotate middleware source code with specialization directives • Create a domain-specific language (DSL) to capture middleware component variability Middleware Specialization Phase • PLA application developer • Analyses & determines the type of specializations applicable • FOCUS transformation engine selects the appropriate transformations & uses the annotations to automate specializations FOCUS addresses specialization challenges by building specialization language, tool, & process to capture and automate middleware specializations

FOCUS Motivation • FOCUS design motivated by template code-generation approaches Jostraca (www.jostraca.org) and Java Emitter Templates (www.eclipse.org) • Both have code generation directives in the form of meta-data, e.g. <%@ jet package="hello" class="GreetingTemplate" %> Hello, World • FOCUS approach augments template-based approaches by: • Enhancing directives: Enables placing meta-data within code directly • Supporting Code Evolution: FOCUS allows specializations and code to evolve in parallel minimizing skew • Minimizing accidental complexity:FOCUS allows ease of specialization specification via XML, does not require understanding of complicated specialization language syntax • Transformational approaches: Stratego-XT (www.program-transformations.org/stratego), DMS (www.semdesigns.com) support a broader range of specializations to be specified in abstract and concrete language syntax • The goal of this work isn’t on specialization languages per se, but instead to quantify the improvements that come from applying various types of specializations to middleware

FOCUS Goals Shield middleware developers from tool maturity issues Extensible, easy to use; minimize accidental complexities in developing specializations Specialized code should not incur performance/foot-print overhead due to the specialization Toolkit based on COTS components and technologies Should not be bothered about actual specialization details Work across different middleware implementations in Java, C++, etc Support middleware specialization with evolution Generated code should work across different compilers, platforms, etc. Should not rely on external libraries

C++ or Java compiler FOCUS Dynamics • PLA application developer selects the features to be specialized • Perl-based FOCUS transformation engine selects the appropriate specialization file from the repository & executes the specialization Feature Selection FOCUS Customization Engine: Transforms annotations Source-to-Source Transformations

Example System Basic Single Processor (BasicSP) –distributed real-time & embedded (DRE) application scenario based on Boeing Bold Stroke Timer Component – triggers periodic refresh rates GPS Component – generates periodic position updates Airframe Component – processes input from the GPS component & feeds to Navigation display Navigation Display – displays GPS position updates Bold Stroke PLA Scenario Representative DRE application: “rate based” Events  Control information Operation  simple data CoSMIC/examples/BasicSP ACE_wrappers/TAO/CIAO/DaNCE/examples/BasicSP

Identifying Ahead of Time Known System Invariance Specification Invariance Single method interfaces: Sends same operation on wire 2 Framework Invariance Deployment Platform Invariance Protocol: A specific protocol used Do not support native exceptions 3 1 A specific Reactor used

Framework Specialization Reactor_Impl select () Select Reactor WFMO_Reactor Thread_Pool Reactor select () select () select () The resolution of this challenge illustrates how middleware frameworks can be specialized Invariance • Middleware components such as Reactor, protocol, components remain same for a PLA scenario Reactor Pattern • Reactor pattern separates the event detection from the demultiplexing • Allows a single-thread to do both activities • Multiple Reactor implementations; need only one! Reactor Specialization • Remove indirection, e.g., Reactor_Impl base class completely (all virtual methods concrete) • No changes to component interface. Does not break compatibility This specialization is an example of Aspect Weaving specialization, where crosscutting middleware features are customized

Capturing Specialization in FSL (1/2) • FOCUS uses an XML DTD to create a DSL for capturing specializations • Use Reactor specialization as an example to bring out FSL language features • Tags used to represent different transformations • <substitute>  Substitution capability • <search> .. </search> <replace> … </replace> Capability to search & replace (supports regular expressions within tags) • <remove>  Remove code within the tag (supports regular expressions) <substitute> <search>ACE_Reactor_Impl</search> <replace>ACE_Select_Reactor_Impl</replace> </substitute> Removes indirection in the Reactor framework Once indirections is removed, virtualness can be eliminated <remove>virtual</remove> These tags do not require any annotations to the middleware source code

Capturing Specialization in FSL (2/2) • <add>  Add code based on a “hook” • <hook> Annotated point within middleware where code needs to be woven • <data> Code to be woven at that point • <comment-region>  Comment a region out between hooks • <start-hook> <end-hook> Annotations that define where to start & stop copy Bring in concrete Reactor declarations into the base Reactor component <add> <hook>REACTOR_SPL_INCLUDE_HOOK</hook> <data>#include "ace/Select_Reactor.h"</data> </add> Eliminate code that creates a “vanilla Reactor component” <comment> <start-hook> TAO_REACTOR_SPL_COMMENT_HOOK_START </start-hook> <end-hook> TAO_REACTOR_SPL_COMMENT_HOOK_END </end-hook> </comment> • Annotations help identify join points & relieve FOCUS from implementing a full fledged language parser • FOCUS is language agnostic, i.e., <data> can be written in any language

Instrumenting Middleware Implementation Middleware Annotations • Middleware developer annotates middleware with hooks • Hooks represented as “meta-comments,” i.e., opaque to compilers ACE_Reactor_Impl * TAO_Advanced_Resource_Factory::allocate_reactor_impl (void) const { ACE_Reactor_Impl *impl = 0; //@@ TAO_REACTOR_SPL_COMMENT_HOOK_START switch (this->reactor_type_) { ………… } //@@ TAO_REACTOR_SPL_COMMENT_HOOK_END } <?xml version='1.0'?> <transform> 11: <module="TAO/tao"> 12: <file name="advanced_resource.cpp"> 13: <comment> 14: <start-hook>TAO_REACTOR_SPL_COMMENT_HOOK_START </start-hook> 15: <end-hook>TAO_REACTOR_SPL_COMMENT_HOOK_END </end-hook> 16: </comment> 17: </file> 18: </module> 23: .... </transform>

FOCUS Specialization Phase Annotated middleware source Transformed middleware source //File: Reactor.h //@@ REACTOR_SPL_INCLUDE_HOOK class Reactor { public: virtual int run_reactor_event_loop (); // Other public methods .... private: ACE_Reactor_Impl *reactor_impl_; // ...... }; //File: Reactor.h //@@ REACTOR_SPL_INCLUDE_HOOK // Code woven by FOCUS: #include "ace/Select_Reactor.h" // END Code woven by FOCUS class Reactor { public: int run_reactor_event_loop (); // Other public methods .... private: // Code woven by FOCUS: ACE_Select_Reactor *reactor_impl_; // End Code woven by FOCUS // ...... }; #include concrete header file Removed virtual keyword Replaced ACE_Reactor_Impl with ACE_Select_Reactor • These specializations don’t affect framework “business logic”, just its structure • Existing framework is still available when developers need OO capabilities

Aspect Weaving Specialization Results • Average Measures: • Reactor 4.5% • Protocol: 5% Experiment • Client/server communicate by sending a simple long • End-to-end latency measures for: • General-purpose optimized TAO and • Reactor, Protocol frameworks specialized with FOCUS compared Jitter better; specialization does not compromise predictability Worst case measures improve by ~50 µsecs • Result Synopsis • Average latency measures improve by ~5% for both protocol and reactor specializations Specialization also applied to other frameworks in TAO: Messaging, Wait Strategies • Jitter improves though not considerably • 99% values closer to average and worst-case values are better indicating better predictability

Scenario Invariance Specialization Resolution of this challenge illustrates how specification imposed functionality can be specialized Invariance • The same operation is invoked on a given connection Normal layered path Specialize Middleware Paths • Create middleware fast paths based on different invariants Optimized Fast path processing • Request Processing Fast Paths • Normal layered path Uses general-purpose optimization for request lookup • Optimized fast path  Bypasses middleware layers to directly perform operation invocation This specializations is an example of Memoization + Layer-folding specialization, where a pre-computed result is cached and used to bypass middleware layers

Automating Dispatch Resolution Specialization Automating Specialization via FOCUS • On a per connection basis determine the target dispatcher; • For subsequent requests, directly use the dispatcher to process requests FSL’s add capability leveraged Normal path; the dispatch is cached in the process Consequences • Specialization compliant with CORBA • POA Current not maintained • Default_Servant and Servant_Manager POA policies not honored with this optimization • Rarely used with DRE applications <add> <hook>TAO_DISPATCH_OPT_ADD_HOOK</hook> <data> 1. if (__dispatch_resolved__) 2.{ 3: //First invocation normal path 4: __dispatch_resolved_ = 0; 5: } 6: else 7:{ // All invocations go through --- Optimized path 8: this->request_disaptcher__.dispatch (…) 9:} <data> </add> FOCUS enables specializations to be specified in the corresponding language syntax Directly use cached dispatcher

Results for Layer-folding Specialization Average end-to-end measures improved by ~16% Average path measures improved by ~40% Experiment • End-to-end latency measures for: • General-purpose optimized TAO with active demultiplexing and perfect hashing • Specialized TAO with layer folding specialization enabled Worst case path measure improved by ~20% Dispersion improves by a factor of ~1.5 for both cases Worst case end-to-end latency improved by ~14% • Path specialized latency measures • Path defined as code-path when a request is received until the upcall is dispatched on the skeleton Specialization applied at the server side; this optimization can be applied at the client side as well

Deployment Platform Specializations The resolution of this challenge illustrates how deployment generality can be resolved to improve QoS Invariance • Endianess of the platforms remains the same for the PLA scenario Deployment Platform Generality • Platform characteristics affect QoS • TANGO Timesys linux + gcc • ACE vxWorks + Greenhills compiler • Certain versions of vxWorks do not support exceptions Specialize Middleware for Deployment platform • Run benchmark tests and programs automatically using autoconf before ORB initialization to determine right hardware, OS and compiler characteristics This specialization is a variation of Constant Propagation where the constant (platform characteristic) is used to tailor the middleware

Automating Platform Specializations Deployment platform specialization • Exception support: • Autoconf’s AC_COMPILE_IF_ELSE macro to check if compiler supports native exception • Loop unrolling optimization • Autoconf’s AC_RUN_IF_ELSE macros to compile and run benchmark to quantify memcpy performance improvements Variable that will be checked to set a compilation flag ACE_CACHE_CHECK([if ACE memcpy needs loop unrolling], [ace_cv_memcpy_loop_unroll], [AC_RUN_IFELSE([AC_LANG_SOURCE([[ /// …. Program that will run the benchmark to //// determine if an optimization is better Directive to run the benchmark Provide the actual source language that will be compiled and run This approach can be applied to automatically discover OS specific system calls and compiler settings that maximize QoS

Results for Autoconf Specialization Average latency improves ~17% Deployment platform specialization • Exception support: • Loop unrolling optimization • During ORB configuration, autoconf used to use emulated exceptions and un-roll loops for memcpy Predictability improved over general purpose middleware Worst-case measures improve by ~50 µs Other types of optimization include sendfile optimization on Unix and Linux platforms to avoid copy between TCP and application buffers

Specialization Catalog • Client Side Specialization • Request Header Caching • Pre-creating Requests • Target Location • Avoid demarshaling checks • Server Side Specialization • Specialize Request Processing • Avoid aligning data types while marshaling • ORB Component Specialization • Multiple types of Reactor • Pluggable protocol framework www.dre.vanderbilt.edu/~arvindk/MW-Spl.pdf

Cumulative Specialization Results Average end-to-end measures improved by ~43% Worst-case measures improved by ~45% Jitter results twice as better as general-purpose optimized TAO End-to-end client side throughput improved by ~65%. Results are notable as base line used is already optimized general-purpose TAO implementation

FOCUS Strengths & Drawbacks Drawbacks • Does not provide full-fledged language parser, i.e., join points need to be identified via annotations or via regular expressions • Static compile-time specialization; Requires recompilation of source code • Aspect C++, DMS use this approach • The need to synchronize annotations with specialization files; Modification of source code requires change to specialization files • Ameliorated with QA techniques by having continuous builds; Limitation exists even with aspects • Correctness of transformations have to be validated externally; unlike AspectJ Strengths • FOCUS provides a lightweight, zero (run-time) overhead approach for customizing middleware • FOCUS is designed to work across different languages (Java, C++) • XML-based approach to capture rules • Easy language extension, ability to add new features easily • If/when C++ aspect technologies mature, can transform them into aspect rules via XSLT transforms • Execute transformations via scripting • Perl regular expression capability really mature for our transformation • Ease of automation – integrate with other quality assurance techniques FOCUS soon available in the latest ACE+TAO distribution in ACE_wrappers/bin directory. Temporary location: www.dre.vanderbilt.edu/~arvindk/FOCUS

Concluding Remarks (1/2) Implications on QoS • Specializations applied such that no dependencies/conflicts exists • As more specializations are developed it is necessary to document dependencies between specializations Implications on ORB maturity • Optimizations improve QoS over and above general-purpose optimizations • Other ORBs, e.g., JacORB, do not use such optimizations; performance gains in this case would be more • Preliminary results [Daugherty] on ZEN show that speed up was better compared to TAO TAO uses active demultiplexing and perfect hashing for O(1) demux time [Daugherty] Gary Daugherty, A Proposal for the Specialization of HA/DRE Systems, ACM PEPM 04

POA POA POA S3 S4 S5 S5 S1 S2 20 35 40 50 Concluding Remarks (2/2) Implications on Schedulability • Specializations help high priority tasks finish ahead of their time to complete • Tasks with priority 50 finish early, increasing time available for scheduling priorities with 35 and 10 • Specializations can benefit both hard real-time and soft and softer real-time tasks Specializations increase slack in the system Implications on Adaptability • Specializations do not consider any form of recovery if invariance assumptions fail • Adaptation requires loading general-purpose code, add checks along request processing path; increases jitter for DRE systems Implications on PLA Scenario • All our specializations improve path latencies considerably than end-to-end latency; More the specialized code path is traversed, greater is QoS improvement

Future Work • Middleware implementations accommodate variability by using patterns • Greater benefits can be accrued by applying specialization approaches to other middleware frameworks; locking, buffering and concurrency strategies in middleware. http://www.cs.wustl.edu/~schmidt/POSA/ • Future research will also examine applicability of specializations to Component based middleware implementations

Questions

FOCUS Instrumentation Phase (3/3) • Transformation preserves the hooks, enables multiple specializations to use the same hook • Minimizes clutter in middleware source code ACE_Reactor_Impl * TAO_Advanced_Resource_Factory::allocate_reactor_impl (void) const { ACE_Reactor_Impl *impl = 0; //@@ TAO_REACTOR_SPL_COMMENT_HOOK_START switch (this->reactor_type_) { ………… } //@@ TAO_REACTOR_SPL_COMMENT_HOOK_END } <add> <hook>TAO_REACTOR_SPL_COMMENT_HOOK_END</hook> <data> ACE_NEW_RETURN (impl, ACE_Select_REACTOR ((ACE_Sig_Handler*)0, (ACE_Timer_Queue*)0, 0, (ACE_Select_Reactor_Notify*)0, this->reactor_mask_signals_), 0); Weave code that creates Select_Reactor instead of a normal Reactor • Hook names need not have any meaning • Differs from Javadoc tag approach • A drawback of this approach is the need to synchronize tags with specialization files • However, same actor (i.e., middleware developer) maintains both entities

FOCUS Specialization Language (FSL) • <copy-from-source>  Copy code from a specific point to a specific point between different files • <start-hook>  hook from where to start copying <end-hook>  hook to stop copying <dest-hook>  destination where you want the code to be placed • Aids copying derived class specializations to base class <copy-from-source> <source>Select_Reactor.h</source> <copy-hook-start>ACE_REACTOR_SPL_COPY_HOOK_START </copy-hook-start> <copy-hook-end>ACE_REACTOR_SPL_COPY_HOOK_END </copy-hook-end> <dest-hook>ACE_REACTOR_SPL_METHODS_ADD_HOOK </dest-hook> </copy-from-source> Copies *additional* methods from the derived class onto base class • Processing of the specialization tags is done in the following order: <remove>, <substitute>, <comment>, <add>, & <copy-from-source> • Order can be configured to support different parsing mechanisms • Enables middleware developers to define & enforce an ordering of how the individual transformations will be carried out

FOCUS: Termination Criteria Termination Criteria (Hypothesis) • We will use TAO mature real-time ORB as our gold standard • Greatest benefits will be accrued for scenarios that can turn on all/most of the optimizations • Performance improvements estimated ~ 30 – 40 % improvement in performance • Turning on just one/two optimizations might improve performance by ~ 10 – 15 % improvement • TAO baseline • General-purpose optimized TAO ORB • Active demultiplexing and perfect hashing for O(1) request demultiplexing • Buffer caching and direct collocation optimization • Most optimized configuration for different ORB components • Experiment Setup • A single threaded client and server; simple long exchanged • Pentium III 850 Mhz processor, running Linux Timesys 2.4.7 kernel, 512 MB of main memory, TAO version 1.4.7 compiled with gcc 3.2.3 • Timers at the client and within ORB used to collect data

FOCUS Instrumentation Phase (1/3) Similar to listing, package; AOP defines class directive <?xml version='1.0'?> <transform> 1: <module name="ace"> 2: <file name="Select_Reactor_Base.h"> 3: <add> 4: <hook>REACTOR_SPL_INCLUDE_HOOK</hook> 5: <data>class ACE_Sig_Handler;</data> 6: <data>class ACE_Sig_Action;</data> 7: <data>class ACE_Sig_Set;</data> 8: </add> 9: </file> 10: </module> 11: <module="TAO/tao"> 12: <file name="advanced_resource.cpp"> 13: <comment> 14: <start-hook>TAO_REACTOR_SPL_COMMENT_HOOK_START </start-hook> 15: <end-hook>TAO_REACTOR_SPL_COMMENT_HOOK_END </end-hook> 16: </comment> 17: </file> 18: </module> 23: .... </transform> Specialization transformations • Middleware developers use FSL to define transformations • Portion of XML file for specializing TAO Reactor implementation • Lines 1 – 2: module, package, & file to specialize • Lines 3 – 8: add necessary forward declaration at a hook in middleware • Lines 13 – 16: comment out unspecialized code within the markers Mimics AOP code weaving at join-points. AOP uses regex to identify points AOP around () advice can be employed; incurs run-time overhead though • Annotations help identify join points & relieve FOCUS from implementing a full fledged language parser • FOCUS is language agnostic, i.e., <data> can be written in any language

Model-Driven Technologies Domain-Specific Modeling Languages Meta-data Meta-data Meta-data Deployment and Configuration information Challenge Overview (1/2) • Model Driven Development (MDD) approaches capture PLA characteristics • Functional: Type of protocol, concurrency strategy • QoS: end-to-end latencies, jitter, throughput • Middleware still general-purpose, i.e., provides code for supporting all applications • Ahead of time system information is not used to customize the middleware

Model-Driven Technologies Domain-Specific Modeling Languages Challenge Overview (2/2) Middleware Specialization Challenge • How can the middleware be tailored for the specific PLA architecture modeled? • How can middleware level optimizations identified and automated based on PLA system characteristics? Research addresses this specialization challenge via Domain Specific Language, tools and process for specializing middleware for PLA

Feature-Oriented CUStomizer (FOCUS) Goals Requirements for middleware specialization framework: • Generated code should work across different compilers, platforms, etc. • Should not rely on external libraries – Aspect J • Specialized code should not incur performance/foot-print overhead due to the specialization • AspectJ [Zhang] results revealed that performance suffered from overhead of aspect libraries • Shield from tool maturity issues • AspectC++ does not work with TAO; recently made to work with ACE • Toolkit based on COTS components and technologies that are also open-source • Work across different middleware implementations in Java, C++, etc. • Support specialization with evolution, i.e., open-source models FOCUS Goals: Build specialization language, tool, & process to automate middleware specializations across different platforms [Zhang] “Resolving Feature Convolution in Middleware”, OOPSLA Conference, Vancouver, 2004

Context Specific Middleware Specialization Techniques for Optimizing