Arvind S. Krishna arvindk@dre.vanderbilt Institute for Software Integrated Systems

OPTEML: Optimization Techniques for Enhancing Middleware Quality of Service for Product-line Architectures http://www.dre.vanderbilt.edu/~arvindk/proposal.pdf Arvind S. Krishna arvindk@dre.vanderbilt.edu Institute for Software Integrated Systems Vanderbilt University Nashville, Tennessee

Air Frame GPS AP HUD Nav Nav Nav Nav HUD HUD HUD Air Frame AP AP AP FLIR FLIR Air Frame IFF GPS IFF IFF IFF Cyclic Exec Cyclic Exec Cyclic Exec Cyclic Exec UCAV F/A-18 A/V-8B F-15 Consequence: Small HW/SW changes have big (negative) impact on DRE system QoS & maintenance Motivation Air Frame FLIR FLIR GPS GPS • Legacy distributed real-time & embedded (DRE) systems have historically been: • Stovepiped • Proprietary • Brittle & non-adaptive • Expensive • Vulnerable

F/A 18 product variant A/V 8-B product variant UCAV product variant F-15 product variant Product-line architecture Distribution Middleware Host Infrastructure Middleware OS & Network Protocols Hardware (CPU, Memory, I/O) Motivation Air Frame FLIR AP HUD GPS Nav IFF Domain-specific Services Common Middleware Services • Middleware factors out many reusable general-purpose & domain-specific services from traditional DRE application responsibility • Essential for product-line architectures (PLAs)

F/A 18 product variant A/V 8-B product variant UCAV product variant F-15 product variant Product-line architecture Distribution Middleware Host Infrastructure Middleware OS & Network Protocols Hardware (CPU, Memory, I/O) Motivation Air Frame FLIR AP HUD GPS Nav IFF Domain-specific Services Common Middleware Services • Middleware factors out many reusable general-purpose & domain-specific services from traditional DRE application responsibility • Essential for product-line architectures (PLAs) • However, standards-based, general-purpose, layered middleware is not yet adequate for the most demanding & mission-critical PLA based DRE systems

F/A 18 product variant A/V 8-B product variant UCAV product variant F-15 product variant Product-line architecture Customized Middleware OS & Network Protocols Hardware (CPU, Memory, I/O) Motivation Air Frame FLIR AP HUD GPS Nav IFF • Middleware factors out many reusable general-purpose & domain-specific services from traditional DRE application responsibility • Essential for product-line architectures (PLAs) • However, standards-based, general-purpose, layers middleware is not yet adequate for the most demanding & mission-critical PLA based DRE systems My research optimizes middleware for PLA-based DRE systems

Presentation Road Map • Technology Context • Research Challenges • Related Research • Research Progress • Proposed Research  Middleware Specialization Techniques • Dissertation Timeline • Concluding Remarks

Domain-specific Services Common Middleware Services Distribution Middleware Host Infrastructure Middleware OS & Network Protocols Overview of Product-line Architectures (PLAs) PLA Characteristics: Scope Commonalities & Variabilities (SCV) • SCV analysis is a process that can be applied to identify commonalities & variabilities in a domain to guide development of a PLA [Coplien] Applying SCV to Bold Stroke • Scope: Bold Stroke component architecture, object-oriented framework, & associated components, e.g., GPS, Airframe, & Display Reusable Application Components Reusable Architecture Framework Air Frame FLIR AP HUD GPS Nav IFF James Coplien et al. Commonality & Variability in Software Engineering, IEEE Software 1998

Distribution Middleware Common Middleware Services Domain-specific Services Host Infrastructure Middleware OS & Network Protocols Hardware (CPU, Memory, I/O) Overview of PLAs Applying SCV to Bold Stroke • Commonalities describe the attributes that are common across all members of the family • Common object-oriented framework & set of components • e.g., GPS, Airframe, Navigation Display components • Common middleware infrastructure • e.g., Real-time CORBA & a variant of Lightweight CORBA Component Model (CCM) called Prism

Distribution Middleware Common Middleware Services Domain-specific Services Host Infrastructure Middleware OS & Network Protocols Hardware (CPU, Memory, I/O) Overview of PLAs Applying SCV to Bold Stroke • Variabilities describe the attributes unique to the different members of the family • Product-dependent component implementations (GPS/INS) • Product-dependent component connections • Product-dependent components (Different weapons systems for security concerns) • Different hardware, OS, & network/bus configurations

Component Repository Comp Comp Comp Comp Comp Comp Comp Comp Comp Comp Comp Comp Comp Comp Comp Comp Comp Comp Comp Deployment Platforms Mature PLA Development Process • PLAs define a framework of components that adhere to a common architectural style • Model driven development (MDD) & domain-specific modeling languages (DSMLs) are used to: • Glue components together • Synthesize artifacts for deployment onto platforms, e.g., J2EE, .NET, & CORBA

www.dre.vanderbilt.edu/ACE Middleware Layers for PLAs Middleware Evolution • Middleware supports application requirements in different domains • Often exhibits a standards-based, general-purpose, & layered architecture • Host infrastructure middleware encapsulates native OS mechanisms to create reusable network programming components • Example: ACE toolkit & JVMs • Examples of QoS variabilities • OS priorities, which differ on Sun, VxWorks, Windows, Linux/RT-Linux • Diverse locks & IPC mechanisms

Middleware Layers for PLAs • Distribution middleware defines higher-level distributed programming models whose reusable APIs & components automate & extend native OS capabilities • Avoids hard-coding dependencies on OS, languages, & location • Examples: Real-time CORBA, COM+, Distributed Real-time Java, Java RMI, www.dre.vanderbilt.edu/TAO • Examples of QoS variabilities • Round-trip timeouts, protocol properties, (de)marshaling strategies, concurrency models • # of threads, stack sizes

www.dre.vanderbilt.edu/CIAO Middleware Layers for PLAs • Component middleware is a class of middleware that enables reusable services to be composed, configured, & installed to create applications rapidly & robustly • Components encapsulate application “business” logic • Containers provide execution environment for components • Interact with underlying middleware to leverage middleware services, such as events, fault-tolerance etc • Examples of QoS variabilities • Type of containers (e.g., real-time, transient, persistent) • Services QoS properties (e.g., event channel vs. direct dispatch)

Presentation Road Map • Technology Context • Research Challenges • Related Research • Research Progress • Proposed Work  Middleware Specialization Techniques • Dissertation Timeline • Concluding Remarks

Research Challenges Overview of Research Challenges • Resolving the tension between • Generality Middlewareis designed to be independent of particular application requirements • Specificity PLAs are driven by the functional & QoS requirements for each product variant Product-line Variant Standards-based, General-purpose, Layered Middleware Architecture Customized Middleware Stack

Asynchrony Research Challenges Addressed in OPTEML Middleware fine-grain componentization challenges Configuration evaluation & validation challenges Specialization challenges

Taxonomy of Research Continuum • Optimization dimensions • Applicability General-purpose to more application-specific • A general-purpose optimization can be applied across all PLA variants, while application-specific targets a particular variant • Binding time  Run-time to design-time • Determines when the optimizations are bound, i.e., run-time  systemexecution, design-time  system development Design-time Binding Time Prior research has focused on a continuum of optimizations Run-time General Specific Applicability

General-purpose Optimizations • General-purpose optimizations are broadly applicable algorithmic & data-structural optimizations within the middleware/OS/protocols to improve QoS Design-Time • Applicability Can be used for a range of applications across different domains • Binding time Design-time & run-time Binding Time Run-time General Specific Applicability

Related Research Schmidt et al., “Pattern Oriented Software Architecture 2”, Addison Wesley http://www.dre.vanderbilt.edu/~arvindk/proposal.pdf

General-purpose Optimizations: What is Missing? Unresolved challenges • Different product-lines achieve different benefits/liabilities from enabling/disabling different optimizations • e.g. type of ORB reactor, type of queue, type of connection management strategy etc. • How to systematically determine what optimizations are ideal for a particular deployment & product variant Need to determine right hooks Solution: Configuration-driven Optimizations

Configuration-driven Optimizations Configuration-driven optimizations tune compiler/middleware/web-server configuration knobs to maximize application QoS for a particular use case Hook for the concurrency strategy Hook for the event demuxing strategy Design-time Hook for marshaling strategy • Applicability  Technique is broadly applicable, but a particular configuration may be specific to a particular application use case • Binding time  Typically bound at system initialization-time Binding Time Hook for the connection management strategy Hook for the underlying transport strategy Run-time General Specific Applicability

Related Research

Configuration-driven Optimization: What is Missing? Unresolved challenges • These optimizations cannot eliminate middleware generality • Overhead from specification compliance, redundant checks, & indirection • e.g., benchmarking efforts revealed that even with most optimal configuration, indirection overheads are costly • How to optimize the request processing path to improve performance http://www.atl.external.lmco.com/projects/QoS/ • Krishna et al. “CCMPerf: A benchmarking tool for CORBA Component Model Implementations”, RTAS 2004, Toronto Canada • Krishna et al. “Empirical Evaluation of CORBA Component Model Implementations”, Springer-Verlag Journal on Real-time Systems, March 2005 Solution: Partial Specialization Optimizations

Partial Specialization Optimizations Partial Specialization optimizations customize programming-languages/middleware according to system invariants known ahead-of-time • Applicability Optimizations are highly context-specific • Binding time Optimizations are generally applied at design-time Design-time Binding Time Run-time General Specific Applicability

Related Research

Lack of Tool Support Automatic specialization generally restricted to declarative languages, such as lambda Little support for OO languages such as C++/Java + Partial Specialization: What is Missing? Lack of Middleware approaches • Middleware is designed for generality, these techniques hold promise when applied to middleware • No approaches address middleware-related issues Lack of approaches that deal with optimizations • AspectJ (Resolving Feature Convolution, Zhang et al.) did not focus on improving performance using optimizations • Aspects are a delivery mechanism, the woven code can be suboptimal! • Aspects also add run-time overhead (need to link to library) create portability issues Provides no guarantees on woven code

Asynchrony Challenge #1: Fine-grain Componentization Middleware fine-grain componentization challenges

Synchronous Asynchrony Middleware Feature Componentization Problem Context Each variant in a PLA may require only a subset of all the middleware features • Research Challenges • General-purpose optimizations address performance issues, but do not address feature-driven componentization • Monolithic ORB architectures put all code in one executable • Conditional compilation allows fine-grained componentization but • Increases number of configuration settings • Hard to maintain & add new features

Addressing Middleware Customizability Challenges for PLAs • Hypotheses: Our approach • Enables different levels of customization • Is application transparent • Significantly reduces footprint • Allows easy addition of new features without undue performance overheads • Solution Approach  Micro ORB Architectures • Identify ORB services whose behavior may vary, e.g., • Object Adapters, Anys, Protocols, (de)marshaling, & buffer allocation mechanisms • Move these out of the ORB

Addressing PLA Customizability Challenges • Solution Approach  Micro ORB Architectures • Identify ORB services whose behavior may vary, e.g., • Object Adapters, Anys, Protocols, (de)marshaling & buffer allocation mechanisms • Move these out of the ORB • Implement alternatives as pluggable strategies, i.e.,virtual components [VC] [VC] Corsaro et al. “Virtual Component Pattern,” Pattern Language of Programs”, Monticello, Illinois, 2003

Addressing PLA Customizability Challenges • Solution Approach  Micro ORB Architectures • Identify ORB services whose behavior may vary: • Object Adapters, Anys, Protocols, (de)marshaling & buffer allocation mechanisms • Move these out of the ORB • Implement alternatives as pluggable strategies, i.e., virtual components [VC] • Each product variant can then choose what component it does or does not require • Application Transparent • No changes to CORBA interfaces • No changes to existing implementations

Research Contributions Research Contributions • Levels of customizability coarse-grain vs. fine-grain componentization • Fine-grain approach break component into multiple sub-components • Policy driven approaches to fine-grain componentization • Footprint Fine-grain approach has 50% reduction compared with monolithic approach [DOA 02, 03] • Cost Adding new features for variants modularized (e.g., new protocol) • Performance enables all general purpose-optimizations [RTAS 03] & real-time properties[ICDCS04] along critical path

Challenge #2: Configuration Evaluation & Validation Configuration evaluation & validation challenges

Ad hoc Techniques for Configuration Evaluation Context Each variant in a PLA requires an appropriate set of middleware configurations to satisfy QoS properties

Ad hoc Techniques for Configuration Evaluation/Validation Problem • Configuration-driven optimizations describe the “ends” (mapping requirements to parameters) However, the “means” (process for the mapping) are tedious, error prone & time-consuming for middleware • e.g., for a simple 5 component scenario requires ~ 60 files each ~ 100-500 LOC

Ad hoc Techniques for Configuration Evaluation/Validation Problem • No systematic process to evaluate & validate configuration settings across different platforms • Configuration Evaluation • e.g., how do we ensure the right middleware configuration for maximizing QoS for product-variants? • Configuration Validation • e.g., how do we ensure that configuration is semantically valid across different deployment scenarios?

Addressing Ad hoc Techniques for Evaluation & Validation • Hypotheses: Our approach for different Product-variants • Eliminates accidental complexity in evaluating QoS of middleware configurations • Enables validation of middleware configurations across diverse platforms • Can be applied to identify middleware configurations that most influence end-to-end performance/latency/jitter (i.e., the “main effects”) Solution Approach: Combine MDD Approach • Raise the level of abstraction, i.e., think in terms of middleware configurations rather than lower-level source code • Auto-generate information required to run & evaluate QoS of variants with Quality Assurance Processes • Validate generated code across different platforms

Build & Benchmark Generation DSML BGML • Developed a domain specific modeling language (DSML) in GME to evaluate middleware configurations to maximize application QoS

BGML Tool Features Challenge 1 : How to capture different PLA communication semantics? One-way synchronous/ asynchronous BGML Challenge Resolution • Provide modeling constructs to depict one-way/two-way invocation semantics • Modeling construct to depict events Two way synchronous communication • Interpreters generate platform specific code • Eliminate accidental complexities in understanding IDL-C++ mapping (void) this->remote_ref_-> AcceptWorkOrderResponse (arg0, arg1);

BGML Tool Features Challenge 2 : How to capture different PLA QoS characteristics Latency between ab < x msecs Throughput should be more than y calls/sec BGML Challenge Resolution • Provide modeling constructs to capture latency/throughput and jitter QoS characteristics ACE_Sample_History history (5000); for (i = 0; i < 5000; i++) { ACE_hrtime_t start = ACE_OS::gethrtime (); (void) this->remote_ref_-> AcceptWorkOrderResponse (arg0, arg1); ACE_CHECK; ACE_hrtime_t now = ACE_OS::gethrtime (); history.sample (now - start); } } • Automatic translation into code that samples data; calculates and computes these metrics

BGML Tool Features Challenge 3 : How to capture PLA workloads, e.g., rate-based? Operations at 20Hz/sec Operations at 40 Hz/sec BGML Challenge Resolution • Tasks: Threads that run at given/rate or continuous or are random (interactive load) • TaskSet: Group tasks into sets having a given rate/priority ACE_Barrier barrier (2); // Generate the Background workload AcceptWorkOrderResponse_Workload<T> task0 (this->remote_ref_, arg0_, arg1_, barrier); AcceptWorkOrderResponse_Workload<T> task1 (this->remote_ref_, arg0_, arg1_, barrier); AcceptWorkOrderResponse_Workload<T> task2 (this->remote_ref_, arg0_, arg1_, barrier);

CoSMIC Tool Suite Challenge 5: How to model & generate middleware configurations settings for PLA variants? BGML integrated with the Options Configuration Modeling Language (OCML) Option selection Documentation Pane

CoSMIC Tool Suite Challenge 6: How generate deployment information? BGML integrated with Platform Independent Component Modeling Language Mapping Virtual nodes • This MDD process automates ALL code required to enact a scenario • Deployment Plan – XML deployment information (PICML) • svc.conf – Configuration for each component implementation (OCML) • Benchmark code – source code for executing benchmarks (BGML) • IDL & CIDL files (PICML/BGML) • Build Files – MPC files (www.ociweb.com) (BGML)

MDD Quality Assurance Process Evaluating and Validating Middleware Configurations • Process output integrated with Skoll (www.cs.umd.edu/projects/skoll) • Skoll QA processes validates configurations across different platforms, compilers & OS A: Use MDD process explained earlier to capture PLA QoS concerns B, C: Synthesize middleware configuration, benchmark and deployment information D: Feed test code to Skoll framework E: Run on varied platforms to measure QoS variations F: Maintain database of results from which patterns of configurations can be identified Process time consuming. Do I need to run this process each time?

Main-effects Screening Process Motivation • Identify the important configurations settings that characterize complete configuration space Process Steps • Use MDD process to generate artifacts • Enhance Skoll to use a statistical techniques (Design of Experiments theory) to estimate & validate configuration importance • e.g., G is most important parameter for the scenario followed by H • Enables defining a region for required performance • Penalty for leaving the region X msecs

Research Contributions Model-Driven QA processes for configuration evaluation & validation • Enhances Configuration-driven optimization techniques by providing a reusable, automated process reproducible for different variants [RTAS05] • Addresses accidental complexities in code-generation [ICSR] • Addresses validation challenges across different hosts/configurations [IEEE Software] • Documents configuration main-effects that can drive customizations or further validation [ICSE 2005]

Challenge #3: Middleware Specialization Specialization challenges

Arvind S. Krishna arvindk@dre.vanderbilt Institute for Software Integrated Systems