Harness and H2O Alternative approaches to metacomputing

1. Harness and H2OAlternative approachesto metacomputing Distributed Computing Laboratory Emory University, Atlanta, USA http://www.mathcs.emory.edu/dcl T.Ampula, D. Drzewiecki, T. Janiak, D. Kurzyniec, G. Stuer, T. Wrzosek, V. Sunderam

2. Cooperatingusers App 1 App 2 Applications PVM FT-MPI Comp. Activeobjects ... Programming model Virtual layer DVM-enabling components Provider B Provider A Provider C The Harness II Project • Joint between Emory, UTK, and ORNL • Cooperative Fault Tolerant Distributed Computing • Programming framework: Fault tolerant MPI, lightweight components, service oriented • Flexible, lightweight, middleware • Hosting layer: H2O substrate • Stateless, lightweight

3. Providers Clients Network H2O Abstraction • Providers owning resources • They independently make them available over the network • Clients discover, locate, and utilizeresources • Resource sharing occurs between single provider and single client • Relationships may betailoredas appropriate • Including identity formats, resource allocation, compensation agreements • Clients can themselves be providers • Cascading pairwise relationships maybe formed

4. A Provider host <<create>> B Container Provider Lookup& use Deploy Client Traditional model A B Provider host Deploy <<create>> Container Provider,Client,or Reseller Provider Lookup& use Client Proposed model H2O Framework • Resources provided as services • Service = active software component exposing functionality of the resource • May represent „added value” • Run within a provider’s container (execution context) • May be deployed by any authorized party: provider, client, or third-party reseller • Decoupling • Providers/providers/clients

5. Registration and Discovery UDDI JNDI LDAP DNS GIS e-mail,phone, ... ... Publish Find ... Deploy Provider A nativecode A Deploy Client Provider Client Provider Client Provider Deploy A B A B B Reseller Developer LegacyApp Repository Repository A B A B C C Example usage scenarios • Resource = legacy application • Provider deploys the service • Provider stores the information about the service in a registry • Client discovers the service • Client accesses legacy application through the service • Resource = computational service • Reseller deploys software component into provider’s container • Reseller notifies the client about the offered computational service • Client utilizes the service • Resource = raw CPU power • Client gathers application components • Client deploys components into providers’ containers • Client executes distributed application utilizing providers’ CPU power

6. Pluglet Kernel Model and Implementation Interface StockQuote { double getStockQuote(); } Clients • H2O nomenclature • container = kernel • component = pluglet • Object-oriented model, Java-based prototype implementation • Pluglet = remotely accessible object • Must implement Pluglet interface, may implement Suspendible interface • Used by kernel to signal/trigger pluglet state changes Functionalinterfaces (e.g. StockQuote) Pluglet [Suspendible] Interface Pluglet { void init(ExecutionContext cxt); void start(); void stop(); void destroy(); } Interface Suspendible { void suspend(); void resume(); }

7. E A D B C F H2O kernel Interoperability – the RMIX layer • H2O built on top of RMIX communication substrate • Provides flexible p2p communication layer for H2O applications • Enable various message layer protocols within a single, provider-based framework library • Adopting common RMI semantics • Enable high performance and interoperability • Easy porting between protocols, dynamic protocol negotiation • Offer flexible communication model, but retain RMI simplicity • Asynchronous and one-way calls Java Web Services RPC clients H2O kernel SOAP clients ... RMIX RMIX Networking Networking RPC, IIOP, JRMP, SOAP, …

8. H2O -- GUI • Application to help H2O users manage kernels they use • load or kill a pluglet, suspend or resume • check state of a kernel/pluglet • start or shutdown a kernel

9. UDDIregistry WSDL WSDL WSDL H2O kernel Well-known host A .B ClientApplication C SOAPproxy RMIXhttpd Register Discover pluglet2wsdl Server host Client host SOAP/HTTP H2O Programming and API • Connection and authentication • (Provider instantiates kernel and publishes reference) • User obtains kernel reference and connects to it • Kernel authenticates the client(optionally client auths. kernel) • If successful, client obtains kernel context • Deploying services • Client (or TPR) may use kernel context to upload pluglets • Need to specify: location of binaries (URL path), class name, optionally: additional meta-information • Invoking services • Client invokes functional interface methods Web Services deployment example

10. Example PPE: MPI on Metasystems • Many scenarios: • Firewalls • Non-routable NW’s • Heterogeneous CoCs • Grid-enabled • MPI across firewalls • Replace all runtime connections by tunneling all MPI communication through H2O channels

11. a cluster FT-MPI job FTMPI job layer H2O - IMPI proxy H2O - IMPI proxy H2O proxy layer FT NameServer FT Notifier IMPI server FT-MPI/H2O/IMPI a cluster FT-MPI job

12. Summary and Status • Harness and H2O • Lightweight, component oriented frameworks • Resource sharing across multiple administrative domains • Innovative concepts • Pluglet model for tailoring resource • Adds substantial flexibility to provider-client paradigm • Multiple parallel programming paradigms supported • RMIX layer supports interoperability • without compromising performance • Status • Alpha version available http://www.mathcs.emory.edu/dcl/