Towards Self-Organizing Distributed Computing Frameworks

1. Towards Self-Organizing Distributed Computing Frameworks Vaidy Sunderam Dawid Kurzyniec Tomasz Wrosek Dept. of Math & Computer ScienceEmory University, Atlanta, GA{vss,dawidk,yrd}@mathcs.emory.edu Research supported by NSF and US DoE

2. Outline • Background and context • Goals and motivations • H2O resource sharing model • Heterogeneous distributed computing • Multiple administrative domains • Support for multiple programming paradigms • Unified low level fabric • Summary and conclusions

3. Background and Context • The Harness project • Joint project with ORNL and UTK; follow on to PVM • Heterogeneous metacomputing platform • Facilitating dynamic, extensible, reconfigurable virtual machines • Component oriented; permits rapid replacement of technologies, models, etc • Status • Java-based framework for heterogeneous distributed computing • Support for multiple programming models: PVM, Javaspaces, FT-MPI • Performance compared to native platforms within –5% to +15% • Exploit usual advantages of component architectures • Next stage • Cooperative resource sharing across multiple administrative domains • The H2O project: Foundational substrate for distributed metacomputing

4. Goals and Motivations • Goals: enable efficient resource sharing • Lightweight • Minimal state • Decentralized and/or localized control • Self-organizing: global status implicitly defined by local conditions • Motivations • Middleware is cumbersome and expensive • Steep learning curve • Rigid and fragile due to tight coupling • Need for modularity, upgradeability, adaptability • Proposal • H2O client-provider-tpr architecture

5. H2O Abstraction • Providers own resources • CPU cycles, storage, information, applications, … • They independentlymake resourcesavailable over the network • specify access policies • Clients discover, locate, and utilizeresources • Resource sharing occurs between single provider and single client • Relationships may betailoredas appropriate • Including identity formats, resource allocation, compensation agreements • Aggregation and reselling • Resource aggregation is purely a client abstraction • Clients can be providers (tpr’s) => cascading pairwise relationships Providers Clients Network

6. Major challenges • Clients have different needs • Hide specifics from a provider • May want to just provide raw resources (CPU power, data storage, ...) • Clients (or third-party resellers) may want to „cook” resources before using/serving • Provider independence • Resource sharing is an individual choice – what, how much, when, to whom • No coordination/synchronization between providers • Security and resource management • Providers wish to keep control over shared resources • Clients require privacy, integrity and quality of service • Shallow learning curve • Interoperability • Exchange of data and connectivity with existing software • Web Services, CORBA, .NET, ...

7. A Provider host <<create>> B Container Provider Lookup& use Deploy Client Traditional model A B Provider host Deploy <<create>> Container Provider,Client,or Reseller Provider Lookup& use Client Proposed model Proposed Solution • Resources provided as services • Service = active software component exposing functionality of the resource • May represent „added value” • They run within a provider’s container (execution context) • They may be deployed by any authorized party • Provider, client, or third-party reseller • Different from traditional model • Decoupling • Providers from clients (to a large extent) • Providers from each other

8. Registration and Discovery UDDI JNDI LDAP DNS GIS e-mail,phone, ... ... Publish Find ... Deploy Provider A nativecode A Deploy Client Provider Client Provider Client Provider Deploy A B A B B Reseller Developer LegacyApp Repository Repository A B A B C C Example usage scenarios Resource = legacy application Provider deploys the service Provider stores the information about the service in a registry Client discovers the service Client accesses legacy application through the service Resource = computational service Reseller deploys software component into provider’s container Reseller notifies the client about the offered computational service Client utilizes the service Resource = raw CPU power Client gathers application components Client deploys components into providers’ containers Client executes distributed application utilizing providers’ CPU power

9. Pluglet Kernel Model and Implementation Interface StockQuote { double getStockQuote(); } • H2O nomenclature • container = kernel • component = pluglet • Object-oriented model, Java-based prototype implementation • Pluglet = remotely accessible object • Must implement Pluglet interface, may implement Suspendible interface • Used by kernel to signal/trigger pluglet state changes • In order to do something useful, components must be able to handle remote requests • Remote Method Invocation (RMI) model adapted • Pluglets export functional remote interfaces • Remote interface = set of remotely invokablemethods • Multiple inheritance of remote interfaces Clients Functionalinterfaces (e.g. StockQuote) Pluglet [Suspendible] Interface Pluglet { void init(ExecutionContext cxt); void start(); void stop(); void destroy(); } Interface Suspendible { void suspend(); void resume(); }

10. Pluglet Interoperability • Issues of remote method invocations • Serialization of sophisticated object graphs • Lack of shared address space (pass-by-copy semantics) • Dealing with remote references, distributed garbage collection • Handling remote failures • Which invocation protocol to use?... • JRMP, SOAP, IIOP, ... • Expressiveness vs Interoperability vs Performance • No ultimate choice; important to support multiple protocols • Sub-project: RMIX • Extended RMI framework for Java • Supports multiple, pluggable invocation protocols • Client can negotiate a protocol with a kernel • Allows for multiple, customizable object endpoints • Provides asynchronous invocation modes

11. Deployment of core functionality

12. Security and Resource Management • Providers and resellers must have fine-grained control over resources • Who can do what and when?... • Authentication • JAAS, Pluggable Authentication Modules (PAM) • Facilitating standards (X.509 certificates, Kerberos, NT, Unix authentication) • Allowing for customized or mixed solutions • Authorization + resource control • H2O exploits Java platform safety and security features • control over local filesystem, network adapters, thread usage • Policies for temporal resource control • Extensible XML based specification (by provider) of usage rules • Remote invocations on pluglets are guarded by security proxies • possible to restrict available interfaces and access rights based on invoker identity • Privacy and integrity • Leverage TSL/SSL layer

13. Login and Pluglet loading

14. Multiple programming paradigms • H2O = simple and lightweight, low-level resource sharing framework • Can serve as an engine for diverse programming paradigms • Metacomputing • PVM, MPI, RMI, Other (e.g. OGSA: H2O kernel == OGSA factory) • Client-server • Enterprise applications, Web Services • Task-Farm Computing • SETI@home, UD Cancer Research, ... • Peer-to-Peer • File sharing, messaging, opportunistic compute/storage/service sharing

15. Metacomputing • Clients aggregate resources • that may belong to them • or may come from independent providers • Clients upload components • DVM-enabling components communicate with each other and provide an illusion of Distributed Virtual Machine • Specific pluglets provide concreteprogramming environments • PVM and Fault Tolerant MPI components currently being adapted to H2O • Distributed state managed by DVM-enabling components alone • statelessness at the provider level • resources decoupled and independent Collaboratingresearchers(clients) App 1 App 2 Applications PVM FT-MPI Java RMI Activeobjects ... Programming model Virtual layer DVM-enabling components Provider B Provider A Provider C

16. UDDIregistry WSDL WSDL WSDL H2O kernel Well-known host A .B ClientApplication C SOAPproxy RMIXhttpd Register Discover pluglet2wsdl Server host Client host SOAP/HTTP Client-Server • Provider uploads pluglets into own kernel • Important case: Web Services • Enterprise apps, or high-performance metacomputing, or join multiple DVM’s/clusters • Client communicates with pluglet using SOAP • RMI over SOAP supported as a plugin toa multiprotocol RMIX suite • Published information about the servicehas a form of a WSDL document • H2O toolkit: pluglet2wsdl tool

17. H2O and Web Services • Web Service Description Language (WSDL) • Popular, XML-based, extensible, standardized • May be used for pure interface specification <definitions name=‘weatherservice’ xmlns=…> <service name=‘Weatherservice’> <message name=‘Weather.GetTempr’> <part name=‘zipcode’ type=‘xsd:string’> </message> <message name=‘Weather.GetTemprResponse’> <part name=‘Result’ type=‘xsd:float’> </message> <portType name=‘WeatherSoapPort’> <operation name=‘GetTempr’> <input message=‘wsdlns:Weather.GetTempr’> <output message=‘wsdlns:Weather.GetTemprResponse’> </operation> </portType> </service></definitions>

18. H2O and Web Services • Web Service Description Language (WSDL) • Popular, XML-based, extensible, standardized • May be used for pure interface specification • … or, to define specific remote binding <definitions name=‘weatherservice’ xmlns=…> <service name=‘Weatherservice’> <message name=‘Weather.GetTempr’> <part name=‘zipcode’ type=‘xsd:string’> </message> <message name=‘Weather.GetTemprResponse’> <part name=‘Result’ type=‘xsd:float’> </message> <portType name=‘WeatherSoapPort’> <operation name=‘GetTempr’> <input message=‘wsdlns:Weather.GetTempr’> <output message=‘wsdlns:Weather.GetTemprResponse’> </operation> </portType> <binding name=‘WeatherSoapBinding’ type=‘WeatherSoapPort’> <soap:binding style=‘rpc’ transport=‘…/soap/http’/> <operation name=‘GetTempr’> <soap:operation soapAction=‘…/Weather.GetTempr’/> <input> <soap:body use=‘encoded’ … /> </input> <output> <soap:body use=‘encoded’ … /> </output> </operation> </binding> </service></definitions>

19. H2O and Web Services • Web Service Description Language (WSDL) • Popular, XML-based, extensible, standardized • May be used for pure interface specification • … or, to define specific remote binding • … or, to describe a specific service instance • But … • WS not well matched to HPDC • Deployment issue: dynamism, lifetime mgmt not well supported • Localization issue: local bindings and deployment techniques • Data encoding issue: currently WS are text protocol oriented <definitions name=‘weatherservice’ xmlns=…> <service name=‘Weatherservice’> <message name=‘Weather.GetTempr’> <part name=‘zipcode’ type=‘xsd:string’> </message> <message name=‘Weather.GetTemprResponse’> <part name=‘Result’ type=‘xsd:float’> </message> <portType name=‘WeatherSoapPort’> <operation name=‘GetTempr’> <input message=‘wsdlns:Weather.GetTempr’> <output message=‘wsdlns:Weather.GetTemprResponse’> </operation> </portType> <binding name=‘WeatherSoapBinding’ type=‘WeatherSoapPort’> <soap:binding style=‘rpc’ transport=‘…/soap/http’/> <operation name=‘GetTempr’> <soap:operation soapAction=‘…/Weather.GetTempr’/> <input> <soap:body use=‘encoded’ … /> </input> <output> <soap:body use=‘encoded’ … /> </output> </operation> </binding> <port name=‘WeatherSoapPort’ binding=‘WeatherSoapBinding’> <soap:address location=‘http://www.emory.edu/…’ /> </port> </service></definitions>

20. H2O Extensions to WSDL • Deployment issue • H2O kernel is consistent runtime hosting environment • Lifetime management defined at pluglet level and implemented by kernel • Localization issue • Propose local dispatch e.g. JavaBinding • javaobject://…/protocol similar to rmi:// • Local “registry” within component container • Data encoding issue • Non-XML XDR-based encoding scheme for structured data • Exploit SOAP extensibility: XDR-enc binary attachments <definitions name=‘weatherservice’ xmlns=…> <service name=‘Weatherservice’> <message name=‘Weather.GetTempr’> <part name=‘zipcode’ type=‘xsd:string’> </message> <message name=‘Weather.GetTemprResponse’> <part name=‘Result’ type=‘xsd:float’> </message> <portType name=‘WeatherSoapPort’> <operation name=‘GetTempr’> <input message=‘wsdlns:Weather.GetTempr’> <output message=‘wsdlns:Weather.GetTemprResponse’> </operation> </portType> <binding name=‘WeatherJavaBinding’ type=‘WeatherJavaPort’> . . . <operation name=‘GetTempr’> . . . <input>(zipcode:int) </input> <output>(temperature:float) </output> </operation> </binding><port name=‘WeatherJavaPort’ binding=‘WeatherJavaBinding’> <java:instance location=‘javaobject://MyWeatherSvc/’ /> </port> </service></definitions>

21. Task-Farm Computing • Conventional model • Farmer (e.g. SETI) publishes worker pluglets in a well-known registry • Workers (e.g. volunteers) acquire them and load into their kernels • Worker pluglets register themselves with Farmer’s application • Farmer’s application breaks the computation into tasks and assigns them to workers • Alternative model • Providers setup kernel allowing: • Pluglet-wrapped tasks to run • Dot-sah files to be created • Client discovers and utilizes resources Farmer Application Worker Worker

22. Peer-to-Peer • File sharing • Users load file-sharing pluglets into their kernels • Pluglets of the same kind communicate with each other and establish network of peers • Pluglet-specific discovery, security, accounting, etc. • GUI-based file-sharing client talks to local pluglets – via one of multiple protocols supported by RMIX • Pluggable P2P logic abstracted from the client • Opportunistic resource sharing • Other P2P scenarios, e.g. transport routing, distributed storage

23. Summary • H2O Project: resource sharing infrastructure • Lightweight • Decoupled • Reconfigurable • Self-organizing • Project Status • Early alpha version available • Preliminary performance experiments performed • (Near) Future plans • Develop library of useful pluglets • MPI, PVM • File sharing, communication • Web services hosting • Implement preliminary security and resource control facilities • Initial public release: November 2002