Common Component Architecture and XCAT

1. Common Component Architecture and XCAT Tiziano Fagni High Performance Computing Lab ISTI - CNR

2. What is a Component Architecture? • A systematic way of encapsulating special functionality and behaviors of a piece of software into reusable units. • More than an object model. • It defines the rules for the way objects can be instantiated and composed. • It defines the environment of services that a component can use. • It defines the way in which we discover how to interact with them.

3. Standard Component Architectures • Examples: • Microsoft COM/DCOM, COM++, .NET • the foundation of all Microsoft office products • Java Beans • Java standard for building user interfaces • Enterprise Java Beans • a standard for client-server applications in Java • CORBA 3.0 • The new component spec for CORBA (CCM).

4. Requirements of component architectures for High Performance Computing • Indipendence from programming paradigms and data size • SPMD and shared memory multi-thread • Coarse and fine grain • Heterogeneity • Architecture • Language • Runtime system • Efficient handling of local and remote components • Effective use of mechanism to support high performance interactions • Avoid extra copies, extra communications, extra synchronization • Parallel data transfer

5. What is Common Component Architecture (CCA)? • It is a common architecture for building very large scientific applications from well tested software components. Being a light-weight specification, does not define: • From framework point of view • The way the framework is constructed. • The way the users interact with the framework to connect components together. • From components point of view • The way the components are created, discovery, conneted. • The main motivations for this is that will be many different frameworks that can be used in very different context. • In wide-area grid. • In parallel supercomputer. • In distrubuted environment. • Etc…

6. The two entities of the CCA architecture • Components • Basic unit of software that are composed together to form applications. • Can be any language. Ususally Java, C, Python, C++ or Fortran. • Frameworks • The software subsystems that provides basic services to components • Used to compose components to make apps. • Many implementations: Parallel, Distributed and Grid.

7. Some Concepts in Detail • Ports: the public interfaces of a component • defines the different ways we can interact with a component and the ways the component uses services and other components. setImage(Image I) Uses Ports - interface of a service used by component Image getImage() Image Processing Component adjustColor() calls doFFT(…) setFilter(Filter) Provides Ports - interfaces functions provided by component

8. Building Applications by Composition • Connect uses Ports to Provides Ports. Image database component setImage(…) Image Processing Component getImage() Acme FFT component doFFT(…) adjustColor() Image tool graphical interface component

9. XCat: a CCA framework • XCat is an implementation of CCA developed by Extreme Lab at Indiana University • Dennis Gannon, Randall Bramley, Kenneth Chiu • Several students of the Lab • The version actually developed is 2.0 and supports C++ and Java languages interoperability.

10. XCat implements standard Web service stack

11. XCat services: overview • XCat uses a service based approach to integrate CCA light-weight specification. It defines: • Creation Service • Connection Service • Naming Service • Application Manager Service

12. XCat services: Creation Service • It is a framework specific component that allows one component to instantiate other components • It defines two type of functionalities: • create a new instance of a component • delete an existing instance of a component • The parameters of creation functionality are: • excutable location • host machine to execute on • the mechanism to be used for instantiation (GRAM, rsh, ssh, exec, proc) • command line parameters • etc…

13. XCat services: Connection Service • It is a framework specific component used by components to establish connection between ports of components • It defines the following functionalities: • connection of components port (its own ports with other component ports or connections of ports of known components) • disconnection of components port • export ports of known components

14. XCat services: Naming Service • Currently based on Java RMI Registry API, it provides a registry service used to register and retrieve components. • It defines the following functionalities: • bind and rebind of components • lookup of registered components • unbind of components

15. XCat services: Application Manager Service • It is a scriptable framework component that acts as a controller for current running application. • The functionalities defined depends by current loaded Jython script. • The manager can also communicate with others CCA components and use the services provided by them.

16. XCat security Provides port Component The current version uses SSL with X.509 certificates (CoG library) and supports both authentication and simple authorization model based on access control list XCat framework Authorization Interceptor XSOAP/Transport Response Remote call

17. How do you “compose” two components? • There are three basic approaches: • Events: A component can generate an “event” of a particular type. Other components are added as “listeners” for such events from that component. • Services: A source component “uses” services “provided” by a target components. • Dataflow: A typed data “output” stream is connected to a similarly typed “input” port of another. • In XCAT all three are supported. • The services model is the basic mechanism. • Dataflow is emulated in CCA by “pushing” data from a user to a provider, or by having a user “pull” data from a provider. • Events are managed by a separate mechanism (XMessage/XEvents).

18. Listener host XEvent agent middleware XMessagges agent middleware Network Event Listener Message PublisherAgent Event ListenerAgent Messagesource Directmessageexchange Message PublisherAgent Event Publisher Event PublisherAgent Messages XMessage channel Publisher host Exception handling Log of messaggesthat are to be sent … Store messages on disk for successivequeries XCat event system: XEvent/XMessagges

19. How do you describe the interfaces a port has? • Use a Scientific Interface Definition Language (Babel), a formal description of the interface objects. • Handle the needed of parallel scientific applications (complex numbers, dynamic multidimensional arrays, parallel communication directives, etc…) • Generate efficient glue code • Mediate differences among different languages (C, C++, Java, Fortran, etc…) • Efficient inter languages calls within same address space and, eventually, across different memory spaces (i.e. distributed objects) • Provide software engineering facilities (enumerated data types, namespace management, object-oriented inheritance model) • For the XCAT java version the interface of a component port can be defined as Java interfaces

20. Current XCAT version is almost OGSI compliant • This is possibile by • Addition of an OGSI GridService Port • Addition of Service Data Elements (SDEs) containing references to each of the other provides ports • Uniquely identifying the component using a ComponentID (GSH) • Using a WSDL representation for the GSR • A HandleResolver service for mapping a GSH to a GSR • Modifications to the XCAT Creation service to make appropriate calls to the Factory and GridService ports for lifetime management

21. XCat programming models • XCat Services APIs, together with CCA compliant components, can be used to generate control applications. We can use two possible approachs: • conventional static approach by writing and compiling Java or C++ programs that create and control applications • more dynamic approach by creating applications using Jython script interpreted by Application Manager Service

22. Set of interest CCA components Generic application C1 C2 Cn-1 Cn XCat programming models: standard approach Use World known services Java program Compilation Program control executable Exec Framework services Creation Service Connection Service Naming Service

23. Set of interest CCA components Generic application C1 C2 Cn-1 Cn XCat programming models: scripting approach Application Manager Service Jython script Use Use Framework services Naming Service Creation Service Exec Connection Service Few known services

24. Simple Java example Echo Generator Component A Provides port simpleEchoProvidesPort A Provides port ProvidesGoPort A uses port simpleEchoUsesPort Echo Printer Component Application Manager

25. Simple example in Java: echo provides interface public interface SimpleEcho { public String echoHello(String s) throws RemoteException; } public class SimpleEchoImpl extends ProvidesPort implements SimpleEcho { public String echoHello(String s) throws RemoteException { return "SimpleEchoImpl says: Hello " + s; } }

26. Simple example in Java: printer component public class EchoPrinterComponent implements Component { public void setServices(Services cc){ PortInfo portType = new PortInfoImpl( "simpleEchoProvidesPort", "http://example.com/echo.wsdl"); simpleEchoImpl = new SimpleEchoImpl(); cc.addProvidesPort(simpleEchoImpl,portType); } }

27. Simple example in Java: generator component public class EchoGeneratorComponent implements Component { public void setServices(Services cc){ PortInfo portType = new PortInfoImpl( "simpleEchoUsesPort", "http://example.com/echo.wsdl"); cc.registerUsesPort(portType); } }

28. generatorComponent = EnvObj() generatorComponent.put("exec-name", "simpleGeneratorComponent") generatorComponent.put("exec-fqn", "samples.simpleEchoGenerator.SimpleEchoGeneratorComponent") printerComponent = EnvObj() printerComponent.put("exec-name", "simplePrinterComponent") printerComponent.put("exec-fqn", "samples.simpleEchoPrinter.SimpleEchoPrinterComponent") # create component wrappers generator = cca.createComponent(generatorComponent) printer = cca.createComponent(printerComponent) # assign a machine name printermc = "exodus.extreme.indiana.edu" generatormc = "exodus.extreme.indiana.edu" cca.setMachineName(generator,generatormc) cca.setMachineName(printer, printermc) # create live instances cca.setCreationMechanism(generator, ssh) cca.setCreationMechanism(generator, gram) cca.createInstanceWithTimeOut(printer, 120000) cca.createInstanceWithTimeOut(generator, 120000) # connect their ports cca.connectPorts(generator, "simpleEchoUsesPort", printer, "simpleEchoProvidesPort") # start the components – invoke go() on GoPort usesGoPortClassName = "samples.idl.goPort.UsesGoPort" usesPortType = "http://example.com/go.wsdl" providesPortName = "providesGoPort" methodName = "go" methodParams = zeros(0, Object) cca.invokeMethodOnComponent(generator, usesGoPortClassName, usesPortType, providesPortName, methodName, methodParams) Scripting XCAT Applications

29. XCat: does it function? • Local component • “exec”, “proc": no problem! • Remote component • “ssh”, “rsh”: no problem! • “GRAM” with CoG Kit: FAILED The problem Actually used CoG library support GT2 but is not compatible with GT2.4 that is installed on HPC Lab machines.

30. XCat future roadmap • Update registry service from LDAP to UDDI model • Integrate a factory service to construct automatically complex distributed applications • Update CoG kit to use GT3 • Continue to work towards a framework implementation that fully merges OGSI and XCat

31. Thanks!!