Overview Collaborative system for metacomputing in heterogeneous network environments

1. Metacomputing with the IceT SystemPaul Gray and Vaidy SunderamEmory University, Atlanta, USA{gray,vss}@mathcs.emory.edu • Overview • Collaborative system for metacomputing in heterogeneous network environments • Framework based on process/data mobility and merging/splitting of resources

2. IceT Goals • Metacomputing as a collaborative activity • Cooperating users contribute resources to a pool, with dynamic control over access and usage levels • Applications use external resources subject to owner controls; userid, filesystem access, etc not required • Process and data portability & mobility • Processes/threads should be architecture/machine independent when possible • Active computational units as well as data should be transportable to their needed location

3. Processes Processes Data 0101 1011 1000 0101 1011 1000 Data User User Computational Resources Computational Resources Processes Data 0101 1011 1000 User Computational Resources Metacomputing View • Concurrent Computing • merging and splitting of multiple virtual machines • portability of code and data, uploading, soft install and migration • Base system: Java, Java-C, C, and Fortran; message passing parallel and distributed computing • Extensible, reconfigurable

4. IceT Resource Handling • Unowned resources • Naming/discovery • Access control • Resource limits • Withdrawal issues • Public resources • Access/accounting • Scheduling and load balancing • Communication limits, firewall tunneling etc

5. Initial Implementation • Resources and processes • Commands/classes such as add, delete, spawn, kill etc to configure VM / process management • Additional commands/classes to merge, split and register virtual environments. • Traditional message passing model • Java based base system; IceT package supplied • Methods for configuring resources, process spawning, simple message passing primitives • Java features leveraged by IceT to enable soft installation, native code transport and dynamic loading, migration

6. Programming Overview 01 import IceT.*; 02 03 public class IllustrativeIceTProgram extends TaskProtocols { 04 public void run() { 05 IceT(); /* Enroll in IceT */ ... Create the collaborative environment ... 12 try { 13 remoteGroup = 14 probeIceTResources(local_IceT_wp_server,"Dave's IceT VM"); 15 } 16 catch (IceTResourceNotRegisteredException itrnre) { 17 notify(itrne.toString()); ... Combine resources, creating a multi-level, time-shared, VM ... 21 IceT_ResourceGroup resources = merge(localGroup,remoteGroup); ... Create remote processes ... 42 spawn("DataGenerators",remoteGroup.size(),taskArray);

7. Programming (contd) ... Use Java's Serializable threads, IceT's portability/migration... 56 SerializableDataProcessingThread serializableThread = 57 new SerializableDataProcessingThread(); 58 serializableThread.start(); 59 Buffer buf; 60 while(resources.getLocalWorkLoad() < LOCAL_WORKLOAD_TOLERANCE) { 61 buf = receive(remoteGroup,DATA_TAG); 62 RawData data = (RawData) buf.unpackSerializableObject(); 63 serializableThread.analyze(data); 64 } 65 ... Migrate the thread elsewhere ... 66 buf.reset(); 67 buf.pack( (Serializable) serializableThread); 68 send(buf, resources.getIdleHost(),DATA_PROCESSING_THREAD_TAG); ... Other processing ... 88 exitIceT(); /* Disconnect from the IceT environment */ 89 } 91 } Annotated IceT program skeleton illustrating merging (12-21), spawning (42), and migration (65-68).

8. Uploading/Soft Install • Program components -> external resources • PVM,MPI etc: user provides static binary form of the executable via NFS, FTP, or building onsite • Can be automated but subject to constraints/pitfalls: filesystem access, setup, header files, multiple versions • IceT vision: soft installation or dynamic upload as needed, removal and cleanup after use • For example, . . .

9. Here Spawn This Linux NT Cluster SGI Solaris Cluster Data Servers Data Repository Merge/Upload Example Update Database, Post-process data

10. Leverage several characteristics of Java Standardized bytecode representation ClassLoader mechanism Built in, extendable, security management Substantial extensions and infrastructure Complete analysis of class dependencies, location, retrieval, security Dynamic inclusion/binding of native code Version control and consistency issues Java and IceT

11. Functionality... • The Java-based implementation proved to be well suited for “computationally-lite” distributed computing tasks across network boundaries. • Although speedups were observable for parallelization of tasks over clusters, performance for traditional distributed computing tasks left a lot to be desired.

12. Use Java for the initial introduction of the program collective to the remote host. The wrapper class may be analyzed for class dependencies, shared library usage, security violations, etc. Use C/Fortran codes as the computational engine of the process. Compiled into shared libraries (.so’s or .DLL’s), they can be encapsulated within the program collective and loaded onto the remote resource. Solution: Blend Java with C/Fortran

13. Here Spawn This Linux NT Cluster SGI Solaris Cluster Native Library, FORTRAN BLAS perhaps The (New) Objective:

14. Pros/Cons of using Java

15. Native Code Use • Benefits • Integration of legacy codes and scientific packages • Significant increase in computational performance • Possibilities • Dynamic re-configuration of the metacomputing environment and VM capabilities • Better decoupling of compute kernels from communication, process management, and housekeeping framework

16. Performance • Computational • Java has yet to be established as a viable language for High Performance Computing; interpreted bytecode - heterogeneity detrimental to execution speed. • Java shells + native code: good solution • Infrastructural • Similar drawbacks but not as severe; e.g communication slowdown is relatively small • But why not use native codes for communication? • I.e. Java-based IceT backplane

17. Reconfiguration Experiments • Low level communication substrate • Standard protocols and network classes are unsuitable for parallel algorithms (1-1 vs. many-many) • CCTL: fast and lightweight mulicast protocol developed for collaborative computing frameworks -> loaded into IceT • Higher level MP library • The LAM version of MPI was wrapped using the JCI (Java-to-C Interface) tool as a separate and ongoing effort (V. Getov, Univ. of Westminster). • IceT demonstrated the ability to write/debug applications locally using the local MPI package and subsequently execute production runs on a bare remote cluster.

18. Reconfiguring the Environment Example: Swapping out the underlying communication layer in a numerical computation. Times (in milliseconds) required to transport a double-precision array. Double Precision Array Size 1 10 100 1,000 10,000 100,000 IceT w/ Java- based MP 38 19 34 250 2538 23662 IceT w/ JNI-wrapped CCTL-based MP 3 3 3 9 59 622 Using IceT, the Java-based communication layer was swapped out for the more efficient CCTL. The result is high-performance and portability.

19. MP library Upload/Run Scenario Java Bytecode 2212EBABEFAC libJavaMPI_MPI.so 010011101011010 Local IceT User Raw Cluster

20. IceT: Summary and Status • Uploading & Java/native code mixing • high degree of portability while maintaing high performance • ability to reconfigure environment and model • Merging and splitting • Computational grid is dynamic; IceT is flexible • Resource allocation/management can be system controlled or owner controlled • IceT continues to develop and extend a particular niche in metacomputing