Java-based Technologies to Support Parallel and Distributed Applications Prof Mark Baker ACET, University of Reading Tel: +44 118 378 8615 E-mail: [email protected] Web: http://acet.rdg.ac.uk/~mab Outline The pros and cons of Java. Problems developing Java applications.
Prof Mark Baker
ACET, University of Reading Tel: +44 118 378 8615 E-mail: [email protected]
The authors conclude, "On Intel Pentium hardware, especially with Linux, the performance gap is small enough to be of little or no concern to programmers."
Some Apps You Don’t know
Major RetailersHard to Define, Know it When you See it…
Started off with GridRM (gridrm.org) a wide-area Java-based resource monitoring system:
A scalable infrastructure for gathering native information from computer-based resources,
Agents, gateways, linked in a hierarchical configuration,
Started designing it pre-2000,
No standards in monitoring arena (apart from SNMP), so issue was what to use for our infrastructure?
Choose HTTP and XML as this seemed to provide us with a core, that we could adapt at the end points.Some History!
Local Layer resource monitoring system:
Hierarchy of gateways:
Each manages a site:
Interacts with agents,
Deals with policies and security.
Stores resource and historical data.
Use what is there!
A system to bind together gateways:
Tycho consists of the following components: resource monitoring system:
Mediators that allow producers and consumers to discover each other and establish remote communications,
Consumers that typically subscribe to receive information or events from producers,
Producers that gather and publish information for consumers.
There is an asynchronous messaging API.
In Tycho, producers and/or consumers (clients) can publish their existence in a Virtual Registry (VR).Tycho Architecture
NaradaBrokering: resource monitoring system:
The NaradaBrokering framework is a distributed messaging infrastructure, developed by the Community Grids Lab at Indiana University.
NaradaBrokering is an asynchronous messaging infrastructure with a publish and subscribe based architecture.
NaradaBrokering is Sun JMS compliant. This messaging standard allows application components to exchange unified messages in an asynchronous system.Performance Tests (Messaging)
MDS4: resource monitoring system:
The Globus Toolkit’s Monitoring and Discovery Service (MDS version 4) is a Web Services Resource Framework (WSRF) based implementation of a wide-area information and registry service.
MDS4 provides a framework that can be used to collect, index and expose data about the state of grid resources and services.
Typical uses for MDS4 include making resource data available for decision making in job submission services or notifying an administrator when storage space is running low on a cluster.Registries
R-GMA: resource monitoring system:
A Java-based implementation of the Grid Monitoring Architecture (GMA) for publishing network monitoring information over the wide-area and as an information service.
R-GMA uses a relational model to search, using an SQL-like API, and describe the monitoring information it collects.
based on a consumer/producer paradigm with client data being stored in a directory service, which presents it as a virtual database.
R-GMA uses the term tuples for sets of data being published or consumed.Registries
Query Response Time Versus the Number of Records for a Query When Selecting a Single Random Record From the Registry (S1)
Query Response Time Versus the Number of Records for a Query that Selects All the Records From the Registry (S2)
The high and base levels rely on the resource monitoring system:mpjdev and the xdev level for actual communications and interaction with the underlying networking hardware.
Two implementations of the mpjdev are envisaged:
JNI wrappers to native MPI implementations,
Use of a lower level device, called xdev, to provide access to Java sockets or specialised communication libraries.MPJ Express - Architecture
A 3-dimensional visualization of the Millennium Simulation. resource monitoring system:
The movie shows a journey through the simulated universe.
On the way, we visit a rich cluster of galaxies and fly around it.
During the two minutes of the movie, we travel a distance for which light would need more than 2.4 billion years.Gadget
This is acceptable performance given that Java has many extra safety features including array bounds checking.
Comparison is between a production quality C code against a Java code that could be optimized further.
The performance of Java Gadget-2 reinforces our belief that Java is a viable option for HPC.Performance