Lightweight grid computing workshop, 3rd May 2006
Download
1 / 19

Lightweight grid computing workshop, 3rd May 2006 - PowerPoint PPT Presentation


  • 144 Views
  • Uploaded on

Lightweight grid computing workshop, 3rd May 2006. WEDS: Web services Environment for Distributed Simulation James Suter Centre for Computational Science University College London. Contents. Problems with existing grid middleware Lightweight middleware? WEDS

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about ' Lightweight grid computing workshop, 3rd May 2006' - barb


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript

Lightweight grid computing workshop, 3rd May 2006

WEDS: Web services Environment for Distributed SimulationJames SuterCentre for Computational ScienceUniversity College London


Contents

  • Problems with existing grid middleware

  • Lightweight middleware?

  • WEDS

  • Enabling grid-based computational science

  • Examples


Grid computing
Grid Computing

  • We define grid computing as distributed computing performed transparently across multiple administrative domains.

  • By computing, we refer to any activity involving digital information. Transparency, implying minimal complexity for the users of the technology, has been missing from all existing distributed computing infrastructures.

  • A general computational grid should provide easy access to many different types of resources to enable users to pick and choose those required to achieve their intended scientific objectives.

See: Phil Trans R Soc London A (2005)


Storage devices

UK and US Grid Technologies

Instruments: XMT devices, LUSI,…

Grid Middleware

HPC resources

Scalable MD, MC, mesoscale modelling

User with laptop

Steering

Performance control/monitoring

Visualization engines

VR and/or AG nodes


Grid computing1
Grid Computing

  • Such demands have so far proved hard to meet and as a result very few scientists have wanted to get near so-called grid technology.

  • Important for the development of the grid as a whole that as many diverse groups of scientists begin to use the grid; it is critical that the uptake of grid technologies is outside of specialized grid-centric projects.

  • Existing grid middleware obstructs rather than facilitates access to grid resources. This has hindered scientific progress with the grid.


Problems for users
Problems for users

  • The existing toolkits have an excessively heavy set of software and administrative requirements, even for relatively simple demands from applications.

    • Existing toolkits are painful and difficult to install and maintain.

    • Existing standards bodies and the task forces within the UK e-Science programme are not engaging sufficiently with the applications community.

  • The effort and time required to overcome these problems are such that many scientists are reticent to deal with grid resources.

  • Application scientists generally use legacy applications, often written without knowledge of grid computing. Porting of these codes to the grid environment with ease of installation and facility is crucial to user uptake.


  • Our solution
    Our solution

    • We have therefore developed simple, lightweight Grid middleware that is "good enough" for rapid adoption, rather than waiting for a solution which will, supposedly, suit all needs.

    • Such a toolkit must be

      • substantially more portable, lightweight, and modular in design than current middleware.

      • By being produced in very close collaboration with application scientists and having full documentation, this will allow end-users to port existing codes and use Grid techniques with the minimum of hassle.


    Lightweight middleware
    Lightweight middleware

    • What do we mean by lightweight?

      • Minimal dependencies on third-party software

      • Small learning-curve for new users – removing the need to learn new programming methods

      • Interoperable with other WSRG implementations

    • Easy to write, and so to adapt to new specs, etc.

    • Uses WSRF::Lite (interoperable with other WSRG implementations)


    Lightweight middleware1
    Lightweight middleware

    • OGSI::Lite/WSRF::Lite

      • by Mark McKeown of Manchester University

    • Lightweight OGSI/WSRF implementation, written in Perl

      • uses existing software where possible; simple installation

    • We have developed WEDS--a Web service hosting Environment for Distributed Simulation


    About weds
    About WEDS

    • Developed to make life easier for application scientists

    • Easy to deploy – sits inside a WSRF::Lite container, has no additional software requirements

    • For use within an administrative domain

    • Provides all the tools and glue required to:

      • expose an unaltered binary application as a service

      • create and interact with service instances

    • Broker service manages creation of services, to load balance across a pool of machines

    See Coveney et al., 2004, NeSC Tech Rpt


    Weds architecture
    WEDS Architecture

    • Each resource runs a WSRF::Lite container containing a WEDS machine service and factory services for each hosted application.

    • Each machine that a user wishes to use is registered with a broker service

    • The user contacts the broker with the details of the job to run

    • The broker match-makes the job details with the capabilities advertised by each machine service and decides where to invoke the service

    • The broker passes back the contact details of the service instance to the client

    Client

    Broker

    Machine Service

    Service Factory

    Wrapper Service

    Invoked

    Application

    Managed resource


    More complex uses of weds
    More complex uses of WEDS

    • Under this framework, an executable can be deployed remotely using a 'standard' script as if it were being run locally.

    • An object-oriented representation of a simulation wrapper can be written in Perl, which allows scripts (or users) to start, visualize and control simulations with commands as simple as

      • $g=$sim->getValue('gravity'); or

      • $sim->setValue('pressure', 200);

    • The term 'loosely coupled' is often used and well describes the way a library of Web services could be easily connected to run simulations or analysers, which automatically exchange their results between each other


    The Polysteer Application

    Developed by Daniel Mason (Imperial; ReG partner)

    • New Monte Carlo polymer simulation code

      • Create as many chain conformations as possible

        Task farming of configuration generation

    • Equilibration is difficult from arbitrary start point

      • Need to watch chains relax

        Attach visualisation client

    • Monte Carlo moves are complex

      • Modify parameters on-the-fly to optimise efficiency

        Attach steering client



    Visualisation

    Visualisation

    • Showing each atom is unreadable

    • Potentials treat CHx, Bz as single entities

    • We visualise ellipsoids rather than spheres


    Work flow example fourier transform of molecular dynamics output
    Work flow example: Fourier transform of molecular dynamics output

    • A user script coordinates a workflow between molecular dynamics, Fourier transform and visualization services. The binary data moves directly between elements, not via the user's machine, halving bandwidth demands.

    • Example: MD program LAMMPS interfacing with a Fourier Transform analysis program


    Steering example lb2d
    Steering example: LB2D output

    • LB2D is a workstation class lattice-Boltzmann code written in C simulating multiphase fluid flow in two dimensions. Preparing the code for use with WEDS involved linking to a steering library

    • A single broker and machine service allows a single simulation process to be executed. The simulation involved stepping the value of the parameter g_cc from 2, for the first 200 time-steps, to 0, for the second 200 time-steps, and back to 2 for the final 200 time-steps, giving a simulation run 600 time-steps long in total.


    Summary
    Summary output

    • Lightweight middleware such as WEDS greatly facilitates deployment of applications on grids

    • We’re now working with several “computational user communities” from physics through to biology

    • All our middleware will be in the public domain: Download from www.realitygrid.org/WEDS


    Acknowledgements
    Acknowledgements output

    • Prof. Peter Coveney, Radhika Saskena, Matt Harvey, Laurent Pedesseau, Mark Mc Keown, Stephen Pickles, Daniel Mason, Jonathan Chin

    • EPSRC

    • OMII


    ad