Auger & XtremWeb:
This presentation is the property of its rightful owner.
Sponsored Links
1 / 23

Auger & XtremWeb: Monte Carlo computation on A Global Computing platform O. Lodygensky, G. Fedak, V. Neri, Cordier, F. Cappello PowerPoint PPT Presentation


  • 50 Views
  • Uploaded on
  • Presentation posted in: General

Auger & XtremWeb: Monte Carlo computation on A Global Computing platform O. Lodygensky, G. Fedak, V. Neri, Cordier, F. Cappello Laboratoire de l’Accelerateur Lineaire; Laboratoire de Recherche en Informatique; CNRS, Université Paris sud, France. Sommaire. Introduction XtremWeb

Download Presentation

Auger & XtremWeb: Monte Carlo computation on A Global Computing platform O. Lodygensky, G. Fedak, V. Neri, Cordier, F. Cappello

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Auger xtremweb monte carlo computation on a global computing platform o lodygensky g fedak v neri cordier f cappello

Auger & XtremWeb:

Monte Carlo computation on

A Global Computing platform

O. Lodygensky, G. Fedak, V. Neri,

Cordier, F. Cappello

Laboratoire de l’Accelerateur Lineaire;

Laboratoire de Recherche en Informatique;

CNRS, Université Paris sud, France.


Sommaire

Sommaire

  • Introduction

  • XtremWeb

  • Auger distributed computing

  • Conclusion


Different grid

Different GRID

Nodes

caracteristics

Traditionnal computing centers,

Clusters

  • <100

  • Stables

  • Individually identified

  • Trusted

« GRID »

2 distributed system types

Global computing systems

Windows,

Linux,

Mac OS

  • ~100 000

  • Volatiles

  • No individualident

  • Not trusted

« Desktop GRID » « Internet Computing »

Peer to Peer systems (P2P)

CHEP2003 - O.Lodygensky


Auger xtremweb monte carlo computation on a global computing platform o lodygensky g fedak v neri cordier f cappello

Desktop GRID

One server centralizes scheduling

On volunteer PCs

  • Dedicated applications

    • [email protected], distributed.net,

    • Decrypthon

  • Production projects

    • [email protected], [email protected],

    • Folderol,

  • Open source/research projects

    • XtremWeb, BOINC,

  • Commercial platforms

    • Entropia, Datasynapse,

    • United Devices, Grid systems

Client Application

set params. / get results.

server

parametres

Internet

VolunteerPC

VolunteerPC

Volunteer PC :

load & exec task

CHEP2003 - O.Lodygensky


Auger xtremweb monte carlo computation on a global computing platform o lodygensky g fedak v neri cordier f cappello

Desktop Grid characteristics

  • Scalability : up to 100 k, 1 M hosts

  • Heterogeneity : different hardwares, OSes

  • Volatility : unpredictable participant behaviour

  • [email protected], Napster, Kazaa, etc. : they work well despite volatility.

  • Perenity : developments and upgrades must be easy

  • Performances : [email protected] ~30 Tflops,

  • Kazaa (1 M users : 100kb/s, 1Mb/s  100 Gb/s, 1 Tb/s?).

  • Sécurity :

    • Volunteer PCs and servers integrity ;

    • Prevent application & results corruption ;

    • Authentication.

CHEP2003 - O.Lodygensky


Sommaire1

Sommaire

  • Introduction

  • XtremWeb

  • Auger distributed computing

  • Conclusion

CHEP2003 - O.Lodygensky


Xw architecture

XW : Architecture

  • Centralized

  • Global Computing (Peer to Peer)

  • 3 entities : client/coordinator/worker

Hierarchical

Coordinator

P2P Coordinator

Global Computing

coordinator

(client)

PC

Client/worker

Internet / LAN

PC

Worker

PC

Worker

PC

Client/Worker

CHEP2003 - O.Lodygensky


Xw t echnology

Data

Base

Server

SQL

Java JDBC

Java

XML-RPC

SSL

Java

PHP3-4

GNU autotool

Communication

protocol

Http Server

Worker

Client

Installation

XW : Technology

Pre requisite for installation: database (Mysql),

JAVA > jdk1.2.

CHEP2003 - O.Lodygensky


Xw secu r i t y

XW : Security

Coordinat.

ssh

ssh

Worker

Sandbox

(SBLSM)

Client

LoadedApp

CHEP2003 - O.Lodygensky


Xw fault tolerance model

XW : fault tolerance model

  • Every entity is volatile by essence

  • Connectionless protocols

    => All entities are stand alone

Submit task

Worker1

Client

Get work

Coord.

Sync/Retrieve result

Put result

Sync/Retrieve result

Sync/Get work

Client2

Put result

Worker2

Coord.

Sync/Submit task

Sync/Get work

Sync/Retrieve result

Sync/Put result

CHEP2003 - O.Lodygensky


Sommaire2

Sommaire

  • Introduction

  • XtremWeb

  • Auger distributed computing

  • Conclusion

CHEP2003 - O.Lodygensky


Pierre auger observatory

Pierre Auger Observatory

Understanding the origin of very high cosmic rays:

  • Aires: Air Showers Extended Simulation

    • Sequential, Monte Carlo. Time for a run: 5 to 10 hours

Air shower parameter

database (Lyon, France)

Traditional Super Computing Centers

XtremWeb

Server

CINES (Fr)

Estimated

PC number ~ 5000

air shower

PC worker

Internet

and LAN

Fermi Lab (USA)

PC Client

PC Worker

PC worker Aires

CHEP2003 - O.Lodygensky


Auger xtremweb monte carlo computation on a global computing platform o lodygensky g fedak v neri cordier f cappello

Auger-XW (AIRES): High Energy Physics

  • Application :

  • AIRES

  • Deployment:

  • Coordinator at LRI

  • Madison:700 workers

  • Pentium III, Linux

  • (500 MHz+933 MHz)

  • (Condor pool)

  • Grenoble Icluster: 146 workers

  • (733 Mhz), PBS

  • LRI: 100 workers

  • Pentium III, Athlon, Linux

  • (500MHz, 733MHz, 1.5 GHz)

  • (Condor pool)

Icluster

Grenoble

PBS

Madison

Wisconsin

Condor

U-psud

network

Internet

LRI

Condor Pool

Other

Labs

lri.fr

XW Coordinator

XW Client

CHEP2003 - O.Lodygensky


Auger xtremweb monte carlo computation on a global computing platform o lodygensky g fedak v neri cordier f cappello

Auger-XW (AIRES): High Energy Physics

CHEP2003 - O.Lodygensky


Auger xtremweb monte carlo computation on a global computing platform o lodygensky g fedak v neri cordier f cappello

Auger-XW (AIRES): High Energy Physics

CHEP2003 - O.Lodygensky


Auger xtremweb monte carlo computation on a global computing platform o lodygensky g fedak v neri cordier f cappello

Auger-XW (AIRES): High Energy Physics

CHEP2003 - O.Lodygensky


Auger xtremweb monte carlo computation on a global computing platform o lodygensky g fedak v neri cordier f cappello

Auger-XW (AIRES): High Energy Physics

CHEP2003 - O.Lodygensky


Auger xtremweb monte carlo computation on a global computing platform o lodygensky g fedak v neri cordier f cappello

Auger-XW (AIRES): High Energy Physics

CHEP2003 - O.Lodygensky


Auger xtremweb monte carlo computation on a global computing platform o lodygensky g fedak v neri cordier f cappello

Auger-XW (AIRES): High Energy Physics

CHEP2003 - O.Lodygensky


Auger xtremweb monte carlo computation on a global computing platform o lodygensky g fedak v neri cordier f cappello

Auger-XW (AIRES): High Energy Physics

CHEP2003 - O.Lodygensky


Sommaire3

Sommaire

  • Introduction

  • XtremWeb

  • Auger distributed computing

  • Conclusion

CHEP2003 - O.Lodygensky


Conclusion

Conclusion

XtremWeb : a « desktop Grid » platform

 Fault tolerance. XtremWeb : « connectionless » + « restartable »

 Security : certificats + crypto + sandbox +…

  • What we have learned so far with XtremWeb:

  • Deployment is critical

  • When they understand the computational power potential ,

    users rapidly ask for more resources!!!

    XtremWeb Auger:

  • International Desktop GRID

  • Condor pools with XW as global infrastructure

  • Good performances (ratio 1:60 with several small hosts than the reference)

  • =>Schedulling is a lack of XtremWeb <=

    =>Strong need of results browsing tools <=

    CHEP2003 - O.Lodygensky


    Software

    Software

    • XtremWeb : www.XtremWeb.net

      • Since 2001

      • Acual version : 1.2.rc0

    CHEP2003 - O.Lodygensky


  • Login