The egee project and the future of european grids
This presentation is the property of its rightful owner.
Sponsored Links
1 / 35

The EGEE project and the future of European grids PowerPoint PPT Presentation


  • 96 Views
  • Uploaded on
  • Presentation posted in: General

The EGEE project and the future of European grids. Guy Wormser Director of “Institut des Grilles du CNRS” Aspera Workshop. PROGRAMS. OPERATING SYSTEM. Disks, CPU etc. Grid: Resource Sharing. Share more than information Data, computing power, applications. Middleware handles everything.

Download Presentation

The EGEE project and the future of European grids

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


The egee project and the future of european grids

The EGEE project and the future of European grids

Guy Wormser

Director of “Institut des Grilles du CNRS”

Aspera Workshop


Grid resource sharing

PROGRAMS

OPERATING SYSTEM

Disks, CPU etc

Grid: Resource Sharing

  • Share more than information

  • Data, computing power, applications

  • Middleware handles everything

Your Program

The Grid

Single computer

MIDDLEWARE

User Interface Machine

Your Program

Word/Excel

Games

Email/Web

Resource Broker

Disk

Server

CPU

Cluster

CPU

Cluster

Inauguration de l'Institut des Grilles, Paris, 3rd December 2007


Electricity grid

Distribution Infrastructure

'Standard Interface'

Electricity Grid

Analogy with the Electricity Power Grid

Power Stations

Inauguration de l'Institut des Grilles, Paris, 3rd December 2007


The egee project and the future of european grids

EGEE : Enabling Grids for E-sciencE

  • Goal

    • create a general European Grid production quality infrastructure on top of present and future EU RN infrastructure

  • Build on

    • EU and EU member states major investment in Grid Technology

    • Several pioneering prototype results

    • Largest Grid development team in the world

  • Goal can be achieved for about €100m/4 years on top of the national and regional initiatives

  • Approach

    • Leverage current and planned national and regional Grid programmes (e.g. LCG)

    • Work closely with relevant industrial Grid developers, NRNs and US

applications

EGEE

Geant network

Inauguration de l'Institut des Grilles, Paris, 3rd December 2007


The egee project and the future of european grids

The Large Hadron Collider Project

4 detectors

CMS

ATLAS

LHCb

Inauguration de l'Institut des Grilles, Paris, 3rd December 2007


Bat 40

Bat 40

Inauguration de l'Institut des Grilles, Paris, 3rd December 2007


New solutions are necessary

New solutions are necessary!

Inauguration de l'Institut des Grilles, Paris, 3rd December 2007


How e infrastructrures help e science

GRID .

INFRASTRUCTURE

KNOWLEDGE .

INFRASTRUCTURE

NETWORK .

INFRASTRUCTURE

How e-Infrastructrures help e-Science

  • e-Infrastructures provide easier access for

    • Small research groups

    • Scientists from many different fields

    • Remote and still developing countries

  • To new technologies

    • Produce and store massive amounts of data

    • Transparent access to millions of files across different administrative domains

    • Low cost access to resources

      • Mobilise large amounts of CPU & storage on short notice (PC clusters)

    • High-end facilities (supercomputers)

  • And help to find new ways to collaborate

    • Develops applications using distributedcomplex workflows

    • Eases distributed collaborations

    • Provides new ways of community building

    • Gives easier access to higher education

Inauguration de l'Institut des Grilles, Paris, 3rd December 2007


The egee project and the future of european grids

EGEE

Main Objectives

Operate a large-scale, production quality grid infrastructure for e-Science

Attract new resources and users from industry as wellas sciences

  • Flagship grid infrastructure project co-funded by the European Commission

  • Now in 2nd phase with 91 partners in 32 countries

Inauguration de l'Institut des Grilles, Paris, 3rd December 2007


Egee what do we deliver

EGEE – What do we deliver?

  • Infrastructure operation

    • Currently includes ~250 sites across 45 countries

      • Continuous monitoring of grid services & automated site configuration/management

      • Support many Virtual Organisations from diverse research disciplines

  • Middleware

    • Production quality middleware distributed under business friendly open source licence

      • Implements a service-oriented architecture that virtualises resources

      • Adheres to recommendations on web service inter-operability and evolving towards emerging standards

  • User Support - Managed process from first contact through to production usage

    • Training

    • Expertise in grid-enabling applications

    • Online helpdesk

    • Networking events (User Forum, Conferences etc.)

Inauguration de l'Institut des Grilles, Paris, 3rd December 2007


The egee project and the future of european grids

240 sites

45 countries

41,000 CPUs

5 PetaBytes

>5000 users

>100 VOs

>100,000 jobs/day

  • Archeology

  • Astronomy

  • Astrophysics

  • Civil Protection

  • Comp. Chemistry

  • Earth Sciences

  • Finance

  • Fusion

  • Geophysics

  • High Energy Physics

  • Life Sciences

  • Multimedia

  • Material Sciences

32%

Inauguration de l'Institut des Grilles, Paris, 3rd December 2007


Types of applications

Types of applications

  • Simulation

    • LHC Monte Carlo simulations; Fusion; WISDOM

    • Jobs needing significant processing power; Large number of independent jobs; limited input data; significant output data

  • Bulk Processing

    • HEP ; Processing of satellite data

    • Distributed input data; Large amount of input and output data; Job management (WMS); Metadata services; complex data structures

  • Parallel Jobs

    • Climate models, computational chemistry

    • Large number of independent but communicating jobs; Need for simultaneous access to large number of CPUs; MPI libraries

  • Short-response delays

    • Prototyping new applications; grid Monitoring grid; Interactivity

    • Limited input & output data; processing needs but fast response and quality of service

  • Workflow

    • Medical imaging; flood analysis

    • Complex analysis algorithms; complex dependencies between jobs

  • Commercial Applications

    • Non-open source software; Geocluster (seismic platform); FlexX (molecular docking); Matlab, Mathematics; Idl, …

    • License server associated to an application deployment model

Inauguration de l'Institut des Grilles, Paris, 3rd December 2007


Lhc computing model

Lab m

Uni x

USA

Brookhaven

Uni a

UK

USA

FermiLab

Lab a

France

Tier 1

Physics

Department

Uni n

CERN

Tier2

……….

Italy

Desktop

Lab b

Germany

NL

Lab c

Uni y

Uni b

LHC Computing Model

The LHC Computing Centre

[email protected]

Inauguration de l'Institut des Grilles, Paris, 3rd December 2007


Seismology 1

SEISMOLOGY[1]

Fast Determination of mechanisms of important earthquakes (IPGP: E. Clévédé, G. Patau)

Challenge

Provide results 24h -48h after its occurrence

5 Seisms already ported: Peru, Guadeloupe, Indonesia (Dec.), Japon, Indonesia (Feb.)

  • Application to run on alert

  • Collect data of 30 seismic stations from GEOSCOPE worldwide network

  • Select stations and data

  • Definition of a spatial 3D grid +time

  • Run for example 50-100jobs

Peru earthquake, 23/6/2001, Mw=8.3

Data used: 15 Geoscope Stations

Inauguration de l'Institut des Grilles, Paris, 3rd December 2007


Korba coastal aquifer

Korba coastal aquifer

Management of water resources in Mediterranean area (SWIMED)

G. Lecca (CRS4 Italy), P. Renard (Unine, CH), J. Kerrou (INAT, Tunisia), R. Ababou (IMFT, Fr)

Tunisia

45 km

Cape Bon Peninsula

70km

south-east of Tunis

Inauguration de l'Institut des Grilles, Paris, 3rd December 2007


Geosciences

GEOSCIENCES

  • Generic seismic platform software, based on Geocluster commercial software developed by CGG

  • Includes 400 geophysical modules, implemented on EGEE

  • Used by both academics and private companies.

  • Free of charge for Academics, with charge for R&D

Inauguration de l'Institut des Grilles, Paris, 3rd December 2007


The egee project and the future of european grids

GATE

GEANT4 Application to Tomography Emission

  • Scientific objectives

    Radiotherapy planning for improving the treatment of cancer by ionizing radiations of the tumours.

    Therapy planning is computed from pre-treatment MR scans by accurately locating tumours in 3D and computing radiation doses applied to the patients.

  • Method

    GEANT4 base software to model

    physics of nuclear medicine.

    Use Monte Carlo simulation to

    improve accuracy of computations (as

    compared to the deterministic classical

    approach)

Inauguration de l'Institut des Grilles, Paris, 3rd December 2007


Drug discovery

Drug Discovery

  • WISDOM focuses on in silico drug discovery for neglected and emerging diseases.

  • Malaria — Summer 2005

    • 46 million ligands docked

    • 1 million selected

    • 1TB data produced; 80 CPU-years used in 6 weeks

  • Avian Flu — Spring 2006

    • H5N1 neuraminidase

    • Impact of selected point mutations on eff. of existing drugs

    • Identification of new potential drugs acting on mutated N1

  • Fall 2006

    • Extension to other neglected diseases

Inauguration de l'Institut des Grilles, Paris, 3rd December 2007


High throughput virtual docking

High Throughput Virtual Docking

Chemical compounds : ZINC

Molecular docking : FlexX, Autodock

Targets structures : PDB

Grid infrastructure : EGEE

Chemical compounds :

Chembridge – 500,000

Drug like – 500,000

Millions of chemical

compounds available

in laboratories

High Throughput Screening

1-10$/compound, nearly impossible

Molecular docking (FlexX, Autodock)

~80 CPU years, 1 TB data

Computational data challenge

~6 weeks on ~1000/1600 computers

Targets :

Plasmepsin II (1lee, 1lf2, 1lf3)

Plasmepsin IV (1ls5)

Hits screening

using assays

performed on

living cells

Leads

Clinical testing

Drug

Inauguration de l'Institut des Grilles, Paris, 3rd December 2007


Grid workflow

Storage

Element

Computing

Element

Computing

Element

Grid workflow

  • FlexX license server :

    • 3000 floating licenses given by BioSolveIT to SCAI

    • Maximum number of used licenses was 1008

Results

Compounds list

Software

Site1

Statistics

Parameter settings

Target structures

Compounds sublists

User interface

Site2

Compounds database

Storage

Element

Software

Results

Inauguration de l'Institut des Grilles, Paris, 3rd December 2007


Gptm3d

gPTM3D

3D Medical Image Analysis Software

  • Scientific objectives

    Interactive volume reconstruction on large radiological data.

    PTM3D is an interactive tool for performing computer-assisted 3D segmentation and volume reconstruction and measurement (RSNA 2004)

    Reconstruction of complex organs (e.g. lung) or entire body from modern CT-scans is involved in augmented reality use case e.g. therapy planning.

  • Method

    Starting from an hand-made rough

    Initialization,a snake-basedalgorithm

    segments each slice of a medical volume.

    3D reconstruction is achieved in parallel

    by triangulating contours from consecutive

    slices.

Inauguration de l'Institut des Grilles, Paris, 3rd December 2007


Grids key compettive advantages

Grids key compettive advantages

  • Transparent access to distributed data

    • Exemples Earth sciences, Life sciences

  • Handling of huge datasets

    • Physique des particle Physics, astrophysics, human sciences

  • Large flexibility in computing ressources

    • Disasters management

    • Avian flu, malaria challenges

  • Synergy between the grid network and the human network 

Inauguration de l'Institut des Grilles, Paris, 3rd December 2007


Egee user forum 2007

EGEE User Forum 2007

user forum:~30 sessions100+ presentations20 demos~60 posters

Co-located with OGF20:

900+ attendees~50 booths

Inauguration de l'Institut des Grilles, Paris, 3rd December 2007


Registered collaborating projects

Infrastructures

geographical or thematic coverage

Support Actions

key complementary functions

Applications

improved services for academia, industry and the public

Registered Collaborating Projects

25 projects have registered as of September 2007:web page

Inauguration de l'Institut des Grilles, Paris, 3rd December 2007


Collaborating infrastructures

Collaborating infrastructures

Inauguration de l'Institut des Grilles, Paris, 3rd December 2007


Egee ii to egee iii

EGEE-II to EGEE-III

  • EGEE-III proposal currently under negotiation with European Commission

  • Key objectives

    • Expand/optimise existing EGEE infrastructure, include more resources and user communities

    • Prepare migration from a project-based model to a sustainable federated infrastructure based on National Grid Initiatives

  • 2 year period – spring 2008 to spring 2010

    • No gap between EGEE-II and EGEE-III

  • Similar consortium

    • Now structured on a national basis (National Grid Initiatives/Joint Research Units)

Inauguration de l'Institut des Grilles, Paris, 3rd December 2007


European grid initiative

European Grid Initiative

  • Need to prepare permanent, common Grid infrastructure

  • Ensure the long-term sustainability of the European e-Infrastructure independent of short project funding cycles

  • Coordinate the integration and interaction between National Grid Infrastructures (NGIs)

  • Operate the production Grid infrastructure on a European level for a wide range of scientific disciplines

Must be no gap in the support of the production grid

Inauguration de l'Institut des Grilles, Paris, 3rd December 2007


The egee project and the future of european grids

http://www.eu-egi.org

  • EGI Design Study proposal approved to the European Commission (started 1st September’07)

  • Supported by 30+ National Grid Initiatives (NGIs)

  • 2 year project to prepare the setup and operation of a new organizational model for a sustainable pan-European grid infrastructure

  • Federated model bringing together NGIs to build a European organisation

  • Well defined, complimentary responsibilities between NGIs and EGI

Inauguration de l'Institut des Grilles, Paris, 3rd December 2007


Characteristics of ngis

Characteristics of NGIs

Each NGI

  • recognized national body with a single point-of-contact

  • mobilise national funding and resources

  • operate the national e-Infrastructure

  • support user communities (application independent, and open to new user communities and resource providers)

  • contribute and adhere to international standards and policies

    Responsibilities between NGIs and EGI are clearly

    separated and complementary

Inauguration de l'Institut des Grilles, Paris, 3rd December 2007


The egee project and the future of european grids

37 European NGIs

+ Asia, US, Latin America

+ PRACE

+ OGF-Europe

+ …

Inauguration de l'Institut des Grilles, Paris, 3rd December 2007


Why a grid institute

Why a « Grid Institute »

  • Considerable importance of Grids related activity within CNRS

  • Federate all activities in CNRS related to research on Grids, grids for research and production grids

    • Better visibility

    • Meilleure efficiency

    • Strengthen the links between these domains

  • Provide a single well identified Point of contact for national and international collaborations

    • CNRS representative for all european contracts, for disucssion within french Ministry

    • central core for the emerging French « National Grid Initiative »

    • Parternship with all major French reserach organisms, CEA, CNES, INRIA, …

  • Outreach activities, evangelisation of new scientific communities, training,..

Inauguration de l'Institut des Grilles, Paris, 3rd December 2007


Institut des grilles composition

Institut des Grilles composition

  • 30 laboratories:

    APC, CC_IN2P3, CPPM, CREATIS, LIP, I3S, IBCP,IN2P3_adm, IPGP, IPHC, IPNL, IPNO, IRISA, IRIT, LABRI, LAL, LAPP, LIFL, LIG, LIP6, LLR, LORIA, LPC Clermont, LPNHE, LRI, IPSL, LPSC, LSIT, Subatech, UREC

    • 13 IN2P3 labs linked to EGEE/LCG

    • 11 computing science labs

    • 5 labs linked to various applications

    • Administrative support

  • GDR Architecture Systèmes et Réseaux (ASR)

  • Total of 310 people

Inauguration de l'Institut des Grilles, Paris, 3rd December 2007


Idg objectives and means

IdG objectives and means

  • Scientific animation

    • Organization of the national prospective on the needs of the scientific community related to Grids

    • Call for proposals

  • Dialog Forum between Grilles production grids and research grids

    • Interoperability GRID5000/EGEE

    • Grids Observatory

    • Middleware of the future

  • Training

  • Communication

Inauguration de l'Institut des Grilles, Paris, 3rd December 2007


The national prospective working groups

The national prospective working groups

  • Thematic working groups

    • Planet and Universe sciences; environmental sciences

    • Life sciences

    • Human sciences

    • Chemistry

    • Physics

    • Engineering and computing sciences

    • Subatomic physics (including astroparticles)

  • Transverse working groups

    • Data grids

    • Grids and supercomputers

    • Regional grids

    • Grids and very large research infrastructures

    • Grids and the user

    • Relationship with industry

Inauguration de l'Institut des Grilles, Paris, 3rd December 2007


Summary

Summary

  • Grids are all about sharing – they are a means of working with groups around the world

    • Today we have a window of opportunity to move grids from research prototypes to permanent production systems (as networks did a few years ago)

  • Interoperability is key to providing the level of support required for our user communities

  • EGEE operates the world’s largest multi-disciplinary grid infrastructure for scientific research

    • In constant and significant production use

  • A third phase of EGEE is under preparation

  • Need to prepare the long-term

    • EGEE, collaborating projects, national grid initiatives and user communities are working to define a model for a sustainable grid infrastructure that is independent of short project cycles

  • In France, creation of the CNRS Grid Institute as the nucleus of the French NGI to help implementing this future!

  • Bright opportunities for grids and astroparticles!

www.eu-egee.org

Inauguration de l'Institut des Grilles, Paris, 3rd December 2007


  • Login