grid infrastructure related projects n.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
Grid Infrastructure & Related Projects PowerPoint Presentation
Download Presentation
Grid Infrastructure & Related Projects

Loading in 2 Seconds...

play fullscreen
1 / 41

Grid Infrastructure & Related Projects - PowerPoint PPT Presentation


  • 74 Views
  • Uploaded on

Grid Infrastructure & Related Projects. Eddie Aronovich Tel-Aviv University, School of CS Eddie.Aronovich@cs.tau.ac.il. Acknowledgements. Presentation is based on slides from: Roberto Barbera, University of Catania and INFN (EGEE Tutorial Roma, 02.11.2005)

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Grid Infrastructure & Related Projects' - margie


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
grid infrastructure related projects

Grid Infrastructure & Related Projects

Eddie Aronovich

Tel-Aviv University, School of CS

Eddie.Aronovich@cs.tau.ac.il

acknowledgements
Acknowledgements
  • Presentation is based on slides from:
    • Roberto Barbera, University of Catania and INFN (EGEE Tutorial Roma, 02.11.2005)
    • Mike Mineter, Concepts of grid computing
    • Fabrizio Gagliardi, EGEE Project Director, CERN, Geneva, Switzerland (Naregi Symposium 2005 – Tokyo)
    • Fabrizio Gagliardi, EGEE Project Director, CERN, Geneva, Switzerland (APAC, 27 September 2005)
    • Guy Warner, NeSC Training Team (An Induction to EGEE for GOSC and the NGS NeSC, 8th December 2004 )
    • http://www.beingrid.eu

Eddie Aronovich – CS_IAP, Apr 2007

slide3
What is a grid ?
  • EGEE project
  • Connectivity with other projects
  • How a job is submitted
  • A few more words

Eddie Aronovich – CS_IAP, Apr 2007

when the grid can do
When the Grid can do ?
  • Coordinates Distributed Resources
  • Open for connectivity (using standard, open, general-purpose protocols)
  • Deliver non-trivial qualities of service

Eddie Aronovich – CS_IAP, Apr 2007

a grid

INTERNET

A grid
  • Grid middleware runs on each shared resource
    • Data storage
    • (Usually) batch jobs on pools of processors
  • Users join VO’s
  • Virtual organisation negotiates with sites to agree access to resources
  • Distributed services (both people and middleware) enable the grid

Eddie Aronovich – CS_IAP, Apr 2007

slide6
What is a grid ?
  • EGEE project
  • How a job is submitted
  • A few more words

Eddie Aronovich – CS_IAP, Apr 2007

egee project in 1k words
EGEE project in 1K words

https://goc.grid-support.ac.uk/gridsite/monitoring/

Eddie Aronovich – CS_IAP, Apr 2007

30 grid projects @ egee 06
30+ Grid Projects @ EGEE’06

Eddie Aronovich – CS_IAP, Apr 2007

20 businesses @ egee06
20 + Businesses @ EGEE06

Capitalising on e-Science to make e-Business

Eddie Aronovich – CS_IAP, Apr 2007

related eu projects

EU

GRID

Related EU projects

ISSeG

Eddie Aronovich – CS_IAP, Apr 2007

egee as partner
EGEE as partner
  • Ongoing collaborations
    • with non EU partners in EGEE: US, Israel, Russia, Korea, Taiwan…
    • with other European infrastructure projects:
      • GÉANT
      • DEISA
      • SEE-GRID
    • with other European grid projects
      • DILIGENT
      • GRIDCC
    • with non-European projects:
      • OSG: OpenScienceGrid (USA)
      • NAREGI (Japan)
  • EGEE as incubator
    • >10 related projects have been created

Eddie Aronovich – CS_IAP, Apr 2007

collaborating e infrastructures
Collaborating e-Infrastructures

TWGRID

Potential for linking ~80 countries

Eddie Aronovich – CS_IAP, Apr 2007

e infrastructure
e-Infrastructure
  • implementation blocks

From a talk by Mario Campolargo, Brussels, 30 May 2005

Eddie Aronovich – CS_IAP, Apr 2007

support for digital repositories

GRID .

INFRASTRUCTURE

KNOWLEDGE .

INFRASTRUCTURE

GÉANT .

INFRASTRUCTURE

Support for digital repositories

3 layered model to support access to heterogeneousinformation and connect resources throughcommon shared services

Grids for digital repositories:

  • Sharing of resources
  • Secure Access Control
  • Data management
  • Execution of computationally demanding applications (e.g. multi-media content)

Eddie Aronovich – CS_IAP, Apr 2007

slide15
What is a grid ?
  • EGEE project
  • Connectivity with other projects
  • How a job is submitted
  • A few more words

Eddie Aronovich – CS_IAP, Apr 2007

jobs life major components

Input “sandbox”

DataSets info

Output “sandbox”

SE & CE info

Job Submit Event

Job Query

Publish

Job Status

Storage

Element

Jobs life (major components)

Replica

Catalogue

“User interface”

Information

Service

Resource

Broker

Author.

&Authen.

Input “sandbox” + Broker Info

Output “sandbox”

Logging &

Book-keeping

Computing

Element

Job Status

Eddie Aronovich – CS_IAP, Apr 2007

slide17

UI

RB/WMS

node

Replica

Location

Server

Network

Server

Workload

Manager

Inform.

Service

Job Contr.

Characts.

& status

Computing

Element

Storage

Element

Eddie Aronovich – CS_IAP, Apr 2007

slide18

Job

Status

UI

RB node

submitted

Replica

Location

Server

Network

Server

Workload

Manager

Inform.

Service

UI: allows users to

access the functionalities

of the WMS

(via command line, GUI,

C++ and Java APIs)

Job Contr.

-

CondorG

CE characts

& status

SE characts

& status

Computing

Element

Storage

Element

Eddie Aronovich – CS_IAP, Apr 2007

slide19

edg-job-submit myjob.jdl

Myjob.jdl

JobType = “Normal”;

Executable = "$(CMS)/exe/sum.exe";

InputSandbox = {"/home/user/WP1testC","/home/file*”, "/home/user/DATA/*"};

OutputSandbox = {“sim.err”, “test.out”, “sim.log"};

Requirements = other. GlueHostOperatingSystemName == “linux" &&

other. GlueHostOperatingSystemRelease == "Red Hat 7.3“ && other.GlueCEPolicyMaxCPUTime > 10000;

Rank = other.GlueCEStateFreeCPUs;

Job

Status

UI

RB node

submitted

Replica

Location

Server

Network

Server

Workload

Manager

Inform.

Service

Job Contr.

-

CondorG

CE characts

& status

SE characts

& status

Job Description Language

(JDL) to specify job

characteristics and

requirements

Computing

Element

Storage

Element

Eddie Aronovich – CS_IAP, Apr 2007

slide20

submitted

waiting

UI

NS: network daemon

responsible for accepting

incoming requests

RB node

Job Status

Replica

Location

Server

Network

Server

Job

Input

Sandbox

files

Workload

Manager

Inform.

Service

RB

storage

Job Contr.

-

CondorG

CE characts

& status

SE characts

& status

Computing

Element

Storage

Element

Eddie Aronovich – CS_IAP, Apr 2007

slide21

submitted

waiting

UI

RB node

Job Status

Replica

Location

Server

Network

Server

Job

Workload manager

Inform.

Service

RB

storage

WM: acts to

satisfy the request

Job Contr.

-

CondorG

CE characts

& status

SE characts

& status

Computing

Element

Storage

Element

Eddie Aronovich – CS_IAP, Apr 2007

slide22

Job Status

submitted

waiting

UI

RB node

Replica

Location

Server

Network

Server

Match-

Maker/

Broker

Workload

Manager

Inform.

Service

RB

storage

Where must this

job be

executed ?

Job Contr.

-

CondorG

CE characts

& status

SE characts

& status

Computing

Element

Storage

Element

Eddie Aronovich – CS_IAP, Apr 2007

slide23

submitted

waiting

UI

RB node

Job Status

Matchmaker: responsible

to find the “best” CE

for a job

Replica

Location

Server

Network

Server

Match-

Maker/

Broker

Workload

Manager

Inform.

Service

RB

storage

Job Contr.

-

CondorG

CE characts

& status

SE characts

& status

Computing

Element

Storage

Element

Eddie Aronovich – CS_IAP, Apr 2007

slide24

Where are (which SEs)

the needed data ?

submitted

waiting

UI

RB node

Job Status

Replica

Location

Server

Network

Server

Match-

Maker/

Broker

Workload

Manager

Inform.

Service

RB

storage

What is the

status of the

Grid ?

Job Contr.

-

CondorG

CE characts

& status

SE characts

& status

Computing

Element

Storage

Element

Eddie Aronovich – CS_IAP, Apr 2007

slide25

submitted

waiting

UI

RB node

Job Status

Replica

Location

Server

Network

Server

Match-

Maker/

Broker

Workload

Manager

Inform.

Service

RB

storage

CE choice

Job Contr.

-

CondorG

CE characts

& status

SE characts

& status

Computing

Element

Storage

Element

Eddie Aronovich – CS_IAP, Apr 2007

slide26

submitted

waiting

UI

RB node

Job Status

Replica

Location

Server

Network

Server

Workload

Manager

Inform.

Service

RB

storage

Job

Adapter

Job Contr.

-

CondorG

Job Adapter: responsible for the final “touches”

to the job before performing submission

(e.g. creation of wrapper script, PFN, etc.)

CE characts

& status

SE characts

& status

Computing

Element

Storage

Element

Eddie Aronovich – CS_IAP, Apr 2007

slide27

submitted

waiting

UI

ready

RB node

Job Status

Replica

Location

Server

Network

Server

Workload

Manager

Inform.

Service

RB

storage

Job

Job Contr.

CE characts

& status

Job Controller: responsible for the

actual job management

operations (done via

CondorG)

SE characts

& status

Computing

Element

Storage

Element

Eddie Aronovich – CS_IAP, Apr 2007

slide28

submitted

waiting

UI

ready

scheduled

RB node

Job Status

Replica

Location

Server

Network

Server

Workload

Manager

Inform.

Service

RB

storage

Job Contr.

-

CondorG

CE characts

& status

SE characts

& status

Job

Computing

Element

Storage

Element

Eddie Aronovich – CS_IAP, Apr 2007

slide29

submitted

waiting

UI

ready

scheduled

running

Job

RB node

Job Status

Replica

Location

Server

Network

Server

Workload

Manager

Inform.

Service

RB

storage

Job Contr.

-

CondorG

Input

Sandbox

files

“Grid enabled”

data transfers/

accesses

Storage

Element

Computing

Element

Eddie Aronovich – CS_IAP, Apr 2007

slide30

submitted

waiting

UI

ready

scheduled

running

done

RB node

Job Status

Replica

Location

Server

Network

Server

Workload

Manager

Inform.

Service

RB

storage

Job Contr.

-

CondorG

Output

Sandbox

files

Computing

Element

Storage

Element

Eddie Aronovich – CS_IAP, Apr 2007

slide31

submitted

waiting

UI

ready

scheduled

running

done

RB node

Job Status

edg-job-get-output <dg-job-id>

Replica

Location

Server

Network

Server

Workload

Manager

Inform.

Service

RB

storage

Job Contr.

-

CondorG

Computing

Element

Storage

Element

Eddie Aronovich – CS_IAP, Apr 2007

slide32

UI

RB node

Job Status

submitted

Replica

Location

Server

Network

Server

waiting

RB

storage

ready

Workload

Manager

Output

Sandbox

files

Inform.

Service

scheduled

Job Contr.

-

CondorG

running

done

cleared

Computing

Element

Storage

Element

Eddie Aronovich – CS_IAP, Apr 2007

slide33

UI

RB node

edg-job-status <dg-job-id>

edg-job-get-logging-info <dg-job-id>

Network

Server

LB: receives and stores

job events; processes

corresponding job status

Workload

Manager

Job

status

Logging &

Bookkeeping

Job Contr.

-

CondorG

Log

Monitor

Log of

job events

LM: parses CondorG log

file (where CondorG logs

info about jobs) and notifies LB

Computing

Element

Eddie Aronovich – CS_IAP, Apr 2007

what about security

No Cross-

Domain Trust

Certification

Certification

Authority

Authority

Policy

Policy

Authority

Authority

Sub-Domain B1

Sub-Domain A1

Domain A

Domain B

Task

Federation

Service

GSI

Server X

Server Y

Virtual

Organization

Domain

What about security ?

Eddie Aronovich – CS_IAP, Apr 2007

slide35
What is a grid ?
  • EGEE project
  • Connectivity with other projects
  • How a job is submitted
  • A few more words

Eddie Aronovich – CS_IAP, Apr 2007

used cpu time this year
Used CPU time this year

Eddie Aronovich – CS_IAP, Apr 2007

over the chasm
Over the Chasm

Eddie Aronovich – CS_IAP, Apr 2007

applications in egee
Applications in EGEE
  • Astrophysics - MAGIC, Planck
  • Computational Chemistry
  • Earth Sciences - Earth Observation, Solid Earth Physics, Hydrology, Climate
  • Financial Simulation - E-GRID
  • Fusion
  • Geophysics - EGEODE
  • High Energy Physics - LHC (ALICE, ATLAS, CMS, LHCb) BaBar, CDF, DØ, ZEUS
  • Life Sciences - Bioinformatics (Drug Discovery, GPS@, Xmipp_MLrefine, etc.)
  • Medical imaging (GATE, CDSS, gPTM3D, SiMRI 3D, etc.)
  • Multimedia

press for more details

Eddie Aronovich – CS_IAP, Apr 2007

steps to become grid production
Steps to become grid production
  • Contact IAGneeded contribution:
    • 30CPUs or cores
    • 1TB of storage attached to the processosrs
    • 4 computers for operational activities
  • Plan your site
  • Get a certificates
  • Install the site
  • Certify the site….

Eddie Aronovich – CS_IAP, Apr 2007

contacts
Contacts
  • Israeli Academic Grid (IAG)

http://iag.iucc.ac.il/

  • EGEE Website

http://www.eu-egee.org

  • Grid Demos

https://glite-demo.ct.infn.it/

Eddie Aronovich – CS_IAP, Apr 2007