Distributed processing
Sponsored Links
This presentation is the property of its rightful owner.
1 / 12

Distributed Processing PowerPoint PPT Presentation

  • Uploaded on
  • Presentation posted in: General

Distributed Processing. Craig E. Tull HCG/NERSC/LBNL (US) ATLAS Grid Software Workshop BNL - May 7, 2002. Distributed Processing. ATLAS distributed processing, PPDG year 2 program role of MOP, other middleware & third party tools objectives: deliverables to users

Download Presentation

Distributed Processing

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript

Distributed processing

Distributed Processing

Craig E. Tull


(US) ATLAS Grid Software Workshop

BNL - May 7, 2002

Distributed processing1

Distributed Processing

  • ATLAS distributed processing, PPDG year 2 program

  • role of MOP, other middleware & third party tools

  • objectives: deliverables to users

  • job description language: status, objectives, options

    • EDG JDL

Distributed processing


Local Application

Local Database

Local Computing

Grid Application Layer


Job Management

Data Management

Metadata Management

Object to File Mapper

Collective Services

Information & Monitoring

Replica Manager

Grid Scheduler

Replica Catalog Interface

Replica Optimization

Underlying Grid Services

SQL Database Service

Computing Element Services

Storage Element Services

Replica Catalog

Authorisation, Authentication and Accounting

Service Index


Fabric services




Fault Tolerance


Installation &


Fabric Storage


Resource Management



pink: WP1yellow:WP2

Jul 01 pseudocode for atlas short term uc01


Logical File Name

LFN = "lfn://"hostname"/"any_string

Physical File Name

PFN = "pfn://"hostname"/"path

Transfer File Name

TFN = "gridftp://"PFN_hostname"/path


InputData = {LFN[]}

OutputSE = host.domain.name

Worker Node

LFN[] = WP1.LFNList()

for (i=0;i<LFN.list;i++){

PFN[] = ReplicaCatalog.getPhysicalFileNames(LFN[i])

j = Athena.eventSelectonSrv.determineClosestPF(PFN[])

localFile = GDMP.makeLocal(PFN[j],OutputSE)



PFN[] = getPhysicalFileNames(LFN):

PFN = getBestPhysicalFileName(PFN[], String[] protocols)

TFN = getTransportFileName(PFN, String protocol)

filename = getPosixFileName(TFN)

Sample use case simple grid job

Sample Use Case: Simple Grid Job

  • Submit and run a simple batch job to process one input file to produce one output file

  • The user specifies his job via a JDL file:


    Requirements = TS >= 1GB

    Input.LFN = lfn://atlas.hep/foo.in

    argv1 = TFN(Input.LFN)

    Output.LFN= lfn://atlas.hep/foo.out

    Output.SE = datastore.rl.ac.uk

    argv2 = TFN(Output.LFN)

  • and where the submitted “job” is:


    gridcp $1 $HOME/tmp1

    grep higgs $HOME/tmp1 > $HOME/tmp2

    gridcp $HOME/tmp2 $2

Steps for simple job example

Get LFN to SFN mapping

send job

job done

copy input file, allocate output file

start job

copy output file

Steps for Simple Job Example

Grid Scheduler

Replica Manager


Select CE and SE











site A

site B

Steps to execute this simple grid job

Steps to Executethis Simple Grid Job

  • User submits the job to the Grid Scheduler.

  • Grid Scheduler asks the Replica Manager for list of all PFNs for the specified input file.

  • Grid Scheduler determines if it is possible to run the job at a Compute Element that is “local” to one of the PFNs.

    • If not, it locates the best CE for the job, and creates a new replica of the input file on a SE local to that CE.

  • Grid Scheduler then allocates space for the output file, and “pins” the input file so that is not deleted or staged to tape until after the job has completed.

  • Then the job is submitted to the CEs job queue.

  • When the Grid Scheduler is notified that the job has completed, it tells the Replica Manager to create a copy of the output file at the site specified in the Job Descriptions file.

  • Replica Manager then will tag this copy of the output file the “master”, and make the original file a “replica”.

Wp1 job status

WP1: Job Status

  • SUBMITTED -- The user has submitted the job to the User Interface.

  • WAITING -- The Resource Broker has received the job.

  • READY -- A Computing Element matching job requirements has been selected.

  • SCHEDULED -- The Computing Element has received the job.

  • RUNNING -- The job is running on a Computing Element.

  • CHKPT -- The job has been suspended and check-pointed on a Computing Element.

  • DONE -- The execution of the job has completed.

  • ABORTED -- The job has been terminated.

  • CLEARED -- The user has retrieved all output files successfully. Bookkeeping information is purged some time after the job enters this state.

Wp1 job submission service jss

WP1: Job Submission Service (JSS)

  • strictly coupled with a Resource Broker

    • deployed for each installed RB

  • single interface (non-blocking), used by the RB

    • job_submit()

      • submit a job to the specified Computing Element, managing also input and output sandboxes

    • job_cancel()

      • kill a list of jobs, identified by their dg_jobId.

  • Logging and Bookkeeping (LB) Service - store & manage logging and bookkeeping information generated by Scheduler & JSS components (Information and Monitoring service

    • Bookkeeping: currently active jobs - job definition, expressed in JDL, status, resource consumption, user-defined data(?)

    • Logging - status of the Grid Scheduler & related components. These data are kept for a longer term and are used mainly for debugging, auditing and statistical purposes

Wp1 job description language jdl

Condor classified advertisements (ClassAds) adopted as Job Description Language (JDL)

Semi-structured data model: no specific schema is required.

Symmetry: all entities in the Grid, in particular applications and computing resources, should be expressible in the same language.

Simplicity: the description language should be simple both syntactically and semantically.

Executable = “simula”;

Arguments = “1 2 3”;

StdInput = “simula.config”;

StdOutput = “simula.out”;

StdError = “simula.err”;

InputSandbox = {“/home/joe/simula.config”, “/usr/local/bin/simula”};

OutputSandbox = {“simula.out”, “simula.err”, “core”};

InputData = “LF:test367-2”;

Replica Catalog = “ldap://pcrc.cern.ch:2010/rc=Replica Catalog, dc=pcrc, dc=cern, dc=ch”

DataAccessProtocol = {“file”, “gridftp”};

OutputSE = “lxde01.pd.infn.it”;

Requirements = other.Architecture == “INTEL” && other.OpSys == “LINUX”;

Rank = other.AverageSI00;

WP1: Job Description Language (JDL)

Wp1 sandbox

WP1: Sandbox

  • Working area (input & output) replicated on each CE to which Grid job is submitted.

    • Very convenient & natural.

  • My Concerns:

    • Requires network access (with associated privileges) to all CEs on Grid.

      • Could be a huge security issue with local administrators.

    • Not (yet) coordinated with WP2 services.

    • Sandbox contents not customizable to local (CE/SE/PFN) environment.

    • Temptation to Abuse (not for data files)

Edg jdl


  • job description language: status, objectives, options

  • Status:

    • Working in EDG testbed

  • Objectives:

    • Provide WP1 Scheduler enough information to locate necessary resources (CE, SE, data, software) to execute job.

  • Options:

  • Login