group metting submit job to grid n.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
Group Metting Submit job to Grid PowerPoint Presentation
Download Presentation
Group Metting Submit job to Grid

Loading in 2 Seconds...

play fullscreen
1 / 20

Group Metting Submit job to Grid - PowerPoint PPT Presentation


  • 70 Views
  • Uploaded on

Group Metting Submit job to Grid. 20070704 Grass. outline. Why we need Grid CPUs example : How to run TPCana Before run TPCana. Why we need Grid CPUs. IPAS Condor. LHC Taiwan Tier-1. example : How to run TPCana. example for setup grid environment. STEP1: copy proxy file.

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Group Metting Submit job to Grid' - rianna


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
outline
outline
  • Why we need Grid CPUs
  • example : How to run TPCana
  • Before run TPCana
example for setup grid environment
example for setup grid environment

STEP1: copy proxy file

[grass@hepfarm2 ~]$ ls -l /usrX/grass/.globus/x509up_grass

-rw------- 1 grass spring8 5562 Jul 2 16:26 /usrX/grass/.globus/x509up_grass

\> chmod 600 x509up_grass

STEP2 setup grid environment

## grid environment

\> source /scratch/grid/glite-3.0.26/etc/profile.d/grid_env.csh

\> setenv X509_USER_PROXY /usrX/grass/.globus/x509up_grass

generate drift table online
generate drift table online

Drift table production time

generate drift table 5 times, we get the same drift table

the same size

no difference

upload data
upload data

rawdata size

size of others data is small

manual of globus submit
manual of “globus_submit”

usage: globus_submit --tarFile=file --output=file --start=num --end=num ...

--tarFile=file: path for tar file to be submitted e.g. ./submitme.tgz

--outputFile=file: path for output file to be stored e.g. ./cronus.$.tgz

--cmd=command: command to be executed, normally a shell script e.g. ./cronus.sh $

--start=num: beginning segment number e.g. 1

--end=num: ending segment number e.g. 100

--segments=num: segment number list e.g. 2-5,7,10

--VO=vo: VO name you belong to e.g. twgrid

--verbose: verbose output

--cronus: submit to Cronus, it's necessary in ipas003

--test: do not submit, show JDL to standard output

--email: email address for summary output e.g. tsan@fnal.gov

--globusscheduler: gatekeeper you wan to submit to e.g. lcg00125.grid.sinica.edu.tw:2119/jobmanager-lcgpbs

--globusrsl: additional options to gatekeeper

special character, $, will be replaced with segment number for --outputFile and --cmd

globus submit
globus_submit
  • globus_submit

--tarFile=../submitme.tgz

--outputFile='tpcana.$.tgz'

--cmd='./runTPC.sh /hepdata4/jychen/TPC/data/2004Oct17 $'

--segments=30525-30527,30529

--globusscheduler=lcg00125.grid.sinica.edu.tw:2119/jobmanager-lcgpbs

--verbose --globusrsl='(queue=twgrid)'

grid copy rate
grid copy rate

upload 3.1G file need 85 second

must of file small then 3G

submit 27 run at 23 00
submit 27 run at 23:00

raid: /hepdata4

file server: asscratch2

what we need to do later
What we need to do later
  • Test the best number of upload files in the same time in our file server.(Tsanlung’s suggestion is 16 runs one time )
  • Write a perl split to submit job.
grid copy rate1
grid copy rate

upload 1.2G file need 40 second

must of file small then 3G