Download
grid european scientific computing infrastructure n.
Skip this Video
Loading SlideShow in 5 Seconds..
GRID (European Scientific Computing infrastructure) PowerPoint Presentation
Download Presentation
GRID (European Scientific Computing infrastructure)

GRID (European Scientific Computing infrastructure)

257 Views Download Presentation
Download Presentation

GRID (European Scientific Computing infrastructure)

- - - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript

  1. GRID(European Scientific Computing infrastructure) .

  2. CERN CERN currently comprises 20 European Member States: Austria, Belgium, Bulgaria, the Czech Republic, Denmark, Finland, France, Germany, Greece, Hungary, Italy, The Netherlands, Norway, Poland, Portugal, the Slovak Republic, Spain, Sweden, Switzerland and the UnitedKingdom. Observer and Non-member States: India, Israel, Japan, the Russian Federation, the United States of America, Turkey, Algeria, Argentina, Armenia, Australia, Azerbaijan, Belarus, Brazil, Canada, China, Croatia, Cyprus, Estonia, Georgia, Iceland, India, Iran, Ireland, Mexico, Morocco, Pakistan, Peru, Romania, Serbia, Slovenia, South Africa, South Korea, Taiwan and the Ukraine. Latvija, atšķirībā nn Igaunijas, nav CERN dalībvalsts  studenti no Latvijas nevar vasarās tur praktizēties...

  3. CERN: Tim Berbers Lee invented HTTP (WWW) 1990, 1993

  4. CERN LHC An image of one of the first lead-ion collisions seen by the ALICE experiment on 7 November, 2010.

  5. Data visualisation of a particle collision inside the Large Hadron Collider, recorded by the ALICE experiment.

  6. LHC Atlas: search for Higgs bosonGreatest Mysteries: What Causes Gravity? • E=mc2 ; v=g * t ; • "Much of today's research in elementary particle physics focuses on the search for a particle called the Higgs boson. This particle is the one missing piece of our present understanding of the laws of nature, known as the Standard Model. This model describes three types of forces: electromagnetic interactions, strong interactions, which bind atomic nuclei; and the weak nuclear force, which governs beta decay. (The Standard Model does not describe the fourth force, gravity.) • “Over the next 15 years, we should begin to find a real understanding of the origin of mass.” • "Newton thought that gravity's force was instantaneous. Einstein assumed that it moved at the speed of light, but until now, no one had measured it," • General relativity (GR) suggests that gravitation (unlike electromagnetic forces) is a pure geometric effect of curved space-time, not a force of nature that propagates. • Most scientists assume that gravity travels at the speed of light and there exist gravitational waves and gravitational radiation (single positive experemnt in 2003) • I put my last dollar, the Higgs shall not be found, just as Gravity waves in the Einstein sense won't be found. • I agree with you. But all of the public tax money is on one side of the issue! • The Higgs is more like a plumber with duct tape, holding the standard model together

  7. The ‘Standard Model’ of Particle Physics Tested by experiments at CERN & elsewhere Perfect agreement between theory and experiments in all laboratories ProposedbyAbdus Salam, Glashow and Weinberg

  8. Open Questions beyond the Standard Model What is the origin of particle masses? due to a Higgs boson? Why so many flavours of matter particles? What is the dark matter in the Universe? Unification of fundamental forces? Quantum theory of gravity? LHC LHC LHC LHC LHC

  9. Where do the masses come from ? Why do Things Weigh? Newton: Weight proportional toMass Einstein: Energy related toMass Neither explained origin of Mass 0 Are masses due to Higgs boson? (the physicists’ Holy Grail)

  10. Has the Higgs Boson been Discovered? Interesting hints around Mh = 125 GeV ? CMS sees broad enhancement ATLAS prefers 125 GeV

  11. CERN,100m zem zemes, Atlas • 2007. gadā

  12. CERN Trubā, gar kuru grābstos, protoni (jonizēti ūdeņraža atomi) skrien ar 99,9999% no gaismas ātruma (tikai 3m/s lēnāk par gaismas ātrumu 300000km/s). • 2017. gadā

  13. A global, federated e-Infrastructure BalticGrid NAREGI SEE-GRID OSG EUChinaGrid EGEE infrastructure ~ 200 sites in 39 countries ~ 20 000 CPUs > 5 PB storage > 10 000 concurrent jobs per day > 60 Virtual Organisations EUMedGrid EUIndiaGrid EELA

  14. 98k jobs/day Scale of EGEE Production Service

  15. Grid sertifikāta iegūšana • Iegūt apstiprinātu BalticGrid sertifikātu – pirmais solis ceļā uz Grid izmantošanu • Informācija: http://grid.lumii.lv/section/show/12 • Domain of the Institution (domain.zz):lumii.lv • Common Name (John Smith):Janis Berzins

  16. Certification Procedure

  17. Creating a Certification Request

  18. BalticGridCA-user.cnf # # OpenSSL configuration file for generating certificate requests for Baltic Grid CA. # # This definition stops the following lines choking if HOME isn't # defined. HOME = . ###RANDFILE = $ENV::HOME/.rnd [ req ] default_bits = 1024 default_keyfile = userkey.pem default_md = sha1 # which md to use. distinguished_name = req_distinguished_name string_mask = nombstr [ req_distinguished_name ] 0.domainComponent = Domain Component (org) 0.domainComponent_default = org 1.domainComponent = Domain Component (BalticGrid) 1.domainComponent_default = balticgrid organizationalUnitName = Domain of the Institution (domain.zz) commonName = Common Name (John Smith) commonName_max = 64

  19. -----BEGIN RSA PRIVATE KEY----- Proc-Type: 4,ENCRYPTED DEK-Info: DES-EDE3-CBC,C280CE744C634255 BrT3IotvrbcpTVeqKssGQnpx2dcnqqGIRb0Jt8pJEUjTX24IsdAg+LxOUEJ70y1a aXMgQmFyemRpbnMwgZ8wDQYJKoZIhvcNAQEBBQADgY0AMIGJAoGBANepPbidunic 4dq8iKj1eEDlicCZ51cKX43Hn17Ca+IKvS7cTBavbFicm6mkfNoCO+erZWL3nlrh GXuhUyCHZJctA9Fu37II3ik7SZe6LahCKu55ZrCP9bEXucvQ7giI2FUcgvjEcK/I 9+NnO+chkJwCTafa32SxZsG7MOnwv14XAgMBAAGgADANBgkqhkiG9w0BAQUFAAOB gQC8oV1AQv1jj2D3gb0aBUwA1CaVqJN+bq2wwmeQSP1+rJXicSlfpIEqI8TwoT6F vEt2EnPAtbXpWMjFtbuM816+tEdkrGLw0wfHdlTCwswcRtHn3QVl4jxA/wReb+CY GXuhUyCHZJctA9Fu37II3ik7SZe6LahCKu55ZrCP9bEXucvQ7giI2FUcgvjEcK/I 9+NnO+chkJwCTafa32SxZsG7MOnwv14XAgMBAAGgADANBgkqhkiG9w0BAQUFAAOB gQC8oV1AQv1jj2D3gb0aBUwA1CaVqJN+bq2wwmeQSP1+rJXicSlfpIEqI8TwoT6F vEt2EnPAtbXpWMjFtbuM816+tEdkrGLw0wfHdlTCwswcRtHn3QVl4jxA/wReb+CY l/OAjuw1hvqYG6ZY6n5zmxZsCnViLMIItW2NMJGBR43CrtJuUHly13hf3eTZiIZq GVjHrRPzj8GC6AOBzQ9KkG/Gcale4ALU1czmSIjwAABL1DNUc8nF/w== -----END RSA PRIVATE KEY----- Result -----BEGIN CERTIFICATE REQUEST----- MIIBnjCCAQcCAQAwXjETMBEGCgmSJomT8ixkARkWA29yZzEaMBgGCgmSJomT8ixk ARkWCmJhbHRpY2dyaWQxETAPBgNVBAsTCGx1bWlpLmx2MRgwFgYDVQQDEw9HdW50 aXMgQmFyemRpbnMwgZ8wDQYJKoZIhvcNAQEBBQADgY0AMIGJAoGBANepPbidunic 4dq8iKj1eEDlicCZ51cKX43Hn17Ca+IKvS7cTBavbFicm6mkfNoCO+erZWL3nlrh GXuhUyCHZJctA9Fu37II3ik7SZe6LahCKu55ZrCP9bEXucvQ7giI2FUcgvjEcK/I 9+NnO+chkJwCTafa32SxZsG7MOnwv14XAgMBAAGgADANBgkqhkiG9w0BAQUFAAOB gQC8oV1AQv1jj2D3gb0aBUwA1CaVqJN+bq2wwmeQSP1+rJXicSlfpIEqI8TwoT6F vEt2EnPAtbXpWMjFtbuM816+tEdkrGLw0wfHdlTCwswcRtHn3QVl4jxA/wReb+CY CSSIx0n3iP6KFP7PMzqLMiGm4jbUVoDiA6ZfKq1HAqPHig== -----END CERTIFICATE REQUEST-----

  20. Certificate: Data: Version: 3 (0x2) Serial Number: 13 (0xd) Signature Algorithm: sha1WithRSAEncryption Issuer: O=BalticGrid, CN=Baltic Grid Certification Authority Validity Not Before: Mar 24 12:30:32 2005 GMT Not After : Mar 24 12:30:32 2006 GMT Subject: O=BalticGrid, OU=latnet.lv, CN=Guntis Barzdins Subject Public Key Info: Public Key Algorithm: rsaEncryption RSA Public Key: (1024 bit) Modulus (1024 bit): 00:c1:54:28:7c:de:67:95:b0:7b:53:24:85:a1:c4: dd:b3:b3:12:b4:06:c4:b0:13:93:c0:5b:ad:2a:ad: 0a:8a:6c:d7:f3:c1:65:d5:1a:3f:f2:e8:ed:da:37: a0:52:e0:05:17:3f:ee:45:91:a8:07:8d:8f:7f:96: aa:fc:7c:4f:27:c6:fc:82:b8:89:54:42:60:ea:18: ff:fa:a4:1e:f7:00:22:66:b2:5b:bb:85:c9:a8:12: 87:f3:6f:96:c2:05:c8:a0:eb:9c:54:03:f1:05:c3: f4:27:ab:6b:30:47:dd:4b:12:b8:21:d9:25:fe:e6: 68:70:23:ae:35:15:80:b5:e7 Exponent: 65537 (0x10001) X509v3 extensions: X509v3 Basic Constraints: critical CA:FALSE X509v3 Key Usage: critical Digital Signature, Non Repudiation, Key Encipherment, Data Encipherment X509v3 Subject Key Identifier: B3:0B:DD:96:09:86:37:1F:CF:5D:D5:78:5B:6D:AB:6F:D0:BC:5A:24 X509v3 Authority Key Identifier: keyid:24:4E:75:31:6A:6C:DF:AA:4D:AD:C6:34:39:23:5F:18:DB:17:47:86 DirName:/O=BalticGrid/CN=Baltic Grid Certification Authority serial:00 X509v3 Certificate Policies: Policy: 1.3.6.1.4.1.19974.11.1.0.1 X509v3 Issuer Alternative Name: URI:http://grid.eenet.ee/BalticGridCA/ Signature Algorithm: sha1WithRSAEncryption 67:e8:50:7d:28:84:d7:cb:88:de:4a:14:da:f4:09:16:05:38: 4a:55:23:11:b5:87:77:05:7d:07:d8:1c:03:45:19:6f:6f:97: ef:7d:1b:c8:7f:29:98:c5:d8:35:cf:2e:2e:b2:16:7e:19:8c: 3c:32:79:2d:ed:9a:7b:50:e3:26:df:79:59:84:8f:c6:34:d4: 3a:c1:65:5b:79:2e:6e:eb:62:50:2f:0a:47:00:08:54:ee:54: 6d:91:9f:ff:58:f0:b5:79:aa:68:12:e9:2c:15:9d:06:41:3b: 3f:29:4b:ba:be:e1:ef:e1:aa:7c:83:5b:be:3a:e1:16:5f:02: 65:70:c6:7d:15:7b:e0:43:3e:f9:c1:b3:96:80:fb:a0:aa:a8: 83:79:0e:0b:87:b7:09:b6:60:6d:64:2c:de:de:c3:1c:4c:cc: e5:54:4c:33:26:d9:31:35:29:30:df:8b:7b:e6:a8:31:6e:a4: 57:ef:51:53:6c:df:7b:f6:6d:8e:d0:ad:ba:72:87:17:47:aa: d4:fa:ff:4d:d0:cc:45:a5:28:e5:a3:46:84:cf:c4:4b:94:f8: ba:27:b5:35:e3:79:f8:49:3d:90:b0:41:5d:71:e5:15:6c:25: d3:61:73:31:c8:c5:3d:5e:a1:68:fe:82:9a:4a:0f:ea:5b:13: b4:6a:be:be -----BEGIN CERTIFICATE----- MIIDdTCCAl2gAwIBAgIBDTANBgkqhkiG9w0BAQUFADBDMRMwEQYDVQQKEwpCYWx0 aWNHcmlkMSwwKgYDVQQDEyNCYWx0aWMgR3JpZCBDZXJ0aWZpY2F0aW9uIEF1dGhv cml0eTAeFw0wNTAzMjQxMjMwMzJaFw0wNjAzMjQxMjMwMzJaMEMxEzARBgNVBAoT yH8pmMXYNc8uLrIWfhmMPDJ5Le2ae1DjJt95WYSPxjTUOsFlW3kubutiUC8KRwAI VO5UbZGf/1jwtXmqaBLpLBWdBkE7PylLur7h7+GqfINbvjrhFl8CZXDGfRV74EM+ +cGzloD7oKqog3kOC4e3CbZgbWQs3t7DHEzM5VRMMybZMTUpMN+Le+aoMW6kV+9R U2zfe/ZtjtCtunKHF0eq1Pr/TdDMRaUo5aNGhM/ES5T4uie1NeN5+Ek9kLBBXXHl FWwl02FzMcjFPV6haP6CmkoP6lsTtGq+vg== -----END CERTIFICATE----- Sertifikāts

  21. Getting Started 1. Get a digital certificate Authentication – who you are http://grid.lumii.lv/section/show/12 2. Join a Virtual Organisation (VO) For LHC join LCG and choose a VO Authorisation – what you are allowed to do https://voms.balticgrid.org:8443/voms/balticgrid/ 3. Get access to a local User Interface Machine (UI) and copy your files and certificate there

  22. Šis būs vajadzīgs • ssh zars.latnet.lv • User interface (sertificētie dabūs kontus) • openssl pkcs12 -export -in usercert.pem -inkey userkey.pem –outcertificate.p12 • Izveidot sertifikata kopiju MS Explorer derīgā formāta • http://winscp.net/download/winscp380.exe • Failu kopēšanai starp zars.latnet.lv un PC

  23. Job Preparation Prepare a file of Job Description Language (JDL): Script to run Job Options Input files My C++ Code ############# athena.jdl ################# Executable = "athena.sh"; StdOutput = "athena.out"; StdError = "athena.err"; InputSandbox = {"athena.sh", "MyJobOptions.py", "MyAlg.cxx", "MyAlg.h", "MyAlg_entries.cxx", "MyAlg_load.cxx", "login_requirements", "requirements", "Makefile"}; OutputSandbox = {"athena.out","athena.err", "ntuple.root", "histo.root", "CLIDDBout.txt"}; Requirements = Member("VO-atlas-release-9.0.4", other.GlueHostApplicationSoftwareRunTimeEnvironment); ################################################ Output Files Choose ATLAS Version (Satisfied by ~32 Sites)

  24. Job Submission Make a copy of your certificate to send out (~ once a day): [lloyd@lcgui ~/atlas]$ grid-proxy-init Your identity: /C=UK/O=eScience/OU=QueenMaryLondon/L=Physics/CN=steve lloyd Enter GRID pass phrase for this identity: Creating proxy .............................. Done Your proxy is valid until: Thu Mar 17 03:25:06 2005 [lloyd@lcgui ~/atlas]$ Submit the Job: VO File to hold job IDs JDL [lloyd@lcgui ~/atlas]$ edg-job-submit --vo atlas -o jobIDfile athena.jdl Selected Virtual Organisation name (from --vo option): atlas Connecting to host lxn1188.cern.ch, port 7772 Logging to host lxn1188.cern.ch, port 9002 ================================ edg-job-submit Success ==================================== The job has been successfully submitted to the Network Server. Use edg-job-status command to check job current status. Your job identifier (edg_jobId) is: - https://lxn1188.cern.ch:9000/0uDjtwbBbj8DTRetxYxoqQ The edg_jobId has been saved in the following file: /home/lloyd/atlas/jobIDfile ============================================================================================ [lloyd@lcgui ~/atlas]$

  25. Job Status [lloyd@lcgui ~/atlas]$ edg-job-status -i jobIDfile------------------------------------------------------------------1 : https://lxn1188.cern.ch:9000/tKlZHxqEhuroJUhuhEBtSA2 : https://lxn1188.cern.ch:9000/IJhkSObaAN5XDKBHPQLQyA3 : https://lxn1188.cern.ch:9000/BMEOq90zqALvkriHdVeN7A4 : https://lxn1188.cern.ch:9000/l6wist7SMq6jVePwQjHofg5 : https://lxn1188.cern.ch:9000/wHl9Yl_puz9hZDMe1OYRyQ6 : https://lxn1188.cern.ch:9000/PciXGNuAu7vZfcuWiGS3zQ7 : https://lxn1188.cern.ch:9000/0uDjtwbBbj8DTRetxYxoqQa : allq : quit------------------------------------------------------------------Choose one or more edg_jobId(s) in the list - [1-7]all:7*************************************************************BOOKKEEPING INFORMATION:Status info for the Job : https://lxn1188.cern.ch:9000/0uDjtwbBbj8DTRetxYxoqQCurrent Status: Done (Success)Exit code: 0Status Reason: Job terminated successfullyDestination: lcg00125.grid.sinica.edu.tw:2119/jobmanager-lcgpbs-shortreached on: Wed Mar 16 17:45:41 2005*************************************************************[lloyd@lcgui ~/atlas]$ Find out its status: Ran at: Valencia RAL CERN Taiwan Taiwan

  26. Job Retrieval Retrieve the Output: [lloyd@lcgui ~/atlas]$ edg-job-get-output -dir . -i jobIDfile Retrieving files from host: lxn1188.cern.ch ( for https://lxn1188.cern.ch:9000/0uDjtwbBbj8DTRetxYxoqQ ) ********************************************************************************* JOB GET OUTPUT OUTCOME Output sandbox files for the job: - https://lxn1188.cern.ch:9000/0uDjtwbBbj8DTRetxYxoqQ have been successfully retrieved and stored in the directory: /home/lloyd/atlas/lloyd_0uDjtwbBbj8DTRetxYxoqQ ********************************************************************************* [lloyd@lcgui ~/atlas]$ ls -lt /home/lloyd/atlas/lloyd_0uDjtwbBbj8DTRetxYxoqQ total 11024 -rw-r--r-- 1 lloyd hep 224 Mar 17 10:47 CLIDDBout.txt -rw-r--r-- 1 lloyd hep 69536 Mar 17 10:47 ntuple.root -rw-r--r-- 1 lloyd hep 5372 Mar 17 10:47 athena.err -rw-r--r-- 1 lloyd hep 11185282 Mar 17 10:47 athena.out

  27. Reāls pilns piemērs [guntisb@zars guntisb]$ tar -xvf tutor1.tar[guntisb@zars guntisb]$ cd job1[guntisb@zars job1]$ voms-proxy-initYour identity: /DC=org/DC=balticgrid/OU=lumii.lv/CN=Guntis BarzdinsEnter GRID pass phrase:Creating proxy ......................................................... DoneYour proxy is valid until Wed May 3 23:51:41 2006[guntisb@zars job1]$[guntisb@zars job1]$ edg-job-submit --vo balticgrid job1.jdlSelected Virtual Organisation name (from --vo option): balticgridConnecting to host grid3.mif.vu.lt, port 7772Logging to host grid3.mif.vu.lt, port 9002********************************************************************************************* JOB SUBMIT OUTCOME The job has been successfully submitted to the Network Server. Use edg-job-status command to check job current status. Your job identifier (edg_jobId) is: - https://grid3.mif.vu.lt:9000/NdAqueQcc5aARqLN0vwWng*********************************************************************************************[guntisb@zars job1]$ edg-job-status https://grid3.mif.vu.lt:9000/NdAqueQcc5aARqLN0vwWng*************************************************************BOOKKEEPING INFORMATION:Status info for the Job : https://grid3.mif.vu.lt:9000/NdAqueQcc5aARqLN0vwWngCurrent Status: Ready Status Reason: unavailableDestination: grid2.mif.vu.lt:2119/jobmanager-lcgpbs-balticgridreached on: Wed May 3 13:16:56 2006*************************************************************[guntisb@zars job1]$ edg-job-status https://grid3.mif.vu.lt:9000/NdAqueQcc5aARqLN0vwWng*************************************************************BOOKKEEPING INFORMATION:Status info for the Job : https://grid3.mif.vu.lt:9000/NdAqueQcc5aARqLN0vwWngCurrent Status: Done (Success)Exit code: 0Status Reason: Job terminated successfullyDestination: grid2.mif.vu.lt:2119/jobmanager-lcgpbs-balticgridreached on: Wed May 3 13:22:58 2006*************************************************************[guntisb@zars job1]$ edg-job-get-output https://grid3.mif.vu.lt:9000/NdAqueQcc5aARqLN0vwWngRetrieving files from host: grid3.mif.vu.lt ( for https://grid3.mif.vu.lt:9000/NdAqueQcc5aARqLN0vwWng )********************************************************************************* JOB GET OUTPUT OUTCOME Output sandbox files for the job: - https://grid3.mif.vu.lt:9000/NdAqueQcc5aARqLN0vwWng have been successfully retrieved and stored in the directory: /tmp/jobOutput/guntisb_NdAqueQcc5aARqLN0vwWng*********************************************************************************[guntisb@zars job1]$ mkdir abc[guntisb@zars job1]$ cp /tmp/jobOutput/guntisb_NdAqueQcc5aARqLN0vwWng/* abc/.[guntisb@zars job1]$ ls -al abctotal 12drwxrwxr-x 2 guntisb guntisb 4096 May 3 15:29 .drwxr-xr-x 3 guntisb guntisb 4096 May 3 15:29 ..-rw-rw-r-- 1 guntisb guntisb 0 May 3 15:29 stderr.log-rw-rw-r-- 1 guntisb guntisb 593 May 3 15:29 stdout.log[guntisb@zars job1]$ [guntisb@zars job1]$ more abc/stdout.log********************* Job Start *********************job1 started on Wed May 3 16:18:39 EEST 2006Executing on:Linux stud128.mif 2.4.32-grsec-i686-r1 #1 Št Sau 14 15:54:22 EET 2006 i686 GNU/LinuxCurrent working directory and contents:/home/bg035/globus-tmp.stud128.918.0/WMS_stud128_010893_https_3a_2f_2fgrid3.mif.vu.lt_3a9000_2fNdAqueQcc5aARqLN0vwWngtotal 8-rw-r--r-- 1 bg035 balticgrid 694 May 3 16:18 job1.sh-rw-r--r-- 1 bg035 balticgrid 0 May 3 16:18 stderr.log-rw-r--r-- 1 bg035 balticgrid 357 May 3 16:18 stdout.log********************** Job End **********************[guntisb@zars job1]$

  28. Pilni Grid uzdevumu paraugi • http://www.ltn.lv/~guntis/unix/tutor1.tar • http://www.ltn.lv/~guntis/unix/mpi.tgz • Papildinformācija • http://grid-it.cnaf.infn.it/index.php?jobsubmit&type=1 • Citi EGEE projekta dokumenti

  29. A global, federated e-Infrastructure BalticGrid NAREGI SEE-GRID OSG EUChinaGrid EGEE infrastructure ~ 200 sites in 39 countries ~ 20 000 CPUs > 5 PB storage > 10 000 concurrent jobs per day > 60 Virtual Organisations EUMedGrid EUIndiaGrid EELA

  30. concertation policies support GO4IT MUPBED PORTA OPTICA STUDY EU-QoS LOBSTER GEANT geo. extension user involv. SEEREN2 AUGERACCESS ORIENT TEIN2 EXPRESS EUMEDIS ALICE e-Infrastructures: network layer

  31. concertation policies support ISSEG ICEAGE ETICS data EGEE grid geo. extension user involv. GRIDCC BALTICGRID BIOINFOGRID HEALTHeCHILD EUCHINAGRID SEEGRID applications DILIGENT EUMEDGRID EELA e-Infrastructures: grid layer

  32. concertation policies DATA e-Infrastructures: data layer Standards OGF data grid