1 / 24

Porting Scientific Applications on GRID: CERN Experience

Porting Scientific Applications on GRID: CERN Experience. Patricia M é ndez Lorenzo CERN (IT-PSS/ED) Trieste, 10th February 2006 ICTP/INFM-Democritos Workshop on Porting Scientific Applications on Computational GRIDs. Outlook. ◘ This is an introductory presentation

tarmon
Download Presentation

Porting Scientific Applications on GRID: CERN Experience

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Porting Scientific Applications on GRID: CERN Experience Patricia Méndez Lorenzo CERN (IT-PSS/ED) Trieste, 10th February 2006 ICTP/INFM-Democritos Workshop on Porting Scientific Applications on Computational GRIDs

  2. Outlook ◘ This is an introductory presentation ◘ Let’s see what is CERN, what is LCG, the elements, the actors and how to get involved ◘ This afternoon, during the 2nd talk we will see some practical examples of communities getting involved in the GRID Trieste, 10th February 2006 Patricia Méndez Lorenzo

  3. CERN The European Organization for Nuclear Research The European Laboratory for Particle Physics ◘ Fundamental research in particle physics ◘ Designs, builds & operates large accelerators ◘ Financed by 20 European countries (member states) + others (US, Canada, Russia, India, etc) 2000 staffs + 6000 users from all over the world ◘ Next huge challenge: LHC (starts in 2007) experiment: 2000 physicists, 150 universities, with an operation life greater than 10 years Trieste, 10th February 2006 Patricia Méndez Lorenzo

  4. LHC- Physics Goals ◘ Higgs particle Key particle in the Standard Model that could explain the elementary particle masses ◘ Search for super-symmetric particles and possible extra dimensions Their discovery would be a serious push for Super Symmetric theories or “String Theories” aiming at the unification of the fundamental forces in the Universe ◘ Anti-matter issues Why the Universe is made of matter instead of an equal quantity of matter and antimatter ◘ Understand the early Universe (10 -20 – 10-6 seconds) Soup of quarks and gluons stabilized into nucleons and then nuclei and atoms Trieste, 10th February 2006 Patricia Méndez Lorenzo

  5. The LHC Experiment ALICE, ATLAS, CMS and LHCb Reduce by online computers that filter out a few hundred good events per sec Recorded on disk and magnetic tape at 100-1000 MB/sec: 15 PB/year The LHC: Generation of 40 million particle collisions (events) per second at the center of each for experiments Trieste, 10th February 2006 Patricia Méndez Lorenzo

  6. The LHC Computing Environment LCG (LHC computing Grid) has been developed to build and maintain a storage and analysis infrastructure for the entire high-energy physics community ◘ LHC is beginning the data taking in summer 2007 ➸Enormous volume of data •Few PB/year at the beginning of the machine operation • Several hundred PB yearly produced for all experiments in 2012 ➸ Large amount of processing power ◘ As a solution a LCG world-wide Grid is proposed ➸Established using a world-wide distributed federal Grid ➸Many components, services, software, etc, to coordinate ◘ Takes place at an unprecedented scale ➸Many institutes, experiments and people working closely together ◘ LCG must be ready at full production capacity, functionality and reliability in less than 1 year! LCG is an essential part of the chain allowing the physicists to perform their analyses ➸It has to be a stable, reliable and easy to use service Trieste, 10th February 2006 Patricia Méndez Lorenzo

  7. Tier-0 – the accelerator centre • Data acquisition and initial Processing of raw data • Distribution of data to the different Tier’s Tier-1 – “online” to the dataacquisition process high availability • Managed Mass Storage – grid-enabled data service • Data-heavy analysis • National, regional support Canada – Triumf (Vancouver) France – IN2P3 (Lyon) Germany – Forschunszentrum Karlsruhe Italy – CNAF (Bologna) Netherlands – NIKHEF/SARA (Amsterdam) Nordic countries – distributed Tier-1 Spain – PIC (Barcelona) Taiwan – Academia SInica (Taipei) UK – CLRC (Oxford) US – FermiLab (Illinois) – Brookhaven (NY) LCG: The LHC Computing Grid Tier-2 – ~100 centres in ~40 countries • Simulation • End-user analysis – batch and interactive Trieste, 10th February 2006 Patricia Méndez Lorenzo

  8. 30 sites 3200 cpus 25 Universities 4 National Labs 2800 CPUs Grid3 LCG in the World May 2005 140 Grid sites 34 countries 12000 CPUs 8 PetaBytes Trieste, 10th February 2006 Patricia Méndez Lorenzo

  9. What is the structure we have at CERN? ◘ CERN is the T0 ➸Deployment: We pack and distribute the software to the sites ➸Development: Development of new projects and also part of the software ➸Support: Assistance to experiments and sites EXPERIMENTS Contact through ARDA-EIS ARDA EIS APPLICATIONS LCG DEPLOYMENT SITES Trieste, 10th February 2006 Patricia Méndez Lorenzo

  10. Our Tier1 centers Trieste, 10th February 2006 Patricia Méndez Lorenzo

  11. CE UI UI WN WN WN WN WN WN LFC The Elements of the Middleware Ouputs copied to Storage Resources Sent to the batch system Connections to UI SE Distribution to CPUs Resources Searching RB/BDII Catalogs getting track of the inputs Trieste, 10th February 2006 Patricia Méndez Lorenzo

  12. Current status of the Experiments and the GRID ◘ They have to get ready ➸How to take raw data and how to store and distribute them ➸How to produce simulated data ➸How to analyze data ◘ They are not alone, EGEE/LCG assist them ➸ARDA Group ➸Application area Group EXPERIMENTS SITES Ramp up their capacity to twice the nominal data rates expected for the production phase THIS IS A FULL CHALLENGE FOR ALL OF US THE SUPPORT AMONG THE 3 ACTORS IS FUNDAMENTAL Successfully completed 6 months before the data taking They have to be ready to a high stability level SERVICES: EGEE/LCG Trieste, 10th February 2006 Patricia Méndez Lorenzo

  13. We have a fundamental Challenge in front of us: Service Challenge As we have just seeing experiments and Grid teams are ramping up… but making what exactly? Service Challenges (SC): Grid part ◘ We provide services to the users (done) ◘ But we have to test them to the necessary level of functionality, reliability and scale ◘ We are preparing, hardening and delivering the production of the LCG environment ◘ Moreover we have to run an environment as realistic as possible Data Challenges (DC): Experiments part ◘ Experiments test their LCG based production chains and performance of the Grid fabric ➸Processing data from simulated events ➸Emulating step by step the scale they will have to face during real data taking ◘ From 2005 experiments include SC testing as part of their DC Trieste, 10th February 2006 Patricia Méndez Lorenzo

  14. Assistance to Experiments ARDA: A Realization of Distributed Analysis for LHC ◘ Main Goal: ➸Coordination of activities needed to prototype distributed analysis systems for the LHC ➸It does NOT develop middleware, go one step further, closer to the users ◘ Main Actors: ➸People of each experiment (they know what they need) ➸Middleware development team (they know what they have) ➸Experiment support team: EIS(they can contact both) ◘ New Gridifications: ➸Coordinated by this group (You will go through us, surely) ➸Application of their tools to new communities (We can help you) Trieste, 10th February 2006 Patricia Méndez Lorenzo

  15. OK, now you know the project, its elements and its actors And however you want still to go inside… ☺ Good choice, let’s see how Trieste, 10th February 2006 Patricia Méndez Lorenzo

  16. New Contributions ◘ Several different communities are joining the GRID project ◘ It has applications to all these fields needing large amount of computational and space resources ◘ We are direct supporting (outside LHC exp.): ➸Biomed (medical applications) ➸Geant4 (simulation toolkit) ➸UNOSAT, ITU (UNO projects) ◘ And how to put then inside the EGEE/LCG is a HOT TOPIC at this moment This afternoon Next part of this 1st talk Trieste, 10th February 2006 Patricia Méndez Lorenzo

  17. Before proceeding… ◘ There are several points under discussion at this moment about how to admit new Virtual Organizations arriving to the GRID ➸Last discussion, during the Grid Deployment Board meeting this Wednesday ◘ I going to explain the procedure if the request is associated in somehow with CERN ➸This is my experience ◘ What to do in the rest of cases is being discussed ➸It should have maximal priority inside the EGEE environment ➸From my point of view it must be flexibly ➸I will explain you also my proposal Trieste, 10th February 2006 Patricia Méndez Lorenzo

  18. What we are doing now for Geant4 ◘ Full Geant4 gridification this afternoon ◘ Right now, just see it as a new community… ◘ But a “special” community because: ➸They pretend to run twice per year (Not the whole year) ►To validate their own software ➸It is used as simulation tool by many Grid VOs ►A good validated product, will make experiments life easier ➸Its software is very well known ► stable, reliable, quite long tested ► LCG thought to use this software as a part of the LCG test ➸The tests inside LCG were asked by people placed at CERN ► Geant4 at CERN, LCG support at CERN, easier to gridify and support this community Trieste, 10th February 2006 Patricia Méndez Lorenzo

  19. Immersion of Geant4 in LCG ◘As any other new VO, they will need support ➸ Provided by EGEE/LCG ◘Support is fundamental ➸To involve you quickly and safely ➸Till you gain familiarity it will play the role of software, production and VO manager ➸We dial with the site on your behalf ◘ They will need to gain familiarity with the system and to test it before passing through the whole EGEE/LCG VO procedure ➸You have to learn before taking a decision ➸ This will be a situation to face more and more UNOSAT and more ◘ EGEE/LCG has to face: ➸What to do with light new VOs arriving ➸ The solution should not be “dteam” (deployment team VO) ➸The most important point is already setup: YOU HAVE SUPPORT Trieste, 10th February 2006 Patricia Méndez Lorenzo

  20. Our Challenge Provide a formal procedure to give them support ◘It is not the same if the community contacts directly the support at CERN ◘ This should not be like that and a common policy should be found ➸A regional policy delegation could be the solution ◘ Most of all in terms of VO policy a procedure should be fixed ◘ Something we still have to clarify ➸Are the sites free enough to provide resources to any new VO as local users? ►Well, this is what we have done for Geant4 and this does seem the method to follow Trieste, 10th February 2006 Patricia Méndez Lorenzo

  21. Initial deployment configuration 1) New applications 2) EGAAP Recommended VO candidate Resource proposal Asks for change 4) CICs/ROCs 3) NA4/SA1 VO requirements Current Procedure The solution for us is to begin the support here Geant4 application LCG Procedure What we did 1. Geant4 is quite known 2. It is fully supported 3. It was the 3rd production 4. We had a short time to begin the production Sites EGEE Procedure Trieste, 10th February 2006 Patricia Méndez Lorenzo

  22. What we have done ◘ We presented the Geant4 community last GDB and we asked for local support ➸In my ideal world, once you ask for the entry a support person should be assigned to you to discuss with you your hopes and the viability ➸Let him to deal with the LCG/Sites Management ◘At the same time we were following the official procedure to become VO ➸After testing the product and seeing you want it, this is mandatory ◘In the case of Geant4 ➸We asked directly for the support to Geant4 site per site ➸We got in time in 5 sites ➸We got the total amount of CPUs required for Geant4 (120CPUs) ➸ LCG Efficiency: 99% ➸At any moment the Geant4 production was totally followed not to interfere in a wrong way with the sites ► This is fundamental for the sites ► Your LCG support is responsible of your production ► Many other communities are running and you should not interfere them (and they should not interfere you!) Trieste, 10th February 2006 Patricia Méndez Lorenzo

  23. Light-weight VO registration ◘We are considering to have an incubator VO to assist new communities ➸This VO is standard for all new communities ➸ It allows you to play with the system ➸ During this time your support plays with you ➸ You do not decide the policy of this VO, we do it ➸ You will not make productions inside, you just learn ➸ LCG has already experience, we know how to deal with ◘Warning: You should have a limited time inside ➸ Then you are a integrated VO ➸ You should go through the registrations steps ➸ Now your support makes exactly that: LCG support no more, no less ► With a strict LCG support no problems should be seen ► Experiments happy, sites happy, LCG happy Trieste, 10th February 2006 Patricia Méndez Lorenzo

  24. Final Message LCG was born with a clear objective Assist LHC experiments during their real data taking The project is however extensible and quite attractive for any other community We are thinking at this moment how to procede with the light-weight VOs This is foreseen and it is part of the project SUPPORT is the most important point if you arrive to the GRID... And the infrastructure is already developed Trieste, 10th February 2006 Patricia Méndez Lorenzo

More Related