1 / 22

gLite adoption and opportunities for collaboration with industry

gLite adoption and opportunities for collaboration with industry. Tony Doyle Distributed Computing Workshop Westminster, 21 May 2008. Introduction. Context – PIPPS Projects Who are GridPP? Why do we need a Grid? What is our Grid? What do we offer?. PIPPS Projects.

oburk
Download Presentation

gLite adoption and opportunities for collaboration with industry

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. gLite adoption and opportunities for collaboration with industry Tony Doyle Distributed Computing Workshop Westminster, 21 May 2008

  2. Introduction Context – PIPPS Projects Who are GridPP? Why do we need a Grid? What is our Grid? What do we offer?

  3. PIPPS Projects • David Sinclair and Chris Town (Cambridge Ontology Ltd) and Andy Parker (Cambridge e-Science Centre) • Mini-PIPSS to develop a Content Based Image Retrieval (CBIR) platform powered by gLite • On completion of the Mini-PIPSS project Cambridge Ontology received £535k private equity investment, changed its name to Imense, and is now doing a PIPSS project with Andy Parker • Oleg Soloviev (Econophysica) and Steve Lloyd (QMUL) • Mini-PIPSS to develop a Grid based automated trading platform for the financial industry • Constellation Technologies Ltd and Neil Geddes (RAL) • PIPSS to develop a commercial version of gLite middleware • DiGS and George Beckett (Edinburgh, EPCC) • PIPSS to develop a Data Grid for Cell Biology, sharing biological images between researchers (an example of inter-disciplinary use of software) • Other EGEE-wide Projects • Total Oil testbed studies (Aberdeen) • EU-wide biomed docking studies (anti-malarial and bird-flu drug development)

  4. Who are GridPP? • UK’s contribution to LHC computing: • 19 UK Universities, STFC and CERN • GridPP1 (2001- 2004) £17m • “From Web to Grid” • GridPP2 (2004 - 2008) £16m • “From Prototype to Production” • GridPP3 (2008 – 2011) £25m • “From Production to Exploitation”

  5. Why do particle physicists need the Grid? CERN LHC The world’s most powerful particle accelerator 4 Large Experiments

  6. One year’s data from LHC would fill a stack of CDs 20km high Concorde (15 Km) Mt. Blanc (4.8 Km) Who are GridPP? Why do particle physicists need the Grid? Example from LHC: starting from this event • ~100,000,000 electronic channels • 800,000,000 proton-proton interactions per second • 0.0002 Higgs per second • 10 PBytes of data a year • (10 Million GBytes = 14 Million CDs) We are looking for this “signature” Selectivity: 1 in 1013 Like looking for 1 person in a thousand world populations Or for a needle in 20 million haystacks!

  7. A question of scale

  8. Solution – Build a Grid Solution – Build a Grid • Share more than information • Efficient use of resources at many institutes • Leverage over other sources of funding • Data, computing power, applications • Join local communities • Challenges: • share data between thousands of scientists with multiple interests • link major and minor computer centres • ensure all data accessible anywhere, anytime • grow rapidly, yet remain reliable for more than a decade • cope with different management policies of different centres • ensuredata security • be up and running routinely in 2008

  9. Middleware is the Key Your Program Single PC Grid Your Program PROGRAMS MIDDLEWARE User Interface Machine Word/Excel Games Email/Web Resource Broker Information Service OPERATING SYSTEM CPU Replica Catalogue Disks, CPU etc Bookkeeping Service Middleware is the Operating System of a distributed computing system CPU Cluster CPU Cluster CPU Cluster Disk Server

  10. 11 10 8 9 0 3 4 6 2 5 1 7 VOMS-proxy-init LFC gridui JDL Job Submission Job Retrieval RB BDII Job Status? JS Grid Enabled Resources Grid Enabled Resources Grid Enabled Resources Grid Enabled Resources Logging & Bookkeeping Submitter CPU Nodes CPU Nodes CPU Nodes CPU Nodes Storage Storage Storage Storage Something like this… VOMS WLMS

  11. CERN computer centre Tier 0 Offline farm RAL,UK Spain Germany Italy France Tier 1 National centres Online system Tier 2 Regional groups ScotGrid NorthGrid SouthGrid London Glasgow Edinburgh Durham Institutes Workstations Grid Infrastructure 11 T1 centres Structure chosen for particle physics. Different for others.

  12. Middleware Validation: From Testbed to Production Build System Development Testbed ~15CPU Application Testbed ~1000CPU Certification Testbed ~40CPU Unit Test Production Build Integration Certification add unit tested code to repository Run nightly build & auto. tests Individual WP tests Grid certification Certified public release for use by apps. Users Build system Integration Team Test Group Tagged package WPs Overall release tests Application Certification Certified release selected for deployment Tagged release selected for certification Fix problems Process to test: frameworks support policies documentation platforms/compilers Apps. Representatives Releases candidate Releases candidate Tagged Releases Certified Releases 24x7 Problem reports

  13. Status March 2008 March 2007 Status in 2007: 177 sites 32,412 CPUs ~13 PB storage Status in 2008: 250 sites, 50 countries 55,094 CPUs ~20PB storage

  14. GridPP & IndustryWhat Do We Offer? • Middleware Expertise • Our Grid (for test purposes) Examples: • Adaptable User Interface (GANGA) • Security tools (GridSite) • Accounting tools (R-GMA & APEL)

  15. Middleware Expertise Workload Management Grid Data Management Network Monitoring Information Services Security Storage Interfaces

  16. Our Grid • The UK Grid (via the individual research sites) has been used to test applications for other areas e.g. • biomedical research • financial modelling • device modelling • oil exploration • image processing

  17. Scriptor Job details Logical Folders Job Monitoring Job builder Log window Adaptable User Interface Ganga GUI

  18. Grid Security for the WebWeb platforms for Grids Security Tools • Digital Certificates • Certification Authority • Gridsite identifies users to websites with the digital certificates • GridSiteWiki is an extension to the tool • GridSite is open source (http://www.gridsite.org/)

  19. Accounting tools • RelationalGridMonitoringArchitecture • An information and monitoring system for static and dynamic information about grid resources, applications and networks • Accounting Processor for Event Logs • Provides a summary of the resources consumed based on attributes such as CPU time, Wall Clock Time, Memory and grid user identity

  20. Knowledge Exchange K n o w l e d g e E x c h a n g e Trust Security Business Models Business Community Quality of Service Accounting Research Community Standards Applications Portability Open Source Support Dissemination Software Licence Management

  21. Knowledge Exchange Productise software for your business Research Business Sustain software on behalf of all users “an essential component within the innovation cycle of any knowledge driven economy” Dissemination

  22. Summary Opportunity for knowledge creation through improved IT skills and an enhanced research base GridPP supports locally-led activities (based upon an international core of expertise and ongoing examples of collaboration) GridPP will work with companies to examine different methods of technology transfer and identify the activities that can be used for industry and business

More Related