1 / 18

Campus grids: e-Infrastructure within a University

Campus grids: e-Infrastructure within a University. Mike Mineter National e-Science Centre mjm@nesc.ac.uk. Thanks to:. Mark Calleja, Cambridge David McBride, Imperial College David Wallom, Oxford Jonathan Giddy, Welsh e-Science Centre John Brodholt, UCL

arin
Download Presentation

Campus grids: e-Infrastructure within a University

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Campus grids: e-Infrastructure within a University Mike Mineter National e-Science Centre mjm@nesc.ac.uk

  2. Thanks to: • Mark Calleja, Cambridge • David McBride, Imperial College • David Wallom, Oxford • Jonathan Giddy, Welsh e-Science Centre • John Brodholt, UCL for descriptions of their campus initiatives.

  3. Overview • Goals • Methods & Examples • Some opportunities and implications

  4. Goals of campus grids • Resource utilisation • Computers: in each university x,000 PCs are little used • Research data • Collaboration • Access Grids – meetings across sites • Sharing visualisation, data, programs • Infrastructure for research • From resource sharing in a university to international collaboration with one middleware stack

  5. Overview • Goals • Methods & Examples • Harvesting CPU cycles • Crossing administrative domains • Globus • Storage Resource Broker • Some opportunities and implications

  6. Harvesting CPU time Often-idle processors!! Teaching labs. + Researchers Analyses constrained by CPU time!

  7. Harvesting CPU time • Teaching lab machines lie idle for most of the time • Harvest spare compute cycles to create a low-cost “high throughput computing” (HTC) platform • Goal: run many tasks in a week, month, … • Typically: many similar tasks invoked from workflow or a script • Monte-Carlo • Simulation – parameter sweeps • Pool processors as a batch processing resource • Submit jobs that run when a machine is free • Condor most common approach • http://www.cs.wisc.edu/condor/

  8. submission machines Linux central manager UCL Pool > 1000 CPUS Platforms: Windows XP and 2000 Capacity: 256Mb-1Gb RAM; Intel 3 (1Ghz) – Intel 4 (3.2Ghz) The UCL Condor Pool Different projects can have their own submission machine Note: need Windows executable (.exe file) John Brodholt UCL

  9. CaO-termimated Mineral Surfaces • Calculations: • investigate 10-20 surfaces • 2 to 5 surface terminations • 4 to 16 impurity positions • > 4 concentrations • Total number of calculations • per impurity: 120-2440 {001} surfaces of CaTiO3 TiO2-termimated M. Alfredsson, J.P. Brodholt and G.D. Price et al, Submitted to Nature Materials

  10. Use these surface energies to predict crystal forms as a function of type dopant AND concentration: Maria Alfredsson, UCL

  11. Predicted Morphology of Ni-doped CaTiO3 SEM-picture of Ni-doped CaTiO3 Crystal Morphologies of Ni-doped CaTiO3 (submitted to Nature Materials) Maria Alfredsson, UCL

  12. Campus grids • Resources in many administrative domains • Need basic services that provide: • Authentication, Authorisation mechanisms • Based on certificates • Single sign-on to access many resources • Control of who can do what • Job submission services • Submit jobs to batch queues on clusters or Condor pools • Information systems • So you know what can be used • Ability to share data

  13. Middleware for campus grids • Globus toolkit http://www.globus.org/ • Tools built on Grid Security Infrastructure and include: • Job submission: run a job on a remote computer • Information services: So I know which computer to use • Storage Resource Broker http://www.sdsc.edu/srb/ • Virtual filesystem: for files held in multiple locations • NIEES offers a testbed to give experience with SRB • SRB and Globus Toolkit 2 are part of the National Grid Service stack

  14. Globus is a Toolkit • To submit a job to run on a remote computer globus-job-submit grid-data.rl.ac.uk/jobmanager-pbs /bin/hostname -f https://grid-data.rl.ac.uk:64001/1415/1110129853/ globus-job-status https://grid-data.rl.ac.uk:64001/1415/1110129853/ DONE globus-job-get-output https://grid-data.rl.ac.uk:64001/1415/1110129853/ grid-data12.rl.ac.uk • Needs higher level services • Brokers to allow jobs to be submitted to “a grid” • VO- specific developments e.g. Portals

  15. SRB server SRB server SRB server Resource Driver Resource Driver Resource Driver Storage Resource Broker User sees a virtual filesytem: • Command line (S-Commands) • MS Windows (InQ) • Web based (MySRB). • Java (JARGON) • Web Services (MATRIX) Filesystems in different admin. domains

  16. OxGrid Central Management Broker MDS/ VOM Storage Example: OxGrid, a University Campus Grid • Single entry point for Oxford users to shared and dedicated resources • Seamless access to National Grid Service and OSC for registered users • Single sign-on using PKI technology integrated with current methods OSC NGS Oxford Users College Resources Departmental Resources David Wallom

  17. Observations • Need specific initial user communities >> vague sense this is a good idea! • Engage with systems managers from first thoughts • Operations effort must be factored in and justifiable! Researcher enthusiasm is not enough! • Be alert to National Grid Service deployment of middleware, and potential for interoperability

  18. Summary • Campus grids enable • High throughput computing • Improve resource utilisation • Collaboration – share processes, data, storage • Builds culture, skills and infrastructure for service-oriented research… • Advantages of using the same stack as the National Grid Service • Expertise and support available • Smaller leaps to partnership in and use of the NGS • White Rose Grid pioneered this!... and provides the expertise for others…

More Related