1 / 42

Session 1 Second Part Day 1 Monday 10 th July

Session 1 Second Part Day 1 Monday 10 th July. Malcolm Atkinson. Distributed Systems: Introduction, Context & Challenges. Introduction to Grids. History of Distributed Computing Challenges of Distributed Computing Driving Forces & Motivation for Grids Collaboration

tadeo
Download Presentation

Session 1 Second Part Day 1 Monday 10 th July

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Session 1 Second Part Day 1 Monday 10th July Malcolm Atkinson Ischia, Italy - 9-21 July 2006

  2. Distributed Systems: Introduction, Context & Challenges Ischia, Italy - 9-21 July 2006

  3. Introduction to Grids • History of Distributed Computing • Challenges of Distributed Computing • Driving Forces & Motivation for Grids • Collaboration • Data Deluge • Computational Roadmap • A history of Grids Ischia, Italy - 9-21 July 2006

  4. What is e-Science? • Goal: to enable better research in all disciplines • Method: Develop collaboration supported by advanced distributed computation • to generate, curate and analyse rich data resources • From experiments, observations and simulations • Quality management, preservation and reliable evidence • to develop and explore models and simulations • Computation and data at all scales • Trustworthy, economic, timely and relevant results • to enable dynamic distributed collaboration • Facilitating collaboration with information and resource sharing • Security, trust, reliability, accountability, manageability and agility Our challenge is to develop an integrated approach to all three Ischia, Italy - 9-21 July 2006

  5. Bespoke pioneering distributed systems, e.g. SABRE & SAGE Linklider proposes shared multi-site computing IBM CICS Ethernet TCP Dozens of academic networks Many incompatible protocols CORBA & DCOM IP-based InternetAcademic & Research WWW Distributed Systems History ARPA net 1960 1970 1980 1990 2000 Ischia, Italy - 9-21 July 2006

  6. D-Grid starts Bespoke pioneering distributed systems Many research grids Using many protocols & M/W stacks Linklater proposes shared multi-site computing IBM CICS Web Services EGEE II EGEE EDG ARPA net Unicore Dozens of academic networks Two dominant protocols GlobusStarts CORBA & DCOM Collaboration via shared bio/chem/medical “DBs” IP-based InternetAcademic & Research I-way Condor WWW Distributed Systems to Grids 1960 1970 1980 1990 2000 Ischia, Italy - 9-21 July 2006

  7. Grids: History & Overview Ischia, Italy - 9-21 July 2006

  8. Japan government funds: • Business Grid project • NAREGI project • UK e-Science programme starts • DARPA funds Globus Toolkit & Legion • EU funds UNICORE project • US DoE pioneers grids for scientific research • NSF funds National Technology Grid • NASA starts Information Power Grid • Today: • Grid solutions are common for HPC • Grid-based business solutions are becoming common • Required technologies & standards are evolving A Grid Computing Timeline US Grid Forum forms at SC ‘98 Grid Forums merge, form GGF European & AP Grid Forums I-Way: SuperComputing ‘95 OGSA-WG formed “Physiology” paper “Anatomy” paper GGF & EGAform OGF OGSA v1.0 Source: Hiro Kishimoto GGF17 Keynote May 2006 Ischia, Italy - 9-21 July 2006

  9. R2AD License Webserver Printer Database What is a Grid? A grid is a system consisting of • Distributed but connected resources and • Software and/or hardware that provides and manages logically seamless access to those resources to meet desired objectives Handheld Server Supercomputer Workstation Cluster Data Center Source: Hiro Kishimoto GGF17 Keynote May 2006 Ischia, Italy - 9-21 July 2006

  10. Utility Computing • Computing “services” • No knowledge of provider • Enabled by grid technology Grid & Related Paradigms • Cluster • Tightly coupled • Homogeneous • Cooperative working • Distributed Computing • Loosely coupled • Heterogeneous • Single Administration • Grid Computing • Large scale • Cross-organizational • Geographical distribution • Distributed Management Source: Hiro Kishimoto GGF17 Keynote May 2006 Ischia, Italy - 9-21 July 2006

  11. How Are Grids Used? High-performance computing Collaborative design E-Business High-energy physics Financial modeling Life sciences Data center automation E-Science Collaborative data-sharing Drug discovery Source: Hiro Kishimoto GGF17 Keynote May 2006 Ischia, Italy - 9-21 July 2006

  12. Grids In Use: E-Science Examples • Data sharing and integration • Life sciences, sharing standard data-sets, combining collaborative data-sets • Medical informatics, integrating hospital information systems for better care and better science • Sciences, high-energy physics • Simulation-based science and engineering • Earthquake simulation • Capability computing • Life sciences, molecular modeling, tomography • Engineering, materials science • Sciences, astronomy, physics • High-throughput, capacity computing for • Life sciences: BLAST, CHARMM, drug screening • Engineering: aircraft design, materials, biomedical • Sciences: high-energy physics, economic modeling Source: Hiro Kishimoto GGF17 Keynote May 2006 Ischia, Italy - 9-21 July 2006

  13. Views of Grids Ischia, Italy - 9-21 July 2006

  14. Grids: integrating & providing homogeneity • (Potentially) Generic & Industry Supported • Mechanism for combining manyheterogeneous resources so they can be used remotely • Data resources • Computation resources • Instruments • Research processes & procedures • Restricting choices as to how it may be done • Harder for provider to make localised decisions • Deployment can be challenging • Providing more homogeneity through virtualisation • Should be easier to compose services • More opportunity to amortise costs • A component of e-Infrastructure Deliberately choosing consistent interfaces, protocols & management controls across a set of compatible services Ischia, Italy - 9-21 July 2006

  15. Grids as a Foundation for Solutions Much to be done by developers of applications & services and by resource providers • The grid per se doesn’t provide • Supported e-Science methods • Supported data & information resources • Computations • Convenient access • Grids help providers of these • International & national secure e-Infrastructure • Standards for interoperation • Standard APIs to promote re-use • But Research Support must be built • What is needed? • Who should do it? Ischia, Italy - 9-21 July 2006

  16. Grids as a Foundation for Solutions Much to be done by developers of applications & services and by resource providers • Therefore must support many categories of user: • Application & Service Developers • Deployers & Operations teams • Higher-level abstraction builders & gateway providers • The grid per se doesn’t provide • Supported e-Science methods • Supported data & information resources • Computations • Convenient access • Grids help providers of these • International & national secure e-Infrastructure • Standards for interoperation • Standard APIs to promote re-use • But Research Support must be built • What is needed? • Who should do it? Ischia, Italy - 9-21 July 2006

  17. Motives for Grids Ischia, Italy - 9-21 July 2006

  18. Why use / build Grids? • Research Arguments • Enables new ways of working • New distributed & collaborative research • Unprecedented scale and resources • Economic Arguments • Reduced system management costs • Shared resources  better utilisation • Pooled resources increased capacity • Load sharing & utility computing • Cheaper disaster recovery Ischia, Italy - 9-21 July 2006

  19. Why use / build Grids? • Computer Science Arguments • New attempt at an old hard problem • Frustrating ignorance existing results • New scale, new dynamics, new scope • Engineering Arguments • Enable autonomous organisations to • Write complementary software components • Set up run & use complementary services • Share operational responsibility • General & consistent environment forAbstraction, Automation, Optimisation & Tools • Generally available code mobility Ischia, Italy - 9-21 July 2006

  20. Why use / build Grids? • Political & Management Arguments • Stimulate innovation • Promote intra-organisation collaboration • Promote inter-enterprise collaboration Ischia, Italy - 9-21 July 2006

  21. Key drivers for Grids Ischia, Italy - 9-21 July 2006

  22. Drivers for Grids • Collaboration • Expertise is distributed • Necessary to achieve critical mass of effort • Necessary to raise sufficient resources • Computational Power • Rapid growth in number of processors • Powered by Moore’s law + device roadmap • Challenge to transform models to exploit this • Deluge of Data • Growth in scale: Number and Size of resources • Growth in complexity • Policy drives greater availability Ischia, Italy - 9-21 July 2006

  23. Collaboration is key Ischia, Italy - 9-21 July 2006

  24. SyntenyGrid Service blast + Biomedical Research Informatics Delivered by Grid Enabled Services Portal http://www.brc.dcs.gla.ac.uk/projects/bridges/

  25. Electronic Patient Records Radiology reporting systems Patients Letters Screening Case Information Assessment/ Symptomatic Biopsy 2ndary Capture Or FFD X-Rays and Case Information Symptomatic/Assessment Information eDiaMoND Grid Case and Reading Information CAD SMF 3D Images Training Case and Reading Information Digital Reading Manage Training Cases Perform Training SMF CAD Temporal Comparison eDiaMoND: Screening for Breast Cancer 1 Trust  Many Trusts Collaborative Working Audit capability Epidemiology • Other Modalities • MRI • PET • Ultrasound Better access to Case information And digital tools Supplement Mentoring With access to digital Training cases and sharing Of information across clinics Overview of e-Infrastructure Provided by eDiamond project: Prof. Sir Mike Brady et al.

  26. climateprediction.net and GENIE • Largest climate model ensemble • >45,000 users, >1,000,000 model years Response of Atlantic circulation to freshwater forcing 10K 2K

  27. Integrative Biology Tackling two Grand Challenge research questions: • What causes heart disease? • How does a cancer form and grow? Together these diseases cause 61% of all UK deaths • Will builda powerful, fault-tolerant Grid • infrastructure for biomedical science • Enabling biomedicalresearchers to use distributed resources such as high-performancecomputers, databases and visualisationtools to develop complex models of howthese killer diseases develop. Courtesy of David Gavaghan & IB Team

  28. IB Partners Courtesy of David Gavaghan & IB Team

  29. Foundations of Collaboration • Strong commitment by individuals • To work together • To take on communication challenges • Mutual respect & mutual trust • Distributed technology • To support information interchange • To support resource sharing • To support data integration • To support trust building • Sufficient time • Common goals • Complementary knowledge, skills & data Can we predictwhen it will work? Can we findremedies when itdoesn’t? Ischia, Italy - 9-21 July 2006

  30. Grid & Collaboration Questions • Without collaboration little is achievable • But must collaboration precede successful grid applications? • Or will persistently and pervasively available grids stimulate collaborations? • If we deliver support for collaborative teams, will we also support the individual researcher? • Can we use grids to democratise computation? Ischia, Italy - 9-21 July 2006

  31.  determining ion channel contribution to the timing of action potentials  resolving the ‘neural code’ from the timing of action potential activity  examining integration within networks of differing dimensions CARMEN - Scales of Integration Understanding the brain may be the greatest informatics challenge of the 21st century

  32. CARMEN Consortium Leadership & Infrastructure Colin Ingram Paul Watson Leslie Smith Jim Austin

  33. Ad Aertsen (Freiburg) Neural network modelling and large-scale simulations George Gerstein(Pennsylvania) Analysis of spike pattern trains Daniel Gardner(Cornell) Lead for the US NIH, Neuroscience Information Framework and Brain ML Sten Grillner(Karolinska Institute) Chairman of the OECD,International Neuroinformatics Coordinating Facility CARMEN Consortium International Partners Shiro Usui(RIKEN Brain Science Institute) Lead for the Japan Node of the International Neuroinformatics Coordinating Facility

  34. CARMEN Consortium Commercial Partners - applications in the pharmaceutical sector - interfacing of data acquisition software - application of infrastructure - commercialisation of tools

  35. The Challenge Toshiba 04 Device diversification 90nm: HP, LOP, LSTP 45nm: UTB SOI 32nm: Double gate Ischia, Italy - 9-21 July 2006

  36. University Partners Advanced Processor Technologies Group (APTGUM) Device Modelling Group (DMGUG) Electronic Systems Design Group (ESDGUS) Intelligent Systems Group (ISGUY) National e-Science Centre (NeSC) Microsystems Technology Group (MSTGUG) Mixed-Mode Design Group in IMNS (MMDGUE) e-Science NorthWest Centre (eSNW) Ischia, Italy - 9-21 July 2006

  37. Industrial Partners Global EDS vendor and world TCAD leader 600 licences of grid implementation, model implementation UK fabless design company and world microprocessor leader Core IP, simulation tools, staff time UK fabless design company and world mixed mode leader Additional PhD studentship for mixed mode design Global semiconductor player with strong UK presence Access to technology, device data, processing Global semiconductor player with strong UK presence Access to technology, device data, processing Global semiconductor player with UK presence CASE studentship, interconnects Trade association of the microelectronics industry in the UK Recruiting new industrial partners and dissemination

  38. Summary & After the Break Ischia, Italy - 9-21 July 2006

  39. Grids in context • Part of a long-term drive for distributed computing • A new and ambitious form • Search for trade-offs & multiple uses • Leads to many varieties • Multiple stake holders • Many good reasons for building & using grids • Question • Will we have many grids or a consistent general purpose “foundation” grid? • What are the minimum standards across the grids • Collaboration is a key driver & enabler Ischia, Italy - 9-21 July 2006

  40. Minimum Grid Functionalities • Supports distributed computation • Data and computation • Over a variety of • hardware components (servers, data stores, …) • Software components (services: resource managers, computation and data services) • With regularity that can be exploited • By applications • By other middleware & tools • By providers and operations • It will normally have security mechanisms • To develop and sustain trust regimes Ischia, Italy - 9-21 July 2006

  41. After the break • Distributed system challenges • Service Oriented Architectures • Examples of the basic architecture Ischia, Italy - 9-21 July 2006

  42. Questions & Comments Ischia, Italy - 9-21 July 2006

More Related