网格计算技术
This presentation is the property of its rightful owner.
Sponsored Links
1 / 77

网格计算技术 第一章 绪 论 PowerPoint PPT Presentation


  • 76 Views
  • Uploaded on
  • Presentation posted in: General

网格计算技术 第一章 绪 论. 龚 斌 山东大学计算机科学与技术学院 山东省高性能计算中心. 主要内容. 网格概述 网格分类 网格的应用 网格研究现状 网格相关组织. What is Grid Computing?. The term Grid comes from an analogy to the Electric Grid. Pervasive access to power.

Download Presentation

网格计算技术 第一章 绪 论

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


6952243

网格计算技术第一章 绪 论

龚 斌

山东大学计算机科学与技术学院

山东省高性能计算中心


6952243

主要内容

  • 网格概述

  • 网格分类

  • 网格的应用

  • 网格研究现状

  • 网格相关组织


What is grid computing

What is Grid Computing?

  • The term Grid comes from an analogy to the Electric Grid.

    • Pervasive access to power.

    • Similarly, Grid will provide pervasive, consistent, and inexpensive access to advanced computational resources.

  • Grid computing usually associated with geographically distributed resources connected by high-performance networks.


What is grid computing1

What is Grid Computing?

  • Grids are about sharing computational resources amongmultiple organizations/owners, multiple users no matter where they are located

    • They are usually not controlled centrally

    • The resources, their software and their management differs considerably across grid

    • They are expected to be extensible and the infrastructure enabling them needs to scale to very large systems

  • So standards and standardized infrastructure is vital


What is grid computing2

What is Grid Computing?

Here is one early definition

  • The transparent, systematic and effective utilisation of geographically distributed heterogeneous resources (both hardware and software) for applications in science and commerce

  • Nowadays “geography” not so important, but resources in different Internet (admin) domains


More recent thoughts

More Recent Thoughts

  • Flexible, secure, coordinated resource sharing among dynamic collections of individuals, institutions, and resource

    From “The Anatomy of the Grid: Enabling Scalable Virtual Organizations”

  • Enable communities (“virtual organizations”) to share geographically distributed resources as they pursue common goals -- assuming the absence of…

    • central location

    • central control

    • Omniscience(全知,上帝)

    • existing trust relationships


6952243

  • Grids are very much about large-scale resource sharing.

    • Spanning administrative boundaries.

  • Problem solving in dynamic, multi-institutional environment.

  • Sharing for example:

    • Direct access to computers, software, data, and other resources.

    • Sharing is highly controlled, clear definitions of exactly what is shared, who is allowed to share, and the conditions under which sharing occurs.


Not a new idea

Not A New Idea

  • Late 70’s – Networked operating systems

  • Late 80’s – Distributed operating system

  • Early 90’s – Heterogeneous computing

  • Mid 90’s - Metacomputing

  • Then the “Grid”– Foster and Kesselman, 1999

  • Also called parallel distributed computing


Resource

Resource 资源

  • Computation

  • Storage

  • Communications

  • Software and licenses

  • Special equipment, capacities, architectures, policies


6952243

什么是网格?

网格(Grid)是构筑在Internet上的一组新兴技术,它将高速互联网、高性能计算机、大型数据库、传感器、远程设备等融为一体,为科技人员和普通老百姓提供更多的资源。Internet主要为人们提供E-mail、网页浏览等通信功能,而网格功能更多更强,能让人们透明地使用计算、存储、信息处理等其他资源。

1998, The Grid: Blueprint for a New Computing Infrastructure.

Ian Foster :美国阿岗国家实验室资深

科学家、美国计算网格项目负责人


6952243

两种网络应用方式


6952243

网 格

  • 耦合各种资源的基础结构

    • 计算机

    • 软件

    • 数据库 (例如, 人类基因库)

    • 各种仪器 (例如,射电望远镜)

    • 人 (物理学家、数学家、计算机专家)

  • 跨越局域网/广域网边界

    • 企业, 组织, Internet

  • 统一集成的单一资源


6952243

国家信息基础设施-数字信息基础设施

本世纪

网络资源联合计算MetaComputer

八十年代

一个无缝集成的计算和协同环境Computational Grid

九十年代中期

广域资源的联合处理服务Grid:含义变化

应用广阔

本世纪初

Grid的由来


6952243

数据、信息和知识


6952243

网格应用需求背景

  • E-Science

    • 大规模构造和挖掘大型的观测和模拟数据库

    • 开发计算机模拟和分析

    • 访问远程的特殊设备

    • 分布各地的合作研究人员近实时的交换信息

  • E-Business

    • 企业计算是高度分布、异构、企业之间(B2B)

    • 企业计算需要大量计算和数据服务

    • 各种资源和服务的提供商出现,租赁业务是可行的


6952243

网格应用需求背景

  • E-Government

    • 各个政府部门的信息互通和互操作

    • 部门之间资源和信息共享和整合

  • E-Entertainment

    • 大规模的视频点播:分布、海量

    • 在线游戏:成千上万人、交互、处理

  • E-Education

    • 教育信息化

    • 主动学习方法和交互式的教学方式

    • 网络上作模拟实验


6952243

电网

电站

电站

电站

矿山

水坝

油井


6952243

电力网格和计算网格的能力对比


6952243

电力网与网格比较


6952243

判断是否网格的三原则:

  • coordinates resources that are not subject to centralized control

    非网格:Web, Sun Grid Engine

  • using standard, open, general-purpose protocols and interfaces

    非网格: P2P , Condor

  • to deliver nontrivial qualities of service

    是网格:GriPhyN, DataGrid, IPG, DAS-2

    2002, What is the Grid? A Three Point Checklist.


6952243

《福布斯》杂志:电脑业趋势

  • 美国《福布斯》杂志科技版于2001年9月发表了一组文章,分析了信息技术40年的发展历史和今后20年的趋势,指出网络计算将极大地改变人们的工作和生活,

    • 信息技术的下一波大浪潮将在2004-2005年度出现,并造就2005-2020 十五年的黄金时代

    • 这个大浪潮将极大地改变我们的工作和生活。到2020年,由此产生的互联网将成长为一个20万亿美元产值的大工业。

    • 信息市场从2000年的一万亿美元增长20倍。

    • 这一波浪潮的本质特征,就是万维网(World Wide Web)升华为网格(Great Global Grid)。


6952243

按需计算

(网格)

对国民经济和社会发展的影响

因特网

客户机-服务器

我们目前所处阶段

大型机-终端

时间

专家使用

广泛使用

早期流行

公众认识

2020年计算机技术将广泛普及


6952243

因特网

Web

网格

计算机

计算机

网页

网页

应用

应用

网格技术的目标:应用层面的互连互通—消除资源孤岛


6952243

网格是下一波大浪潮

  • 第一波

  • 1960-现在

  • 计算机联通

  • Internet

  • telnet

    ftp

    E-mail

  • 第二波

  • 1980-现在

  • 网页联通

  • WWW

  • Http://

  • 第三波

  • 1993-现在

  • 资源全面联通

  • Grid

  • Grid://

  • 计算资源

    存储资源

    信息资源

    知识资源


Www ggg

GGG(Great Global Grid)

Information and Knowledge Grid

Computing and Data Grid

Other Grid Models

Business Grid

CDN

RTEC

Web Service

Semantic Web

Knowledge

Management

Ontology

Information

Platform

TeraGrid

IPG

GIG

ASCI Grid

Data Grid

P2P

Parasitic Computing

HP eSpeak

IBM Web Services

Microsof.Net

Sun ONE

WWWGGG


6952243

网格的本质

  • 本质

    • 资源共享

    • 协同工作

  • 在多个机构动态形成的虚拟组织中共享资源和协同解决问题。


6952243

网格的特点

  • 虚拟性—虚拟组织VO

  • 集成性—集成创新

  • 协商性

  • 分布与共享

  • 自相似性

  • 动态性与多样性

  • 自治性与管理的多重性


6952243

网格的主体和客体

  • 网格主体:网格活动的驱动者,是网格中具有自主活动能力的实体,网格中的活动指令都是由网格主体发出的。

    • 网格用户

    • 网格应用

  • 网格客体:网格中不能自主活动的实体,是网格活动进行的场所、操作实现的工具等。

  • 网格策略:哪个主体,在什么时间,对哪个客体进行什么操作?


6952243

网格主体、客体和策略间的关系

客体

主体

策略


6952243

网格的分类

  • 按网格客体分类

  • 按网格主体分类

  • 按网格策略分类


6952243

按网格客体分类

  • 资源网格—计算网格、数据网格

  • 信息网格

  • 知识网格


6952243

网格应用

知识网格

信息网格

资源网格(计算网格,数据网格)

网格物理基础(Internet,局域网)


6952243

高性能计算机

系统的共享存取

计算网格

信息服务网格

应用软件和

信息资源的共享存取

数据库和文件系统的共享存取

数据网格

网格的分类


6952243

按网格主体分类

  • 科学研究网格:Grid Physics Network, EDGrid

  • 游戏网格:butterfly.net

  • 制造网格

  • 访问网格:AccessGrid

  • 。。。


6952243

按网格策略分类

  • 网格研究刚开始,最终定位不清

  • 研究人员对网格策略有不同的观点

  • 策略分类很少研究


6952243

网格分类

根据网格所管理的资源类型

  • 计算网格:强调计算力获取、管理等的网格;

  • 数据网格:强调数据存储、管理、传输、处理的网格;

  • 信息网格:强调信息存储、管理、传输、处理的网格;

  • 服务网格:强调应用服务集成的网格;

  • 知识网格:强调知识存储、管理、传输、处理的网格;

  • 语义网格:强调语义解析的网格,实现语义互操作;


Classes of grid services types of grids

Classes of Grid Services / Types of Grids

Utility Grid

  • Computational Services – CPU cycles

    • Pooling computing power: [email protected], TeraGrid, AusGrid, ChinaGrid, IndiaGrid, UK Grid,…

  • Data Services

    • Collaborative data sharing generated by instruments, sensors, persons: LHC Grid, Napster

  • Application Services

    • Access to remote software/libraries and license management—NetSolve

  • Interaction Services

    • eLearning, Virtual Tables, Group Communication (Access Grid), Gaming

  • Knowledge Services

    • The way knowledge is acquired, processed and managed—data mining.

  • Utility Computing Services

    • Towards a market-based Grid computing: Leasing and delivering Grid services as ICT utilities.

Users

Knowledge Grid

Interaction Grid

ASP Grid

Data Grid

infrastructure

Computational Grid


6952243

IBM的网格远景:现在的计算机


6952243

未来:因特网是计算机!


6952243

UI

JDL

作业提交过程示例

Replica

Catalogue

(RC)

Information

Service (IS)

Resource

Broker (RB)

Storage

Element

(SE)

Logging &

Book-keeping

(LB)

Job Submission

Service (JSS)

Compute

Element CE)


6952243

Input Sandbox

UI

JDL

Job Submit

Event

Job Status

作业提交过程示例

submitted

Replica

Catalogue

(RC)

Information

Service (IS)

Resource

Broker (RB)

Storage

Element

(SE)

Logging &

Book-keeping

(LB)

Job Submission

Service (JSS)

Compute

Element (CE)


6952243

waiting

UI

JDL

Job Status

作业提交过程示例

submitted

Replica

Catalogue

(RC)

Information

Service (IS)

Resource

Broker (RB)

Storage

Element

(SE)

Logging &

Book-keeping

(LB)

Job Submission

Service (JSS)

Compute

Element (CE)


6952243

UI

JDL

ready

Job Status

作业提交过程示例

submitted

Replica

Catalogue

(RC)

Information

Service (IS)

waiting

Resource

Broker

(RB)

Storage

Element

(SE)

Logging &

Book-keeping

(LB)

Job Submission

Service (JSS)

Compute

Element (CE)


6952243

UI

JDL

scheduled

BrokerInfo

Job Status

作业提交过程示例

submitted

Replica

Catalogue

(RC)

Information

Service (IS)

waiting

ready

Resource

Broker (RB)

Storage

Element (SE)

Logging &

Book-keeping

(LB)

Job Submission

Service

(JSS)

Compute

Element (CE)


6952243

UI

JDL

Input Sandbox

running

Job Status

作业提交过程示例

submitted

Replica

Catalogue

(RC)

Information

Service (IS)

waiting

ready

scheduled

Resource

Broker (RB)

Storage

Element

(SE)

Logging &

Book-keeping

(LB)

Job Submission

Service (JSS)

Compute

Element (CE)


6952243

UI

JDL

running

Job Status

Job Status

作业提交过程示例

submitted

Replica

Catalogue

(RC)

Information

Service (IS)

waiting

ready

scheduled

Resource

Broker (RB)

Storage

Element

(SE)

Logging &

Book-keeping

(LB)

Job Submission

Service (JSS)

Compute

Element (CE)


6952243

UI

JDL

done

Job Status

Job Status

作业提交过程示例

submitted

Replica

Catalogue

Information

Service

waiting

ready

scheduled

Resource

Broker

running

Storage

Element

Logging &

Book-keeping

Job Submission

Service

Compute

Element


6952243

UI

JDL

outputready

Output Sandbox

Job Status

Job Status

作业提交过程示例

submitted

Replica

Catalogue

Information

Service

waiting

ready

scheduled

Resource

Broker

running

Storage

Element

done

Logging &

Book-keeping

Job Submission

Service

Compute

Element


6952243

UI

JDL

Output Sandbox

cleared

Job Status

作业提交过程示例

submitted

Replica

Catalogue

(RC)

Information

Service (IS)

waiting

ready

scheduled

Resource

Broker (RB)

running

Storage

Element

(SE)

done

Logging &

Book-keeping

(LB)

Job Submission

Service (JS)

outputready

Compute

Element (CE)


Grid scenario

我们看到一些东西,

但太落弱.

请模拟加强信号

并识别!

NCSA

Potsdam

WashU

1Tbit/sec

Hong

Kong

RZG

Thessaloniki

LANL

LANL

Now..

Grid Scenario

资源代理:

NCSA + Carching

但是,需要

10Gbit/sec…

OK! 资源预计:

需要 5TB, 2TF.

什么地方可完成此任务?

资源代理:

LANL 是最好的匹配…


6952243

PC Devices

software

Supercomputer

网格挑战

单一视图?

Storage

Instruments

统一访问?

统一管理?

Info.

Sensors

按需服务?

Networks

Applications

Persons


6952243

  • 服务网格

  • Virtualization of services

  • Dynamic service provisioning

  • Self-healing of services

  • Integratable with Enterprise applications

共享程度

  • 合作网格

  • DOE, UK Grid & DoD

  • 协同共享

  • 公用的数据中心

  • 动态的提供资源

  • 企业网格

  • Toshiba, TI, GM

  • Cluster-to-cluster sharing management

  • Reliable file transfer & staging

  • User account mapping, Firewalls, Kerboros

互联网服务提供方

企业间及合作伙伴

企业内部

time

2000

2008

2004

1996

网格三个阶段


Using a grid a user s perspective

Using a grid: A user’s perspective

  • Enrolling and installing grid software

  • Logging onto the grid

  • Queries and submitting jobs

  • Data configuration

  • Monitoring progress and recovery

  • Reserving resources


Using a grid an administrator s perspective

Using a grid: An administrator’s perspective

  • Planning

  • Installation

  • Managing enrollment of donors and users

  • Certificate authority

  • Resource management

  • Data sharing


Using a grid an application developer s perspective

Using a grid: An application developer’s perspective

  • Globus

  • Grid Middleware

  • Grid Portal etc.


Building a grid the project manager s view

Building a Grid -The Project Manager’s View

  • Keys to success:

    • Realize that grids are built, not bought!

    • Early identification of business drivers and potential applications for the grid project

  • Doing these things should help you quickly identify:

    • Is there a good business case for building a grid?

    • What’s the right kind of grid to build?

      • Desktop or Server Aggregation?

      • Integrated or Toolkit?


Building a grid the project manager s view1

Building a Grid -The Project Manager’s View

  • Use a Lifecycle Project Model, e.g.

    • Requirements: identify apps, users and their needs

    • Initial Planning: scope out hardware, middleware

    • Prototype: build a testbed

    • Review results

    • Final Planning: gap analysis for production implementation

    • Deploy: purchase & install hardware, software; training for users

    • Maintain: break-fix, identify and gridify other apps

    • (Iterate!)


Building a grid the systems administrator s view

Building a Grid -The Systems Administrator’s View

  • Establish installation and operational standards

  • Establish security infrastructure to manage grid identities

  • Establish resource registry infrastructure

  • Install grid middleware and configure for appropriate services, e.g.

    • Compute engines

    • Data sources

  • Establish grid identities for services and users

  • Work with users to gridify their applications


The grid a brief history

The Grid:A Brief History

  • Early 90s

    • Gigabit testbeds, metacomputing

  • Mid to late 90s

    • Early experiments (e.g., I-WAY), academic software projects (e.g., Globus, Legion), application experiments

  • 2002

    • Dozens of application communities & projects

    • Major infrastructure deployments

    • Significant technology base (esp. Globus ToolkitTM)

    • Growing industrial interest

    • Global Grid Forum: ~500 people, 20+ countries


The evolution of the grid

The evolution of the Grid

  • The First Generation

    • The mid 1990’s

    • To provide computational resources to a range of high-performance applications.

    • FAFNER: RSA public key encryption algorithm

      • Setup to factor RSA130 using a new numerical technique called the Number Field Sieve(NFS) factoring method using computational Web servers.

    • I-WAY(The Information wide area year): an experimental high-performance network linking many high-performance computers and advanced visualization environments(CAVE).

    • To produce metacomputing environment by integrating resources from opposite ends of the computing spectrum.


Fafner factoring via network enabled recursion

FAFNERFactoring via Network-Enabled Recursion

  • RSA公钥加密算法(Rivest, Shamri以及Adelman )

    • 前提:很大的数很难进行因数分解

  • 实现方法:

    • 通过Web进行因数分解(原来EMAIL)

    • 并行算法:NFS(Number Field Sieve)

    • 瓶颈:Web服务器的层次化

    • 效果:很小的计算能力也可以有贡献


I way information wide area year

I-WAYInformation Wide Area Year

  • 不是建造新的网络,而是集成已有的网络

  • ATM(Asynchronous Transfer Mode) 网络技术

  • 接口管理I-POP(Installed point-of-presence )服务器

    • I-Soft的标准软件环境,试图解决异构,扩展性,性能以及安全等问题。

    • CRB(Computational Resource Broker)

    • 低层的通信库Nexus

    • 应用

      • 超级计算

      • 访问远程资源

      • 虚拟现实

      • 视频、Web等


6952243

对比


The i way experiment 1995

The I-WAY Experiment (1995)

Source: Linda Winkler and Richard Foster


The evolution of the grid cont

The evolution of the Grid(cont.)

  • The Second of Generation

    • There are three main issues:

      • Heterogeneity、Scalability、adaptability

    • Requirements for the data and computation infrastructure

      • Administrative hierarchy

      • Communication services

      • Information services

      • Naming services

      • Distributed fie system and caching

      • Security and authorization

      • System status and fault tolerance

      • Resource management and scheduling

      • User and administrative GUI


The evolution of the grid cont1

The evolution of the Grid(cont.)

  • Core technologies

    • Globus: provides a software infrastructure that enables applications to handle distributed heterogeneous computing resources as a single virtual machine.

    • Legion: provided the software infrastructure so that a system of heterogeneous, geographically distributed, high-performance machines could interact seamlessly.


The evolution of the grid cont2

The evolution of the Grid(cont.)

  • Distributed object systems

    • CORBA

    • Jini and RMI

    • The common component architecture forum


The evolution of the grid cont3

The evolution of the Grid(cont.)

  • Grid resource brokers and schedulers

    • Batch and scheduling systems

      • Condor

      • The portable batch system(PBS)

      • The SUN Grid engine(SGE)

      • The load sharing facility(LSF)

    • Storage resource broker: to provided ‘uniform access to distributed storage’ across a range of storage devices via a well-defined API.

    • Nimrod/G resource broker and GRACE


The evolution of the grid cont4

The evolution of the Grid(cont.)

  • Grid portals

    • Allows application scientist and researchers to access resources specific to a particular domain of interest via a Web interface.

    • The NPACI HotPage

    • The SDSC Grid port toolkit

    • Grid Portal Development Kit (GPDK)

    • Gridsphere


The evolution of the grid cont5

The evolution of the Grid(cont.)

  • Integrated systems

    • Cactus

    • DataGrid

    • UNICORE

    • Webflow

  • Peer-to-Peer computing

    • JXTA


The evolution of the grid cont6

Grid computing

E-Science

Agents

e-Business

Web services

The evolution of the Grid(cont.)

  • The Third Generation

    • Serviced-oriented architectures

      • Web services

      • The Open Grid Services Architecture(OGSA) framework

      • Agents


Some characteristics of grids

Some Characteristics of Grids

Numerousresources

Owned by multiple organizations & individuals

Connected by

heterogeneous,

multi-level networks

Different security

requirements

& policies

Different resource

management

policies

Geographically

distributed

Unreliable resources and environments

Resources are

heterogeneous

Slide by Hiro


What are grid benefits

What are Grid benefits?

  • Resource sharing across multiple administrative boundaries

    • Effective utilisation of the (existing) resources

    • Dynamic provisioning

  • Application Acceleration

  • Scalability

  • Reliability

  • Virtualisation:

    • applications, services, resources,…


  • Login