architectural models for resource management in the grid l.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
Architectural Models for Resource Management in the Grid PowerPoint Presentation
Download Presentation
Architectural Models for Resource Management in the Grid

Loading in 2 Seconds...

play fullscreen
1 / 30

Architectural Models for Resource Management in the Grid - PowerPoint PPT Presentation


  • 410 Views
  • Uploaded on

$grid Architectural Models for Resource Management in the Grid Rajkumar Buyya Monash University, Australia Steve Chapin, Syracuse University, USA David DiNucci Elepar.com, USA Outline Changes in Computing Landscape Resource Management Issues Architectural Models

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Architectural Models for Resource Management in the Grid' - mike_john


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
architectural models for resource management in the grid

$grid

Architectural Models for Resource Management in the Grid

Rajkumar Buyya

Monash University, Australia

Steve Chapin,

Syracuse University, USA

David DiNucci

Elepar.com, USA

outline
Outline
  • Changes in Computing Landscape
  • Resource Management Issues
  • Architectural Models
  • Hierarchical Resource Management
  • Abstract Owner
  • Market Model
  • Economy Grid
  • Nimrod/G Resource Broker
  • Conclusions
computing power hpc drivers
Computing Power (HPC) Drivers
  • Solving grand challenge applications using computer modeling, simulation and analysis

Aerospace

Internet &

Ecommerce

Life Sciences

Digital Biology

CAD/CAM

Military Applications

Military Applications

Military Applications

computing platforms

2100

2100

2100

2100

2100

2100

2100

2100

2100

Computing Platforms

?

PERFORMANCE

Breaking Administrative Barriers

  • Individual
  • Group
  • Department
  • Campus
  • State
  • National
  • Globe
  • Inter Planet
  • Universe

Desktop

SMPs or SuperComputers

Global

Cluster/Grid

Inter Planet

Cluster/Grid ??

Local

Cluster

Enterprise

Cluster/Grid

towards grid computing
Towards Grid Computing

Unification of geographically distributed resources

what is grid
What is Grid ?
  • An infrastructure that couples:
    • Computers – PCs, workstations, clusters, supercomputers, laptops, notebooks,

mobile devices, PDA, etc;

    • Software – e.g., ASPs renting expensive special purpose applications on demand;
    • Catalogued data and databases – e.g. transparent access to human genome database;
    • Special devices – e.g., radio telescope – SETI@Home searching for life in galaxy, Austrophysics@Swinburne for pulsars)
    • People/collaborators.
  • & offers dependable, consistent, & pervasive access to resources.
sources of complexity in grid resource management
Sources of Complexity in Grid Resource Management
  • No single administrative control.
  • No single ownership policy:
    • Each resource owner has their own policies or scheduling mechanisms;
    • Users must honour them (particularly external Grid users).
  • Heterogeneity of resources.
  • Dynamic availability – may appear and disappear…
sources of complexity in grid resource management9
Sources of Complexity in Grid Resource Management
  • Unreliable resource – disappear from view!
  • No uniform cost model - varies from one user’s resource to another and from time of day.
  • No single access mechanism – Web, custom interfaces, command line…
grid resource management issues
Grid Resource Management Issues
  • Authentication (once).
  • Specify (code, resources, etc.).
  • Discover resources.
  • Negotiate authorization, acceptable use, Cost, etc.
  • Acquire resources.
  • Schedule Jobs.
  • Initiate computation.
  • Steer computation.
  • Access remote data-sets.
  • Collaborate with results.
  • Account for usage.
  • Discover resources.
  • Negotiate authorisation,
  • acceptable use, Cost, etc.
  • Acquire resources.
  • Schedule jobs.
  • Initiate computation.
  • Steer computation.

Domain 1

Domain 2

Ack: Globus..

slide12

Global

Scheduler

Global

Scheduler

Global

Scheduler

Global

Scheduler

Global

Scheduler

Hierarchical Resource Management

Access/Admission Control Agent

User

Grid Information Service

Persistent Job Control

Agent

Connection Cloud

Monitor

Local

Scheduler

Deployment Agent

Domain

Resource

Manager or Control Agent

Control Domain

- Task

Resource

who owns the grid
Who owns the GRID?

Talk to

people

Powerappliances

Use GRID

resources

I want to:

My interface is:

I arrange serviceand payments with a:

(may be many choices)

Abstract

Owner (AO)

Phone co.

Electric co.

HPC

Networks

Instruments

People

Antennae

Cable/fiber

Switches

Generators

Power lines

Transformers

But resources I “get”may belong to others:

ao is owner or broker
AO is owner or broker

User

  • User negotiates with AO through “order window”
  • That AO may own some resources, and/or it may broker with other AOs for those resources
  • After negotiation, resources are delivered through “pickup window”

Requests

Resources

Order

Window

Pickup

Window

AO

Order

Order

Pickup

Pickup

Manager

ResourceManager

Sales

Delivery

AO3

Physical

Resource

AO2

AO1

ao resources
AO Resources
  • Resources are objects
  • Classes are
    • Instrument
      • Data source, sink, transform
      • e.g. programs, people, files, data collection devices
    • Channel
      • Moves data among instruments
    • Complexes of above
  • Attributes define sizes, times, connections, etc.

Instrument

(File)

Instrument

(Program)

Channels

Instrument

(File)

Instrument

(Program)

Instrument

(Telescope)

Instrument

(Person)

negotiating with an ao
Negotiating with an AO

Make dummy resource

(with attributes set to constants, variables, or “don’t care”) + bid + delivery plan+ variable constraints

Pick one,

Try again,

Or give up

Assign tasksto resource,use, relinquish

User

Perhapslater...

Delivery

Window

Order Window

Resource candidates

(values for variables/attributes+ asking price for each)

AO

Resource

many testbeds who pays

$grid

Many Testbeds ? & who pays ?

GUSTO

EcoGrid

Legion Testbed

NASA IPG

testbeds so far observations
Testbeds so far -- observations
  • Who contributed to resources & why ?
    • Volunteers: for fun, challenge, fame, public good like SETI@Home & distributed.net projects.
    • Collaborators: sharing resources while developing new technologies of common interest – Globus, Legion, Ecogrid.
  • How long ?
    • Short duration: GUSTO decommissioned.
  • What do we need ? Grid Marketplace!
    • Regulates demand and supply, offers incentive for being players, simple, scalable solution, quasi-deterministic – proven model in real-world.
users in grid economy strategy
Users in Grid Economy & Strategy
  • Grid Consumers
    • Execute jobs for solving varying problem size and complexity
    • Benefit by selecting and aggregating resources wisely
    • Tradeoff timeframe and cost
      • Strategy: minimise expenses
  • Grid Providers
    • Contribute “idle” resource for executing consumer jobs
    • Benefit by maximizing resource utilisation
    • Tradeoff local requirements & market opportunity
      • Strategy: maximise returns on services
building of a economy grid brokerage system
Building of a Economy Grid “brokerage” system…..

Who pays for that ???

Users!

Foundation for the Grid Economy

grid architecture for computational economy
Grid Architecture for Computational Economy

Information Server(s)

Grid Market Services

Sign-on

Health

Monitor

Info ?

Grid Node N

Grid Explorer

Application

Secure

Job Control

Agent

Grid Node1

Schedule Advisor

QoS

Pricing Algorithms

Trade Server

Trading

Trade Manager

Accounting

Resource

Reservation

Misc. services

Deployment Agent

JobExec

Resource Allocation

Storage

Grid User

Grid Resource Broker

R1

R2

Rm

Grid Middleware

Services

Grid Service Providers

economic models for trading
Economic Models for Trading
  • Commodity Market Model
  • Posted Prices Models
  • Bargaining Model
  • Tendering (Contract Net) Model
  • Auction Model
    • English, first-price sealed-bid, second-price sealded-bid (Vickrey), and Dutch.
  • Proportional Resource Sharing Model
  • Shareholder Model
  • Partnership Model
economy grid globus grace
Economy Grid = Globus + GRACE

Applications

Grid

Apps.

Science

Engineering

Commerce

Portals

ActiveSheet

High-level Services and Tools

GlobusView

Grid Status

Grid

Tools

Nimrod/G

DUROC

MPI-G

MPI-IO

CC++

globusrun

Core Services

Heartbeat Monitor

Nexus

GRACE-TS

Grid

Middleware

GRAM

Globus Security Interface

GASS

DUROC

MDS

GBank

GARA

GMD

Grid

Fabric

Local Services

GRD

QBank

JVM

Condor

TCP

UDP

eCash

LSF

PBS

Linux

Irix

Solaris

what is nimrod g
What is Nimrod/G ?
  • A resource broker for managing and steering task farming (parametric sweep) applications on computational Grids based on deadline and computational economy.
  • Key Features
    • A single window to manage & control experiment
    • Resource Discovery
    • Trade for Resources
    • Scheduling
    • Steering & data management
  • It allows to study the behaviour of some of the output variables against a range of different input scenarios.
nimrod g grid broker architecture
Nimrod/G Grid Broker Architecture

Nimrod/G Client

Nimrod/G Client

Nimrod/G Client

Nimrod/G Engine

Schedule Advisor

Trading Manager

Grid

Store

Grid Dispatcher

Grid Explorer

Grid Middleware

Globus,Legion, Condor-g,, Ninf,etc.

TM TS

GE GIS

Grid Information Server(s)

RM & TS

RM & TS

RM & TS

G

C

L

G

Legion enabled node.

Globus enabled node.

L

C

RM: Local Resource Manager, TS: Trade Server

Condor enabled node.

a nimrod g client

Cost

A Nimrod/G Client

Deadline

Legion hosts

Globus Hosts

Bezek is in both

Globus and Legion Domains

slide28

Global Economy Grid

Australia

North America

Monash Uni.:

ANL: SGI/Sun/SP2

USC-ISI: SGI

UVa: Linux Cluster

Manitoba: Cluster

Nimrod/G

Linux cluster

Globus+Legion

+Condor/G

Globus/Legion

GRACE_TS

Solaris WS

Internet

Europe

ZIB/FUB: T3E/Mosix

Cardiff: Sun E6500

Paderborn: HPCLine

Lecce: Compaq SC

CNR: Cluster

CERN: Cluster

Asia/Japan

Tokyo I-Tech.:

ETL, Tuskuba

Linux cluster

Globus +

GRACE_TS

Globus +

GRACE_TS

conclusions
Conclusions
  • Proposed three models for Grid resource management architecture
    • Hierarchical, AO, & Market-model
  • The future systems are likely follow a model that combines all these models.
  • The future computing (HPC) infrastructure is going to be a Grid of Clusters.
  • Peer-to-Peer/Grid has already become a darling of venture capitalists.
  • The impact of Grid on 21st century economy will be the same as electricity on 20th century economy.