common execution environment
Download
Skip this Video
Download Presentation
Common Execution Environment

Loading in 2 Seconds...

play fullscreen
1 / 23

Common Execution Environment - PowerPoint PPT Presentation


  • 86 Views
  • Uploaded on

Common Execution Environment. Kate Keahey OOI Cyberinfrastructure Life Cycle Objectives Milestone Review, Release 1 San Diego, CA February 23-25, 2010. Overview. Use Cases Architecture Overview CEI Components in R1 CEI Elements Elastic Provisioning Unit Infrastructure as a Service

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about ' Common Execution Environment' - shalin


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
common execution environment

Common Execution Environment

Kate Keahey

OOI Cyberinfrastructure Life Cycle Objectives

Milestone Review, Release 1

San Diego, CA

February 23-25, 2010

overview
Overview
  • Use Cases
  • Architecture Overview
  • CEI Components in R1
  • CEI Elements
    • Elastic Provisioning Unit
    • Infrastructure as a Service
    • Deployable Type management

OOI CI LCO Review, Feb 2010

use case overview
Use Case Overview
  • Highly Available Services
  • Rapidly provision resources
  • - Scale to demand

OOI CI LCO Review, Feb 2010

architecture overview
Architecture Overview

VM

(Deployable Unit)

VM

(Deployable Unit)

Application

Software

(Deployable Type)

VM

(Deployable Unit)

EPU Worker

(Operational Unit)

EPU Worker

(Operational Unit)

EPU

HA Service

(OOI Application)

EPU Worker

(Operational Unit)

Availability and

Scaling Strategy

OOI CI LCO Review, Feb 2010

architecture overview1
Architecture Overview

OOI CI LCO Review, Feb 2010

taskable resources
Taskable Resources

Deployable Type

e.g., a description of an environment in rBuilder

Deployable Unit

e.g., a VM image

Start/Stop

Monitorable

Controllable

Taskable Resources

HA Service

Operational Unit

Process

e.g., a core or application service

Simple or complex deployed appliance, e.g., a deployed VM instance, a virtual cluster, platforms for workflow execution

e.g., an executing program or a long-running service

OOI CI LCO Review, Feb 2010

architecture overview2
Architecture Overview

OOI CI LCO Review, Feb 2010

architecture overview3
Architecture Overview

OOI CI LCO Review, Feb 2010

cei service interactions
CEI Service Interactions

Registry

Planner

Fault Monitoring

and

Compensation

Provisioner

Controller

(“Executive”)

Resource

(Taskable)

OOI CI LCO Review, Feb 2010

2970-00001 OV2 CEI

cei service interaction
CEI Service Interaction

Deployable

Type

Registry

Monitoring

Information

Planner

VM info

Other Info

Exchange

Proc Info

Provisioner

Controller

IaaS Adapter

IaaS

Context Broker

OOI CI LCO Review, Feb 2010

elastic processing unit
Elastic Processing Unit
  • Elastic Processing Unit (EPU): High-Availability Services with policy based scaling to demand

EPU Worker

N1

EPU Worker

N2

EPU Worker

NX

EPU Controller

[N, Deployable Unit]

Register HA service

Msg to N

OOI CI LCO Review, Feb 2010

elastic processing unit1
Elastic Processing Unit

Deployable

Type

Registry

Monitoring

Information

Planner

Will this whole thing work (even in a small way)?

VM info

Other Info

Exchange

Proc Info

Provisioner

Controller

IaaS Adapter

IaaS

Context Broker

OOI CI LCO Review, Feb 2010

epu prototype 1
EPU: Prototype 1
  • EPU design and prototype:
    • Policy-configurable planner/scheduler
    • Scaling policy: size of work queue
    • All components integrated with Magnet/AMQP

OOI CI LCO Review, Feb 2010

epu prototype 2
EPU: Prototype 2
  • Updates technology choices
    • Uses Redis to distribute work to VM instances and as Information Aggregator
  • New policy
    • load
  • Demonstration later

OOI CI LCO Review, Feb 2010

epu technology choices
EPU: Technology Choices
  • IaaS Adapter: still evaluating
    • Deltacloud, jcloud, libcloud, dasein, fog
    • Standards: OCCI-WG
  • Nimbus Context Broker
  • Redis for Information storage

OOI CI LCO Review, Feb 2010

epu elaboration iterations
EPU: Elaboration Iterations
  • Atomic HA for EPU controller
  • Detailed design on interaction with COI
  • Scalability, scalability, scalability
  • Finalize technology choices
  • Prototype end-to-end EPU
  • Stress testing and harness
  • Investigate storage/compute/network
  • Refine implementation

OOI CI LCO Review, Feb 2010

infrastructure as a service
Infrastructure-as-a-Service

Deployable

Type

Registry

Monitoring

Information

Planner

VM info

Other Info

Exchange

Proc Info

Can we adapt existing resources?

Provisioner

Will it work with COI?

Controller

Where are we going to get IaaS resources?

IaaS Adapter

IaaS

Context Broker

OOI CI LCO Review, Feb 2010

infrastructure as a service1
Infrastructure-as-a-Service
  • Targeted deployment platforms
    • Commercial (AWS, rackspace)
    • Scientific (FutureGrid, Magellan, …)
  • IaaS for science
    • The Nimbus Workspace Service
    • Evaluate Ganeti/SGE/Condor
  • Nimbus
    • Make implementation technology-independent
    • Make implementation WSRF-independent
    • Provide user management functions

OOI CI LCO Review, Feb 2010

deployable types
Deployable Types

Deployable

Type

Registry

Monitoring

Information

Planner

complexity?

VM info

Other Info

Exchange

Proc Info

Provisioner

Controller

IaaS Adapter

IaaS

Will it work with COI?

scalability?

Context Broker

OOI CI LCO Review, Feb 2010

deployable types1
Deployable Types
  • Complexity
    • Document requirements and use cases
      • Web App, Hadoop, Job Queue, NFS, Torque
    • Design overall approach
    • Preliminary roundup and evaluation of relevant technologies
      • Puppet, Chef, CohesiveFT, rBuilder
      • Working choices: rBuilder and Chef
    • Prototype integration of Chef with the Nimbus Context Broker

OOI CI LCO Review, Feb 2010

deployable types2
Deployable Types
  • Protocol-independence
    • Make the Context Broker WSRF-independent
    • REST interfaces
  • Scalability and Sustainability
    • Prototype scalability harness
    • Scalability tests
      • Contextualizing up to 500 nodes
      • Challenges in harness design and efficiency

OOI CI LCO Review, Feb 2010

dt elaboration iterations
DT: Elaboration Iterations
  • Finalize technology choices for management of Deployable Types/Units
  • Deployable Type Registry
  • More comprehensive harness and more contextualization scalability tests
  • Understanding performance

OOI CI LCO Review, Feb 2010

slide23

Thanks !

OOI CI LCO Review, Feb 2010

ad