Programming the grid with components
Download
1 / 32

Programming the Grid with Components - PowerPoint PPT Presentation


  • 58 Views
  • Uploaded on

Programming the Grid with Components. Madhu Govindaraju Aleksander Slominski Dennis Gannon Sriram Krishnan. Outline. Software Components XCAT Component and Services model Web Services Programming model OGSI. Software Components. Analogy with hardware

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about ' Programming the Grid with Components' - joelle-carson


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
Programming the grid with components

Programming the Grid with Components

Madhu Govindaraju

Aleksander Slominski

Dennis Gannon

Sriram Krishnan

Extreme! Computing Lab, Dept. of Computer Science, Indiana University


Outline
Outline

  • Software Components

  • XCAT

    • Component and Services model

    • Web Services

    • Programming model

    • OGSI

Extreme! Computing Lab, Dept. of Computer Science, Indiana University


Software components
Software Components

  • Analogy with hardware

  • Standard design paradigm for app development

  • Goal: to build applications by composition of well tested and well behaved subsystems (components).

    • simplify application design process

    • promote code reusability

    • plug and play

Extreme! Computing Lab, Dept. of Computer Science, Indiana University


Component architecture
Component Architecture

  • A Component Architecture consists of two parts:

    • Components

      • software objects that implement a set of required behaviors

    • Frameworks

      • A runtime environment

      • A set of services used by components

Extreme! Computing Lab, Dept. of Computer Science, Indiana University


Industry standards
Industry Standards

  • COM/DCOM

    • interoperability for Microsoft applications

  • Java Beans, Enterprise Java Beans (EJB)

    • Java based desktop and enterprise applications

  • OMG CORBA Component Model (CCM)

    • language neutral superset of EJB

Extreme! Computing Lab, Dept. of Computer Science, Indiana University


Academic research
Academic Research

  • SciRun from Utah

    • scalable parallel applications and viz

  • Webflow from Syracuse

    • graphical composition palette

  • CCAT and XCAT from Indiana University

    • framework for Grid based applications

Extreme! Computing Lab, Dept. of Computer Science, Indiana University


Cca common component architecture
CCA: Common Component Architecture

  • CCA: Initiative to define minimal specification for scientific components

    • Department of Energy (National Labs)

    • Few universities

  • Aim: build components for high performance computing

  • Draws ideas from CCM and other models as a baseline

  • Targeting

    • Parallel

    • Distributed

Extreme! Computing Lab, Dept. of Computer Science, Indiana University


The common component architecture
The Common Component Architecture

  • Key Idea: dynamic composition by linking “ports”

    • Provides ports: interface of “services” provided

    • Uses ports: a call-site for a service to be provided by another component.

Uses port-

Provides

Port

Extreme! Computing Lab, Dept. of Computer Science, Indiana University


What is new about cca
What is new about CCA?

  • Minimal specification

  • Envision connections as dynamic

    • Can add, remove and connect ports at run-time

  • Tailored to build Problem Solving Environments

    • End user manipulates connections

  • Not specific to any underlying distributed object model

  • Notion of collective ports

    • logically one connection

    • implementation: multiple network connections

Extreme! Computing Lab, Dept. of Computer Science, Indiana University


Components on the grid issues
Components on the Grid: Issues

  • Discovering static information about components

  • Launching components

  • Discovering references for running instances

  • Communication protocols

  • Messages and Events

  • Authentication and Authorization

  • Encapsulating legacy applications

  • Efficient scheduling

  • Run-time environment

Extreme! Computing Lab, Dept. of Computer Science, Indiana University


Grid programming model needs
Grid Programming Model: Needs

  • Need a set of APIs, protocols, libraries and tools that allow access to Grid resources

  • Examples: Globus, Condor

    • provide infrastructure

    • do not provide programming model

  • What is missing?

    • How can we make it easier to “program” the Grid?

Extreme! Computing Lab, Dept. of Computer Science, Indiana University


XCAT

  • Implementation of the CCA specification

  • Designed for distributed applications

  • Allows creation of Grid application from components

  • Wraps Grid Services as components

  • Test-bed for CCA

    • Is there an alternative to OMG IDL?

Extreme! Computing Lab, Dept. of Computer Science, Indiana University


Component communication
Component Communication

  • How do components communicate?

    • Use Remote Procedure Call (RPC) Mechanism

      • XCAT uses SOAP 1.1

      • XCAT ports can serve as web services

    • Events/Messages

      • Objects encoded as XML documents

Extreme! Computing Lab, Dept. of Computer Science, Indiana University


Xcat services architecture
XCAT Services Architecture

  • Default services for all components

  • XCAT services

    • Directory

      • locate components based on port types and other attributes

    • Registry

      • locate running instances of components

    • Creation

      • create running instance of a component

    • Connection

      • connect ports of two running instances

    • Events

      • publish/subscribe framework for messages

Extreme! Computing Lab, Dept. of Computer Science, Indiana University


Ids information directory service
IDS: Information Directory Service

  • Store Component descriptions in XML

  • Used to search databases and remote repositories

MySQL Database

Request component info

Information

Directory

Service

Component

RDBS

File system

Return XML specs.

LDAP

Extreme! Computing Lab, Dept. of Computer Science, Indiana University


Creation service
Creation Service

  • Creates a running instance of another component

  • Encapsulates authentication issues

  • Supports GRAM, SSH and Local-Exec

Globus

resource Y

Launch an instance of

component X on resource Y

Creation

Service

Component

X

Returns: remote reference

to new component instance

Extreme! Computing Lab, Dept. of Computer Science, Indiana University


Registry service
Registry Service

  • Stores instance information (runtime configurations)

  • Used to advertise and search other instances

  • Tradeoff: distributed vs. centralized

Find a running instance

of component X

Registry

Service

Component

Returns: remote reference

to component instance

Extreme! Computing Lab, Dept. of Computer Science, Indiana University


Connection service
Connection Service

  • A component that can be used to connect a “uses” port of one component to the “provides” port of another

  • Can export ports of other components as one’s own

X

Connection

Service

Component

A

Connect port A of component X

to port B of component Y

B

Y

Extreme! Computing Lab, Dept. of Computer Science, Indiana University


Event service

x

x

Event Service

  • Channel for publish-subscribe event model

  • Creation and connection events

  • Application users can use it for asynchronous information transfer

C

Subscribe to

listen for events

of type X

Event Channel

Service

Component

x

Notification of publication

of events of type X

A

x

B

Extreme! Computing Lab, Dept. of Computer Science, Indiana University


Features of events
Features of Events

  • Event/Notification Services are an essential part of all distributed systems

  • Application Events

    • “I am done.” “I just wrote to a file,” “here is a result.”

  • Event Publishers & Listeners

    • Some Listeners subscribe, others poll

    • Must have persistent event channels

      • record events for later analysis

      • application history logs

      • allow for retrieval in chunks

Extreme! Computing Lab, Dept. of Computer Science, Indiana University


Xcat other features
XCAT: Other Features

  • Security:

    • Every remote method call is intercepted

    • port, method and parameter level

    • authentication based on SSL

    • authorization based on simple ACL

  • Component Handle:

    • in XML and can be converted to WSDL

    • GSH /GSR

  • Exceptions

    • Supports throw and catch across components

Extreme! Computing Lab, Dept. of Computer Science, Indiana University


Xcat processes
XCAT Processes

Framework

Impl.

Creation

Service

Creation

Service

Framework

Impl.

Scripts,

Front-end,

Tools

Connection

Service

Connection

Service

Scripts,

Front-end

Tools

Application

Code

Other

Services

Other

Services

Application

Code

Port

Wrapper Code

Extreme! Computing Lab, Dept. of Computer Science, Indiana University


Xcat web services stack
XCAT-Web Services Stack

Extreme! Computing Lab, Dept. of Computer Science, Indiana University


Xcat programming examples
XCAT Programming: Examples

  • Builder: A tool used to select and link components together

  • Builder Options

    • Swing based GUI

    • Jython scripting

    • Portal

  • XCAT

    • Jython scripting

Extreme! Computing Lab, Dept. of Computer Science, Indiana University


CCAT Gui image

Extreme! Computing Lab, Dept. of Computer Science, Indiana University


Scripting xcat applications
Scripting XCAT Applications

import xcat

stringDump = xcat.createComponent(‘StringDump’)

printer = xcat.createComponent(‘Printer’)

xcat.setCreationMechanism(stringDump, ‘gram’)

xcat.setCreationMechanism(printer, ‘ssh’)

xcat.setMachineName(stringDump, “exodus”)

xcat.setMachineName(printer, “rainier”)

Extreme! Computing Lab, Dept. of Computer Science, Indiana University


Scripting xcat applications contd
Scripting XCAT Applications: contd

printer.put(“exec-fqn”,

“samples.printer.Printer”)

stringDump.put(“exec-fqn”, “samples.stringDump.StringDump)

xcat.createInstance(printer)

xcat.createInstance(stringDump)

xcat.connectPorts(stringDump, ‘outputString’,

printer, ‘inputString’)

Extreme! Computing Lab, Dept. of Computer Science, Indiana University


Encapsulating legacy apps
Encapsulating Legacy Apps

  • Common Case

    • Legacy App that reads files and writes files

    • Use a “scripted component”

      • Component runs a python script

    • The App Script

      • Stages files

      • Launches and monitors application

      • Writes output files

      • publishes event streams

Control

Event

Stream

XCAT comp

App

Script

application

output

files

input

files

Extreme! Computing Lab, Dept. of Computer Science, Indiana University


Scripting grid computations
Scripting Grid Computations

  • Grid Scripts are executable documents that

    • Orchestrate Computations

      • Select resources

      • Supply application parameters and launch remote jobs

      • Subscribe to event streams

      • Create components

      • Connect components

Extreme! Computing Lab, Dept. of Computer Science, Indiana University


Composition using components
Composition Using Components

  • Two ways of composing applications

    • Composition in space:

      • one component/service directly invokes the services of another

    • Composition in time:

      • A workflow engine schedules tasks that involve accessing remote services and responding to events

Extreme! Computing Lab, Dept. of Computer Science, Indiana University


Xcat ogsi and web services

GS

GS

Component A

Component B

XCAT, OGSI and Web Services

Grid Service Client

Standard

Grid Service

Extreme! Computing Lab, Dept. of Computer Science, Indiana University


Xcat and ogsi compliance
XCAT and OGSI compliance

  • Add a GridService port to each component

  • Merge component handle with GSH/GSR

  • OGSI messaging is non-reliable push model

    • XCAT has a reliable, persistent model

  • OGSI has factory portType for instantiating new services

    • XCAT has an extended distributed factory model

Extreme! Computing Lab, Dept. of Computer Science, Indiana University


ad