Triana
This presentation is the property of its rightful owner.
Sponsored Links
1 / 45

Triana PowerPoint PPT Presentation


  • 181 Views
  • Uploaded on
  • Presentation posted in: Sports / Games

Triana. Dr Matthew Shields, School of Computer Science, Cardiff University. A Problem Solving Environment and Grid Workflow Tool. Course Outline. Introduction History and Overview Tools and Toolboxes, Components and Groups How to compose a workflow Triana as a workflow editor.

Download Presentation

Triana

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Triana

Triana

Dr Matthew Shields, School of Computer Science, Cardiff University

A Problem Solving Environment and Grid Workflow Tool


Course outline

Course Outline

  • Introduction

    • History and Overview

    • Tools and Toolboxes, Components and Groups

    • How to compose a workflow

    • Triana as a workflow editor


Course outline cont

Course Outline cont.

  • Service-oriented Components

    • Web services as components

    • Web service workflow

    • Peer 2 Peer services as components

    • Distributed service workflow

  • Grid-oriented Components

    • Grid file and job primitives as components

    • Complex Grid workflow


Install triana

Install Triana

  • Unpack the triana.tar.gz to a location of your choice

  • Set env $TRIANA (Unix) or %TRIANA% (Windows) to the unpacked directory

  • Run $TRIANA/bin/triana (Unix) or %TRIANA/bin/triana.bat (Windows)

  • Course website http://www.trianacode.org/coregrid/


About triana

About Triana

  • Distributed Problem Solving Environment

    • Workflow Based

  • Multiple Component Domains

    • Signal Processing (original domain)

    • Image Processing

    • Text Processing

    • Audio

  • Composite Components


Our research motivation

Our Research Motivation

  • Based on research within the Triana project:

    • Triana - a vehicle for investigating research into distributed systems

    • a workflow-based Problem Solving Environment

  • Goals:

    • To make the access of distributed services as seamless as possible

    • To connect heterogeneous Grids

    • To abstract the core capabilities needed for service-based computing (in P2P, Web services or Grid Computing)


Research scope

Research Scope

  • Triana is currently used in many projects

  • GridOneD - Radio Astronomy

  • DART - Distributed Audio Retrieval with Triana

  • WHiP - Workflows Hosted in Portals

  • Data-Mining Grid

  • BioDiversity World

    • See http://www.trianacode.org/collaborations


Components and definitions

Components and Definitions

  • Component is unit of execution

  • Components are defined in XML files:

    • Naming information

    • Input and output ports

    • Parameter information

  • Why Components?

    • To simplify the application design process and to speed up application development


Triana workflow

Triana Workflow

  • Triana is inherently flow based

    • Data flow - data arriving at component triggers execution

    • Control flow - control commands trigger execution

  • Decentralised execution

    • Data or Control messages sent along communication “pipes” from sender to receiver causes receiver to execute

    • Synchronous or Asynchronous messaging (Implementation dependant)

    • Multiple inputs can block or trigger immediately (Component designer defined)


Taskgraph

Taskgraph

  • Internal object based workflow graph representation

    • Taskgraph - DAG

    • Tasks

    • Connections

  • External XML representation

    • Simple XML syntax

    • List of participating Task definitions

    • Parent/Child connection

    • Hierarchical (Compound components)


Simple workflow example

Simple Workflow Example

Try It! $TRIANA/bin/triana


Example continued

Example Continued

  • Image is converted to Grey scale and then thresholded (set image less than 136 to black and everything higher to white).

  • difference is then taken between this image and a image where the white lines are thinned out (i.e. detail is taken away).

  • The resulting image is basically the detail of the white parts which were pruned by the ShrinkWhite unit which gives an outline of an image.

  • Group the unit and save it as a new compound component


Coalescing binary search

Coalescing Binary Search

GEO 600 Coalescing Binary Search Algorithm implemented as a Triana workflow


Triana the gat and the gap

Grid services

Triana, the GAT and the GAP

Service Based

Computing:

Grid Computing:

Deployment, discovery and communication with distributed services e.g. P2P and (GSI) Web services

Job Submission, File services

A Graphical Grid Computing Environment or Portal

GAP Interface

GAT Interface

P2PS

JXTA

Web

Services

Condor

Unicore

GridFTP

GRMS

WSRF

Globus RLS

PBS

.NET

GridLab

P2PS

Discovery

UDDI

JXTA

Discovery

SOAP

P2PS

Pipes

SSH

SGE

LDR

Other..

JXTA

Pipes


Gap interface

GAP Interface

P2PS

JXTA

Web

Services

P2PS

Discovery

UDDI

JXTA

Discovery

SOAP

P2PS

Pipes

JXTA

Pipes

GAP Interface

  • Motivation by GAT (GridLab.org)

  • A Simple Service based API, for

    • Service Deployment,

    • Service Discovery

    • Pipe Based Communication

  • Static application interface with multiple middleware bindings

    • P2PS (name…?)

    • JXTA

    • Web services


Service oriented comms

Service Discovery

Dynamic?

Decentralized?

Communication

Message Format

SOAP?

Transport Protocol

TCP?

UDP?

Service Oriented Comms

en_fr

hello

network

bonjour

BabelFish

GAP

babelfish.

altavista.

com


Gap interface1

GAP Interface

  • Simple Java API

    • Peer-to-Peer Discovery

    • Pipe-Based Communication

  • Multiple Bindings

    • JXTA

    • P2PS

    • Web Services (WSPeer)


P2p gap bindings

P2P GAP Bindings

  • Two Main Bindings

    • Jxta

      • legacy implementation, due to upgrade soon…

    • P2PS

      • Simple (lightweight) P2P System

      • Uses in Triana, NRL (sensor nets) and Simulations (within NS-2)

      • Useful in highly dynamic environments

      • Based on some Jxta principles i.e. pipes, rendezvous nodes, dynamic discovery, etc

      • Capable of building small-world networks (centralised-decentralised)


P2ps overview

Discovery Service

Broadcast/Locate Adverts + Queries in Discovery Subnet

Pipe Service

Connect Pipes using Endpoint Resolvers

Rendezvous Service

Send/Receive Adverts + Queries from known Rendezvous Peers

Resolvers

Pluggable Network Interfaces

For Ns2 simulation environment

P2PS Overview

P2PS Peer

Discovery

Service

Rendezvous

Service

Pipe

Service

UDP Resolver

TCP Resolver

??? Resolver

UDP

TCP

???

Network


Service creation discovery and invocation

Service Creation, Discovery, and invocation

  • Drag a Audio.Output.Play unit to a new scratchpad

  • Right click on the Play unit and select create service

    • Select Local Service [P2PS] and press Create

    • Leave the default settings and click ok

  • Use menu File -> New to create a new scratchpad

  • Select menu Services -> Discover Services

    • Type Play in the search name panel and click ok

  • You should see a number of remote Play units

  • Drag one of the remote Play units to the new scratch pad

  • Drag a Audio.Input.LoadSound unit and connect to the play

  • Download a sound file from the tutorial site http://www.trianacode.org/coregrid/

  • Double click the LoadSound unit or right click and select properties, choose the sound file, click ok, click the play button.


Service deployment

Service Deployment

  • Peer-to-peer services

  • Distributing Audio !!!

  • Generic example - can try and distribute using other networks

  • Uses multicast or rendezvous to discover services

  • Shows the basic interface to distribution

  • Can publish compound/group componnents

  • Publish as Web services

    • Need a running UDDI server for service publication and discovery


Wspeer

WSPeer

  • High Level Interface to Web Services

    • Discovery

    • Invocation

    • Deployment

    • Hosting

  • Abstract from usual Web Service Discovery and Communication Mechanisms (i.e. UDDI and HTTP)

    • P2PS Web Service Discovery?

  • Uses Apache AXIS as SOAP Engine

  • Extends Capabilities of Apache AXIS

    • Stubless Invocation (including complex types)

    • Non Standard Transports (i.e. P2PS)


Wspeer1

locate

publish

publish

locate

deploy

deploy

invoke

invoke

launch server

UDDI

HTTP

Server

WSPeer

Application

deploy

publish

locate

invoke

WSPeer – P2PS

WSPeer – HTTP/UDDI


Web service choreography

Web Service Choreography

  • Web services

  • Connecting Web services and integrating with other Triana units

  • Xmethods - http://www.xmethods.com/

  • Services -> Import Services

    • Bible Verses

    • http://www.stgregorioschurchdc.org/wsdl/Bible.wsdl

  • Drag the readBible service to a new scratch pad

  • Connect a Common.String.StringGen and a Common.String.StringViewer component

  • Double click StringGen for the input window

    • Psalm 119:105

    • Matthew 5:3-12,44-48

    • Matthew 5:3-12;John 1:1-5

  • Press Play, double click StringViewer to see the output


Multiple services

Multiple Services

  • Xmethods - http://www.xmethods.com/

  • Services -> Import Services

    • Urdu Translator

    • http://www.apniurdu.com/SOAP/Urdu2.wsdl

  • Drag the Translate service to the scratch pad

  • Drag a Common.Control.Duplicator unit the scratch pad

  • Connect readBible to Duplicator

  • connect one output from Duplicator to the StringViewer

  • Connect the other output to the Translator service

  • Drag a new StringViewer to the scratch pad and connect the output of the Translator to that

  • Press play and double click the string viewers

  • Try different inputs for StringGen


Web services resource framework

Web Services Resource Framework

  • WSRF defines conventions for accessing stateful resources (WS-Resources) via Web service interactions

  • Resources are refered to using EPR in message headers

  • In Triana we refer to EPR/WS-addresses as a “context” for extendibility, i.e. WS-Security context

  • Example using Globus 4 CounterService example

  • Import http://d220.astro.cf.ac.uk:8080/wsrf/services/CounterService?wsdl


Triana

  • The link between context and WSTypeGen is a “trigger” node to create a control dependency

  • Right click on WSTypeGen1, select Node Editor

  • Select “In Params” tab, click add, select “Trigger node only” select ok

  • Double click the WSTypeGen unit, type a number into the “int” field

  • To set the context on the Add service, double click or right click and select “Set Context”

  • Press play

  • Drag another WSTypeGen unit and Add service onto the scratch pad, set another integer and play again


Gridlab gat

GridLab GAT

  • Application-level API (Gridlab.org)

  • Abstract ‘Common’ Capabilities required by Grid Applications

    • Resource Discovery

    • Job Submission

    • File Transfer

  • Capabilities extracted through a number of Grid Application Scenarios.

  • Makes life as a Grid programmer easier …

  • Motivation for the GGF SAGA research group (Simple API for Grid Applications)


Using the gat

Using the GAT

GATContext context = new GATContext();

Preferences prefs = new Preferences();

prefs.put("ResourceBroker.adaptor.name", "grms");

URI exe = new URI("file:////bin/hostname");

File outFile = GAT.createFile(context, prefs, new URI("file:///out")

SoftwareDescription sd = new SoftwareDescription();

sd.setLocation(exe);

sd.setStdout(outFile);

ResourceDescription rd = new HardwareResourceDescription();

rd.put("hostname", "bouscat.cs.cf.ac.uk");

JobDescription jd = new JobDescription(sd, rd);

ResourceBroker broker = GAT.createResourceBroker(context, prefs);

Job job = broker.submitJob(jd);


Visual gat

Visual GAT

  • GAT Primitives represented as Workflow Objects

    • Job Tasks

    • File Tasks

  • Grid Data Flow implicit in Workflow Connections

    • File Task -> File Task = Data Transfer

    • File Task -> Job Task = Pre-Stage File

    • File Task -> Local Task = File Read

  • Grid Operations handled by GAT Adaptors

    • Grid File -> Grid File = GridFTP

    • HTTP File -> Local Task = HTTP

    • Grid File -> GRMS Job = GRMS Pre-Stage


Grid workflow in triana

Grid Workflow in Triana

File Write

File

(Local/Remote)

Pre-Stage

Job

(Local/Remote)

Post-Stage

Temporary

File

File Read

Local Java

Unit


Visual gat alternative

Visual GAT Alternative

  • Grid Operations Expressed Explicitly

    • GridFTP Tasks

    • File Write Tasks

    • File Read Tasks

  • Not Intuitive

  • Not Future Proof


File input http

GridLab GAT

File CPI

Resource Broker CPI

HTTP

Adaptor

GridFTP

Adaptor

Data

Movement

Local

Adaptor

GRMS

Adaptor

GRAM

Adaptor

Local

Adaptor

File Input (HTTP)

Triana

WaveToy.par

www.

gridlab.

org


File staging gridftp

File Staging (GridFTP)

Triana

GridLab GAT

File CPI

Resource Broker CPI

HTTP

Adaptor

GridFTP

Adaptor

Data

Movement

Local

Adaptor

GRMS

Adaptor

GRAM

Adaptor

Local

Adaptor

WaveToy.par

www.

gridlab.

org

Testbed


Job submission 1

Job Submission 1

Triana

GridLab GAT

File CPI

Resource Broker CPI

HTTP

Adaptor

GridFTP

Adaptor

Data

Movement

Local

Adaptor

GRMS

Adaptor

GRAM

Adaptor

Local

Adaptor

GAS

GRMS

Cactus

www.

gridlab.

org

Testbed

Data

Mvment

Testbed

WaveToy.par

Delphoi

Mercury

iGrid


Job submission 2

Job Submission 2

Triana

GridLab GAT

File CPI

Resource Broker CPI

Instant

Mess

HTTP

Adaptor

GridFTP

Adaptor

Data

Movement

Local

Adaptor

GRMS

Adaptor

GRAM

Adaptor

Local

Adaptor

Portal

GAS

GRMS

Cactus

www.

gridlab.

org

Testbed

Data

Mvment

Testbed

Delphoi

Mercury

iGrid


Job submission gram

Job Submission (GRAM)

Triana

GridLab GAT

File CPI

Resource Broker CPI

Instant

Mess

HTTP

Adaptor

GridFTP

Adaptor

Data

Movement

Local

Adaptor

GRMS

Adaptor

GRAM

Adaptor

Local

Adaptor

Portal

GRAM

Cactus

www.

gridlab.

org

Testbed

Testbed


Gridmonsteer

WSPeer

User Interface

Web Service

GridMonSteer

Resource

Broker

(e.g. GRMS)

Grid Side

submit

run

User Side

Application

(e.g. Triana)

GridMonSteer (gms)

gms http://www.xyx.org:4802/axis/gms cactus

monitor

cactus

phi_xy.jpg

phi_yz.jpg

phi_xy.jpg

phi_yz.jpg

rundir

http://www.xyz.org/axis/gms

invoke using SOAP


Triana grid workflow summary

Triana Grid Workflow Summary

  • GridLab GAT

    • Application-level API

    • Abstracts Common Capabilities required by Grid Applications such as Triana

  • Visual GAT

    • GAT Primitives represented in Triana as Workflow Objects

    • Grid Data Flow implicit in Workflow Connections

  • GridMonSteer

    • Generic Architecture for Monitoring and Steering Legacy Applications


Future

Future

  • Java GAT 2.0 (coming soon Vrije Universiteit)

    • globus-4 web services, a windows shares adaptor (SAMBA/CIFS), an initial set of glite adaptors

  • Mrs DART - Distributed Audio Retrieval with Triana

    • Distribute music analysis workflows to P2P network to analyse user digital audio collections and make content based recommendations

  • Alchemist

    • Generic version of DART framework for distributed scientific search and analysis

  • WHiP - Workflows Hosted in Portals

    • Share workflows in collaborative Web portals for scientific collaboration and knowledge capture


Links

Links

  • Triana - http://www.trianacode.org/

  • WSPeer - http://www.wspeer.org/

  • WHiP - http://www.whipplugin.org/

  • Mrs DART - http://www.mrsdart.com/

  • Java GAT - http://www.cs.vu.nl/ibis/javagat.html


  • Login