condor birdbath soap interface to condor
Download
Skip this Video
Download Presentation
Condor BirdBath SOAP Interface to Condor

Loading in 2 Seconds...

play fullscreen
1 / 11

Condor BirdBath SOAP Interface to Condor - PowerPoint PPT Presentation


  • 124 Views
  • Uploaded on

Condor BirdBath SOAP Interface to Condor. Charaka Goonatilake Department of Computer Science University College London c.goonatilake@cs.ucl.ac.uk. Background. Motivations. Adding Web Service interfaces to Condor daemons

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Condor BirdBath SOAP Interface to Condor' - JasminFlorian


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
condor birdbath soap interface to condor

Condor BirdBathSOAP Interface to Condor

Charaka Goonatilake

Department of Computer Science

University College London

c.goonatilake@cs.ucl.ac.uk

motivations
Motivations
  • Adding Web Service interfaces to Condor daemons
  • Facilitate the development of third-party applications capable of interacting with Condor (remotely).
    • E.g. build higher-level application specific scheduler that submits jobs to multiple Condor pools based on application semantics
    • These can be built using a wide range of languages/SOAP packages
    • BirdBath has been tested on:
      • Java (Apache Axis, XSUL)
      • Python (ZSI)
      • C# (.Net)
      • C/C++ (gSOAP)
  • Condor accessible from platforms where its command-line tools are not supported/installed
the hypothetical big picture

Schedd

Schedd

Schedd

The (Hypothetical) Big Picture

Site D

Condor-G

Globus/gridSAM

Client

SOAP

SOAP

flocking

Site A

Site B

Site C

condor soap interfaces

Shadow

Condor SOAP Interfaces
  • Now available in Condor 6.7.5
  • Obtain resource ClassAds
  • Obtain all daemon ClassAds
  • Begin/commit/abort transaction
  • Submit Job
  • Obtain Job ClassAds
  • Get/Send file
  • Removed/hold/release job

Execution

machine

Execution

machine

Submission

machine(s)

Central manager

Schedd

Collector

Negotiator

Execution

machine

Execution

machine

clients for birdbath
Clients for BirdBath
  • WSDLs available that describe each daemon interface
  • SOAP libraries (stub code) can be generated automatically from WSDLs

Custom Client

holdJob()

Submission

machine(s)

SOAP library

Schedd

example query collector
Example: Query Collector
  • Obtain information (ClassAds) about all resources that have over 512MB RAM

collectorLocator = new CondorCollectorLocator();

collector = locator.getcondorCollector(new URL(“http://kotturoti.cs.ucl.ac.uk:9618”);

classAds = collector.queryStartdAds(“Memory>512”);

example job submission
Example: Job Submission
    • Create New Transaction
    • Create New JobId
    • Send Files ({input files, binaries})
    • Create Job Description
    • Submit Job
    • Commit Transaction
  • Transaction based process
  • Uses two phase commit to promote reliability and robustness
  • Files transferred in chunks – Base64 encoded SOAP attachments
  • DAGMan jobs supported
omii collaboration gridsam plugin

Client

Client

<?xml version="1.0" encoding="UTF-8"?>

<JobDefinition xmlns="http://…">

<JobDescription>

<Application>

<Executable>/bin/echo</Executable>

<Argument>hello world</Argument>

</Application>

</JobDescription>

</JobDefinition>

JSDL

JSDL

SOAP

SOAP

GridSAM

GridSAM

Condor

Condor

SGE

SGE

Fork

Fork

OMII Collaboration: gridSAM plugin
  • Job Submission & Monitoring Web Service that submits to range of resource managers (e.g. Condor, SGE, Globus)
  • Uses JSDL (Job Submission Description Language) for standardised job description
gridsam plugin

Our plugin accesses Condor via the SOAP interfaces

Condor-WS

  • Building JSDL to ClassAd conversion libraries

ClassAd

SOAP

Submission

machine(s)

Schedd

gridSAM plugin
  • Existing Condor plugin uses shell-based submission via command-line tools and monitoring of log files

Client

JSDL

SOAP

Middle-tier

GridSAM

Condor

SGE

Fork

future work
Future Work
  • Bundle plug-in into gridSAM
    • Also provide separate JSDL to ClassAd conversion library
  • Extensions to other daemons (what functionality to expose?)
    • For example, allow checkpoint data to be retrieved through the shadow in order to migrate jobs across pools
  • Roll out daemons on e-minerals mini-grid
ad