Developer apis to condor a tutorial on condor s web service interface
1 / 39

- PowerPoint PPT Presentation

  • Updated On :

Developer APIs to Condor + A Tutorial on Condor’s Web Service Interface. Interfacing Applications w/ Condor. Suppose you have an application which needs a lot of compute cycles You want this application to utilize a pool of machines How can this be done?. Some Condor APIs.

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
Download Presentation

PowerPoint Slideshow about '' - jaden

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
Developer apis to condor a tutorial on condor s web service interface

Developer APIs to Condor+A Tutorial on Condor’s Web Service Interface

Interfacing applications w condor
Interfacing Applications w/ Condor

  • Suppose you have an application which needs a lot of compute cycles

  • You want this application to utilize a pool of machines

  • How can this be done?

Some condor apis
Some Condor APIs

  • MW (previous talk)

  • Command Line tools

    • condor_submit, condor_q, etc


  • Condor GAHP

  • Condor Perl Module

  • SOAP

Command line tools
Command Line Tools

  • Don’t underestimate them!

  • Your program can create a submit file on disk and simply invoke condor_submit:

    system(“echo universe=VANILLA > /tmp/condor.sub”);

    system(“echo executable=myprog >> /tmp/condor.sub”);

    . . .

    system(“echo queue >> /tmp/condor.sub”);

    system(“condor_submit /tmp/condor.sub”);

Command line tools1
Command Line Tools

  • Your program can create a submit file and give it to condor_submit through stdin:

    PERL: fopen(SUBMIT, “|condor_submit”);

    print SUBMIT “universe=VANILLA\n”;

    . . .

    C/C++: int s = popen(“condor_submit”, “r+”);

    write(s, “universe=VANILLA\n”, 17/*len*/);

    . . .

Command line tools2
Command Line Tools

  • Using the +Attribute with condor_submit:

    universe = VANILLA

    executable = /bin/hostname

    output = job.out

    log = job.log

    +webuser = “zmiller”


Command line tools3
Command Line Tools

  • Use -constraint and –format with condor_q:

    % condor_q -constraint ‘webuser==“zmiller”’

    -- Submitter: : <> :


    213503.0 zmiller 10/11 06:00 0+00:00:00 I 0 0.0 hostname

    % condor_q -constraint 'webuser=="zmiller"' -format "%i\t" ClusterId -format "%s\n" Cmd

    213503 /bin/hostname

Command line tools4
Command Line Tools

  • condor_wait will watch a job log file and wait for a certain (or all) jobs to complete:

    system(“condor_wait job.log”);

  • can specify a timeout

Command line tools5
Command Line Tools

  • condor_q and condor_status –xml option

  • So it is relatively simple to build on top of Condor’s command line tools alone, and can be accessed from many different languages (C, PERL, python, PHP, etc).

  • However…


  • DRMAA is a GGF standardized job-submission API

  • Has C (and now Java) bindings

  • Is not Condor-specific -- your app could submit to any job scheduler with minimal changes (probably just linking in a different library)

  • SourceForge Project


  • Easy to use, but

  • Unfortunately, the DRMAA API does not support some very important features, such as:

    • Two-phase commit

    • Fault tolerance

    • Transactions

Condor gahp
Condor GAHP

  • The Condor GAHP is a relatively low-level protocol based on simple ASCII messages through stdin and stdout

  • Supports a rich feature set including two-phase commits, transactions, and optional asynchronous notification of events

  • Is available in Condor 6.7.X

Gahp cont
GAHP, cont


R: $GahpVersion: 1.0.0 Nov 26 2001 NCSA\ CoG\ Gahpd $


R: E


R: E




R: S $GahpVersion: 1.0.0 Nov 26 2001 NCSA\ CoG\ Gahpd $

S: INITIALIZE_FROM_FILE /tmp/grid_proxy_554523.txt

R: S


R: S


R: S 0


R: S 1

R: 100 0


R: S

Condor perl module
Condor Perl Module

  • Perl module to parse the “job log file”

  • Recommended instead of polling w/ condor_q

  • Call-back event model

  • (Note: job log can be written in XML)

Soap interface

  • Simple Object Access Protocol

    • Mechanism for doing RPC using XML

      (typically over HTTP or HTTPS)

    • A World Wide Web Consortium (W3C) standard

  • SOAP Toolkit: Transform a WSDL to a client library

Benefits of a condor soap api
Benefits of a Condor SOAP API

  • Condor becomes a service

    • Can be accessed with standard web service tools

  • Condor accessible from platforms where its command-line tools are not supported

  • Talk to Condor with your favorite language and SOAP toolkit

Condor soap api functionality
Condor SOAP API functionality

  • Submit jobs

  • Retrieve job output

  • Remove/hold/release jobs

  • Query machine status

  • Query job status

Getting machine status via soap


over HTTP

Getting machine status via SOAP

Your program



Machine List

SOAP library

The api

  • Core API, described with WSDL, is designed to be as flexible as possible

    • File transfer is done in chunks

    • Transactions are explicit

  • Wrapper libraries aim to make common tasks as simple as possible

    • Currently in Java and C#

    • Expose an object-oriented interface

Things we will cover
Things we will cover

  • Condor setup

  • Necessary tools

  • Job Submission

  • Job Querying

  • Job Retrieval

  • Authentication with SSL and X.509

    • An important addition in late 6.7

Condor setup
Condor setup

  • Start with a working condor_config

  • The SOAP interface is off by default

    • Turn it on by adding ENABLE_SOAP=TRUE

  • Access to the SOAP interface is denied by default

    • Set ALLOW_SOAP and DENY_SOAP, they work like ALLOW_READ/WRITE/…

    • See section 3.7.4 of the v6.7 manual for a description

    • Example: ALLOW_SOAP=*/*

Necessary tools
Necessary tools

  • You need a SOAP toolkit

    • Apache Axis (Java) -

    • Microsoft .Net -

    • gSOAP (C/C++) -

    • ZSI (Python) -

    • SOAP::Lite (Perl) -

  • You need Condor’s WSDL files

    • Find them in lib/webservice/ in your Condor release

  • Put the two together to generate a client library

    • $ java org.apache.axis.wsdl.WSDL2Java condorSchedd.wsdl

  • Compile that client library

    • $ javac condor/*.java

All our examples are in Java using Apache Axis

Helpful tools
Helpful tools

  • The core API has some complex spots

  • A wrapper library is available in Java and C#

    • Makes the API a bit easier to use (e.g. simpler file transfer & job ad submission)

    • Makes the API more OO, no need to remember and pass around transaction ids

  • We are going to use the Java wrapper library for our examples

    • You can download it from

    • Will be included in Condor release

Submitting a job

Explicit bits

Implicit bits

Submitting a job

  • The CLI way…


universe = vanilla

executable = /bin/cp

arguments = cp.sub cp.worked

should_transfer_files = yes

transfer_input_files = cp.sub

when_to_transfer_output = on_exit

queue 1

clusterid = X

procid = Y

owner = matt

requirements = Z

$ condor_submit cp.sub

Submitting a job1

Repeat to submit multiple clusters

Repeat to submit multiple jobs in a single cluster

Submitting a job

  • The SOAP way…

    • Begin transaction

    • Create cluster

    • Create job

    • Send files

    • Describe job

    • Commit transaction

Submission from java

1. Begin transaction

2. Create cluster

3. Create job

4&5. Send files & describe job

6. Commit transaction

Submission from Java

Schedd schedd = new Schedd(“http://…”);

Transaction xact = schedd.createTransaction();


int cluster = xact.createCluster();

int job = xact.createJob(cluster);

File[] files = { new File(“cp.sub”) };

xact.submit(cluster, job, “owner”, UniverseType.VANILLA, “/bin/cp”, “cp.sub cp.worked”, “requirements”, null, files);


Submission from java1

Schedd’s location

Max time between calls (seconds)

Job owner, e.g. “matt”

Requirements, e.g. “OpSys==\“Linux\””

Extra attributes, e.g. Out=“stdout.txt” or Err=“stderr.txt”

Submission from Java

Schedd schedd = new Schedd(“http://…”);

Transaction xact = schedd.createTransaction();


int cluster = xact.createCluster();

int job = xact.createJob(cluster);

File[] files = { new File("cp.sub") };

xact.submit(cluster, job, “owner”, UniverseType.VANILLA, “/bin/cp”, “cp.sub cp.worked”, “requirements”, null, files);


Querying jobs
Querying jobs

  • The CLI way…

$ condor_q

-- Submitter: localhost : <> : localhost


1.0 matt 10/27 14:45 0+02:46:42 C 0 1.8 sleep 10000

42 jobs; 1 idle, 1 running, 1 held, 1 unexpanded

Querying jobs1

Also, getJobAds given a constraint, e.g. “Owner==\“matt\””

Querying jobs

  • The SOAP way from Java…

String[] statusName = { “”, “Idle”, “Running”, “Removed”, “Completed”, “Held” };

int cluster = 1;

int job = 0;

Schedd schedd = new Schedd(“http://…”);

ClassAd ad = new ClassAd(schedd.getJobAd(cluster, job));

int status = Integer.valueOf(ad.get(“JobStatus”));

System.out.println(“Job is “ + statusName[status]);

Retrieving a job
Retrieving a job “Owner==\“matt\””

  • The CLI way..

  • Well, if you are submitting to a local Schedd, the Schedd will have all of a job’s output written back for you

  • If you are doing remote submission you need condor_transfer_data, which takes a constraint and transfers all files in spool directories of matching jobs

Retrieving a job1

Discover available files “Owner==\“matt\””

Remote file

Local file

Retrieving a job

  • The SOAP way in Java…

int cluster = 1;

int job = 0;

Schedd schedd = new Schedd(“http://…”);

Transaction xact = schedd.createTransaction();


FileInfo[] files = xact.listSpool(cluster, job);

for (FileInfo file : files) {

xact.getFile(cluster, job, file.getName(), file.getSize(), new File(file.getName()));



Authentication for soap
Authentication for SOAP “Owner==\“matt\””

  • Authentication is done via mutual SSL authentication

    • Both the client and server have certificates and identify themselves

  • Possible in late-late 6.7 (available by 6.8)

  • It is not always necessary, e.g. in some controlled environments (a portal) where the submitting component is trusted

  • A necessity in an open environment -- remember that the submit call takes the job’s owner as a parameter

    • Imagine what happens if anyone can submit to a Schedd running as root…

Authentication setup
Authentication setup “Owner==\“matt\””

  • Create and sign some certificates

  • Use OpenSSL to create a CA

    • -newca

  • Create a server cert and password-less key

    • -newreq && -sign

    • mv newcert.pem server-cert.pem

    • openssl rsa -in newreq.pem -out server-key.pem

  • Create a client cert and key

    • -newreq && -sign && mv newcert.pem client-cert.pem && mv newreq.pem client-key.pem

Authentication config
Authentication config “Owner==\“matt\””

  • Config options…

    • ENABLE_SOAP_SSL is FALSE by default


      • Set this to a different port for each SUBSYS you want to talk to over ssl, the default is a random port

      • Example: SCHEDD_SOAP_SSL_PORT=1980

    • SOAP_SSL_SERVER_KEYFILE is required and has no default

      • The file containing the server’s certificate AND private key, i.e. “keyfile” after

        cat server-cert.pem server-key.pem > keyfile

Authentication config1
Authentication config “Owner==\“matt\””

  • Config options continue…

    • SOAP_SSL_CA_FILE is required

      • The file containing public CA certificates used in signing client certificates, e.g. demoCA/cacert.pem

  • All options except SOAP_SSL_PORT have an optional SUBSYS_* version

    • For instance, turn on SSL for everyone except the Collector with



One last bit of config
One last bit of config “Owner==\“matt\””

  • The certificates we generated have a principal name, which is not standard across many authentication mechanisms

  • Condor maps authenticated names (here, principal names) to canonical names that are authentication method independent

  • This is done through mapfiles, given by SEC_CANONICAL_MAPFILE and SEC_USER_MAPFILE

  • Canonical map: SSL .*emailAddress=(.*)* \1

  • User map: (.*) \1

  • “SSL” is the authentication method, “.*emailAddress….*” is a pattern to match against authenticated names, and “\1” is the canonical name, in this case the username on the email in the principal

Https with java
HTTPS with Java “Owner==\“matt\””

  • Setup keys…

    • keytool -import -keystore truststore -trustcacerts -file demoCA/cacert.pem

    • openssl pkcs12 -export -inkey client-key.pem -in client-cert.pem -out keystore

  • All the previous code stays the same, just set some properties


    • Example: java Example https://…

Questions? “Owner==\“matt\””