1 / 25

The gLite API – PART I

The gLite API – PART I. Giuseppe LA ROCCA INFN Catania giuseppe.larocca@ct.infn.it ACGRID-II School 2-14 November 2009 Kuala Lumpur - Malaysia. W M P r o x y J a v a A P I ver 3.1.0. Software Requirements Java Class org.glite.wms.wmproxy.WMProxyAPI Support to cog-jglobus-1.4

hunsakert
Download Presentation

The gLite API – PART I

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. The gLite API – PART I Giuseppe LA ROCCA INFN Catania giuseppe.larocca@ct.infn.it ACGRID-II School 2-14 November 2009 Kuala Lumpur - Malaysia

  2. W M P r o x y J a v a A P I ver 3.1.0 • Software Requirements • Java Class org.glite.wms.wmproxy.WMProxyAPI • Support to cog-jglobus-1.4 • Hands-on

  3. Installation guide API JAVA /1 0. Start from a fresh install of Scientific Linux CERN other platforms are also compatible butyet not properly tested Axis (http://ws.apache.org/axis/) where the needed jars available are: axis-1.4-2006-04-22.jar axis2-1.0-2006-05-05.jar axis-jaxrpc-1.2-2005-05-03.jar axis-wsdl4j-1.2-2005-04-12.jar 2. Geronimo J2EE libraries: geronimo-activation_1.0.2_spec-1.1-2006-06-14.jar geronimo-j2ee_1.4_spec-1.1-2006-06-14.jar

  4. 3. Crypto libraries: cryptix-asn1.jar cryptix32.jar bcprov.jar puretls.jar 4. gLite libraries: glite-jdl-api-java.jar glite-wms-wmproxy-api-java.jar glite-security-delegation-java.jar glite-security-trustmanager.jar glite-security-util-java.ja 5. GLOBUS and other libraries: cog-jglobus.jar classad.jar commons-discovery-0.2-2004-03-23.jar commons-logging-api-1.1.jar log4j-1.2.14.jar Installation guide API JAVA /2 W A R N I N G Make sure all the listed jar(s) are included in your CLASSPATH environment variable !!

  5. Warning!! All the needed gLite packages can be found here http://eticssoft.web.cern.ch/eticssoft/repository/org.glite All the needed external packages can be found here http://eticssoft.web.cern.ch/eticssoft/repository/externals Installation guide API JAVA /3

  6. The JDL to be matched must contains some mandatory Requirements and Rank attributes; when using command line interface these are added by default by the UI, but in our case user has to take care to check if his/her JDL file contains them: Requirements = (other.GlueCEStateStatus == "Production"); Rank = (-other.GlueCEStateEstimatedResponseTime); Basic Requirement

  7. The main Java class of this API is: org.glite.wms.wmproxy.WMProxyAPI; A client object can be created using one of these following constructor(s): public WMProxyAPI(String url, String proxyFile) public WMProxyAPI(String url, String proxyFile, String CAcerts) public WMProxyAPI(String url, InputStream proxyStream) public WMProxyAPI(String url, InputStream proxyStream, String CAcerts) org.glite.wms.wmproxy.WMProxyAPI

  8. Therefore, using one of the four constructors WMProxyAPI Client = new WMProxyAPI( … ); a new instance is created. This instance will be created to invoke any other WMProxy services. For example: WMProxyAPI Client = new WMProxyAPI(https://wms-gilda-01.ct.infn.it:7443/glite_wms_wmproxy_server,“/tmp/x509up_u504”);

  9. In case of failure, one of the following exceptions is thrown: org.glite.wms.wmproxy.AuthenticationFaultType org.glite.wms.wmproxy.JobUnknownFaultType org.glite.wms.wmproxy.AuthorizationFaultType org.glite.wms.wmproxy.NoSuitableResourcesFaultType org.glite.wms.wmproxy.GetQuotaManagementFaultType org.glite.wms.wmproxy.ProxyFileException org.glite.wms.wmproxy.GrstDelegationException org.glite.wms.wmproxy.GenericFaultType org.gridsite.www.namespaces.delegation_1.DelegationExceptionType

  10. Before calling submission or list match services, client proxy credentials are requested to be transferred from the client to the server through a delegation process. The following Java code performs this delegation process: The obtained delegation identifier can be reused for several calls to the WMProxy server; Delegation of user credential

  11. The following Java code submits a job and returns a JobID Submit a job to WMProxy

  12. Submission of simple jobs, DAGs, collections and parametric jobs to the WMProxy service requires as input a job description file in which job characteristics and requirements are expressed by means of Condor class-ad-like expressions. In this description the users can specify some files that are needed by the submitted jobs during the execution on the remote Computing Elements. These files are listed in the JDL InputSandbox attribute specifying for each of them the complete URI of its location. If the job does NOT have any file in the InputSandbox to be transferred from the submitting machine to the WMProxy node, the submission can be performed also calling the jobSubmit service: jobId = client.jobSubmit(jdlString, delegationID); Difference between submission and registration /1

  13. Otherwise, these following steps are needed: a preliminary server registration:jobId = client.jobRegister(jdlString, delegationID); transfer of files in the InputSandbox from the client machine to the WMS node; call the jobStart service to trigger the submission: client.jobStart(jobId); Difference between submission and registration /2

  14. In order to allow grid user to invoke, within Java code, the UrlCopyclass to copy files from the WMS Server to the UI and vice versa the following Java package is requested cog-jglobus-1.4-bin.tar.gz The above package, with all the needed documentation, can be downloaded from this web page http://dev.globus.org/wiki/CoG_JGlobus_1.4 Software Requirements

  15. Executable = "/bin/sh"; Arguments = "start_hostname.sh"; StdOutput = "hostname.out"; StdError = "hostname.err"; InputSandbox = {"start_hostname.sh"}; OutputSandbox = {"hostname.err","hostname.out"}; Requirements = (other.GlueCEStateStatus == “Production”); Rank = (-other.GlueCEStateEstimatedResponseTime); RetryCount = 3; #!/bin/sh sleep 5 hostname -f hostname.jdl start_hostname.sh

  16. $ java WMProxyGetProxyAndSubmit \ https://gilda-wms-01.ct.infn.it:7443/glite_wms_wmproxy_server \ hostname.jdl start_hostname.sh | grep -v "ERROR" Connecting to the service https://gilda-wms-01.ct.infn.it:7443/glite_wms_wmproxy_server ====================== glite-wms-job-submission Success ====================== The job has been successfully submitted to the WMProxy Your job identifier is: https://gilda-lb.ct.infn.it:9000/P-H1wp0RpTnos2qGyQ2ETQ =============================================================================== Submit a new request to WMProxy /1 Usage : java WMProxyGetProxyAndSubmit -h[elp] java WMProxyGetProxyAndSubmit <wmproxy_server> <jdl_file > [ <InputSandboxFiles> ] where: <wmproxy_server> ... the entry point of the WMProxy Server to contact (e.g. : https://gilda-wms-01.ct.infn.it:7443/glite_wms_wmproxy_server) (e.g. : https://gilda-wms-02.ct.infn.it:7443/glite_wms_wmproxy_server) <jdl_file> ... the jdl file to submit to the grid <InputSandboxFiles> ... The list of file(s) to transfer

  17. The script, thanks to the UrlCopy Class, performs the copy of the InputSandbox files to the reduced path of the WMS as you can see: cd /var/glite/SandboxDir/P-/ ll https_3a_2f_2fgilda-wms-01.ct.infn.it_3a9000_2fP-H1wp0RpTnos2qGyQ2ETQ/input/ -rwxrwxr-x 1 gilda001 glite 30 Jan 11 09:05 start_hostname.sh Submit a new request to WMProxy /2

  18. When the job finishs you can retrieve the output file(s) as follow: $ java WMProxyGetOutputAndPurge \ https://gilda-wms-01.ct.infn.it:7443/glite_wms_wmproxy_server \ https://gilda-lb.ct.infn.it:9000/P-H1wp0RpTnos2qGyQ2ETQ \ $PWD/ | grep -v "ERROR" Connecting to the service https://gilda-wms-01.ct.infn.it:7443/glite_wms_wmproxy_server ====================== glite-wms-job-output Success ====================== JOB GET OUTPUT OUTCOME Output sandbox files for the job: https://gilda-lb.ct.infn.it:9000/P-H1wp0RpTnos2qGyQ2ETQ have been successfully retrieved and stored in the directory: /home/larocca/Api-Java-WMProxy/ =========================================================================== Retrieve output file(s)

  19. The following Java code cancels a list of jobIDs Cancelling a job

  20. A job can be canceled before it ends using the following Java class: $ java WMProxyCancelJobs \ https://gilda-wms-01.ct.infn.it:7443/glite_wms_wmproxy_server \ https://gilda-lb.ct.infn.it:9000/dwgBZqwoPKkl7v_OK6kiQw \ | grep -v "ERROR" Connecting to the service https://gilda-wms-01.ct.infn.it:7443/glite_wms_wmproxy_server ======================== glite-wms-job-cancel Success ============================= The cancellation request has been successfully submitted for the following job(s): https://gilda-lb.ct.infn.it:9000/dwgBZqwoPKkl7v_OK6kiQw ==================================================================================== Cancelling a job

  21. The following Java code returns a list of CEs available for job submission Listing CE ID(s) that match a job

  22. It is possible to see which CEs are eligible to run a job described by a given JDL using the following Java API: $ java WMProxyGetProxyAndListMatch \ https://gilda-wms-01.ct.infn.it:7443/glite_wms_wmproxy_server \ hostname.jdl start_hostname.sh | grep -v "ERROR" Connecting to the service https://gilda-lb.ct.infn.it:7443/glite_wms_wmproxy_server COMPUTING ELEMENT IDs LIST ========================================================================== The following CE(s) matching your job requirements have been found: - gilda-01.pd.infn.it:2119/jobmanager-lcgpbs-gilda - gn0.hpcc.sztaki.hu:2119/jobmanager-lcgpbs-gilda - grid010.ct.infn.it:2119/jobmanager-lcgpbs-infinite - grid010.ct.infn.it:2119/jobmanager-lcgpbs-long - grid010.ct.infn.it:2119/jobmanager-lcgpbs-short - grid011f.cnaf.infn.it:2119/jobmanager-lcgpbs-gilda - iceage-ce-01.ct.infn.it:2119/jobmanager-lcgpbs-infinite - iceage-ce-01.ct.infn.it:2119/jobmanager-lcgpbs-long - iceage-ce-01.ct.infn.it:2119/jobmanager-lcgpbs-short - ce.hpc.iit.bme.hu:2119/jobmanager-lcgpbs-gilda ========================================================================== Listing CE ID(s) that match a job

  23. Java API Documentation http://trinity.datamat.it/projects/EGEE/wiki/apidoc/3.1/htmljava/index.html

  24. References Datamat – WMProxy quickstart http://trinity.datamat.it/projects/EGEE/wiki/wiki.php?n=WMProxyClient.QuickStart JDL Attributes guide for WMProxy https://edms.cern.ch/document/590869/1 WMProxy user guide https://edms.cern.ch/document/674643/1 GILDA wiki https://grid.ct.infn.it/twiki/bin/view/GILDA/ApiJavaWMProxy https://grid.ct.infn.it/twiki/bin/view/GILDA/WMProxyCPPAPI

  25. Hands-on $ cd ${HOME}/Api-Java-WMProxy To compile Java code run: $ source wmproxy.sh Submit and collect results with the twoJava classes: WMProxyGetProxyAndSubmit WMProxyGetOutputAndPurge • Connect to the training infrastructure using the information reported in the tutorial sheet • Run the hands-on available in this web link: http://www.euasiagrid.org/wiki/index.php/WMProxy_Java_API • Enjoy!

More Related