1 / 7

Production Repository Service

The Production Repository service enables secure remote access to production definitions, job generation, browsing and status monitoring. It includes implicit locking, security based on DISET infrastructure, logging of commands, and a potential web-based user interface.

trudyr
Download Presentation

Production Repository Service

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Production Repository Service Presenter: Gennady Kuznetsov & Andrey Tsaregorodsev ……

  2. Introduction • The Production Repository service is designed to provide the following capabilities: • Storing definitions of productions with remote secure access by productionmanagers • Job generation according to production definition and submission to the DIRAC WMS; • Production definitions browsing and status monitoring; • The major difference compared to the previous version will be: • jobs will not be prepared in advance, but generated on demand; • access to the repository will be via “Production Repository” Service and not in a file system; • implicit locking of the Productions to efficiently resolve conflicts of concurrent access; • security based on DISET infrastructure; • logging of all important commands accepted by Repository Service; • investigating possibility of introducing Web based user interface Presenter Name:Gennady Kuznetsov

  3. Rep.Dbase Productions Sub. Requests Vectors Repository Service Production Manager Production Repository Service Publish Production Processing Submission Request Service Publish Submission Request Publish Vector of Files to process Job Generator TransformationService Tempdisk Job Submission DIRAC WMS Presenter Name:Gennady Kuznetsov

  4. Database Layout1 PRODUCTION TABLE production_id production_name production_description workflow_name created_by timestamp production_status last_job_submitted jobs_prepeared jobs_total active_request_id production_body The persistent data required for the service will be stored in MySQL data base. The following layout proposed for the database structure. Stores productions created by Production Manager Presenter Name:Gennady Kuznetsov

  5. Database Layout2 SUBMISSIONREQUEST TABLE request_id production_id created_by timestamp jobn_start_from jobn_quantity jobn_counter user_proxy Stores Submission Requests, made by Production Manager on duty, and stores his proxy. Stores input vectors used to create jobs (one table per Production) VECTOR_{prod_id} TABLE job_id timestamp vector_body Presenter Name:Gennady Kuznetsov

  6. Rep.Dbase Productions Sub. Requests Vectors Remote Repository publishProduction (production);Defines specific Production deleteProduction (production);stopProduction (production);startProduction (production);setTotalJobs (production, n_jobs); Sets maximum number of jobs for the Production prepareJob (production, in_vector); Creates Jobs input vectors (one per Job) submitJob (production, n_jobs, proxy); Creates Submission Request and sends jobs to DIRAC WMS getSubmissionStatus (submissionID); Gets the Submission Request status information DIRAC WMS Presenter Name:Gennady Kuznetsov

  7. Scenario • A typical production scenario will contain the following steps: • Production manager defines the production workflow in the Production Console and publishes it in the Production Repository ( publishProduction() ) • For the given production the total intended number of jobs is defined ( setTotalJobs() ) • For MC jobs this preparation phase is done. For the reprocessing jobs the production definition is complemented by providing Input Data Vectors; this is done typically by a Transformation Agent automatically or by a production manager by hand ( prepareJob() ) • Production manager browsers the current production definitions and status and submits jobs according to the state of the resources ( submitJob() ). A progress bar is provided to follow the submission process (getSubmissionStatus() ). This operation can be replaced by an automatic application (not part of the Repository Service) which will feed jobs to the WMS taking into account its occupancy. • Productions can be locked/unlocked according to operational needs. • A possibility to resubmit jobs is provided. Presenter Name:Gennady Kuznetsov

More Related