1 / 10

Some ideas for the future

Some ideas for the future. Francesco.Giacomini@cnaf.infn.it WP1 Meeting Milano, 11 th September 2001. Overview. Complex jobs Dependency on Condor Use of existing frameworks. Complex Jobs: Definition. A complex job is a set of jobs with dependencies among them.

amity
Download Presentation

Some ideas for the future

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Some ideas for the future Francesco.Giacomini@cnaf.infn.it WP1 Meeting Milano, 11th September 2001

  2. Overview • Complex jobs • Dependency on Condor • Use of existing frameworks

  3. Complex Jobs: Definition A complex job is a set of jobs with dependencies among them. • Dependencies are temporal and are given, for example, by I/O requirements. • subjob B needs as input the output of subjob A • The definition is very vague and generic.

  4. Complex Jobs:(initial) Assumptions • Dependencies are not cyclical • Otherwise subjobs have to execute concurrently (co-allocation problem) • The definition is not recursive • A subjob cannot be a complex job • Otherwise the semantics of a dependency is more complicated

  5. Complex Jobs: Example A B F • The complex job can be represented as a Direct Acyclical Graph (DAG) C D E

  6. Complex Jobs: Simple Solution • Represent a DAG with a nested ClassAd (?) • “Linearise” the DAG • for the example: A C D B E F • use a “topological sort” algorithm • the solution is not unique • For each subjob send it to the Resource Broker independently • Data representing a dependency is saved in a “temporary” and “anonymous” file

  7. Complex Jobs: Other Issues • Job identifiers • Interaction with LB • What to do in case a subjob fails • Various types of dependencies

  8. Dependency on Condor • In my opinion we should get rid as soon as possible of our dependency on those parts of Condor which will not likely become Open Source (TM) • Starting with Condor-G • job submission is not particularly difficult directly on top of Globus • job monitoring should be done based on info in the LB

  9. Implementing ComputingElement • Implemented the ComputingElement interface directly on top of Globus, so that one can do: ComputingElement ce(“pcfg.cnaf.infn.it”); Job job; job.set_attribute(“executable”, “/bin/sleep”); job.set_attribute(“arguments”, “10”); ce.submit(job); ce.cancel(job);

  10. Using Existing Frameworks • We should evaluate and possibly adopt existing (OO) application frameworks, instead or reinventing the wheel • there was a comment on this from one of the D1.2 reviewers • One possibility is the Adaptive Communication Environment (ACE) (www.cs.wustl.edu/~schmidt/ACE.html)

More Related