1 / 8

Enhancing Scientific Workflows: Specifications and Future Directions

The breakout session led by Matthew Addis at the NeSC workshop on December 5, 2003, aimed to deepen the understanding of workflow languages and engines in scientific applications. The goal was to identify the requirements needed for effective scientific workflows, enabling a comparison of existing projects. Key areas of focus included performance, scheduling, fault tolerance, and manageability. The session emphasized the necessity of collaboration in developing a common architecture for workflow systems, targeting improvements in scalability, monitoring, and adaptability to enhance scientific research.

ava-huff
Download Presentation

Enhancing Scientific Workflows: Specifications and Future Directions

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Workflow languages and engines breakout Matthew Addis IT Innovation 5 December 2003 NeSC workshop on workflow services

  2. Workflow languages and engines breakout • Objective: • Better understand the requirements for workflow languages and engines in scientific applications • Allow comparison of work already done by projects employing scientific workflow/dataflow • Approach • Identify the different areas/types of requirements • Identify the different levels at which these requirements might exist using a ‘stack’ type approach

  3. Requirements areas • Performance • Scheduling • Discovery • Events/monitoring/reporting • Fault tolerance • Scalability • Launching/invocation/execution • Steering/interaction/control • Manageability

  4. Stack

  5. Positioning of existing systems

  6. Some characteristics to consider when differentiating existing approaches Execution policies/approaches, e.g. data flows Models and structures: e.g. DAGs Data model and types We need to pull together and reuse existing body of work in this area

  7. Plumbing group • Streams are important in science • Not supported in commercial systems • Dependability • Detection of failures • Propagation of exceptions • Handling • Optimisation • Quality of service • Networks, computers, sets of resources • Choices of data formats and transfer mechanisms • Manageability • Monitor what’s going on • Control over execution and services • Dynamic adaptation of workflow

  8. Next steps • Establish email discussion group • More work on describing existing systems with respect to the areas we’ve identified • Report • Ideally: • Common research and development • What is the smallest reference architecture that satisfies most of what we want

More Related