Towards intelligent workflow planning for neuroimaging analyses
Download
1 / 20

Towards Intelligent Workflow Planning for Neuroimaging Analyses - PowerPoint PPT Presentation


  • 64 Views
  • Uploaded on

Towards Intelligent Workflow Planning for Neuroimaging Analyses. Irfan Habib, Ashiq Anjum, Peter Bloodsworth, Richard McClatchey Centre for Complex Cooperative Systems, BIT, University of the West of England, Bristol. Introduction.

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about ' Towards Intelligent Workflow Planning for Neuroimaging Analyses' - marsha


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
Towards intelligent workflow planning for neuroimaging analyses
Towards Intelligent Workflow Planning for Neuroimaging Analyses

Irfan Habib, Ashiq Anjum, Peter Bloodsworth, Richard McClatchey

Centre for Complex Cooperative Systems, BIT, University of the West of England, Bristol


Introduction
Introduction Analyses

  • Recent progress in neuroimaging techniques and data formats has led to an explosive growth in neuroimaging data

  • Analysis of this data can facilitate research in neuro-degenerative diseases.


Commercial Partners Analyses

Academic Partners

Clinical Users

http://www.neugrid.eu



CIVET produces 1100% more data than it consumes, and intermediate data usage is more than 4000%.

Without optimisation runtime of a single workflow is 8 hrs


Civet pipeline
CIVET Pipeline intermediate data usage is more than 4000%.

85% of All Tasks in CIVET execute in less than 512 secs


Civet pipeline1
CIVET Pipeline intermediate data usage is more than 4000%.

These 85% of tasks in CIVET perform just 8% of the computation


Existing approaches
Existing Approaches intermediate data usage is more than 4000%.

  • State-of-the-art approaches for workflow planning include:

    • Data-based Methods: Data elimination, data diffusion

    • Task-based Approaches: Task Clustering

    • Scheduling-based Approaches


Task clustering
Task Clustering intermediate data usage is more than 4000%.

CIVET

Normalised Workflow turnaround time (with respect to standard CIVET on SGE Cluster)


Task clustering1
Task Clustering intermediate data usage is more than 4000%.

CIVET

Normalised Cumulative Data Retrieval (with respect to standard CIVET on SGE Cluster)


What are the issues
What are the issues? intermediate data usage is more than 4000%.

  • Different clustering strategies work for different types of workflows.

  • A specific automated horizontal task clustering strategy created a computationally efficient workflow in this case.


What are the issues? intermediate data usage is more than 4000%.

Coarse-grained Tasks with High-level of data-interdependencies

More Coarse Grained Tasks

Fine-grained Tasks with Low-level of data-interdependencies

Higher Data Affinity


What are the issues? intermediate data usage is more than 4000%.

  • Creating an efficient workflow plan involves consideration of several trade-offs!

    • Various parameters need to be optimised: Data efficiency, scheduling latency, workflow turn-around time, network latencies.

  • Hence workflow planning is a multi-dimensional optimisation problem.


This paper proposes an initial single-objective genetic algorithm based workflow planning approach.


B1 algorithm based workflow planning approach.

C2

C4

C3

B2

C3


B1 algorithm based workflow planning approach.

B1

B1

B1

B1

B1

C4

C4

C4

C4

C4

C2

C3

C3

C3

C3

C3

C4

Enact Workflow

Grid

C3

Store Provenance Data

B2

Provenance Storage

C3

Randomly Planned User Submitted Workflows


Fitness algorithm based workflow planning approach.Calculation

Selection

Genetic operators

Pipeline Service Planner

Provenance Data


Implementation of the Approach algorithm based workflow planning approach.

  • The workflow planning approach will first be simulated in SimGRID.

  • Various parameters for the planning approach will be tweaked and evaluated

    • Type of selection producing the quickest convergence towards efficiency

    • Extending fitness functions for multi-objectives


Conclusion
Conclusion algorithm based workflow planning approach.

  • Several workflow planning techniques exist, however prior knowledge about the nature of the workflow is required to select an appropriate technique.

  • This paper proposes a single-objective evolutionary workflow planning approach to optimise workflow turn-around times.

  • The approach will be first implemented in a SimGrid environment and results will be shared in future publications.


ad