slide1 l.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
10.30 - 11.00 Introduction to myGrid 11.00 – 12.30 tutorial (part 1) 12.30 – 1.15 Break 1.15 – 2.30 Tutorial (part2) 2.3 PowerPoint Presentation
Download Presentation
10.30 - 11.00 Introduction to myGrid 11.00 – 12.30 tutorial (part 1) 12.30 – 1.15 Break 1.15 – 2.30 Tutorial (part2) 2.3

Loading in 2 Seconds...

play fullscreen
1 / 44

10.30 - 11.00 Introduction to myGrid 11.00 – 12.30 tutorial (part 1) 12.30 – 1.15 Break 1.15 – 2.30 Tutorial (part2) 2.3 - PowerPoint PPT Presentation


  • 217 Views
  • Uploaded on

Taverna training day in Sheffield 10.30 - 11.00 Introduction to myGrid 11.00 – 12.30 tutorial (part 1) 12.30 – 1.15 Break 1.15 – 2.30 Tutorial (part2) 2.30 – 2.40 break 2.40 – 3.00 sharing my workflows An Introduction to Taverna Workflows Franck Tanoh my Grid University of Manchester

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about '10.30 - 11.00 Introduction to myGrid 11.00 – 12.30 tutorial (part 1) 12.30 – 1.15 Break 1.15 – 2.30 Tutorial (part2) 2.3' - EllenMixel


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
slide1

Taverna training day in Sheffield

10.30 - 11.00 Introduction to myGrid

11.00 – 12.30 tutorial (part 1)

12.30 – 1.15 Break

1.15 – 2.30 Tutorial (part2)

2.30 – 2.40 break

2.40 – 3.00 sharing my workflows

an introduction to taverna workflows

An Introduction to Taverna Workflows

Franck Tanoh

myGrid

University of Manchester

what is my grid
What is myGrid?
  • myGrid is a suite components to support in silico experiments in biology
  • Taverna workbench = myGrid user interface
  • Originally designed to support bioinformatics

Expanded into new areas:

Chemoinformatics

Health Informatics

Medical Imaging

Integrative Biology

  • Open source – and always will be
history
History

EPSRC funded UK eScience Program Pilot Project

omii uk
OMII-UK
  • University of Manchester (myGrid) joined with the Universities of Edinburgh (OGSA-DAI) and Southampton (OMII phase 1) in March 2006
  • OMII-UK aims to provide software and support to enable a sustained future for the UK e-Science community and its international collaborators.
  • A guarantee of development and support
the life science community
The Life Science Community

In silico Biology is an open Community

  • Open access to data
  • Open access to resources
  • Open access to tools
  • Open access to applications

Global in silico biological research

the community problems
The Community Problems
  • Everything is Distributed
    • Data, Resources and Scientists
  • Heterogeneous data
  • Very few standards
    • I/O formats, data representation, annotation
    • Everything is a string!

Integration of data and interoperability of resources is difficult

lots of resources
Lots of Resources

NAR 2007 – 968 databases

traditional bioinformatics
Traditional Bioinformatics

12181 acatttctac caacagtgga tgaggttgtt ggtctatgtt ctcaccaaat ttggtgttgt 12241 cagtctttta aattttaacc tttagagaag agtcatacag tcaatagcct tttttagctt 12301 gaccatccta atagatacac agtggtgtct cactgtgatt ttaatttgca ttttcctgct 12361 gactaattat gttgagcttg ttaccattta gacaacttca ttagagaagt gtctaatatt 12421 taggtgactt gcctgttttt ttttaattgg gatcttaatt tttttaaatt attgatttgt 12481 aggagctatt tatatattct ggatacaagt tctttatcag atacacagtt tgtgactatt 12541 ttcttataag tctgtggttt ttatattaat gtttttattg atgactgttt tttacaattg 12601 tggttaagta tacatgacat aaaacggatt atcttaacca ttttaaaatg taaaattcga 12661 tggcattaag tacatccaca atattgtgca actatcacca ctatcatact ccaaaagggc 12721 atccaatacc cattaagctg tcactcccca atctcccatt ttcccacccc tgacaatcaa 12781 taacccattt tctgtctcta tggatttgcc tgttctggat attcatatta atagaatcaa

cutting and pasting
Cutting and Pasting
  • Advantages:
    • Low Technology on both server and client side
    • Very Robust: Hard to break.
    • Data Integration happens along the way
  • Disadvantages:
    • Time Consuming (and painful!)
      • Can be repeated rarely
      • Limited to small data sets.
    • Error Prone:
      • Poor repeatability

How do you do this for a genome/proteome/metabolome of information!

pipeline programming
Pipeline Programming
  • Advantages
    • Repeatable
    • Allows automation
    • Quick, reliable, efficient
  • Disadvantages
    • Requires programming skills
    • Difficult to modify
    • Requires local tool and database installation
    • Requires tool and database maintenance!!!
what we want as a solution
What we want as a solution

A system that is:

  • Allows automation
  • Allows easy repetition, verification and sharing of experiments
  • Works on distributed resource
  • Requires few programming skills
  • Runs on a local desktop / laptop
my grid as a solution
myGrid as a solution

myGrid allows the automated orchestration of in silico experiments over distributed resources from the scientist’s desktop

Built on computer science technologies of:

  • Web services
  • Workflows
  • Semantic web technologies
web services
Web Services

Web services support machine-to-machine interaction over a network.

Web services are a:

    • technology and standard for exposing code / databases with an API that can be consumed by a third party remotely.
    • describes how to interact with it.

They are:

  • Self-contained
  • Self-describing
  • Modular
  • Platform independent
workflows

RepeatMasker

Web service

GenScan

Web Service

BlastWeb Service

Workflows
  • General technique for describing and enacting a process
  • Describes what you want to do, not how you want to do it
  • High level description of the experiment
workflows16
Workflows

Workflow language specifies how bioinformatics processes fit together.

High level workflow diagram separated from any lower level coding – you don’t have to be a coder to build workflows.

Workflow is a kind of script or protocol that you configure when you run it.

Easier to explain, share, relocate, reuse and repurpose.

Workflow <=> Model

Workflow is the integrator of knowledge

The METHODS section of a scientific publication

workflow advantages
Workflow Advantages
  • Automation
    • Capturing processes in an explicit manner
    • Tedium! Computers don’t get bored/distracted/hungry/impatient!
    • Saves repeated time and effort
  • Modification, maintenance, substitution and personalisation
  • Easy to share, explain, relocate, reuse and build
  • Releases Scientists/Bioinformaticians to do other work
  • Record
    • Provenance: what the data is like, where it came from, its quality
    • Management of data (LSID - Life Science Identifiers)
different workflow systems
Different Workflow Systems
  • Kepler
  • Triana
  • DiscoveryNet
  • Taverna
  • Geodise
  • Pegasus
  • Pipeline Pilot

Each has differences in action, language, access restrictions, subject areas

slide19

SOAPLAB

Web Service

Any Application

Web Service

e.g. DDBJ BLAST

Taverna Workflow Components

Scufl Simple Conceptual Unified Flow Language

Taverna Writing, running workflows & examining results

SOAPLAB Makes applications available

slide20

An Open World

  • Open domain services and resources.
  • Taverna accesses 3000+ services
  • Third party – we don’t own them – we didn’t build them
  • All the major providers
    • NCBI, DDBJ, EBI …
  • Enforce NO common data model.
  • Quality Web Services considered desirable
slide21

Services Landscape

SoapLab

Standard soap

services

BioMoby

Taverna web

services

API consumer

Biomart

Local Java

Beanshell

Nested workflow

shield the scientist bury the complexity
Shield the Scientist – Bury the Complexity

Taverna

Workbench

Application

Scufl Model

Simple Conceptual Unified Flow Language

Workflow Execution

Workflow enactor

what can you do with my grid
What can you do with myGrid?
  • >40000 downloads
  • Users worldwide

US, Singapore, UK, Europe, Australia

  • Systems biology
  • Proteomics
  • Gene/protein annotation
  • Microarray data analysis
  • Medical image analysis
  • Heart simulations
  • High throughput screening
  • Genotype/Phenotype studies
  • Health Informatics
  • Astronomy
  • Chemoinformatics
  • Data integration
trypanosomiasis in africa
Trypanosomiasis in Africa

Steve Kemp

Andy Brass

Paul Fisher

http://www.genomics.liv.ac.uk/tryps/trypsindex.html

trypanosomiasis study
Trypanosomiasis Study
  • A form of Sleeping sickness in cattle – Known as n’gana
  • Caused by Trypanosoma brucei
  • Can we breed cattle resistant to n’gana infection?
  • What are the causes of the differences between resistant and susceptible strains?
trypanosomiasis study26
Trypanosomiasis Study

Understanding Phenotype

  • Comparing resistant vs susceptible strains – Microarrays

Understanding Genotype

  • Mapping quantitative traits – Classical genetics QTL

Need to access microarray data, genomic sequence information, pathway databases AND integrate the results

slide27

Key:

A – Retrieve genes in QTL region

B – Annotate genes with external database Ids

C – Cross-reference Ids with KEGG gene ids

D – Retrieve microarray data from MaxD database

E – For each KEGG gene get the pathways it’s involved in

F – For each pathway get a description of what it does

G – For each KEGG gene get a description of what it does

results
Results
  • Identified a pathway for which its correlating gene (Daxx) is believed to play a role in trypanosomiasis resistance.
  • Manual analysis on the microarray and QTL data had failed to identify this gene as a candidate.
why was the workflow approach successful
Why was the Workflow Approach Successful?
  • Workflow analysed each piece of data systematically
    • Eliminated user bias and premature filtering of datasets and results leading to single sided, expert-driven hypotheses
  • The size of the QTL and amount of the microarray data made a manual approach impractical
  • Workflows capture exactly where data came from and how it was analysed
  • Workflow output produced a manageable amount of data for the biologists to interpret and verify
    • “make sense of this data” -> “does this make sense?”
trichuris muris mouse whipworm infection parasite model of the human parasite trichuris trichuria
Trichuris muris(mouse whipworm) infectionparasite model of the human parasite - Trichuris trichuria)
  • Identified the biological pathways involved in sex dependence in the mouse model, previously believed to be involved in the ability of mice to expel the parasite.
  • Manual experimentation: Two year study of candidate genes, processes unidentified
  • Workflows: trypanosomiasis cattle experiment was reused without change.
  • Analysis of the resulting data by a biologist found the processes in a couple of days.

Joanne Pennock, Richard Grencis

University of manchester

workflow reuse workflows are scientific protocols share them

Addisons Disease

SNP design

Protein annotation

Microarray analysis

Workflow Reuse – Workflows are Scientific Protocols – Share them!
semantic web technologies
Semantic Web Technologies
  • myGrid built on Web Services, Workflows AND semantic web technologies
  • Semantic web technologies are used to:
    • Find appropriate services during workflow design
    • Find similar workflows for reuse and repurposing
    • Record the process and outcome of an experiment, in context

->>>> the experimental provenance

finding services
Finding Services

There are over 3000 distributed services. How do we find an appropriate one?

Find services by their function instead of their name

  • We need to annotate services by their functions.
  • The services might be distributed, but a registry of service descriptions can be central and queried
feta semantic discovery
Feta Semantic Discovery
  • Feta is the myGrid component that can query the service annotations and find services

Questions we can ask:

Find me all the services that perform a multiple sequence alignment And accept protein sequences in FASTA format as input

my grid ontology
myGrid Ontology

Upper level

ontology

Contributes to

Task

ontology

Informatics

ontology

Molecular Biology

ontology

Bioinformatics

ontology

sequence

protein_structure_feature

Web Service

ontology

biological_sequence

Similarity Search Service

protein_sequence

BLAST service

nucleotide_sequence

DNA_sequence

BLASTp service

InterProScan service

Specialises

annotations
Annotations
  • Feta has been available for over a year
  • Only just been included in the release
  • Need critical mass of service annotations before release
  • By demonstrating the use of service annotation, we aim to encourage service providers to provide the annotations in the future
  • Annotation experiments with users and domain experts
  • Domain expert annotations much better
    • We now have a domain expert for full-time service annotation
data management
Data Management
  • Workflows can generate vast amount of data - how can we manage and track it?
  • We need to manage
    • data AND
    • metadata AND
    • experiment provenance
  • Workflow experiments may consist of many workflows of the same, or different experiments.
  • Scientists need to check back over past results, compare workflow runs and share workflow runs with colleagues
provenance the my grid logbook
Provenance – the myGrid logbook

Smart Tea

  • Who, What, Where, When, Why?, How?
  • Context
  • Interpretation
  • Logging & Debugging
  • Reproducibility and repeatability
  • Evidence & Audit
  • Non-repudiation
  • Credit and Attribution
  • Credibility
  • Accurate reuse and interpretation
  • Just good scientific practice

BioMOBY

conclusions
Conclusions

Web services and workflows are powerful technologies for in silico science

  • automation
  • high throughput experiments
  • systematic analysis
  • Interoperability of distributed resources
contact us
Contact Us
  • Taverna development is user-driven
  • Please tell us what you would like to see via the mailing lists:
    • Taverna-Users and Taverna-Hackers
  • Download software and find out more at:

http://www.mygrid.org.uk

http://taverna.sourceforge.net

my grid acknowledgements
myGrid acknowledgements

Carole Goble, Norman Paton, Robert Stevens, Anil Wipat, David De Roure, Steve Pettifer

  • OMII-UK Tom Oinn, Katy Wolstencroft, Daniele Turi, June Finch, Stuart Owen, David Withers, Stian Soiland, Franck Tanoh, Matthew Gamble, Alan Williams, Ian Dunlop
  • Research Martin Szomszor, Duncan Hull, Jun Zhao, Pinar Alper, Antoon Goderis, Alastair Hampshire, Qiuwei Yu, Wang Kaixuan.
  • Current contributors Matthew Pocock, James Marsh, Khalid Belhajjame, PsyGrid project, Bergen people, EMBRACE people.
  • User Advocates and their bosses Simon Pearce, Claire Jennings, Hannah Tipney, May Tassabehji, Andy Brass, Paul Fisher, Peter Li, Simon Hubbard, Tracy Craddock, Doug Kell, Marco Roos, Matthew Pocock, Mark Wilkinson
  • Past Contributors Matthew Addis, Nedim Alpdemir, Tim Carver, Rich Cawley, Neil Davis, Alvaro Fernandes, Justin Ferris, Robert Gaizaukaus, Kevin Glover, Chris Greenhalgh, Mark Greenwood, Yikun Guo, Ananth Krishna, Phillip Lord, Darren Marvin, Simon Miles, Luc Moreau, Arijit Mukherjee, Juri Papay, Savas Parastatidis, Milena Radenkovic, Stefan Rennick-Egglestone, Peter Rice, Martin Senger, Nick Sharman, Victor Tan, Paul Watson, and Chris Wroe.
  • IndustrialDennis Quan, Sean Martin, Michael Niemi (IBM), Chimatica.
  • Funding EPSRC, Wellcome Trust.