1 / 24

Pyrosequencing for Metagenomics: accessing and organizing raw data

Pyrosequencing for Metagenomics: accessing and organizing raw data. Giuseppe D’Auria. FISABIO, Valencia. Norwich 08-12 September 2014. Practice workflow. Extract fasta and quality files belonging to each dataset. Organize data and folders. Searching for rRNAs. Recruitment protocol

barryedwards
Download Presentation

Pyrosequencing for Metagenomics: accessing and organizing raw data

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Pyrosequencing for Metagenomics: accessing and organizing raw data Giuseppe D’Auria FISABIO, Valencia Norwich 08-12 September 2014

  2. Practice workflow Extract fasta and quality files belonging to each dataset Organize data and folders Searching for rRNAs Recruitment protocol by MUMmer Clusterize 16S rRNA Assembly protocol via MIRA Annotate 16S rRNA Search for tRNA We will start from a single sff (standard flowgram format) file containing a metagenome and a metatranscriptome experiments labelled by two MIDs (Multiplex Identifiers)

  3. Practice workflow Extract fasta and quality files belonging to each dataset Organize data and folders We will start from a single sff (standard flowgram format) file containing a metagenome and a metatranscriptome experiments labelled by two MIDs (Multiplex Identifiers)

  4. Extract fasta and quality files belonging to each dataset Excercise 2 Use sff_extract to extract sequences from sff -c parameter to remove adaptor sequences and make possible MIDs to be identified 2) Use bin_fasta_on_mid_primers.pl to separate mids QUALITY-Mid1 FASTA-Mid1 QUALITY-Mid2 FASTA-Mid2 QUALITY-MidX FASTA-MidX Extracting MIDs → FASTA file → Fasta Qual → mid_fasta_file Identify Mids and separate Fasta and Fasta Quality files bin_fasta_on_mid_primers.pl SFF http://sourceforge.net/projects/mira-assembler/files/MIRA/

  5. Extract fasta and quality files belonging to each dataset embo@embo-VirtualBox:~$ # Go to data folder embo@embo-VirtualBox:~$ cd data embo@embo-VirtualBox:~/data$ # Create project2 folder embo@embo-VirtualBox:~/data$ mkdir project2 embo@embo-VirtualBox:~$ # Go to project2 folder embo@embo-VirtualBox:~/data$ cd project2 embo@embo-VirtualBox:~/data/project2 # Link SFF file embo@embo-VirtualBox:~/data/project2$ ln -s ~/data/Sequences/dataset2.sff ~/data/project2/dataset2.sff embo@embo-VirtualBox:~/data/project2$ # Extract FASTQ and QUALITY from sff embo@embo-VirtualBox:~/data/project2$ sff_extract -c -A dataset2.sff embo@embo-VirtualBox:~/data/project2$ # Sort reads by MIDs embo@embo-VirtualBox:~/data/project2$ bin_fasta_on_mid_primers.pl -r dataset2.fasta -q dataset2.fasta.qual -m ../Sequences/mids.fas -b out Open the terminal out_midi_CCAACC  Metagenome out_midi_CGCCAT  Metatranscriptome

  6. Extract fasta and quality files belonging to each dataset embo@embo-VirtualBox:~/data/project2$ # Create Metagenome folder embo@embo-VirtualBox:~/data/project2$ mkdir metage embo@embo-VirtualBox:~/data/project2$ # Create Metatranscriptome folder embo@embo-VirtualBox:~/data/project2$ mkdir metatra embo@embo-VirtualBox:~/data/project2$ # Move project files in folders embo@embo-VirtualBox:~/data/project2$ mv out_midi_CCAACC.fasta* metage/ embo@embo-VirtualBox:~/data/project2$ mv out_midi_CGCCAT.fasta* metatra/ embo@embo-VirtualBox:~/data/project2$ # Go to Metagenome folder embo@embo-VirtualBox:~/data/project2/metage$ cd metage embo@embo-VirtualBox:~/data/project2/metage$ # Take a look at the folder embo@embo-VirtualBox:~/data/project2/metage$ ls -ltr Open the terminal

  7. Practice workflow Extract fasta and quality files belonging to each dataset Organize data and folders Recruitment protocol by MUMmer We will start from a single sff (standard flowgram format) file containing a metagenome and a metatranscriptome experiments labelled by two MIDs (Multiplex Identifiers)

  8. Mapping and recruitment graph embo@embo-VirtualBox:~/data/project2/metage$ # Link file to simpler name embo@embo-VirtualBox:~/data/project2/metage$ ln -s out_midi_CCAACC.fasta metage.fas embo@embo-VirtualBox:~/data/project2/metage$ # Mapping of reads on reference genome embo@embo-VirtualBox:~/data/project2/metage$ # Obtaining mapping coordinates embo@embo-VirtualBox:~/data/project2/metage$ nucmer --prefix=recruit ../../References/reference.fasta metage.fas --coords embo@embo-VirtualBox:~/data/project2/metage$ # Obtaining mapping image (postscript) embo@embo-VirtualBox:~/data/project2/metage$ mummerplot recruit.delta -R ../../References/reference.fasta -Q metage.fas --coverage --postscript -p recruit embo@embo-VirtualBox:~/data/project2/metage$ # Visualizing mapping embo@embo-VirtualBox:~/data/project2/metage$ evince recruit.ps & Open the terminal http://mummer.sourceforge.net/

  9. Practice workflow Extract fasta and quality files belonging to each dataset Organize data and folders Recruitment protocol by MUMmer Assembly protocol via MIRA We will start from a single sff (standard flowgram format) file containing a metagenome and a metatranscriptome experiments labelled by two MIDs (Multiplex Identifiers)

  10. Assembly protocol via MIRA http://chevreux.org/projects_mira.html http://sourceforge.net/apps/mediawiki/mira-assembler Assmebly viewer http://bioinf.scri.ac.uk/tablet/ embo@embo-VirtualBox:~/data/project2/metage$ # Linking metagenome file for assembly embo@embo-VirtualBox:~/data/project2/metage$ ln -s out_midi_CCAACC.fasta metage_in.454.fasta embo@embo-VirtualBox:~/data/project2/metage$ ln -s out_midi_CCAACC.fasta.qual metage_in.454.fasta.qual embo@embo-VirtualBox:~/data/project2/metage$ ln -s ../dataset2.xml metage_traceinfo_in.454.xml embo@embo-VirtualBox:~/data/project2/metage$ # Start denovo assembly embo@embo-VirtualBox:~/data/project2/metage$ mira --project=metage --job=denovo,genome,draft,454 454_SETTINGS -LR:ft=fasta embo@embo-VirtualBox:~/data/project2/metage$ # Goto results folder embo@embo-VirtualBox:~/data/project2/metage$ cd metage_assembly embo@embo-VirtualBox:~/data/project2/metage/metage_assembly$ cd metage_d_results embo@embo-VirtualBox:~/data/project2/metage/metage_assembly/metage_d_results$ # Take a look at the results embo@embo-VirtualBox:~/data/project2/metage/metage_assembly/metage_d_results$ tablet metage_out.ace &

  11. Practice workflow Extract fasta and quality files belonging to each dataset Organize data and folders Searching for rRNAs Recruitment protocol by MUMmer Assembly protocol via MIRA We will start from a single sff (standard flowgram format) file containing a metagenome and a metatranscriptome experiments labelled by two MIDs (Multiplex Identifiers)

  12. Searching for rRNAs http://weizhong-lab.ucsd.edu/meta_rna/ extract_sequences_by_list  One of my perl scripts embo@embo-VirtualBox:~/data/project2/metage/metage_assembly/metage_d_results$ cd ../../../ embo@embo-VirtualBox:~/data/project2$ cd metatra embo@embo-VirtualBox:~/data/project2/metatra$ # Link needed files embo@embo-VirtualBox:~/data/project2/metatra$ ln -s out_midi_CGCCAT.fasta metatra.fas embo@embo-VirtualBox:~/data/project2/metatra$ # Searching for 16S sequences embo@embo-VirtualBox:~/data/project2/metatra$ rna_hmm3.py -i metatra.fas -m ssu -o metatra_16S -L embo@embo-VirtualBox:~/data/project2/metatra$ ../../References/hmm3 embo@embo-VirtualBox:~/data/project2/metatra$ # Extract 16S sequences from the 16S table embo@embo-VirtualBox:~/data/project2/metatra$ extract_sequences_by_list.pl -f metatra.fas -t metatra_16S -c 0 -o -d 1

  13. Practice workflow Extract fasta and quality files belonging to each dataset Organize data and folders Searching for rRNAs Recruitment protocol by MUMmer Clusterize 16S rRNA Assembly protocol via MIRA We will start from a single sff (standard flowgram format) file containing a metagenome and a metatranscriptome experiments labelled by two MIDs (Multiplex Identifiers)

  14. Clustering http://weizhong-lab.ucsd.edu/cd-hit/ cd-hit_translate  Oneother of my perl scripts embo@embo-VirtualBox:~/data/project2/metatra$ # Filtering out chimeras embo@embo-VirtualBox:~/data/project2/metatra$ #ChimeraSlayer.pl --query_FASTA 16S.list.fasta embo@embo-VirtualBox:~/data/project2/metatra$ # Clustering 16S sequences embo@embo-VirtualBox:~/data/project2/metatra$ cdhit -i 16S.list.fasta -o 16Sc90s90 -c 0.9 -s 0.9 -bak 1 embo@embo-VirtualBox:~/data/project2/metatra$ cd-hit_translate.pl 16Sc90s90.bak.clstr > 16S.tab

  15. Practice workflow Extract fasta and quality files belonging to each dataset Organize data and folders Searching for rRNAs Recruitment protocol by MUMmer Clusterize 16S rRNA Assembly protocol via MIRA Annotate 16S rRNA We will start from a single sff (standard flowgram format) file containing a metagenome and a metatranscriptome experiments labelled by two MIDs (Multiplex Identifiers)

  16. Annotate 16S rRNA embo@embo-VirtualBox:~/data/project2/metatra$ # 16S assignation by RDP_classifie embo@embo-VirtualBox:~/data/project2/metatra$ java -jar ~/Software/rdp_classifier_2.2/rdp_classifier-2.2.jar -q 16S.remain.fasta -o 16S_rdp -f fixrank http://rdp.cme.msu.edu/index.jsp

  17. Practice workflow Extract fasta and quality files belonging to each dataset Organize data and folders Searching for rRNAs Recruitment protocol by MUMmer Clusterize 16S rRNA Assembly protocol via MIRA Annotate 16S rRNA Search for tRNA We will start from a single sff (standard flowgram format) file containing a metagenome and a metatranscriptome experiments labelled by two MIDs (Multiplex Identifiers)

  18. Searching for tRNAs http://lowelab.ucsc.edu/tRNAscan-SE/ extract_sequences_by_list.pl  Another of my perl scripts embo@embo-VirtualBox:~/data/project2/metatra$ # Searching for tRNAs embo@embo-VirtualBox:~/data/project2/metatra$ tRNAscan-SE -B 16S.remain.fasta > tRNAs.tab embo@embo-VirtualBox:~/data/project2/metatra$ # Extract tRNAs sequences from the tRNAs table embo@embo-VirtualBox:~/data/project2/metatra$ extract_sequences_by_list.pl -f 16S.remain.fasta -t tRNAs.tab -c 0 -o tRNAs -d 1

  19. Running out of physical limits

  20. For INTREPID and BRAVE people http://www.perl.org/

  21. Perl is a scripting language widely used for system administration and programming on the World Wide Web. It originated in the UNIX community and has a strong UNIX slant, but usage on Windows has grown rapidly. ActivePerl is a quality-assured binary distribution of Perl for popular UNIX platforms and Windows. perl (small 'p') is the program used to interpret the Perl language.

  22. For INTREPID and BRAVE people II http://www.r-project.org/ R is a free software environment for statistical computing and graphics. It compiles and runs on a wide variety of UNIX platforms, Windows and MacOS.

  23. http://www.bioconductor.org/ Thank you again for your attention..........

More Related