1 / 37

February 12, 2011

http://nemo.nic.uoregon.edu. February 12, 2011. NEMO All-Hands Meeting: Database and Portal Kurt Mueller and Jason Sydes. Agenda. FTP site / file database overview NEMO portal Present Future. Database – FTP site. Place to store files Raw waveforms (input files)

ryan-gibson
Download Presentation

February 12, 2011

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. http://nemo.nic.uoregon.edu February 12, 2011 NEMO All-Hands Meeting: Database and Portal Kurt Mueller and Jason Sydes

  2. Agenda • FTP site / file database overview • NEMO portal • Present • Future

  3. Database – FTP site • Place to store files • Raw waveforms (input files) • Intermediate analysis results • Parameter files • Final output files

  4. Organization • Organize files by lab, experiment, file type: LAB DIRECTORY EXPERIMENT DIRECTORY INPUT_DATA DIRECTORY ANALYSIS_1 DIRECTORY ANALYSIS_2 DIRECTORY

  5. Example JTC (lab-level folder) JTC-PMN1 (experiment-level folder) JTC-PMN1_Input_Data (subdir) 'JTC-PMN1.raw' (input data file) JTC-PMN1_Decomposition_2011-01-15 (subdir) 'JTC-PMN1_tPCA.m' (metascript file) 'JTC-PMN1_DecompObj-20110115.mat' (MATLAB output file) 'JTC-PMN1_tPCA.raw' (simple binary output data file)

  6. Uploading files – ftp client

  7. Data in FTP is both managed and unmanaged • Mixture of two different areas • Unmanaged • Portal managed • Possible area for documentation? • What role will FTP server will serve in future? • Can browse and download portal data via FTP • Readonly!

  8. Data in portal accessible via FTP

  9. Graphical ftp clients • Mac OS X + Windows + Linux • Filezilla - http://filezilla-project.org/ • Mac OS X + Windows • Cyberduck – free - http://cyberduck.ch/ • Mac OS X • Transmit - $34 - http://www.panic.com/transmit/

  10. Goals of portal • Online record of experiment metadata • Repository of experiment data files • Tight coupling to NEMO ontology • Provenance of all data • NEMO analysis pipeline execution • Querying on metadata now in progress

  11. Record experiment metadata Experiment Conditions Sessions Task Stimuli Responses Data files Publications Subject groups

  12. Quick Sampling of Portal

  13. Portal database tables

  14. Repository of experiment data files • New upload widget provides progress feedback (standard http upload does not)

  15. OpenID access to NEMO resources • SSO: Single Sign On • Login once, get access to • NEMO portal • NEMO wiki • NEMO sourceforge page • Learn about it: http://openid.net/ • How to create a new SSO account: http://nemo.nic.uoregon.edu/wiki/New_Nemo_User

  16. Role-based access control (RBAC)

  17. Database coupled to OntologyExample Table: Stimulus

  18. Database coupled to Ontology Example “Ontology”: Stimulus

  19. Database coupled to Ontology

  20. Database coupled to OntologyExample mapping: Stimulus

  21. Database coupled to Ontology Example term: stimulus_intensity

  22. Database coupled to OntologyExample tooltip

  23. Database coupled to OntologyQuick syncing

  24. Open Provenance Model(http://openprovenance.org/)

  25. Open Provenance Model– subset used –

  26. Open Provenance ModelExample

  27. Open Provenance ModelSimplified Example

  28. Workflow – MATLAB in portal • Current workflow: • Conduct experiment, producing raw waveforms • Process data with NEMO Toolkit scripts, locally • Upload raw and processed data to ftp or portal • Create an experiment in the portal, associate data with experiment

  29. Worfklow – MATLAB in portal • Future workflow (in design) • Conduct experiment, producing raw waveforms • Create an experiment in the portal • Upload raw data through portal • Execute MATLAB pipeline through portal • Automatic storage of output data • Automatic provenance tracking • Single place for MATLAB scripts to live • Run on big iron

  30. Questions?

  31. Thank you!

More Related