Deep Project Management with XNAT Vanderbilt University

1. Deep Project Management with XNATVanderbilt University 2013-08-20 Benjamin Yvernault, Bennett Landman, Brian Boyd,

2. Requirements At this point, you should know how to : • Register and Log in to XNAT • See your projects and join projects • Create project / sharing data • Manage access permissions • Browse data and view image previews • Search for projects, subjects, and data • Download imaging data, processed, and stored meta-data • Delete scans / sessions / projects • Use the Pre-Archive • Upload scans through DicomBrowser • Quality Assurance

3. Set up your computer Xnatsetup (basic) will on your computer : • Install Python package to run the command tools : • Instructions on the wiki • Install lxml and httplib2 using easy_install or pip • Install Pyxnat from a git version : git clone git://github.com/sburns/PyCap.gitPyCapcd PyCappython setup.pyinstall • Set up your .bashrc or .profile : • Set up the XNAT variables :export XNAT_USER = bob (your xnat username)export XNAT_PASS = bob (your xnat password)export XNAT_HOST = http://xnat.vanderbilt.edu/xnat • Add the tools to PATH :export PATH=/PathToMasimatlab/trunk/xnatspiders/Xnat_tools:$PATH Remember!

4. After this short course, you will be able to: • Get our code from Masimatlab • Set up your computer (python packages, …) • Download data from XNAT using command line tools • Upload data using XNAT command tools • Dcm4chee to send big DICOM database to XNAT • Use XNAT command line tools • Project mirroring XNAT->XNAT / full

5. Download from XNAT • You can download using what you learn from the previous course : • Download file • Manage file • You can use the Xnatdownloadcommandline tool developed by our lab that downloads resources from scans or/and assessors.

6. Xnatdownload • Python commandlinetool developed by our lab • Present in the masimatlabsvn repository:trunk/xnatspiders/Xnat_tools/Xnatdownload • Different options to precise your download • Use: Xnatdownload -h to see the options

7. XNAT view

8. Upload Data to XNAT • DICOM can be uploaded using: • DicomBrowser (previous course) • dcm4che (next slides) • Xnatupload is a commandline tool developed by our lab to upload any resources to XNAT using three methods.

9. Dcm4che (1) • Open Source clinical Image and object management • Programed in Java • Tutorial on how to upload : http://xnat.vanderbilt.edu in Advanced resources. • Join Masimatlab on NITRC to obtain the code for Dicom4che • Dcmsndpresent in the masimatlab/trunk/xnatspiders/dcm4che_tools/dcm4che-2.0.27/bin/ • E.G: dcmsndVandyXNAT@xnat.vanderbilt.edu:8104 /FullPathTemporatoryFolder/*

10. XNAT reading DICOM Remember! • XNAT provides a DicomServer: • Read information from the header • Add the Dicom to the PreArchive with the right project/subject/session • XNAT uses tags from DICOM to identify where it should be: • Usually tag (0010,4000) Patient Comments • Need to be: “Project:ProjectIDXNAT;Subject:SubjectIDXNAT;Session:SessionIDXNAT” • E.G: Project:VUSTP;Subject:VUSTP4;Session:VUSTP4a • Check this link to know the tags : http://mindhive.mit.edu/node/1352

11. Dcm4che (2) • Use an IDE for java developers like Eclipse • Create a new project and configure the java build path to add all the externals JARs and Class folder from masimatlab • Change the SNARLdcmEdit.javafile that will change in all of your DICOM the tag of identification for XNAT (0010,4000) • Follow the steps in http://xnat.vanderbilt.edu “Sending DICOM to Vanderbilt XNAT” link • Send the DICOMs to XNAT using dcmsnd • dcmsndVandyXNAT@xnat.vanderbilt.edy:8104/path/*

12. Xnatupload • Python commandline tool developed by our lab • Present in the masimatlabsvn repository:trunk/xnatspiders/Xnat_tools/Xnatupload • It can upload any resources (not only DICOM) • Three methods in the option: • Files in the same folder, using specific names • Files spread into a specific tree hierarchy • Upload only on existing subjects/sessions/scans/assessors

13. First Method of Upload • This method uploads only resources for scans. • Options : --up1 • Conditions: • all the file are in the same folder • Name the files like : • Project-x-subject-x-experiment-x-scan-x-scantype-x-series_description-x-resource.extensionE.G: VUSTP-x-VUSTP5-x-VUSTP5a-x-301-x-T1-x-3W_T1-x-NIFTI.nii.gz • The project needs to exist first on XNAT • You need to select a type of session (MR/CT/PET) • Call the script :Xnatupload -p VUSTP -d /tmp/vustp_data/ --up1 --sess MR

14. Second Method of Upload • This method uploads only resources for scans. • Options : --up2 • Conditions: • One Folder with a specific structure • The structure needs to be : • Folder/subject/experiment/scan-x-scantype-x-series_description/resource/file.extensionE.G: MainFolder/VUSTP5/VUSTP5a/301-x-T1-x-3W_T1/NIFTI/file.nii.gz • The project needs to exist first on XNAT • You need to select a type of session (MR/CT/PET) • Call the script :Xnatupload -p VUSTP -d /tmp/vustp_data/ --up2 --sess MR

15. Third Method of Upload • This method uploads only (does not create any subject/experiment/scan or assessor on XNAT) • Options : -o/--onlyUpload • Conditions: • One Folder with all the files in it (multi project) • The files needs to have a name like : • Project-x-subject-x-experiment-x-scan-x-resource.extensionE.G: VUSTP-x-VUSTP5-x-VUSTP5a-x-301-x-NIFTI.nii.gz • Use the option -T to specify the kind of data (scans or assessors) • The location needs to exist on XNAT • Call the script :Xnatupload -p VUSTP -d /tmp/vustp_data/ --only -T 1

16. XNAT view

17. XNAT command tools • Xnatdownload and Xnatupload (already presented) • Xnatreport • Creates report on project(s) from XNAT • Xnatquery • Queries and displays XNAT at any level • Xnatcheck • Checks if a resource exists on specific assessors or scans • Checks the status of assessors • XnatSwitchProcessStatus • Xnatsetup

18. Xnatreport • Use Xnatreport -h to see the options • E.G: Xnatreport –p VUSTP

19. Xnatquery • Option --info for a report more focus on the processes. E.G: Xnatquery –p VUSTP --info • Help: Xnatquery –h • Xnatquery will display on the terminal all the objects at the level you asked.

20. Xnatcheck • Xnatcheck allows you to check the existence of a resource on a project • Help: Xnatcheck -h

21. XNAT view

22. XnatSwitchProcessStatus • The status on an assessor gives information about the job status of this process. Our system use this status to launch jobs. • XnatSwitchProcessStatus allows you to change the status of several assessors at a time. • Help: XnatSwitchProcessStatus -h

23. XNAT view

24. Xnatsetup • Command tool to Setup your computer • Different setups: • Basic : • Install python packages: httplib2 / lxml / pyxnat • Set up your .bashrc or .profile • Advance: • Set up path for spiders • Redcap: • Install python packages: PyCap and Pandas • Set up your .bashrc or .profile • Some specific spiders set up to check if you have everything to run them.

25. xnat_copy • Mirrors a Project from one XNAT instance to another • Copies files & attributes for all Subjects/Sessions/Scans xnat_copy [options] project -cf recheck existing files -ca recheck existing attributes

26. Interactive Assignment • Create a NITRC account and join the Masimatlab team to get the last version of masimatlabsvn repository. • Download the code from masimatlab on NITRC • Set up your computer (manually or with Xnatsetup): • Download all the NIFTI from VUSTP using Xnatdownload • Upload the VUSTP data into a new project using Xnatupload and one of the method • Check if a fMRIQA process in VUSTP doesn’t have a PDF • Xnatquery --info on VUSTP

27. Next time • Create a project on REDCapand ask for API • What is a spider: • source code • Which spiders already exist? • Run spiders from your computer • ACCRE: run spiders (status / failed / ….)