1 / 19

Automatic Processing Pipelines with XNAT and REDCap Vanderbilt University

Automatic Processing Pipelines with XNAT and REDCap Vanderbilt University. 2013-08-20 Benjamin Yvernault, Bennett Landman , Brian Boyd, . Requirements. At this point, you should know how to : Create project / sharing data Browse data and view image previews

blaise
Download Presentation

Automatic Processing Pipelines with XNAT and REDCap Vanderbilt University

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Automatic Processing Pipelines with XNATand REDCapVanderbilt University 2013-08-20 Benjamin Yvernault, Bennett Landman, Brian Boyd,

  2. Requirements At this point, you should know how to : • Create project / sharing data • Browse data and view image previews • Search for projects, subjects, and data • Download imaging data, processed, and stored meta-data • Delete scans / sessions / projects • Use the Pre-Archive • Upload scans DICOMs • Quality Assurance

  3. After this short course, you will be able to: • Create a project on REDCap • Set up the design of your REDCap project • Request API access for your project on REDCap • What is a spider: • source code • Which spiders already exist? • Run spiders from your computer • Run a spider on your local machine • ACCRE: run spiders (status / failed / ….)

  4. Setting REDCap project for XNAT processing

  5. REDCap: Setup the main instrument Variable Name: project_xnat-> type: Text Box Variable Name: subject_xnat -> type: Text Box Variable Name: experiment_xnat -> type: Text Box Variable Name: scan_xnat -> type: Text Box Variable Name: scan_type_xnat-> type: Text Box Variable Name: scan_sd_type-> type: Text Box Variable Name: script_name -> type: Text Box Variable Name: script_version -> type: Text Box Variable Name: pdf_report -> type: File Upload

  6. REDCap: Upload main instrument Take time to create all this fields each time you have a new project. Variable Name: project_xnat-> type: Text Box Variable Name: subject_xnat -> type: Text Box Variable Name: experiment_xnat -> type: Text Box Variable Name: scan_xnat -> type: Text Box Variable Name: scan_type_xnat-> type: Text Box Variable Name: scan_sd_type-> type: Text Box Variable Name: script_name -> type: Text Box Variable Name: script_version -> type: Text Box Variable Name: pdf_report -> type: File Upload Upload the main instrument dictionary as a first step. File location: http://xnat.vanderbilt.edu -> Everything about spiders -> How to upload to REDCap -> link with the csv file.

  7. REDCap: Download Shared Libraries

  8. REDCap: API Key • To send data from XNAT to REDCap, you need to have an API key for your project on REDCap

  9. REDCap Project Online Designer

  10. Spiders … • Script executing a process on XNAT • Download images / data from XNAT • Execute a process on them • Copy the data to a directory • A specific spider to upload results on XNAT • All the spiders on http://xnat.vanderbilt.edu • Source code in the project Masimatlab on NITRC • Three kind of Spiders : • Spider that creates results for a Scan • Spider that corresponds to an assessor • Infrastructure spiders

  11. Scan Spiders • Converter Spider : • Spider_dcm2nii_all.py (using dcm2nii) • Spider_dcm2nii_all_v2.py (using r2agui) • Changing attributes: • Spider_set_scan_type.py • Spider_Series_Description.py • Extracting/creating files: • Spider_physlog.py • Spider_Make_Preview_from_nifti.py

  12. Process Spiders • This spiders can work on a scan level (one assessor per scan) or an experiment level (one assessor per session) • Spider that generates an assessor: • Spider_fMRQA.py • Spider_VBMQA.py • Spider_dtiQA_v2.py • Spider_labelFusion.py • Spider_intra_sess_reg.py • Spider_nonrigi_reg_to_ATLAS.py • Spider_FreesurferReconAllWithQC.py • Spider_ON_segmentation.py

  13. Infrastructure Spider • Spider_report_for_projects.py to create a report on projects (same than Xnatreport) • Spider_send_data_to_redcap.py to send data from complet processes on XNAT to REDCap • Spider_Process_Upload to upload the processes that finished. Any process Spiders will copy the results needed to a specific directory. You should run it when you need to upload processes.

  14. Spider flow Spider_process_upload.py / Xnatprocessupload Python script Download data Spiders Any language script: - Matlab/SPM - Perl - shell script Folder Run process UPLOAD_SPIDER_DIR

  15. How to run a spider? • Spiders can run on your local machine if you have the requirements. • E.G: Most of the spiders use xvfb-run • python Spider_fMRIQA.py -h to see the options • Your .bashrc or .profile set up with your XNAT logs • Python packages install • Specific python package to run spiders on ACCRE develop by our lab.

  16. ACCRE and Spiders • Advanced Computing Center for Research & Education • ACCRE is a Cluster for Vanderbilt University • Python script running every night on the cluster • Submitting one job per assessor • Status on the assessor related to the job status: • NEED_INPUTS-> the inputs files are not present on XNAT • NEED_TO_RUN-> ready to be submitted on ACCRE • JOB_RUNNING -> job submitted and running on ACCRE • JOB_FAILED -> job stop with an error • READY_TO_UPLOAD-> job ended and copy the results to the Upload directory • UPLOADING-> Spider_Process_Upload running and uploading results • READY_TO_COMPLETE-> a script will check and set the walltime and memory used by the job on ACCRE. • COMPLETE-> jobs done (processes finished and uploaded) with results on XNAT

  17. ACCRE and CCI package Spider_process_upload.py / Xnatprocessupload Crontab job running every night a python script Get assessors that need to run CCI Package Folder UPLOAD_SPIDER_DIR Run jobs for each assessors (Spider flow on ACCRE)

  18. Status sync between XNAT and REDCap

  19. Interactive Assignment • Add to your .bashrcthe line (or .profile on Mac OS): export UPLOAD_SPIDER_DIR=path • Create a project on REDCap • Upload the dictionary from http://xnat.vanderbilt.edu • Everything about Spiders -> How to upload on REDCap • Download the shared libraries for fMRIQA / VBMQA / dtiQA • Run the Spider VBMQA on one T1 in the project you created in the last courses (it’s going to take 10mins) • Look at the results and upload them using Xnatprocessupload • Delete the project

More Related