Program Update April 11, 2013 Andrew J. Buckler, MS Principal Investigator, QI-Bench

1. Program Update April 11, 2013 Andrew J. Buckler, MS Principal Investigator, QI-Bench With Funding Support provided by National Institute of Standards and Technology

2. Agenda • Summary and close-out of 1H FY13 reporting period: • Coveringwhat’sbeenaccomplishedfrom multiple pointsofview • Progress since F2F: • QIBA Challenge support • Architectural components • Publications 2

3. FY 2012(n=110) Autumn 2012(n=42) • Established initial structure • Initial Specify, Formulate, and Iterate • Substantial work in Execute and Analyze • First work on test-beds • V&V issues in Execute • Substantial work in Analyze library • Deployment support • Advanced test-beds Winter 2013(n=13) In Queue(n=23) • Architectural design for next generation • Expansion to contour-based analysis • First comprehensive analysis library runs on vCT test-bed • PACS interface to RDSM • Early work on triple store • QI-Bench REST • First workstation support • Realization of QI-Bench cohesive architecture • Migrate worked example test-bed into triples 3 3 3

5. 5 5 5

7. Label map files from second 1B study (steps only needed to accommodate 1B) Segmentation Objects: 1.2…UID.dcm Extract tags such as SUBJID, readerID, Acqrep Convert from dcm to nii format Tertiary Analysis Rename to Seg_SUBJID_0_ACQREP_rdrID.dcm Register all seg_SUBJID_0_ACQREP_rdrID.nii with respect to one of them across different acqrep Extract tags such as pixel spacing, matrix Secondary Analysis Calculate volumes Register all seg_SUBJID_0_ACQREP_rdrID.nii with respect to one of them within same acqrep Primary Analysis For each SUBJID and for each ACREP STAPLE  Weights.nii Binarize  Weights-bin.nii Pair-wise comparison between seg_SUBJID_0_ACREP_rdrID.nii-r.nii and Weights-bin.nii Prepare input files for statistical modules Bland Altman, CCC Linear Mixed Effect Probability Density Function of Error Create input files for analysis of metrics: Intersection, Union, Jaccard, DICE 7 7 7 7

8. Files rdg_SUBJID_participantID.csv and seg_SUBJID_TPINDEX_participantID from participants • Supported segmentation object formats: • Analyze • mhd • mha • nrrd • nii • QIBA Challenge Support Primary Analysis Prepare input files for statistical modules Tertiary Analysis Register all seg_SUBJID_0_ACQREP_rdrID.nii with respect to one of them across different acqrep Bland Altman, CCC Linear Mixed Effect Probability Density Function of Error Secondary Analysis Register all seg_SUBJID_0_ACQREP_rdrID.nii with respect to one of them within same acqrep (steps only needed for 3A) Disregard distractors For each SUBJID and for each ACREP STAPLE  Weights.nii Binarize  Weights-bin.nii Pair-wise comparison between seg_SUBJID_0_ACREP_rdrID.nii-r.nii and Weights-bin.nii Re-assign TPINDEX to ACQREP Roll-up to form s_files, including cov and dcm metadata Create input files for analysis of metrics: Intersection, Union, Jaccard, DICE 8 8 8 8

9. wernsing DEVELOPMENTPROGRESSREPORT 9 9

10. 10

11. 11

12. 12

13. 13

14. 14

15. 15

16. Development plan Continue with core development. 1) Controller to support… 2) QI-Bench API 3) Connections to Computing Services 4) Connections to Business Logic and Data Services Parallel work Creating the initial Slicer plug-in Refining and augmenting the QI-Bench API Workflows and custom actors in Kepler Connections to the API Web applications 16

17. Recent Publications 17

18. 18

19. Value proposition of QI-Bench • Efficiently collect and exploit evidence establishing standards for optimized quantitative imaging: • Users want confidence in the read-outs • Pharma wants to use them as endpoints • Device/SW companies want to market products that produce them without huge costs • Public wants to trust the decisions that they contribute to • By providing a verification framework to develop precompetitive specifications and support test harnesses to curate and utilize reference data • Doing so as an accessible and open resource facilitates collaboration among diverse stakeholders 19

20. Summary:QI-Bench Contributions • We make it practical to increase the magnitude of data for increased statistical significance. • We provide practical means to grapple with massive data sets. • We address the problem of efficient use of resources to assess limits of generalizability. • We make formal specification accessible to diverse groups of experts that are not skilled or interested in knowledge engineering. • We map both medical as well as technical domain expertise into representations well suited to emerging capabilities of the semantic web. • We enable a mechanism to assess compliance with standards or requirements within specific contexts for use. • We take a “toolbox” approach to statistical analysis. • We provide the capability in a manner which is accessible to varying levels of collaborative models, from individual companies or institutions to larger consortia or public-private partnerships to fully open public access. 20