1 / 17

Agenda : - Update GANTT IV Delphi round Hippo axes orientation Harmonized Protocol Ongoing

V PMT meeting – Jan 4, 2011. Agenda : - Update GANTT IV Delphi round Hippo axes orientation Harmonized Protocol Ongoing Validation vs Pathology Publication Issues. VARIABILITY EVALUATION. GOLD STANDARD. 5 expert tracers. 20 naive tracers. Local Protocol :

metta
Download Presentation

Agenda : - Update GANTT IV Delphi round Hippo axes orientation Harmonized Protocol Ongoing

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. V PMT meeting – Jan 4, 2011 • Agenda: • - Update GANTT • IV Delphi round • Hippo axes orientation • Harmonized Protocol • Ongoing • Validation vs Pathology • Publication Issues

  2. VARIABILITY EVALUATION GOLD STANDARD 5 expert tracers 20 naive tracers Local Protocol: Experimental set (1.5T ADNI): 2 x each of the 5 Scheltens’s atrophy score x 2 sides (SAME on 3T ADNI scans) (total for each rater: 40 hippos) Benchmark Harmonized hippos: 1.5T ADNI scans 2 x each of the 5 Scheltens’s atrophy score x 2 sides (SAME on 3T ADNI scans) (total for each rater: 40 hippos) Qualification (20 tracers) Harmonized Protocol: Experimental set (1.5T ADNI): 2 x each of the 5 Scheltens’s atrophy score x 2 sides (SAME on 3T ADNI scans) (total for each rater: 40 hippos) global and local 95% confidence intervals Harmonized Protocol: 1.5T ADNI scans 2 sides x 5 Scheltens’s atrophy scores x 3 time points (0-12°month-24°month) x 3 scanners + retracing for timepoint 1 (SAME on 3T ADNI scans) (total for each rater: 240 hippos – including 40 hippos already traced) RM-ANOVA: test of main effects side, trace-retrace, atrophy, time, scanner, rater The best 5 naive tracers RM-ANOVA: test of rater and rater by center terms

  3. V PMT meeting – Dec 21, 2011

  4. IV Delphi round – separating A/F Agreement for the definition of the criterion for excluding the white matter of the fornix 8% vs 82% Median 8 Fisher= <0.0005 Binomial= 0.003 disagree agree

  5. IV Delphi round – Image Orientation Agreement for orienting images along the mean axis between left and right 25% vs 6% vs 69% Median= 8 Fisher= 0.026 Binomial= 0.118 left x left & right x right mean angle

  6. IV Delphi round – Image Orientation • The preliminary quantitative investigation, used by panelists for their decisions, was carried out on AC-PC oriented images. Therefore we: • traced the left Hippo of one AD and one control on a AC-PC oriented image • re-traced the left Hippo of the same AD on the same AC-PC oriented image • - traced the left Hippo of same AD and control on images oriented along the mean angle of hippocampal long axes; • The results show that the difference between tracings made on differently oriented images is about 2%, but this includes the difference due to re-tracing on the same image, which is 1.7%. •  Data from preliminary phase can be considered valid, independently on orientation.

  7. Harmonized Protocol: release 0.0 The Harmonized Protocol based on the features defined in the Delphi rounds has been written, sent to Master Tracers, and to Panelists. We asked Master Tracers and Panelists to read carefully the protocol, and propose corrections, suggestions or comments to make it as clear and complete as possible (without interfering with panelists’ decisions). The corrected version needs to be ready before the beginning of the qualification of naive tracers. Masters are using the Harmonized Protocol for the segmentation of benchmark images.

  8. Validation versus pathology 1 tracer Local Protocol: 1.5T 3D T1-weighted scans from (Bobinski et al., 2000) pathologically verified set (total for rater: 30 hippos) WHICH SAMPLE? Qualification on benchmark images (total for each rater: 40 hippos) Harmonized Protocol: 1.5T 3D T1-weighted scans from (Bobinski et al., 2000) pathologically verified set (total for rater: 30 hippos) RM-ANOVA: test of protocol main effect

  9. Publication Issues

  10. Next periodical meetings AAN – April 24, 2012: Delphi Paper; Italian-ADNI Paper using prototype-protocol; Publication policy AAIC - July 2012: Harmonized Protocol with reliability values

  11. 1 tracer VARIABILITY EVALUATION GOLD STANDARD VALIDATION vs PATHOLOGY 5 expert tracers 20 naive tracers Training (tracing 20 hippos on 1.5T ADNI scans with each SU) (SAME on 3T ADNI scans) Local Protocol: Experimental set (1.5T ADNI): 2 x each of the 5 Scheltens’s atrophy score x 2 sides (SAME on 3T ADNI scans) (total for each rater: 40 hippos) Local Protocol: 1.5T 3D T1-weighted scans from (Bobinski et al., 2000) pathologically verified set (total for rater: 30 hippos) Delphi panel → harmonized prot Benchmark Harmonized hippos: 1.5T ADNI scans 2 x each of the 5 Scheltens’s atrophy score x 2 sides (SAME on 3T ADNI scans) (total for each rater: 40 hippos) Qualification (20 tracers) Qualification Harmonized Protocol: 1.5T 3D T1-weighted scans from (Bobinski et al., 2000) pathologically verified set (total for rater: 30 hippos) Harmonized Protocol: Experimental set (1.5T ADNI): 2 x each of the 5 Scheltens’s atrophy score x 2 sides (SAME on 3T ADNI scans) (total for each rater: 40 hippos) global and local 95% confidence intervals AAIC July 2012 Harmonized Protocol: 1.5T ADNI scans 2 sides x 5 Scheltens’s atrophy scores x 3 time points (0-12°month-24°month) x 3 scanners + retracing for timepoint 1 (SAME on 3T ADNI scans) (total for each rater: 240 hippos – including 40 hippos already traced) RM-ANOVA: test of protocol main effect RM-ANOVA: test of main effects side, trace-retrace, atrophy, time, scanner, rater The best 5 naive tracers RM-ANOVA: test of rater and rater by center terms

  12. Paper operationalization Operationalization of differences among protocols for manual hippocampal segmentation M Boccardi, M Bocchetta, R Ganzola, N Robitaille, A Redolfi, S Duchesne, CR Jack , GB Frisoni, and the Alzheimer's Disease Neuroimaging Initiative. Collaborators: G Bartzokis, JG Csernansky, MJ de Leon, L deToledo-Morrell, RJ Killiany, S Lehéricy, N Malykhin, J Pantel, JC Pruessner, H Soininen, C Watson To be submitted to Alzheimer’s and Dementia

  13. Papers describing the project Survey of protocols (preliminary phase; published, JAD 2011) Operationalization (preliminary phase; to be completed) Delphi consensus Master tracers’ practice and reliability Development of certification platform Validation data and Protocol definition Validation vs pathology

  14. 1 tracer VARIABILITY EVALUATION GOLD STANDARD VALIDATION vs PATHOLOGY 5 expert tracers 20 naive tracers Training (tracing 20 hippos on 1.5T ADNI scans with each SU) (SAME on 3T ADNI scans) Local Protocol: Experimental set (1.5T ADNI): 2 x each of the 5 Scheltens’s atrophy score x 2 sides (SAME on 3T ADNI scans) (total for each rater: 40 hippos) Local Protocol: 1.5T 3D T1-weighted scans from (Bobinski et al., 2000) pathologically verified set (total for rater: 30 hippos) Delphi panel → harmonized prot Benchmark Harmonized hippos: 1.5T ADNI scans 2 x each of the 5 Scheltens’s atrophy score x 2 sides (SAME on 3T ADNI scans) (total for each rater: 40 hippos) Qualification (20 tracers) Qualification Harmonized Protocol: 1.5T 3D T1-weighted scans from (Bobinski et al., 2000) pathologically verified set (total for rater: 30 hippos) Harmonized Protocol: Experimental set (1.5T ADNI): 2 x each of the 5 Scheltens’s atrophy score x 2 sides (SAME on 3T ADNI scans) (total for each rater: 40 hippos) global and local 95% confidence intervals Harmonized Protocol: 1.5T ADNI scans 2 sides x 5 Scheltens’s atrophy scores x 3 time points (0-12°month-24°month) x 3 scanners + retracing for timepoint 1 (SAME on 3T ADNI scans) (total for each rater: 240 hippos – including 40 hippos already traced) RM-ANOVA: test of protocol main effect RM-ANOVA: test of main effects side, trace-retrace, atrophy, time, scanner, rater The best 5 naive tracers RM-ANOVA: test of rater and rater by center terms

  15. Communication Policy (SD) COMMUNICATIONS STEERING COMMITTEE This is made of: (i) the Principal Investigator, (ii) the co-Principal Inestigator, (iii) the leader of the Statistical Working Group. Decisions will be taken by majority. APPLICABILITY Planned communications reporting project results will be agreed among members of the Publication Steering Committee. For each paper, a writing team will be identified, made of a lead author, senior author, and coauthors. Scientific communication directly based on data from the Harmonization Project will go through review process (as per ADNI) Scientific communications where the protocol is used by a tracer, certified or not, should simply reference the following article: XXX AUTHORSHIP A Group will be formed (“Harmonization … Group”) for easier reference in indexed publications Include group membership ACKNOWLEDGMENTS In addition to key personnel from the Coordinating centre and the Statistical Working Group centre, the following teams can be identified: Authors of protocols, Delphi panelists, Expert tracers, Naive tracers, and P.I.s of EADC-ADNI (and other) centres. Papers where members of above teams contribute primary work will include them as coauthors whenever possible or allowed by the journal's publication policies. When not allowed, group authorship will be acknoweledged in the authors' list (...for the EADC-ADNI Working Group on A Harmonized Protocol for Hippocampal Volumetry) and individual team members listed as Collaborators or in the Acknowledgements. In any case, all possible efforts will be made to guarantee participants the highest possible vilsibility.  The Acknowledgment section will include the full list of the EADC-ADNI (and others) centre P.I.s and Board Members (see Appendix 1). DIGITAL DATA The project will produce digital data (e.g. probabilistic masks, reference images, gold standard training set of hippocampal tracings). These will be made public in planned releases through the project’s online repositories. Public distribution will equally apply to academic centres and for/not-for profit entities. (like ADNI?!) POLICY SUBJECT TO PERIODICAL REVIEW Date, time, version APPENDIX 1. Full list of EADC-ADNI (and others) centres, and Board Members.

  16. Appendix 1 P.I.s: Giovanni B Frisoni, Clifford R. Jack Jr

  17. 1 tracer VARIABILITY EVALUATION GOLD STANDARD VALIDATION vs PATHOLOGY 5 expert tracers 20 naive tracers Training (tracing 20 hippos on 1.5T ADNI scans with each SU) (SAME on 3T ADNI scans) Local Protocol: Experimental set (1.5T ADNI): 2 x each of the 5 Scheltens’s atrophy score x 2 sides (SAME on 3T ADNI scans) (total for each rater: 40 hippos) Local Protocol: 1.5T 3D T1-weighted scans from (Bobinski et al., 2000) pathologically verified set (total for rater: 30 hippos) Delphi panel → harmonized prot Benchmark Harmonized hippos: 1.5T ADNI scans 2 x each of the 5 Scheltens’s atrophy score x 2 sides (SAME on 3T ADNI scans) (total for each rater: 40 hippos) Qualification (20 tracers) Qualification Harmonized Protocol: 1.5T 3D T1-weighted scans from (Bobinski et al., 2000) pathologically verified set (total for rater: 30 hippos) Harmonized Protocol: Experimental set (1.5T ADNI): 2 x each of the 5 Scheltens’s atrophy score x 2 sides (SAME on 3T ADNI scans) (total for each rater: 40 hippos) global and local 95% confidence intervals Harmonized Protocol: 1.5T ADNI scans 2 sides x 5 Scheltens’s atrophy scores x 3 time points (0-12°month-24°month) x 3 scanners + retracing for timepoint 1 (SAME on 3T ADNI scans) (total for each rater: 240 hippos – including 40 hippos already traced) RM-ANOVA: test of protocol main effect RM-ANOVA: test of main effects side, trace-retrace, atrophy, time, scanner, rater The best 5 naive tracers RM-ANOVA: test of rater and rater by center terms

More Related