The Insight Journal

1. The Insight Journal Kitware Inc.

2. Credits • Julien Jomier – UNC Caddlab • Stephen Aylward – UNC Caddlab • Zack Galbreath – Kitware • Andy Cedilnik – Kitware • Bill Hoffman - Kitware

3. The Insight Journal • Technical work must be reproducible • Papers should be publicly accessible • The Peer-Review process must be open

4. Let’s Implement this paper…. Supervisor “You just need to implement the code and try it…” Happy Grad Student, unaware of her destiny

5. Let’s Implement this paper…. Six months later… • Parameters are missing • Images are not available • Implementation details are missing • Abundance of pretty equations • Abundance of pretty screenshots Unhappy Grad Student, aware of her destiny

6. Time to face theTruth

8. Publications are a measureof scientific productivity • They disseminate knowledge • They allow others to reproduce our results • They are validated by the peer-review process

9. Why do we reallywant to publish ?

10. Because we needto have publications in our CV

11. We have met the enemy… and he is us !

12. “Publish or Perish” Who invented this ?and Why ?

13. “Publish or Perish” Was invented by those who needed to evaluate researcher’s productivity.

14. “Publish or Perish” Empowers those who read your CV to grade you by simply counting lines in the “Publications” section.

15. “Publish or Perish” The group of best educated people in the world has been alienated with a simple trick

16. The Scientific Method Hypothesis Confidence Experiment Outcome HypothesisRejection Galileo Galilei

17. The Scientific Method The Logic of Scientific Discovery by Karl Popper

20. Image Processing Method Parameters InputImage(s) Algorithm OutputImage(s)

21. Process Flow DSpace Automatic Testing Environment Sends Code StoresProject Insight Journal Manager Submits Project Submits Entry Author Posts Results Reviews Project Dashboard Reviewer

22. Submission PDF doc Journal Repository Code Input Data Author Web Site Build Machines Results Data

27. Comparison of output images • Publicly available • Tests for non-trivial images • Reviewers and readers can use it as a first view of the paper. Results Verification

29. DSpace • Open Source • Developed by MIT & HP • Persistant URIs • Prevents data from becoming obsolete DSpace StoresProject Insight Journal Manager

30. Insight Journal Manager • Open Access • Peer Review • Searchable Insight Journal Manager Author Reviewer Reviewer Reviewer Reviewer Peer-Review is ONLY useful if it enforces reproducibility

31. Testing Environment • Security Issues • Open source tools • Dashboard Automatic Testing Environment Insight Journal Manager Dashboard

32. Enjoy the Insight Journal !

34. Implementation Details

35. Insight Journal Manager • http://insight-journal.org • Web interface: PHP/Java/HTML • Central place for submissions, reviews, browsing/searching • No delay between submissions & reviews • Public reviews

36. Issues • Title, comments. • Creation/Paper Due/Notification/Publication dates • Tied to a DSpace collection • Submission through DSpace + comments

37. Adding reviews • Rating • Level of Affiliation • Expertise • Anonymous Review • Title • Review

38. Browsing reviews • No delays • Users can post comments • Authors can reply to comments • PDF can be downloaded

39. Scoring Weighted average of: • Sensitivity (STAPLE algorithm) • Helpfulness • # submissions/max submissions • # reviews/max reviews • # anonymous reviews

40. What’s next? • Moderated submissions • SVN repository for code fixing • …

41. Safe Testing • Virtualization! • Lots of options: User-Mode-Linux, coLinux, VMWare, chroot • Xen: http://www.cl.cam.ac.uk/Research/SRG/netos/xen/

42. Tools used in the testing process • Xen • Debian • CMake • Python • Dashboards

43. Testing Details • two CPUs and two partitions, so two tests can be run simultaneously • VMs can only access their own partition; no network access • Mutexes prevent filesystem corruption • Submit test results to dashboard • Restore pristine filesystem after every test

44. Enjoy the Insight Journal !