Indexing and Mining Biological Images

1. Indexing and Mining Biological Images Christos Faloutsos CMU

2. Outline • Motivation - Problem Definition • Proposed method • Experiments • Conclusions

3. ViVo with Ambuj Singh, Vebjorn Ljosa, Arnab Bhattacharya (UCSB) Jia-Yu Tim Pan, HJ Yang (CMU)

4. Detachment Development 1 day after detachment 3 days after detachment Normal 3 months after detachment 7 days after detachment 28 days after detachment

5. Data and Problem • (Data) Retinal images taken from cats • (Problem) What happens in retina after detachment? • What tissues (regions) are involved? • How do they change over time? • How will a program convey this info? • More than classification“we want to learn what classifier learned”

6. skip Why study retinal detachment • Common damage to retina • No effective treatment • Surgery or drugs (<100% recovery) • Need to understand more about detachment development

7. skip Retina, its image, and the detachment • retina Layers of tissues stained by 3 antibodies (R,G,B)

8. skip Computer Scientist’s View of Retinal Detachment normal detachment 7 days after

9. Detachment Development 1 day after detachment 3 days after detachment Normal 3 months after detachment 7 days after detachment 28 days after detachment

10. How do the treatments do? 28 days after reattachment surgery 6 days after O2 treatment

11. Outline • Motivation - Problem Definition • Proposed method • Experiments • Conclusions

12. Main idea • extract characteristic visual ‘words’ • Equivalent to characteristic keywords, in a collection of text documents

13. Visual Vocabulary (ViVo) generation Visualvocabulary Independent component analysis (ICA) Tile image Extract color structure features

14. skip Proposed method: ViVo • Textures are different. • Wavelet (Daubechies-4), MPEG-7 color structure • Local variation: partitioned into 64x64 “tiles”. [f1, …, fm] “tile-vector”

15. skip ViVos

16. Outline • Motivation - Problem Definition • Proposed method • Experiments • Conclusions

17. Evaluation of ViVo method • how meaningful are the discovered ViVos? • can they help in classification? • generality? • how else can they help biologists create hypotheses?

18. Example ViVos

19. Goals: • how meaningful are the discovered ViVos? • can they help in classification? • generality? • how else can they help biologists create hypotheses?

20. Quality of ViVo – by classification Predicted Truth 86% accuracy 46 ViVos (90% energy)

21. Goals: • how meaningful are the discovered ViVos? • can they help in classification? • generality? • how else can they help biologists create hypotheses?

22. ViVos for protein images

23. Protein images – MPEG7 CS Predicted Truth 84% accuracy 4 ViVos (93% energy) 1-NN classifier

24. Evaluation of ViVo method • how meaningful are the discovered ViVos? • can they help in classification? • generality? • how else can they help biologists create hypotheses? ‘ViVo-annotation’!

25. Automatic ViVo-annotation of images • A tile represents a ViVo vk if the largest coefficient of the tile is along the kth basis vector • A ViVo vk represents a class ci if the majority of its tiles are in that class • For each image, the representative ViVos for the class are automatically highlighted

26. Which tissue is significant on 7-day?

27. 6 days after O2 treatment

28. 28 days after surgery

29. Conclusions: • how meaningful are the discovered ViVos? • can they help in classification? • generality? • how else can they help biologists create hypotheses?

30. Outcome/status • What are the key results so far? • ViVos: Automatic Visual Vocabulary generation for biomedical image mining, Bhattacharya, Ljosa, Pan, Yang, Faloutsos, Singh (under review) • Software – MATLAB code • Tutorial in SIGMOD’05 (Murphy+Faloutsos)