Acknowledgements: Barry Kalman Stan Kwasny Koong-Nah Chung John Heuser

1. Detection of Intracellular Organelles in Digitized Electronmicroscopic Images Using Wavelets and Neural NetworksFinal Report for CS513John Olsen and Wei Yan Acknowledgements: Barry Kalman Stan Kwasny Koong-Nah Chung John Heuser

2. The Experiment • We used Kalman & Kwasny’s neural network tool to create a set of trained feedforward neural networks (FFNNs) that detect particular intracellular organelles, called caveolae, in scanning EM images.

7. Performance is 84% correct

8. Data Flow Diagram

9. Wavelet Transform • Equivalent to the 1st order derivative of a smoothing function. • Analogous to the Fourier Transform. • Multiple levels of Resolution. • 2-Dimensions • Image component: The set of wavelet coefficients for a particular dimension at a particular resolution. • Image components: Multiple representations of the entire image at different resolutions.

10. Four-level, 2-D Wavelet Hierarchy Image L1 L2 L3 L4 320 x 240 160 x 120 80 x 60 40 x 30 20 x 15 1 4 16 64 256 76,800 19,200 4,800 1200 300 - 2 2 2 3 - 38,400 9,600 2,400 900 Level Image size(grains) Grain size(pixels) Total Grains No. of Comps (x, y, r) No. of Coeffs • Maximum no. of coeffs at any one level, direction = no of grains in image. For our data, the max no. of coefficients/image = 51, 300. • A Component consists of the set of wavelet coefficients for a particular direction and level of resolution. • Total of 9 components: 2 for each level of decomposition, 1 for undecomposed components at the last level (DC, very low frequency).

11. Why Wavelets ? • Enhance contrast edges. • Multi-Resolution emphasizes features of different sizes. • Data reduction from thresholding.

12. Wavelet Coefficient Thresholding • For all images, dropped coefficients with magnitudes < 0.4 • 16 million coefficients reduced to 1.6 million. • Threshold determined by experience and experiment: Set of coefficients after thresholding reverse transformed to image, and this image is visually compared to the original.

14. LOSRAAM Training Set • Each component is made a separate vector: 9 components per image * 199 images = 1791 components. • 1.6 M coefficients distributed over 1791 components. • Each coefficient represented by a doublet: [magnitude, normalized index into component array]. • Coefficients fed into LOSRAAM Neural Network one by one.

15. The LOSRAAM ANN • Linear Output Recurrent Recursive Auto-Associative Memory • AAM: Targeting the outputs to be the same as the inputs. • Recurrence: a portion of the input is the activation pattern from the hidden layer of the previous iteration. • Unsupervised Learning: A criterion for judging outputs is determined. The ANN learns a mapping of inputs to outputs that fits the criterion.

17. LOSRAAM Training Statistics • Running time 55 hours. • Sun Sparc, 4 parallel processors. • Error function: initial value = 1,800,000 final value = 401.

18. Clustering LOSRAAM State Vectors • Image structure produces a sequence of activation patterns of the 4 unit hidden layer. • Activation patterns represented by a sequence of vectors in 4-D space. • The trajectory of these vectors is represented by a series of point values in 4-D space. • These points are clustered into “centers of attraction”. • Clustering is a “fuzzy” process. • Our LOSRAAM data yielded 4 centers of attraction.

19. Fuzzy Transition Matricesand Fuzzy Feature Vectors • An FTM is a 4 x 4 matrix of transitions between centers of attraction. • FTMs were computed for each component, for each image. • Thus, each image is represented by 9 FTMs, each containing 16 elements, 144 elements total. • Linearization of FTMs gives 144 element FFVs, one per image.

20. Singular Valued Decomposition of FFVs • SVD is a data reduction and conditioning technique. • Combine FFVs for all images into a 144 x 199 matrix. • Using SVD, identified 9 of the 144 columns that accounted for > 99% of the variance in the entire training data set. • These 1791 values used to train the Feed Forward ANNs. 144 elements 9 elements 199 FFVs 199 reduced FFVs

21. Feed-Forward Neural Network • Supervised Learning: Input and output are provided. The FFNN learns mapping by example. • We used a 9-1-1 FFNN.

22. 9-1-1 Feed-Forward Neural Network Output Unit Fully skip-connected Hidden Unit Input Units Image’s 9 element FFV

23. FFNN Training • Input 199 training patterns (9 values each), one at a time. • Periodically check performance with PAC set. (A set of 16 images, 8 positive, 8 negative). • Harvest weights if performance on training set and on PAC are both > 85%. • If PAC test failed, weights are discarded. • Trained 5 FFNNs to criterion performance on training set and PAC.

24. FFNN Training Issues • Overtraining. First time training with a two-hidden-unit FFNN led to 100% performance for both training and PAC, but only 65% performance on the test set. • Attempt to raise harvest criterion to 88% for one-hidden-unit FFNN failed. No trained networks were produced.

25. Data Flow Diagram

26. Can performance be improved? • Human performance ~ 100%. • ANN must handle several different sources of variation: position, view angle, size, number, flatness. • Increase the size and depth of the training set, e.g., 500 images. • Add positional ‘hint’ units, number hint units, etc.

27. Significance of Results to Kalman and Kwasny’s ANN • Scanning EM images are rich in detail, perhaps more so than mammograms. • Absolute performance on EM images is better than performance on mammograms, particularly with respect to specificity. • The results suggest that the ANN is capable of using the rich detail found in EM images to achieve a low false positive rate. Mammograms EMs False Pos. 44% 14% False Neg. 25% 14%

29. Potential Applications • Automated scanning of EM images. • Large scale screening of histological slides for abnormal cell morphologies.