Introduction to Deep Learning for Biomedical Informatics

1. Introduction to Deep Learning for Biomedical Informatics Yushuo Niu Computer Science & Engineering Department The University of Connecticut CSE4095/5810: Introduction to Biomedical Informatics Professor: StevenA. Demurjian’s

2. Outline What is Machine Learning Why is Deep Learning taking of The great Enthusiasm and Dynamism Different Deep Learning Architectures Most popular Software Packages Conclusion

3. What is Machine Learning • Using data to answer questions

4. Machine Learning Algorithm Why is Machine Learning important? How is Machine Learning used today? Database mining Large datasets from growth of Medical Records, Biology and Engineering Application can’t program by hand Self-driving vehicle, Natural Language Processing (NLP), Computer Vision Self-customizing programs Amazon, Netflix recommendations and Google search What is Machine Learning

5. Machine Learning Algorithm Category: Supervised Learning Unsupervised Learning Others: Reinforcement learning, Recommender system What is Machine Learning

6. Comparison between classic machine learning and deep learning Why is Deep Learning taking off

7. Model of a neuron Why is Deep Learning taking off

8. The benefit of implicit features Micro-array Based Cancer Classification Why is Deep Learning taking off

9. Scale drives deep learning process Only Deep Learning systems continue to increase their performance with increasing size of dataset Why is Deep Learning taking off

10. The great enthusiasm and dynamism • A rapid surge of interest of publication about deep learning in sub-areas of Biomedical Informatics • Publication statistics are obtained from Google Scholar

11. Different Deep Learning Architectures • Several architectures stand out in popularity • Percentage of the most commonly used deep learning methods for health informatics

12. Different Deep Learning Architectures • Deep Neural Network • Architecture

13. Different Deep Learning Architectures • Deep Neural Network • Description • More than two hidden layers • General deep framework used for classification and regression • Pros • Successfully applied in many fields • Cons • Training is computationally expensive and slow • vanishing of the gradient

14. Different Deep Learning Architectures • Deep Neural Network • Application • Bioinformatics • Drug Design, RNA binding protein, DNA methylation • Medical Imaging • Alzheimer/MCI diagnosis, Hemorrhage detection, Tumor detection • Pervasive Sensing • Food intake, Energy expenditure • Medical Information • Data mining • Public Health • Air pollutant prediction

15. Different Deep Learning Architectures • Deep Autoencoder • Architecture

16. Different Deep Learning Architectures • Deep Autoencoder • Description • Mainly designed for fexture extraction or dimensionality reduction • Has the same number of input and output nodes • Unsupervised Learning • Pros • Dose not require labelled data for training • Cons • Require a pre-training stage • Suffer from vanishing of the error

17. Different Deep Learning Architectures • Deep Autoencoder • Application • Bioinformatics • Cancer Diagnosis • Medical Imaging • 3D brain reconstruction, Cell clustering • Medical Information • Data mining • Public Health • Predicting demographic info

18. Different Deep Learning Architectures • Deep Belief Network • Architecture

19. Different Deep Learning Architectures • Deep Belief Network • Description • Has undirected connections just at the top two layers • Allows unsupervised and supervised • Pros • Proposes a layer-by-layer greedy learning strategy to initialize the network • Inferences tractable maximizing the likelihood directly • Cons • Training procedure is computationally expensive due to the initialization process and sampling

20. Different Deep Learning Architectures • Deep Belief Network • Application • Bioinformatics • Gene selection/classification, Compound-Protein interaction • Medical Imaging • Brain tissues classification, Tissue classification • Pervasive Sensing • Human activity recognition, Obstacle detection, Signlanguage recognition • Medical Information • Prediction of disease • Public Health • Lifestyle diseases

21. Different Deep Learning Architectures • Deep Boltzmann Machine • Architecture

22. Different Deep Learning Architectures • Deep Boltzmann Machine • Description • Possesses undirected connections (conditionally independent) between all layers of the network • Uses a stochastic maximum likelihood algorithm to maximize the lower bound of the likelihood • Pros • Incorporates top-down feedback for a more robust inferences with ambiguous inputs • Cons • Time complexity for the inference is higher than Deep belief Network • Optimization of the parameters is not practical for large datasets

23. Different Deep Learning Architectures • Deep Boltzmann Machine • Application • Bioinformatics • Compound-Protein interaction • Medical Imaging • Tissues classification • Pervasive Sensing • Hand gesture recognition, Obstacle detection, Signlanguage recognition • Medical Information • Prediction of disease • Public Health • Lifestyle diseases

24. Different Deep Learning Architectures • Recurrent Neural Network • Architecture

25. Different Deep Learning Architectures • Recurrent Neural Network • Description • Capable of analyzing stream of data • Output depends on the previous computations • Shares the same weights across all steps • Pros • Can memorize sequential events • Can model time dependencies • Great success in many Natural Language Processing • Cons • Suffer vanishing gradient and exploding gradient problems

26. Different Deep Learning Architectures • Recurrent Neural Network • Application • Medical Information • Prediction of disease Prediction of disease • Human behaviour monitoring • Data mining

27. Different Deep Learning Architectures • Convolutional Neural Network • Architecture

28. Different Deep Learning Architectures • Convolutional Neural Network • Description • It is well suited for 2D data such as images • Every hidden convolutional filter transforms its input to a 3D output volume • Pros • Few neuron connections required with respect to a typical NN • Cons • It may require many layers • It usually requires a large dataset of labelled images

29. Different Deep Learning Architectures • Convolutional Neural Network • Application • Medical Imaging • Neural cells classification, Brain tissues classification • Pervasive Sensing • Human activity recognition, Food intake • Medical Information • Human behaviour monitoring • Public Health • Infectious disease epidemics

30. Most popular software packages

31. Conclusion • With a large influx of Biomedical data, rapid surge of interest in deep learning • Reduce Human intervention • With EHRs, increase the reliability of clinical decision support

32. Conclusion