1 / 16

Evolution of Deep Learning Architectures: A Comprehensive Overview

Explore the evolution of deep learning architectures from LeNet in 1998 to the latest developments like AlexNet, VGG16, Inception, and DenseNet. Understand the concepts of residual networks, network depth vs. width, and the impact on model performance.

amcleod
Download Presentation

Evolution of Deep Learning Architectures: A Comprehensive Overview

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Deep learning architectures Usman Roshan NJIT Figures from https://www.jeremyjordan.me/convnet-architectures/ https://medium.com/@sidereal/cnns-architectures-lenet-alexnet-vgg-googlenet-resnet-and-more-666091488df5

  2. Evolution of networks

  3. LeNet Deep learner by YannLeCunn in 1998 to train on MNIST

  4. AlexNet Won ImageNet in 2012

  5. VGG16 Introduced in 2014 by Oxford, won ImageNet classification plus localization

  6. Inception (GoogLeNet) I Introduced in 2014 by Google, won ImageNet classification only

  7. Inception II (Inception cell) Basic unit of inception

  8. Inception III (efficient large kernel width) Two stacked 3x3 kernels have a similar effect of a single 5x5 kernel with fewer parameters. Do a 3x3 kernel with successive 1x3 and 3x1 convolutions

  9. Inception IV

  10. Residual networks • Deep(er) networks suffer from degradation: high error than shallow networks. One reason is vanishing gradients • Residual connections introduced in 2015 to alleviate this problem (won ImageNet the same year)

  11. Residual networks II Relu before residual addition gives better results than after

  12. Residual networks

  13. Residual networks

  14. Deep vs. wide networks • Deep: few nodes per layer, many layers • Wide: few layers, many nodes per layer • Turns out that a 3-layer can express functions that a 2-layer network would require exponential nodes (The Power of Depth for Feedforward Neural Networks, Eldan and Shamir, COLT 16)

  15. DenseNet Concatenate output of a layer to all successive layers

  16. DenseNet vs. ResNet

More Related