1 / 13

Unsupervised feature learning for audio classification using convolutional deep belief networks

Unsupervised feature learning for audio classification using convolutional deep belief networks. Honglak Lee, Yan Largman, Peter Pham and Andrew Y. Ng. Presented by Bo Chen, 5.7,2010. Outline. 1. What’s Deep Learning? 2. Why use Deep Learning? 3. Foundations of Deep Learning

jeroen
Download Presentation

Unsupervised feature learning for audio classification using convolutional deep belief networks

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Unsupervised feature learning for audio classification using convolutional deep belief networks Honglak Lee, Yan Largman, Peter Pham and Andrew Y. Ng Presented by Bo Chen, 5.7,2010

  2. Outline • 1. What’s Deep Learning? • 2. Why use Deep Learning? • 3. Foundations of Deep Learning • 4. Convolutional Deep Belief Networks • 5. Results

  3. Deep Architecture • Deep architectures: compositions of many layers of adaptive non-linear components. Difficulty: parameter searching (local minima) • Deep belief nets: probabilistic generative models that are composed of multiple layers of stochastic, latent variables. (Hinton et al., 2006) Deep Learning Wiki

  4. Why Use Deep Learning • Insufficient depth can hurt Usually our experiences tell us that one-layer machine only gives us a set of general dictionary elements, unless a huge number of dictionary elements. • The brain has a deep architecture • Cognitive processes seem deep • Learn a feature hierarchies or the complicated functions that can represent high-level abstractions For example, PixelsEdgletsMotifsPartsObjectsScenes Some from Yoshua Bengio’s course notes and Yann Lecun, et.al.,2010

  5. One-layer dictionary 30 16x16 dictionary elements and reconstructed images 250 16x16 dictionary elements and reconstructed images

  6. Restricted Boltzmann Machine Binary-valued Energy function Real-valued Contrastive divergence is used to solve the problem. (Hinton et al., 2006) Figure from R Salakhutdinov et. al. 

  7. Deep Architectures RBM in the different layers can be independently trained.

  8. Convolutional Network Architecture Intuitively, in each layer the weight matrix will catch the most consistent ‘structures’ through all of the images. Figure from Yann LeCun et. al, 1998

  9. 3-dimensional Dictionary elements in the second layer D: the first-layer dictionary element E: the second-layer dictionary element S: the convolution of the image and the first-layer elements. The dictionary element in the second layer is a 3-dimensional matrix.

  10. Convolutional RBM with Probabilistic Max-Pooling Layer Max-pooling Layer

  11. Convolutional Deep Belief Networks : the weight matrix Connecting pooling unit Pk to detection unit H’l.

  12. Results on Natural Images

  13. Results Caltech101 Images

More Related