Modeling image patches with a directed hierarchy of Markov random fields

Dave Bradley 30 Jan

I will be talking about deep learning in general and Restricted Boltzman Machines (RBMs) in particular. Deep learning refers to learning machines that have several intermediate representation layers. Traditionally, with the exception of convolutional neural networks, these deep machines have been hard to train. Recently, however, greedy layer-wise pretraining with unsupervised data has been found to produce good results for a variety of deep architectures. I will be covering some recent work presented at NIPS that use RBMs in deep architectures to model natural images (Osindero and Hinton 2007) and recognize the orientation of faces (Salakhutdinov and Hinton 2007). I will also cover an interesting empirical evaluation of RBMs that was presented at ICML (Larochelle et al. 2007) and takes a more critical view of the challenges remaining in deep learning.
Papers

Osindero, S. and Hinton, G. E. Modeling image patches with a directed hierarchy of Markov random fields. Advances in Neural Information Processing Systems 20, 2007

Salakhutdinov, R. R. and Hinton, G. E. Using Deep Belief Nets to Learn Covariance Kernels for Gaussian Processes. Advances in Neural Information Processing Systems 20 , 2007

Hugo Larochelle, Dumitru Erhan, Aaron Courville, James Bergstra and Yoshua Bengio, An Empirical Evaluation of Deep Architectures on Problems with Many Factors of Variation, International Conference on Machine Learning proceedings, 2007