ECE599/692 Home
ECE599/692 - Deep Learning


Course Information

Syllabus

Reference

Datasets


Textbooks

  • [Goodfellow] Ian Goodfellow, Yoshua Bengio, Aaron Courville, Deep Learning, MIT Press.
  • [Nielsen] Michael A. Nielsen, Neural Network and Deep Learning, Determination Press, 2015.
  • [RNN-Tutorial] Recurrent Neural Network Tutorial
  • [Snyder&Qi:2017] Wesley E. Snyder, Hairong Qi, Foundations of Computer Vision, Cambridge University Press, 2017.
  • [Duda&Hart:2001] R. O. Duda, P. E. Hart, D. G. Stork, Pattern Classification, 2nd Edition, John Wiley, 2001.

Online Courses

Papers

  • [HAN:2016] Z. Yang, D. Yang, C. Dyer, X. He, A. Smola and E.H. Hovy, "Hierarchical attention networks for document classificatio," Proceedings of the HLT-NAACL conference. San Diego, CA, 2016.
  • [TextCNN:2016] A. Conneau, H. Schwenk, L. Barrault, Y. Lecun, "Very deep convolutional networks for text classification," 2016.
  • [ConditionalGAN:2014] M. Mirza, S. Osindero, "Conditional generative adversarial nets," 2014.
  • [DCGAN:2016] A. Radford, L. Metz, S. Chintala, "Unsupervised representation learning with deep convolutional generative adversarial networks," ICLR, 2016.
  • [Arjovsky:2017] M. Arjovsky, L. Bottou, "Towards principled methods for training generative adversarial networks," ICLR, 2017.
  • [Goodfellow:2016] I. Goodfellow, "NIPS 2016 Tutorial: Generative adversarial networks," NIPS, 2016.
  • [Goodfellow:2014] I.J. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, Y. Bengio, "Generative adversarial networks," NIPS, 2014
  • [Bengio:2014] Y. Bengio, A. Courville, P. Vincent, "Representation learning: A review and new perspectives," 2014.
  • [Hinton:2006a] G.E. Hinton, S. Osindero, Y.W. Teh, "A fast learning algorithm for deep belief nets," Neural Computing, 2006.
  • [Hinton:2006b] G.E. Hinton, R.R. Salakhutdinov, "Reducing the dimensionality of data with neural networks," Science, 313, July 2006.
  • [DAE:2008] P. Vincent, H. Larochelle, Y. Bengio, P.A. Manzagol, "Extracting and composing robust features with denoising autoencoders," ICML, 2008.
  • [SAE:2008] M.A. Ranzato, Y.L. Roureau, Y. LeCun, "Sparse feature learning for deep belief networks," NIPS, 2008
  • [sDAE:2010] P. Vincent, H. Larochelle, I. Lajoie, Y. Bengio and P. Manzagol, "Stacked denoising autoencoders: Learning useful representations in a deep network with a local denoising criterion," Journal of Machine Learning Research, 2010. (Github repository)
  • [GNN:2018] P.W. Battaglia, J.B. Hamrick, et al., "Relational inductive biases, deep learning, and graph networks," https://arxiv.org/pdf/1806.01261.pdf, June 2018.
  • [Capsule:2017] S. Sabour, N. Frosst, and G.E. Hinton. "Dynamic routing between capsules," NIPS, 2017.
  • [SENet:cvpr:2018] Jie Hu, Li Shen, Gang Sun, "Squeeze-and-Excitation networks," CVPR, 2018
  • [ResNet:eccv:2016] Kaiming He, Xiangyu Zhang, Shaoqing Ren, Jian Sun, "Identity mappings in deep residual networks," ECCV, 2016.
  • [ResNet:2015] Kaiming He, Xiangyu Zhang, Shaoqing Ren, Jian Sun, "Deep residual learning for image recognition," 2015.
  • [VGGNet:2014] K. simonyan, A. Zisserman, "Very deep convolutional networks for large-scale image recognition," 2014.
  • [GoogLeNet:2014] C. Szegedy, W. Liu, Y. Jia, P. Sermanet, S. Reed, D. Anguelov, D. Erhan, V. Vanhoucke, A. Rabinovich, "Going deeper with convolutions," 2014.
  • [AlexNet:2012] A. Krizhevsky, I. Sutskever, G.E. Hinton, "ImageNet classification with deep convolutional neural networks," Advances in Neural Information Processing Systems, pages 1097-1105, 2012. (ImageNet)
  • [Hinton:2012] G. Hinton, N. Srivastava, A. Krizhevsky, I. Sutskever, R. Salakhutdinov, "Improving neural networks by preventing co-adaptation of feature detectors," 2012. (Dropout)
  • [Benjio:2012] Y. Bengio, "Practical recommendations for gradient-based training of deep architectures," 2012 (Hyper-parameters)
  • [LeCun:1998] Y. LeCun, L. Bottou, Y. Bengio, P. Haffner, "Gradient-based learning applied to document recognition," Proceedings of the IEEE, November 1998.
  • [LeCun:1989] Y. LeCun, B. Boser, J.S. Denker, D. Henderson, R.E. Howard, W. Hubbard, L.D. Jackel, "Backpropagation applied to handwritten zip code recognition," Neural Computation, 1(4):541-551, 1989. (first paper on CNN).
  • [LMRL:1998] T. S. Lee, D. Mumford, R. Romero, V. A.F. Lamme, "The role of the primary visual cortex in higher level vision," Vision Research 38: 2429-2454, 1998.
  • [RumelhartHintonWilliams:1986] D.E. Rumelhart, G.E. Hinton, R.J. Williams, "Learning representations by back-propagating errors," Nature, 323(9):533-536, October 1986. (BP)

Student Paper Samples

  • [Rahimpour:icip17] Alireza Rahimpour, Liu Liu, Ali Taalimi, Yang Song, Hairong Qi, "Person re-identification using visual attention," IEEE International Conference on Image Processing (ICIP), Beijing, September 2017.
  • [Liu:icip17] Liu Liu, Alireza Rahimpour, Ali Taalimi, Hairong Qi, "Binary representation learning via direct binary embedding" IEEE International Conference on Image Processing (ICIP), Beijing, September 2017.

Compute Environment

Conferences

  • Top five computer science conferences
  • ICML: International Conference on Machine Learning
  • NIPS: Neural Information Processing Systems
  • ICLR: International Conference on Learning Representations
  • CVPR: Computer Vision and Pattern Recognition
  • ICCV: International Conference on Computer Vision
  • ECCV: European Conference on Computer Vision
  • ICPR: International Conference of Pattern Recognition
  • WACV: Winter Conference on Applications of Computer Vision
  • ICIP: International Conference on Image Processing