Project 4 - Denoising Autoencoder (Due 10/26)

Objectives:

The objective of this project is two-fold, to obtain a deeper understanding of the training process through the implementation of a denoising autoencoder from scratch, and to experience the importance of initialization in CNN training.

Data set used:

CIFAR-10.

This site gives you an overview of what CIFAR-10 is and how to download the dataset.
This site gives you an up-to-date who is the best in CIFAR-10.

Requirements:

Task 1: Implement a denoising autoencoder
- You can use an open-source distributions as a starting point.
- Add Gaussian noise as one of the noise models to the corruption module.
- Instead of sigmoid and tanh, use ReLu as the activation function and compare the difference
Task 2: Apply the denoising autoencoder on CIFAR-10 and use the output from DAE as input to VGGNet you implemented in project 3. Compare and comment on the performance gain if any.
Task 3: Use the weight trained from DAE on CIFAR-10 to initialize the VGGNet you constructed in Task 1 of Project 3. Compare and comment on the performance gain and convergence trend.

Report

Please submit the following through Canvas before midnight on the due date.

Provide the performance comparison graphs and discussion/comments on the differences.
(For 692 students only) Read [sDAE:2010] and play around the stacked denoising autoencoder implementation. Write a one-page (with double space) report on your understanding of supervised vs. unsupervised learning, how unsupervised learning had improved the performance of the training of supervised network, the discussion of RBM and DBN sort of cooled down especially after the rise of variants of CNN, provide your point of view on the development trend. Find the paper on variational autoencoder that represents the state-of-the-art on AE development. Provide your brief understanding.