Project 1 - MNIST Digit Recognition Using Multi-Layer Neural Networks (Due 09/07)

Objectives:

The objective of this project is two-fold. First, to help you get started on TensorFlow and how to use Google Cloud. Second, to have a thorough understanding of the traditional neural network and backpropagation.

Data set used:

MNIST

Requirements:

Task 1: Get familiar with Google Cloud and TensorFlow.
Task 2: Digit recognition with TensorFlow.
Implement a multi-layer neural network to recognize digits using MNIST.
- Go through the sample code (testMNIST.py) and the step-by-step instruction here.
- Study this modified version (testMNISTmod.py) of the code where an object-oriented programming stype is used. You should always use this style in future designs.
- Evaluate testMNISTmod.py and plot figures illustrating accuracy vs. different hyper-parameter setups (e.g., different numbers of hidden layers, different learning rate). The requirement of this task is not clearly defined which is intentional. You need to think of all possible ways to evaluate the effect of the hyper-parameters to accuracy, convergence speed, etc.
Task 3: Digit recognition with Numpy.
Go through Nielsen's book [Nielsen] - Chapter 1, where he used Numpy to implement a multi-layer neural network from scratch. Study and run his code and do the same evaluation as in Task 2.

Report

You are not required to submit a full-scale report this time. Please submit the following through Canvas before midnight on the due date.

The evaluation graphs that you generated from both Tasks 2 and 3.
What's your conclusion based on the evaluations in Tasks 2 and 3? Write a 1-page report.
(For 692 students only) Describe your understanding of the following four subjects: batch vs. online processing, gradient descent vs. stochastic gradient descent, perceptron vs. sigmoid neurons, and feedforward vs. backpropagation. You need to describe each subject in no more than 3 sentences. This part of the report should not exceed 1 page.