Project 1 - MNIST Digit Recognition Using Multi-Layer
Neural Networks (Due 09/07)
Objectives:
The objective of this project is two-fold. First, to help you
get started on TensorFlow and how to use Google Cloud. Second, to have a thorough understanding of
the traditional neural network and backpropagation.
Data set used:
MNIST
Requirements:
- Task 1: Get familiar with Google Cloud and TensorFlow.
- Task 2: Digit recognition with TensorFlow.
Implement a multi-layer neural network to recognize
digits using MNIST.
- Go through the sample code (testMNIST.py) and the step-by-step
instruction here.
- Study this modified version (testMNISTmod.py) of the code where an
object-oriented programming stype is used. You should always use
this style in future designs.
- Evaluate testMNISTmod.py and plot figures illustrating
accuracy vs. different hyper-parameter setups (e.g., different
numbers of hidden layers, different learning rate). The
requirement of this task is not clearly defined which is
intentional. You need to think of all possible ways to evaluate
the effect of the hyper-parameters to accuracy, convergence speed,
etc.
- Task 3: Digit recognition with Numpy.
Go through Nielsen's book [Nielsen] - Chapter 1, where
he used Numpy to implement a multi-layer neural network from
scratch. Study and run his code and do the same evaluation as
in Task 2.
Report
You are not required to submit a full-scale report this time. Please
submit the following through Canvas before midnight on the due date.
- The evaluation graphs that you generated from both Tasks 2 and
3.
- What's your conclusion based on the evaluations in Tasks 2 and
3? Write a 1-page report.
- (For 692 students only) Describe your understanding of the following four subjects: batch vs. online processing,
gradient descent vs. stochastic gradient descent, perceptron
vs. sigmoid neurons, and feedforward vs. backpropagation. You need to
describe each subject in no more than 3 sentences. This part of the
report should not exceed 1 page.