VII. Neural Networks and Learning

Supervised Learning

	Produce desired outputs for training inputs
	Generalize reasonably & appropriately to other inputs
	Good example: pattern recognition
	Feedforward multilayer networks

Feedforward Network

Typical Artificial Neuron

Typical Artificial Neuron

Equations

Single-Layer Perceptron

Variables

Single Layer Perceptron Equations

2D Weight Vector

N-Dimensional Weight Vector

Goal of Perceptron Learning

	Suppose we have training patterns x1, x2, É, xP with corresponding desired outputs y1, y2, É, yP
	where xp ë {0, 1}n, yp ë {0, 1}
	We want to find w, q such that yp = Q(w×xp Ð q) for p = 1, É, P

Treating Threshold as Weight

Treating Threshold as Weight

Augmented Vectors

Reformulation as Positive Examples

Adjustment of Weight Vector

Outline of Perceptron Learning Algorithm

	initialize weight vector randomly
	until all patterns classified correctly, do:
		for p = 1, É, P do:
			if zp classified correctly, do nothing
			else adjust weight vector to be closer to correct classification

Weight Adjustment

Improvement in Performance

Perceptron Learning Theorem

	If there is a set of weights that will solve the problem,
	then the PLA will eventually find it
	(for a sufficiently small learning rate)
	Note: only applies if positive & negative examples are linearly separable

Classification Power of Multilayer Perceptrons

	Perceptrons can function as logic gates
	Therefore MLP can form intersections, unions, differences of linearly-separable regions
	Classes can be arbitrary hyperpolyhedra
	Minsky & Papert criticism of perceptrons
	No one succeeded in developing a MLP learning algorithm

Credit Assignment Problem