Date 
Topic/s Covered 
Lecture Notes
& Handouts 
Reading
Assignments (before class) 
Homework &
Projects 
Thr  Aug 23 
Class overview, Introduction 


Tue  Aug 28 
Evaluative Feedback 



Thr Aug 30 
Review of discretetime probability fundamentals 


Tue Sept 4 
Discrete Time Markov Chains 

Due: Sept. 11 

Thr Sep 6 
The Reinforcement Learning Problem 



Tue Sep 11 
Class Cancelled 



Thr Sep 13 
The Reinforcement Learning Problem (cont) 

Tue Sep 18 
Markov Decision Processes (MDPs) and Optimality Criterion in
MDPs, SW/HW Considerations 
Due: Sept. 27 

Thr Sep 20 
Finite Horizon MDP 


Tue Sep 25 
Dynamic Programming, Policy Iteration, Value Iteration 

Due Oct 16 

Thr Sep 27 
Monte Carlo Methods 



Tue Oct 2 
Monte Carlo Methods (cont) 



Thr Oct 4 
Monte Carlo Methods (cont) 



Tue Oct 9 
InClass Midterm FH 511 




Fall break No Class 


Tue Oct 16 
Temporal Difference Learning 


Thr Oct 18 
ActorCritic Model, Eligibility Traces 

Due: October 30 

Tue Oct 23 
Generalization & Function Approximation 

Due Nov 6 

Thr Oct 25 
Generalization & Function Approximation (cont) 



Tue Oct 30 
Neural Networks Introduction, Feedforward Neural Networks 


Thr Nov 1 
Neural Networks (cont) 


Tue Nov 6 
Planning in RL 

Due: Nov. 15 

Thr Nov 8 
Partially Observable MDPs (POMDPs) 


Tue Nov 13 
Recurrent Neural Networks 



Thr Nov 15 
Case Studies, Apprenticeship learning 

Due: Nov. 27 

Tue Nov 20 
NeuroDynamic Programming, Policy Gradient methods 



Thr Nov 22 
Thanksgiving No Class 


Tue Nov 27 
Student Presentations 



Thr Nov 29 
Student Presentations 



Tue Dec 4 
Student Presentations 



Friday Dec 7 
Final
project reports are due 



Matlab codes
that cover most topics addressed in the course textbook can be found at: http://www.waxworksmath.com/Authors/N_Z/Sutton/sutton.html
Last Update: November 16, 2012