Date 
Topic/s Covered 
Lecture Notes
& Handouts 
Reading
Assignments (before class) 
Homework &
Projects 
Thr  Aug 20 
Class overview, Introduction 
Word Clouds on Interests
and Expectations 


Tue  Aug 25 
Evaluative Feedback 



Thr Aug 27 
Review of discretetime probability fundamentals 


Due: September 8 
Tue Sept 1 
Discrete Time Markov Chains 



Thr Sep 3 
The Reinforcement Learning Problem 



Tue Sep 8 
The Reinforcement Learning Problem (cont) 



Thr Sep 10 
Markov Decision Processes (MDPs) and Optimality Criterion in
MDPs, SW/HW Considerations 



Tue Sep 15 
Finite Horizon MDP 



Thr Sep 17 
(Aaron Mishtal) 



Tue Sep 22 
Class Cancelled 



Thr Sep 24 
Monte Carlo Methods 



Tue Sept 29 
Monte Carlo Methods (cont) 



Thr Oct 1 
Monte Carlo Methods (cont) 



Tue Oct 6 
InClass Midterm MK
405 




Temporal Difference Learning 



Tue Oct 13 
Fall break No Class 



Thr Oct 15 
ActorCritic Model, Eligibility Traces (Aaron Mishtal) 



Tue Oct 20 
Generalization & Function Approximation 



Thr Oct 22 
Generalization & Function Approximation (cont) 



Tue Oct 27 
Neural Networks Introduction, Feedforward Neural Networks 



Thr Oct 29 
Neural Networks (cont) 



Tue Nov 3 
Planning in RL 



Thr Nov 5 
Partially Observable MDPs (POMDPs) 



Tue Nov 10 
Recurrent Neural Networks 



Thr Nov 12 
Inverse Reinforcement Learning, Imitation Learning in RL 



Tue Nov 17 
Deep Reinforcement Learning 



Thr Nov 19 
Student Presentations 



Tue Nov 24 
Student Presentations 



Thr Nov 26 
Thanksgiving No Class 



Tue Dec 1 
Student Presentations 



Friday Dec 4 
Final
project reports are due 



Matlab codes that cover most
topics addressed in the course textbook can be found at: http://www.waxworksmath.com/Authors/N_Z/Sutton/sutton.html
Last Update: August 27, 2015