Date 
Topic/s Covered 
Lecture Notes
& Handouts 
Reading
Assignments (before class) 
Homework &
Projects 
Thr  Aug 20 
Class overview, Introduction 
Word Clouds on Interests
and Expectations 


Tue  Aug 25 
Evaluative Feedback 



Thr – Aug 27 
Review of discretetime probability fundamentals 


Due: September 8 
Tue – Sept 1 
Discrete Time Markov Chains 



Thr – Sep 3 
The Reinforcement Learning Problem 



Tue – Sep 8 
The Reinforcement Learning Problem (cont’) 



Thr – Sep 10 
Markov Decision Processes (MDPs) and Optimality Criterion in
MDPs, SW/HW Considerations 

Due: September 24 

Tue – Sep 15 
Finite Horizon MDP 



Thr – Sep 17 




Tue – Sep 22 
Class Cancelled 



Thr – Sep 24 
Monte Carlo Methods 


Due: Oct 8 
Tue – Sept 29 
Monte Carlo Methods (cont’) Temporal Difference Learning 



Thr – Oct 1 
Temporal Difference Learning (cont’) 



Tue – Oct 6 
InClass Midterm – MK
405 




Temporal Difference Learning (cont’) 


Homework #3 Due: Oct 20 
Tue – Oct 13 
ActorCritic Model, Eligibility Traces 



Thr – Oct 15 
Fall break – No Class 


Project #2 Due: October 29 
Tue – Oct 20 
Generalization & Function Approximation 


Homework #4 Due: Nov 3 
Thr – Oct 22 
Generalization & Function Approximation (cont’) 



Tue – Oct 27 
Neural Networks – Introduction, Feedforward Neural Networks 



Thr – Oct 29 
Neural Networks (cont’) 



Tue – Nov 3 
Planning in RL 


Homework #5 Due: Nov 17 
Thr – Nov 5 
Partially Observable MDPs (POMDPs) 



Tue – Nov 10 
Recurrent Neural Networks, LSTMs 



Thr – Nov 12 
Inverse Reinforcement Learning, Imitation Learning in RL 



Tue – Nov 17 
Deep Reinforcement Learning 



Thr – Nov 19 
TBD 



Tue – Nov 24 
TBD 



Thr – Nov 26 
Thanksgiving – No Class 



Tue – Dec 1 
TBD 



Monday – December 7 
MK 405 – 10:15am to 12:15pm 



Matlab codes that cover most
topics addressed in the course textbook can be found at: http://www.waxworksmath.com/Authors/N_Z/Sutton/sutton.html
Last Update: September 17, 2015