Date 
Topic/s Covered 
Lecture Notes
& Handouts 
Reading
Assignments (before class) 
Homework &
Projects 
Thr  Aug 20 
Class overview, Introduction 
Word Clouds on Interests
and Expectations 


Tue  Aug 25 
Evaluative Feedback 



Thr – Aug 27 
Review of discretetime probability fundamentals 


Due: September 8 
Tue – Sept 1 
Discrete Time Markov Chains 



Thr – Sep 3 
The Reinforcement Learning Problem 



Tue – Sep 8 
The Reinforcement Learning Problem (cont’) 



Thr – Sep 10 
Markov Decision Processes (MDPs) and Optimality Criterion in
MDPs, SW/HW Considerations 

Due: September 24 

Tue – Sep 15 
Finite Horizon MDP 



Thr – Sep 17 




Tue – Sep 22 
Class Cancelled 



Thr – Sep 24 
Monte Carlo Methods 


Due: Oct 8 
Tue – Sept 29 
Monte Carlo Methods (cont’) Temporal Difference Learning 



Thr – Oct 1 
Temporal Difference Learning (cont’) 



Tue – Oct 6 
InClass Midterm – MK
405 




Temporal Difference Learning (cont’) 



Tue – Oct 13 
ActorCritic Model, Eligibility Traces 


Due: Oct 22 
Thr – Oct 15 
Fall break – No Class 



Tue – Oct 20 
Eligibility traces (cont’) 


Due: Nov 3 
Thr – Oct 22 
Intro to Neural Networks 



Tue – Oct 27 
Planning and Learning 


Due: Nov 5 
Thr – Oct 29 
Case studies in RL 



Tue – Nov 3 
Recurrent Neural Networks, LSTMs 



Thr – Nov 5 
Partially Observable MDPs (POMDPs) 


Due: Nov 19 
Tue – Nov 10 
Open topics in RL and its Applications 


Final project
presentations schedule 
Thr – Nov 12 
Deep reinforcement learning 



Tue – Nov 17 
Project presentations 



Wed – Nov 18 3:35pm – 4:25pm 
Project presentations Location: MK 525 



Thr – Nov 19 
Project presentations 



Tue – Nov 24 
Project presentations 



Wed – Nov 25 1:15pm – 3:30pm 
Project Presentations  Tickle Engineering Bldg Room 402 



Thr – Nov 26 
Thanksgiving – No Class 



Tue – Dec 11:10pm
– 1:45pm 
Project presentations 



Monday – December 7 
Final project reports are due 



Matlab codes that cover most
topics addressed in the course textbook can be found at: http://www.waxworksmath.com/Authors/N_Z/Sutton/sutton.html
Last Update: November 20, 2015