ECE 517: Reinforcement Learning in Artificial Intelligence – Fall 2012

* Schedules are subject to changes, please check back frequently
 

Date

Topic/s Covered

Lecture Notes & Handouts

Reading Assignments (before class)

Homework & Projects

Thr - Aug 23

Class overview, Introduction

Lecture #1

* A good reference book on Probability theory)

* Matlab references [1][2]

 

 

Tue - Aug 28

Evaluative Feedback

Lecture #2

 

 

Thr – Aug 30

Review of discrete-time probability fundamentals

Lecture #3

 

Tue – Sept 4

Discrete Time Markov Chains

Lecture #4

 

HW#1

Due: Sept. 11

Thr – Sep 6

The Reinforcement Learning Problem

Lecture #5

 

 

 

Tue – Sep 11

 

Class Cancelled

 

 

 

Thr – Sep 13

The Reinforcement Learning Problem (cont’)

 

Tue – Sep 18

Markov Decision Processes (MDPs) and Optimality Criterion in MDPs, SW/HW Considerations

Lecture #6

HW#2

Due: Sept. 27

Thr – Sep 20

Finite Horizon MDP

Lecture #7

 

Tue – Sep 25

Dynamic Programming, Policy Iteration, Value Iteration

Lecture #8

 

 

Project #1

Due Oct 16

Thr – Sep 27

Monte Carlo Methods

Lecture #9

 

 

Tue – Oct 2

Monte Carlo Methods (cont’)

 

 

 

Thr – Oct 4

Monte Carlo Methods (cont’)

 

 

 

Tue – Oct 9

In-Class Midterm – FH 511

 

 


Thr – Oct 11

Fall break – No Class

 

 

Tue – Oct 16

Temporal Difference Learning

Lecture #10

 

Thr – Oct 18

Actor-Critic Model, Eligibility Traces

Lecture #11

 

HW #3

Due: October 30

Tue – Oct 23

Generalization & Function Approximation

Lecture #12

 

 

Project #2

Due Nov 6

Thr – Oct 25

Generalization & Function Approximation (cont’)

 

 

Tue – Oct 30

Neural Networks – Introduction, Feedforward Neural Networks

Lecture #13

 

Thr – Nov 1

Neural Networks (cont’)

 

List of projects to choose from

Tue – Nov 6

Planning in RL

Lecture #14

 

HW #4

Due: Nov. 15

Thr – Nov 8

Partially Observable MDPs (POMDPs)

 

Projects Assignments

Tue – Nov 13

Recurrent Neural Networks

 

 

Thr – Nov 15

Case Studies, Apprenticeship learning

 

HW #5

Due: Nov. 27

Tue – Nov 20

Neuro-Dynamic Programming, Policy Gradient methods

 

 

Thr – Nov 22

Thanksgiving – No Class

 

Tue – Nov 27

Student Presentations

 

 

 

Thr – Nov 29

Student Presentations

 

 

 

Tue – Dec 4

Student Presentations

 

 

 

Friday – Dec 7

Final project reports are due

 

 

 

Matlab codes that cover most topics addressed in the course textbook can be found at: http://www.waxworksmath.com/Authors/N_Z/Sutton/sutton.html

Last Update: November 16, 2012