The University of Tennessee at Knoxville

Department of Electrical Engineering & Computer Science



ECE 517: Reinforcement Learning in Artificial Intelligence


Fall 2015


Course Homepage: ECE-517


General Information


Lectures: T/TR 11:10 12:25 PM (MK 405)

Instructor: Dr. Itamar Arel


Office: MK 608

Office Hours: T/Tr 2:00PM 3:00PM

Tel: (865)-974-3891


TA: Aaron Mishtal Office located in MK 606

Office hours: MWF: 12:00 2:00pm



Course Description


Reinforcement learning (RL) is an exciting and relatively new machine learning discipline, which corresponds to a broad class of methods that allow a system to learn how to behave in environments that are incompletely known/specified based on reward signals. A key concept in RL is that the intelligent agent learns autonomously, based on acquired experience, rather than by being externally instructed or supervised. This course will address various topics pertaining to the theory and practice of RL.




Background in linear algebra and probability theory, as well as knowledge of the Matlab programming, are required.


Reference Material


Lecture notes will be posted and made available at the course website. The majority of the material will appear in the lecture notes and supplemental reading material (papers). The course textbook will be:

m  R. Sutton and A. Barto, Reinforcement Learning: An Introduction, 2015, 2nd Ed. (available online at:



The course will include two small projects and a final project. The final project will also involve an in-class presentation by the students.


Assignment Sets


There will be five assignment sets issued. Students are expected to complete all of the assigned problems. Assignment sets are to be submitted at the beginning of the class in which they are due.




There will be one in-class midterm exam. No make-up exams will be offered.




The following topics will be covered:

q  Foundations of machine learning & autonomous cognitive systems

q  Markov decision processes (MDP)

q  Neural networks

q  Dynamic programming

q  Temporal difference (TD) learning

q  Monte Carlo reinforcement learning methods

q  Eligibility traces

q  Hardware & software implementation considerations

q  Deep Reinforcement Learning


Final Grade



90 100


87 89


80 86


77 79


70 76


60 69


0 59


Grading Policy (tentative)


Small projects (2 12.5 points) = 25%

Assignment sets (5 5 points) = 25%

Midterm exam = 20%

Final project = 30%




Students are encouraged to frequently visit the course website for announcements and updates.


Last Update: August 18, 2015