AERE504: Course Info

Lectures

• Tuesdays and Thursdays 2:10pm-3:25pm in 2202 Howe.

• The 2018 Fall semester is from 08/26/2019 to 12/19/2019.

Major Themes

• There is a principled mathematical framework for defining rational behavior.

• Key algorithms for solving decision problems leverage decomposition and recursion.

• The computational techniques discussed in class can lead to superior decisions that are sometimes counterintuitive.

• Successful application of these principles depends on the choice of representation and approximation.

• The same computational approaches can be applied to very different application domains.

Learning Outcomes

• You will gain a broad fundamental understanding of the mathematical models and solution methods for decision making (exercises, two midterms, take-home exam).

• You will be able to implement and extend key algorithms for learning and decision making (two programming projects).

• You will be able to identify an application of the theory in this course and formulate it mathematically (proposal).

• You will gain a deep understanding of an area of particular interest and apply it to a problem (final project).

• You will be able to critique approaches to solving decision problems (peer review).

Textbook

Mykel J. Kochenderfer, Decision Making Under Uncertainty: Theory and Application, MIT Press, 2015.

References

• Olivier Sigaud and Olivier Buffet, Markov Decision Processes in Artificial Intelligence, Wiley, 2010.
  

• Richard Sutton and Andrew Barto, Reinforcement Learning: An Introduction, MIT Press, 2006 or 2016.
  

• David Silver, UCL Course on RL, 2015.
  

• Michael Littman, POMDPs for Dummies, 1999.
  

• Dan Klein and Pieter Abbeel, UC Berkeley CS 188.
  

Grading

Projects: 30%

• 15% Project 1

• 15% Project 2

Midterms: 45%

• 15% Midterm 1

• 15% Midterm 2

• 15% Take-home exam

Final project: 25%

• 5% Proposal

• 15% Paper

• 5% Peer Review

Grades are posted on Canvas.

Late Policy

The Late Policy is that there are No Late days, so please plan ahead under uncertainty.

Exams

Dates

• Midterm 1 (in class) on Sept. 17

• Midterm 2 (in class) on Oct. 31

• Take-home Exam (at home) due on Nov. 19, 11:59PM

Notes
Students are allowed to bring only 1 double sided page (letter size) of typed or handwritten notes to the midterm.

Students with Disabilities

Students with disabilities: “Students are encouraged to schedule their examinations with accommodations to be taken with their instructor or TA, when possible. However, Professors/Instructors can work with our staff at the Exam Accommodation Center (EAC) located in the Hixson-Lied Student Success Center by completing the “Exam Accommodation Referral” form. If you have further questions you can call 515-294-7220. Exam Accommodations Center hours are 8:00 a.m. - 5:00 p.m. Monday - Friday. Please note: One week advance notice is needed to coordinate exams at the EAC.” More information may be found here.