Skip to main navigation Skip to search Skip to main content

Stochastic Decision Theory

Course

URL study guide

https://tue.osiris-student.nl/onderwijscatalogus/extern/cursus?cursuscode=2MMS50&collegejaar=2025&taal=en

Description

Many decision and optimization problems that arise in practical scenarios are stochastic in nature, in that they involve information or phenomena that are intrinsically random or extremely difficult to predict with any degree of certainty. Markov decision processes and stochastic dynamic programming models provide a powerful methodological framework for modeling and solving such problems.  The course covers the three main problem formulations (finite horizon, infinite-horizon average reward/cost, and infinite-horizon discounted reward/cost) as well as the three main
computational approaches (value iteration, policy iteration and linear programming).  The course also treats Gittins-index policies for so-called multi-armed bandit problems and various learning algorithms for optimizing exploration-exploitation trade-offs in scenarios with unknown parameter values.  In addition, the course presents several miscellaneous topics and techniques in stochastic decision and optimization problems, such as newsboy problems,
achievable performance regions, optimal stopping problems and stochastic approximation methods.

Objectives

The goal of the course is to familiarize students with the basic mathematical concepts and computational techniques for stochastic decision and optimization problems, and illustrate the application of these methods in various scenarios. The methodological framework of Markov decision processes and stochastic dynamic programming models will play a central role, and the students are expected to obtain knowledge of the main problem formulations and be able to apply the the main computational approaches in that domain to stylized problems. The use of these problem formulations and computational techniques will be illustrated in the context of various specific examples. The students will also gain familiarity with so-called multi-armed bandit models and Gittins indices for stochastic optimization in scenarios with unknown parameter values, and learn how to apply these approaches to specific problem instances. In addition, the students are expected to acquaint themselves with several miscellaneous topics, such as newsboy problems, achievable performance regions, optimal stopping problems, and stochastic approximation methods.
 
 

Method of Assessment

Four written homework assignments
Course period1/09/15 → 31/08/26
Course formatCourse