- Part II
Required background
Basics of discrete-time Markov chains. Basics of Matlab programming. See Background material.
Syllabus
Introduction to sequential decision problems. Dynamic programming for deterministic models. Markov decision processes. Finite horizon problems. Infinite horizon problems. Risk averse control.
Teaching material
Further readings
-
[PDM] “Processi decisionali markoviani” In: Modelli e metodi decisionali in condizioni di incertezza e rischio. G. Ghiani, E. Manni (2009). Ch. 5, pp. 171–201. McGraw-Hill. (in Italian)
-
[DP] “Dynamic programming” In: Encyclopedia of Physical Science and Technology. M.L. Puterman (2003). pp. 673 – 696. Academic Press.
-
[MDP] “Markov Decision Processes: Discrete Stochastic Dynamic Programming” M. L. Puterman - Wiley-Interscience, 1994
-
[DPOC] “Dynamic Programming and Optimal Control Vol. I-II” D. P. Bertsekas, Athena Scientific, 2007
-
D. J. White, “A survey of applications of Markov decision processes”, The Journal of the Operational Research Society, vol. 44, no. 11, pp. 1073-1096. (full-text available from computers connected to the university network)
-
Some applications of Dynamic Programming to Computer Science problems
Background material
-
[DES] “Introduction to Discrete Event Systems” (Chapter 7, Discrete-time Markov Chains) C. G. Cassandras, S. Lafortune, Springer, 2008
-
Download or re-activate Matlab with UNISI license