Courses:

Dynamic Programming and Stochastic Control >> Content Detail

Calendar

This section provides the course's lecture topics and corresponding reading assignments.


LEC #	TOPIC	TEXTBOOK READING ASSIGNMENT


1	Introduction to Dynamic Programming, Examples, Problem Formulation	Sections 1.1, 1.2


2	The Dynamic Programming Algorithm	Sections 1.3, 1.4


3	Deterministic Systems and the Shortest Path Problem	Sections 2.1, 2.2, 2.3


4	Shortest Path Algorithms	Section 4.1


5	Linear Quadratic Problems, Inventory Control	Sections 4.2, 4.4


6	Stopping and Scheduling Problems	Sections 4.5, 4.6


7	Deterministic Continuous-Time Optimal Control	Sections 3.1, 3.2


8	The Pontryagin Minimum Principle I	Section 3.3


9	The Pontryagin Minimum Principle II	Section 3.4


10	Imperfect State Information Problems	Section 5.1


11	Linear Quadratic Problems with Imperfect State Information	Section 5.2


12	Imperfect State Information Problems for Finite-State Systems	Sections 5.4, 5.5


13	Suboptimal Control I	Sections 6.1, 6.2


14	Suboptimal Control II	Section 6.3


15	Rollout Algorithms	Section 6.4


16	Additional Topics in Suboptimal Control	Section 6.5


17	Infinite Horizon Problems - Stochastic Shortest Path Problems	Sections 7.1, 7.2


18	Infinite Horizon Problems	Sections 7.2, 7.3


19	Stochastic Shortest Path and Discounted Problems	Section 7.4


20	Average Cost Problems	Section 7.5


21	Semi-Markov Problems	Vol. 2, Section 1.5


22	Neuro-Dynamic Programming I	Vol. 2, Section 2.3


23	Neuro-Dynamic Programming II	Vol. 2, Section 2.3


24	Neuro-Dynamic Programming III