Dynamic Programming and Optimal Control: Course information

This is home page for a course of 10 sessions that I will be presenting at London Business School in winter 2013.

This is a provisional information. The number of students is likely to be small and so I will try to tailor the material to your interests and needs once I know you better.

Aims

My aim is to equip you with research-useful knowledge concerning theory, modelling and solution tools in the field of dynamic programming and optimal control. The applications are to systems that evolve over time, such as can be found in physical and biological sciences, economics, operations management, computer science and engineering. One learns by being exposed to a lot of problems. So we shall cover many such: “research planning”, “insects as optimizers”, “planning for retirement”, “picking a parking space”, “optimal gambling”, “control of queues”, and “steering a space craft to the moon”.

I will presume that you have a mathematical background and that you enjoy solving mathematical problems. The most important pre-requisites are a knowledge of basic probability and linear algebra. A knowledge of Markov chains and Lagrangian optimization methods would be helpful.

We will have 10 sessions of length 2:45. A typical session will consist of a 55 minute lecture, about 20 minutes of discussion, questions or examples, a break of 15 minutes, a further 55 minute lecture, and 20 minutes of discussion and questions.

My draft sessions plan is provisional and will probably change once I see how many we will be, hear about your background and interests, and reconsider what is feasible. At present the plan shows me doing all the presenting, but I would probably like to replace some of this with presentations of some research papers by participants.

Course notes

The course will be an enhanced version of lectures that I give to final year mathematics students at Cambridge for which there already exists these notes. I will be adding additional material and preparing some new notes for our course. I expect that we will also be looking at some research papers. I will aim to make each session a self-contained unit on one or two particular topics. The note include hyperlinks between references to equations and to keywords in the index, but those may not be visible within your browser, only if you download the notes.

Course blog

I will be writing a course blog in which I will make a post after each of sessions: to emphasize an idea, give a sidebar, correction (!), or answer an interesting question (that perhaps a student sends to me in email). You may comment on these posts.

Comments, Discussion and Questions

I am organising these pages so that you can comment or ask questions on the discussion page (link in the sidebar at the right), and also at the end of each of the blog posts. I expect some good contributions. I will endeavour to answer questions. When you comment, you can do this anonymously if you wish. Just give any name and leave the email address box blank. You can also send me email with questions, suggestions or if you think you have discovered a mistake in the notes.

Assessment

There will be 4 assignments. Each of these will contain 4-5 problems, which you are to work. Some assignments will also include some reading, which you are to summarise in a brief report.
There will be a three-hour long examination, which will be taken in March. There will be 4 questions on the exam and full marks can be obtained by answering 3 questions.
The final mark will be based 20% on your work on the assignments and 80% on the examination.

Recommended books

These books are ones that I have read, and which I would recommend if you want to go deeper. However, I am intending that you buy or read them as part of participating in this course.

Bertsekas, D. P., Dynamic Programming and Optimal Control, Volumes I and II, Prentice Hall, 3rd edition 2005.
Hocking, L. M., Optimal Control: An introduction to the theory and applications, Oxford 1991.
Ross, S., Introduction to Stochastic Dynamic Programming. Academic Press, 1995.
Whittle, P., Optimization Over Time. Volumes I and II. Wiley, 1982-83.

Exam questions

You might like to practice on these Cambridge exam questions in the course Optimization and Control from 2001 to the present and 1995-2006. Some of these questions have amusing real-life applications.

Dynamic Programming and Optimal Control

Course information