Broom balancing in the 21st century

Have a look at this link to see what optimal control can do when it is coupled to today's fast microprocessors and feedback controllers:

Quadrocopter pole acrobatics

Thanks to Felix Horns for finding this link.

Final homework

The final homework is Problems Sheet 4, Questions 4,5,7,8. (We have already done 6.) In all these questions the strategy is to use the necessary conditions of PMP to synthesize the optimal solution. What I mean by "synthesize" is that you write down all the information that PMP says is necessary (in terms of $u(t)$ maximizing, $H(x,u,\lambda)$ to $-\lambda_0(t)$, and $\dot \lambda=-\partial H/\partial x$, and any boundary and transversality conditions) until you convince yourself that the optimal policy can only be one thing. Question 5 is a bit like proving the shortest distance between two points is a straight line. The answer is fairly obvious, but we can prove it is correct by applying the necessary conditions of PMP to rule out the possibility of any other solution.

Please send the solutions to me by next Monday.

Session 9

The maximum principle that I talked about today is due to Lev Pontryagin. It is remarkable that despite being blind he was one of the greatest mathematicians of his generation. The key thing to grasp is that the PMP provides necessary conditions. We use the fact that an adjoint trajectory $\lambda$ exists to deduce properties of, or completely determine, the optimal control and optimally controlled trajectory. To my thinking, the PMP is notoriously badly explained in most books that it appears. I hope I have been able to make it seem more intuitive.  I think that Lagrange multipliers give a helpful interpretation, as does differentiation of the infinitestimal version of the optimality equation.

The rocket car example is a celebrated problem of optimal control theory that is nicely solved using the PMP. There is a nice interactive demo of it that you can try. I am pleased to have finally tracked down the fact that this problem was first solved by D.W. Bushaw, Differential Equations with a  Discontinuous Forcing Term, Ph.D. Thesis, Princeton, 1952. 

In the obituary of Donald W. Bushaw (1926-2012) it is stated that "Don’s Ph.D. thesis is recognized as the beginning of modern optimal control theory."

The name of A. A. Feldbaum (a Russian) is also mentioned in connection with this problem which he solved at about the same time. Pontryagin came up with his maximum principle a few years later, 1956.

The final example today ended with an example of turnpike theory.  This is an important concept in economics and is concerned with the optimal path of capital accumulation. Can you see intuitively why the optimal turnpike does not depend on the utility function $g$ of the agent? The label "turnpike" is due to Dorfman, Samuelson and Solow. I have mentioned it in the blog What makes for a beautiful problem is Science?

My personal experience of the power of PMP came in solving the problem about searching for a moving target that I have previously mentioned.

R. R. Weber. Optimal search for a randomly moving object. J. Appl. Prob. 23:708-717, 1986.

The problem that is solved in this paper in continuous time is still an open problem in discrete time. That shows the power of PMP, in that with it we can solve problems in discrete time that cannot be solved in discrete time