I have added to the notes, on page 77, the argument by which we can verify that in the fishing problem the value function $F(x)$ is concave.
The homework for next week is just three questions: the first three on Problems Sheet 4. See hints below.
The name "Kalman filter" refers to the estimation equation (16.1) and takes its name from Rudolf Kalman (1930 –), who developed it in the years 1958-64. He also coined the terms controllable and observable, and gave the criteria that we have seen in previous lectures. The fact that a system is controllable iff the matrix $[B\ AB\ \cdots\ A^{n-1}B]$ is of full rank is sometimes called Kalman's criteria. In the IEEE biography of Kalman it is stated
You will understand the ideas better once you have worked through the details of a scalar example (in which $n=m=p=1$). You do this in Problems Sheet 4 Question 1. When you do this question, start by supposing that $\hat x_t=\hat x_{t-1}+u_{t-1}+h_t(y_t-\hat x_{t-1})$, and then find the value of $h_t$ that minimizes the variance of $\hat x_t$. You can start by subtracting $x_t=x_{t-1}+u_{t-1}+3\epsilon_t$ and using $y_t=x_{t-1}+2\eta_t$. I also think that Question 2 is helpful in gaining an appreciation of the duality between control and estimation problems. One of the things you will learn from this question is a sufficient condition for the variance of $\hat x_t$ (that is, $V_t$) to tend to a finite limit. Question 3 is about solving a HJB equation (easier than the fishing example).
Notice that the Riccati equation for $V_t$, i.e. $V_t = g\, V_{t-1}$ runs in the opposite time direction to the one we had for $\Pi_t$ in Chapter 13, where $\Pi_{t-1} = f\, \Pi_t$. We are given $V_0$ and $\Pi_h$.
The homework for next week is just three questions: the first three on Problems Sheet 4. See hints below.
The name "Kalman filter" refers to the estimation equation (16.1) and takes its name from Rudolf Kalman (1930 –), who developed it in the years 1958-64. He also coined the terms controllable and observable, and gave the criteria that we have seen in previous lectures. The fact that a system is controllable iff the matrix $[B\ AB\ \cdots\ A^{n-1}B]$ is of full rank is sometimes called Kalman's criteria. In the IEEE biography of Kalman it is stated
The Kalman filter, and its later extensions to nonlinear problems, represents perhaps the most widely applied by-product of modern control theory. It has been used in space vehicle navigation and control (e.g. the Apollo vehicle), radar tracking algorithms for ABM applications, process control, and socioeconomic systems.The theory in this lecture is admittedly quite tricky - partly because the notation. As a test of memory, can you say what roles in the theory are taken by each of these?
$x_t$, $\hat x_t$, $y_t$, $u_t$, $\epsilon_t$, $\eta_t$, $\Delta_t$, $\xi_t$, $\zeta_t$, $A$, $B$, $C$, $R$, $S$, $Q$, $K_t$, $\Pi_t$, $N$, $L$, $M$, $H_t$, $V_t$.
Notice that the Riccati equation for $V_t$, i.e. $V_t = g\, V_{t-1}$ runs in the opposite time direction to the one we had for $\Pi_t$ in Chapter 13, where $\Pi_{t-1} = f\, \Pi_t$. We are given $V_0$ and $\Pi_h$.
