Description
Representation of noise in discrete-time dynamical systems. Prediction and the Markov property. Gaussian processes. Evolution of the main statistics of noisy linear systems: mean, covariance, and their asymptotic behavior. Optimal control and Bellman's principle of optimality. Dynamic programming: deterministic and stochastic formulations. Application to perfectly observed noisy linear systems: linear quadratic optimal regulator. Measurement noise and partially observed systems: minimum-variance and maximum-likelihood estimators; Kalman filter. Optimal control of partially observed stochastic systems. Optimality of the separation principle between filtering and control in linear Gaussian systems. Controlled Markov chains. Optimal stopping problems. Optimal routing in telecommunications.