The Hamiltonian of optimal control theory was developed by Lev Pontryagin as part of his maximum principle. It was inspired by, but is distinct from, the Hamiltonian of classical mechanics. Pontryagin proved that a necessary condition for solving the optimal control problem is that the control should be chosen so as to minimize the Hamiltonian. For details see Pontryagin's maximum principle.
Contents
Notation and Problem statement
A control
where
and the control must satisfy the constraints
Definition of the Hamiltonian
where
For information on the properties of the Hamiltonian, see Pontryagin's maximum principle.
The Hamiltonian in discrete time
When the problem is formulated in discrete time, the Hamiltonian is defined as:
and the costate equations are
(Note that the discrete time Hamiltonian at time
The Hamiltonian of control compared to the Hamiltonian of mechanics
William Rowan Hamilton defined the Hamiltonian as a function of three variables:
where
Hamilton then formulated his equations as
Similarly the Hamiltonian of control theory (as normally defined) is a function of 4 variables
and the associated conditions for a maximum are
This definition agrees with that given by the article by Sussmann and Willems. (see p. 39, equation 14). Sussmann-Willems show how the control Hamiltonian can be used in dynamics e.g. for the brachystochrone problem, but do not mention the prior work of Carathéodory on this approach.