Reference:
K. Máthé,
L. Busoniu,
R. Munos, and
B. De Schutter,
"Optimistic planning with an adaptive number of action switches for
near-optimal nonlinear control," Engineering Applications of
Artificial Intelligence, vol. 67, pp. 355-367, 2018.
Abstract:
We consider infinite-horizon optimal control of nonlinear systems
where the control actions are discrete, and focus on optimistic
planning algorithms from artificial intelligence, which can handle
general nonlinear systems with nonquadratic costs. With the main goal
of reducing computations, we introduce two such algorithms that only
search for constrained action sequences. The constraint prevents the
sequences from switching between different actions more than a limited
number of times. We call the first method optimistic switch-limited
planning (OSP), and develop analysis showing that its fixed number of
switches S leads to polynomial complexity in the search horizon, in
contrast to the exponential complexity of the existing OP algorithm
for deterministic systems; and to a correspondingly faster convergence
towards optimality. Since tuning S is difficult, we introduce an
adaptive variant called OASP that automatically adjusts S so as to
limit computations while ensuring that near-optimal solutions keep
being explored. OSP and OASP are analytically evaluated in
representative special cases, and numerically illustrated in
simulations of a rotational pendulum. To show that the algorithms also
work in challenging applications, OSP is used to control the pendulum
in real time, while OASP is applied for trajectory control of a
simulated quadrotor.