Optimistic planning with a limited number of action switches for near-optimal nonlinear control

Reference:
K. Máthé, L. Busoniu, R. Munos, and B. De Schutter, "Optimistic planning with a limited number of action switches for near-optimal nonlinear control," Proceedings of the 53rd IEEE Conference on Decision and Control, Los Angeles, California, pp. 3518-3523, Dec. 2014.

Abstract:
We consider infinite-horizon optimal control of nonlinear systems where the actions (inputs) are discrete. With the goal of limiting computations, we introduce a search algorithm for action sequences constrained to switch at most a given number of times between different actions. The new algorithm belongs to the optimistic planning class originating in artificial intelligence, and is called optimistic switch-limited planning (OSP). It inherits the generality of the OP class, so it works for nonlinear, nonsmooth systems with nonquadratic costs. We develop analysis showing that the switch constraint leads to polynomial complexity in the search horizon, in contrast to the exponential complexity of state-of-the-art OP; and to a correspondingly faster convergence. The degree of the polynomial varies with the problem and is a meaningful measure for the difficulty of solving it. We study this degree in two representative, opposite cases. In simulations we first apply OSP to a problem where limited-switch sequences are near-optimal, and then in a networked control setting where the switch constraint must be satisfied in closed loop.