**Reference:**

L. Busoniu,
R. Munos,
B. De Schutter, and
R. Babuska,
"Optimistic planning for sparsely stochastic systems," *Proceedings
of the 2011 IEEE Symposium on Adaptive Dynamic Programming and
Reinforcement Learning (ADPRL 2011)*, Paris, France, pp. 48-55,
Apr. 2011.

**Abstract:**

We propose an online planning algorithm for finite-action, sparsely
stochastic Markov decision processes, in which the random state
transitions can only end up in a small number of possible next states.
The algorithm builds a planning tree by iteratively expanding states,
where each expansion exploits sparsity to add all possible successor
states. Each state to expand is actively chosen to improve the
knowledge about action quality, and this allows the algorithm to
return a good action after a strictly limited number of expansions.
More specifically, the active selection method is *optimistic*
in that it chooses the most promising states first, so the novel
algorithm is called *optimistic planning for sparsely stochastic
systems*. We note that the new algorithm can also be seen as
model-predictive (receding-horizon) control. The algorithm obtains
promising numerical results, including the successful online control
of a simulated HIV infection with stochastic drug effectiveness.

Corresponding technical report: pdf file (377 KB)

@inproceedings{BusMun:11-007,

author={L. Bu{\c{s}}oniu and R. Munos and B. {D}e Schutter and R. Babu{\v{s}}ka},

title={Optimistic planning for sparsely stochastic systems},

booktitle={Proceedings of the 2011 IEEE Symposium on Adaptive Dynamic Programming and Reinforcement Learning (ADPRL 2011)},

address={Paris, France},

pages={48--55},

month=apr,

year={2011}

}

Go to the publications overview page.

This page is maintained by Bart De Schutter. Last update: March 21, 2022.