Solving generalized semi-Markov decision processes using continuous phase-type distributions
Abstract
We introduce the generalized semi-Markov decision process (GSMDP) as
an extension of continuous-time MDPs and semi-Markov decision
processes (SMDPs) for modeling stochastic decision processes with
asynchronous events and actions. Using phase-type distributions and
uniformization, we show how an arbitrary GSMDP can be approximated by
a discrete-time MDP, which can then be solved using existing MDP
techniques. The techniques we present can also be seen as an
alternative approach for solving SMDPs, and we demonstrate that the
introduction of phases allows us to generate higher quality policies
than those obtained by standard SMDP solution techniques.
Sample citation
Håkan L. S. Younes and
Reid G. Simmons. 2004.
Solving generalized semi-Markov decision processes using continuous phase-type distributions. In
Proceedings of the Nineteenth National Conference on Artificial Intelligence, 742–747, San Jose, California. AAAI Press.
Full paper (6 pages, 15 references)
Copyright © 2004, American Association for Artificial Intelligence. All rights reserved.
Presentation (34 slides)