Policy Generation for Continuous-time Stochastic Domains with Concurrency

Håkan L. S. Younes Reid G. Simmons

Abstract:

We adopt the framework of Younes, Musliner, & Simmons for planning with concurrency in continuous-time stochastic domains. Our contribution is a set of concrete techniques for policy generation, failure analysis, and repair. These techniques have been implemented in Tempastic, a novel temporal probabilistic planner, and we demonstrate the performance of the planner on two variations of a transportation domain with concurrent actions and exogenous events. Tempastic makes use of a deterministic temporal planner to generate initial policies. Policies are represented using decision trees, and we use incremental decision tree induction to efficiently incorporate changes suggested by the failure analysis.

Sample citation:

Håkan L. S. Younes and Reid G. Simmons. 2004. Policy generation for continuous-time stochastic domains with concurrency. In Proceedings of the Fourteenth International Conference on Automated Planning and Scheduling, edited by Shlomo Zilberstein, Jana Koehler, and Sven Koenig, 325–333, Whistler, Canada. AAAI Press.

Full paper:

pdf (9 pages, 27 references)
Copyright © 2004, American Association for Artificial Intelligence. All rights reserved.

Presentation:

ppt, pdf (25 slides)

Citings:

[ Google Scholar ]
  1. Mausam and Daniel S. Weld. 2008. Planning with durative actions in stochastic domains. Journal of Artificial Intelligence Research 31:33–82.
  2. Mausam, Piergiorgio Bertoli, and Daniel S. Weld. 2007. A hybridized planner for stochastic domains. In Proceedings of the Twentieth International Joint Conference on Artificial Intelligence, 1972–1978.
  3. Sylvie Thiébaux, Charles Gretton, John Slaney, David Price, and Froduald Kabanza. 2006. Decision-theoretic planning with non-Markovian rewards. Journal of Artificial Intelligence Research 25:17–74.
  4. Douglas Aberdeen. 2006. Policy-gradient methods for planning. In Advances in Neural Information Processing Systems 18, 9–16. MIT Press.
  5. Mausam and Daniel S. Weld. 2006. Challenges for temporal planning with uncertain durations. In Proceedings of the Sixteenth International Conference on Planning and Scheduling, 414–417. AAAI Press.
  6. Iain Little, Douglas Aberdeen, and Sylvie Thiébaux. 2005. Prottle: A probabilistic temporal planner. In Proceedings of the Twentieth National Conference on Artificial Intelligence, 1181–1186. AAAI Press.
  7. Mausam and Daniel S. Weld. 2005. Concurrent probabilistic temporal planning. In Proceedings of the Fifteenth International Conference on Automated Planning and Scheduling, 120–129. AAAI Press.
  8. Mausam and Daniel S. Weld. 2004. Solving concurrent Markov decision processes. In Proceedings of the Nineteenth National Conference on Artificial Intelligence, 716–722. AAAI Press.