Record:   Prev Next
Author Mausam
Title Planning with Markov decision processes [electronic resource] : an AI perspective / Mausam and Andrey Kolobov
Imprint San Rafael, Calif. (1537 Fourth Street, San Rafael, CA 94901 USA) : Morgan & Claypool, c2012
book jacket
Descript 1 electronic text (xvi, 194 p.) : ill., digital file
Series Synthesis lectures on artificial intelligence and machine learning, 1939-4616 ; # 17
Synthesis digital library of engineering and computer science
Synthesis lectures on artificial intelligence and machine learning ; # 17. 1939-4616
Note Part of: Synthesis digital library of engineering and computer science
Series from website
Includes bibliographical references (p. 163-185) and index
Preface -- 1. Introduction -- 1.1 Characteristics of an MDP -- 1.2 Connections with different fields -- 1.3 Overview of this book --
2. MDPs -- 2.1 Markov decision processes: definition -- 2.2 Solutions of an MDP -- 2.3 Solution existence -- 2.4 Stochastic shortest-path MDPs -- 2.5 Factored MDPs -- 2.6 Complexity of solving MDPs --
3. Fundamental algorithms -- 3.1 A brute-force algorithm -- 3.2 Policy evaluation -- 3.3 Policy iteration -- 3.4 Value iteration -- 3.5 Prioritization in value iteration -- 3.6 Partitioned value iteration -- 3.7 Linear programming formulation -- 3.8 Infinite-horizon discounted-reward MDPs -- 3.9 Finite-horizon MDPs -- 3.10 MDPs with dead ends --
4. Heuristic search algorithms -- 4.1 Heuristic search and SSP MDPs -- 4.2 FIND-and-REVISE: a schema for heuristic search -- 4.3 LAO and extensions -- 4.4 RTDP and extensions -- 4.5 Heuristics and transition graph pruning -- 4.6 Computing admissible heuristics -- 4.7 Heuristic search and dead ends --
5. Symbolic algorithms -- 5.1 Algebraic decision diagrams -- 5.2 SPUDD: value iteration using ADDs -- 5.3 Symbolic LAO* -- 5.4 Other symbolic algorithms -- 5.5 Other symbolic representations -- 5.6 Approximations using symbolic approaches --
6. Approximation algorithms -- 6.1 Determinization-based techniques -- 6.2 Sampling-based techniques -- 6.3 Heuristic search with inadmissible heuristics -- 6.4 Dimensionality reduction-based techniques -- 6.5 Hierarchical planning -- 6.6 Hybridized planning -- 6.7 A comparison of different algorithms --
7. Advanced notes -- 7.1 MDPs with continuous or hybrid states -- 7.2 MDP with concurrency and durative actions -- 7.3 Relational MDPs -- 7.4 Generalized stochastic shortest path MDPs -- 7.5 Other models -- 7.6 Issues in probabilistic planning -- 7.7 Summary -- Bibliography
Abstract freely available; full-text restricted to subscribers or individual document purchasers
Google scholar
Google book search
Mode of access: World Wide Web
System requirements: Adobe Acrobat Reader
Markov Decision Processes (MDPs) are widely popular in Artificial Intelligence for modeling sequential decision-making scenarios with probabilistic dynamics. They are the framework of choice when designing an intelligent agent that needs to act for long periods of time in an environment where its actions could have uncertain outcomes.MDPs are actively researched in two related subareas of AI, probabilistic planning and reinforcement learning. Probabilistic planning assumes known models for the agent's goals and domain dynamics, and focuses on determining how the agent should behave to achieve its objectives. On the other hand, reinforcement learning additionally learns these models based on the feedback the agent gets from the environment. This book provides a concise introduction to the use of MDPs for solving probabilistic planning problems, with an emphasis on the algorithmic perspective. It covers the whole spectrum of the field, from the basics to state-of-the-art optimal and approximation algorithms.We first describe the theoretical foundations of MDPs and the fundamental solution techniques for them.We then discuss modern optimal algorithms based on heuristic search and the use of structured representations. A major focus of the book is on the numerous approximation schemes for MDPs that have been developed in the AI literature. These include determinization-based approaches, sampling techniques, heuristic functions, dimensionality reduction, and hierarchical representations. Finally, we briefly introduce several extensions of the standard MDP classes that model and solve even more complex planning problems
Also available in print
Title from PDF t.p. (viewed on July 14, 2012)
Link Print version: 9781608458868
Subject Artificial intelligence -- Mathematical models
Markov processes
AI planning
probabilistic planning
uncertainty in AI
sequential decision making under uncertainty
reinforcement learning
Alt Author Kolobov, Andrey
Record:   Prev Next