LEADER 00000nam  2200637 a 4500 
001    201206AIM017 
005    20120714151701.0 
006    m    e   d         
007    cr cn |||m|||a 
008    120714s2012    caua   foab   001 0 eng d 
020    9781608458875 (electronic bk.) 
020    |z9781608458868 (pbk.) 
024 7  10.2200/S00426ED1V01Y201206AIM017|2doi 
035    (OCoLC)799364832 
035    (CaBNVSL)swl00401089 
040    CaBNVSL|cCaBNVSL|dCaBNVSL 
050  4 Q335|bM286 2012 
082 04 006.3|223 
100 0  Mausam 
245 10 Planning with Markov decision processes|h[electronic 
       resource] :|ban AI perspective /|cMausam and Andrey 
       Kolobov 
260    San Rafael, Calif. (1537 Fourth Street, San Rafael, CA  
       94901 USA) :|bMorgan & Claypool,|cc2012 
300    1 electronic text (xvi, 194 p.) :|bill., digital file 
490 1  Synthesis lectures on artificial intelligence and machine 
       learning,|x1939-4616 ;|v# 17 
500    Part of: Synthesis digital library of engineering and 
       computer science 
500    Series from website 
504    Includes bibliographical references (p. 163-185) and index
505 0  Preface -- 1. Introduction -- 1.1 Characteristics of an 
       MDP -- 1.2 Connections with different fields -- 1.3 
       Overview of this book -- 
505 8  2. MDPs -- 2.1 Markov decision processes: definition -- 
       2.2 Solutions of an MDP -- 2.3 Solution existence -- 2.4 
       Stochastic shortest-path MDPs -- 2.5 Factored MDPs -- 2.6 
       Complexity of solving MDPs -- 
505 8  3. Fundamental algorithms -- 3.1 A brute-force algorithm -
       - 3.2 Policy evaluation -- 3.3 Policy iteration -- 3.4 
       Value iteration -- 3.5 Prioritization in value iteration -
       - 3.6 Partitioned value iteration -- 3.7 Linear 
       programming formulation -- 3.8 Infinite-horizon discounted
       -reward MDPs -- 3.9 Finite-horizon MDPs -- 3.10 MDPs with 
       dead ends -- 
505 8  4. Heuristic search algorithms -- 4.1 Heuristic search and
       SSP MDPs -- 4.2 FIND-and-REVISE: a schema for heuristic 
       search -- 4.3 LAO and extensions -- 4.4 RTDP and 
       extensions -- 4.5 Heuristics and transition graph pruning 
       -- 4.6 Computing admissible heuristics -- 4.7 Heuristic 
       search and dead ends -- 
505 8  5. Symbolic algorithms -- 5.1 Algebraic decision diagrams 
       -- 5.2 SPUDD: value iteration using ADDs -- 5.3 Symbolic 
       LAO* -- 5.4 Other symbolic algorithms -- 5.5 Other 
       symbolic representations -- 5.6 Approximations using 
       symbolic approaches -- 
505 8  6. Approximation algorithms -- 6.1 Determinization-based 
       techniques -- 6.2 Sampling-based techniques -- 6.3 
       Heuristic search with inadmissible heuristics -- 6.4 
       Dimensionality reduction-based techniques -- 6.5 
       Hierarchical planning -- 6.6 Hybridized planning -- 6.7 A 
       comparison of different algorithms -- 
505 8  7. Advanced notes -- 7.1 MDPs with continuous or hybrid 
       states -- 7.2 MDP with concurrency and durative actions --
       7.3 Relational MDPs -- 7.4 Generalized stochastic shortest
       path MDPs -- 7.5 Other models -- 7.6 Issues in 
       probabilistic planning -- 7.7 Summary -- Bibliography 
506    Abstract freely available; full-text restricted to 
       subscribers or individual document purchasers 
510 0  Compendex 
510 0  INSPEC 
510 0  Google scholar 
510 0  Google book search 
520 3  Markov Decision Processes (MDPs) are widely popular in 
       Artificial Intelligence for modeling sequential decision-
       making scenarios with probabilistic dynamics. They are the
       framework of choice when designing an intelligent agent 
       that needs to act for long periods of time in an 
       environment where its actions could have uncertain 
       outcomes.MDPs are actively researched in two related 
       subareas of AI, probabilistic planning and reinforcement 
       learning. Probabilistic planning assumes known models for 
       the agent's goals and domain dynamics, and focuses on 
       determining how the agent should behave to achieve its 
       objectives. On the other hand, reinforcement learning 
       additionally learns these models based on the feedback the
       agent gets from the environment. This book provides a 
       concise introduction to the use of MDPs for solving 
       probabilistic planning problems, with an emphasis on the 
       algorithmic perspective. It covers the whole spectrum of 
       the field, from the basics to state-of-the-art optimal and
       approximation algorithms.We first describe the theoretical
       foundations of MDPs and the fundamental solution 
       techniques for them.We then discuss modern optimal 
       algorithms based on heuristic search and the use of 
       structured representations. A major focus of the book is 
       on the numerous approximation schemes for MDPs that have 
       been developed in the AI literature. These include 
       determinization-based approaches, sampling techniques, 
       heuristic functions, dimensionality reduction, and 
       hierarchical representations. Finally, we briefly 
       introduce several extensions of the standard MDP classes 
       that model and solve even more complex planning problems 
530    Also available in print 
538    Mode of access: World Wide Web 
538    System requirements: Adobe Acrobat Reader 
588    Title from PDF t.p. (viewed on July 14, 2012) 
650  0 Artificial intelligence|xMathematical models 
650  0 Markov processes 
653    MDP 
653    AI planning 
653    probabilistic planning 
653    uncertainty in AI 
653    sequential decision making under uncertainty 
653    reinforcement learning 
700 1  Kolobov, Andrey 
776 08 |iPrint version:|z9781608458868 
830  0 Synthesis digital library of engineering and computer 
       science 
830  0 Synthesis lectures on artificial intelligence and machine 
       learning ;|v# 17.|x1939-4616 
856 48 |zeBook(Morgan-IISLIB)|uhttp://dx.doi.org/10.2200/
       S00426ED1V01Y201206AIM017