Last edited by Taujora
Friday, July 31, 2020 | History

2 edition of Exploiting structure to efficiently solve large scale partially observable Markov decision processes. found in the catalog.

Exploiting structure to efficiently solve large scale partially observable Markov decision processes.

Pascal Poupart

Exploiting structure to efficiently solve large scale partially observable Markov decision processes.

by Pascal Poupart

  • 217 Want to read
  • 28 Currently reading

Published .
Written in English


About the Edition

Partially observable Markov decision processes (POMDPs) provide a natural and principled framework to model a wide range of sequential decision making problems under uncertainty. To date, the use of POMDPs in real-world problems has been limited by the poor scalability of existing solution algorithms, which can only solve problems with up to ten thousand states. In fact, the complexity of finding an optimal policy for a finite-horizon discrete POMDP is PSPACE-complete. In practice, two important sources of intractability plague most solution algorithms: Large policy spaces and large state spaces.In practice, it is critical to simultaneously mitigate the impact of complex policy representations and large state spaces. Hence, this thesis describes three approaches that combine techniques capable of dealing with each source of intractability: VDC with BPI, VDC with Perseus (a randomized point-based value iteration algorithm by Spaan and Vlassis [136]), and state abstraction with Perseus. The scalability of those approaches is demonstrated on two problems with more than 33 million states: synthetic network management and a real-world system designed to assist elderly persons with cognitive deficiencies to carry out simple daily tasks such as hand-washing. This represents an important step towards the deployment of POMDP techniques in ever larger, real-world, sequential decision making problems.On the other hand, for many real-world POMDPs it is possible to define effective policies with simple rules of thumb. This suggests that we may be able to find small policies that are near optimal. This thesis first presents a Bounded Policy Iteration (BPI) algorithm to robustly find a good policy represented by a small finite state controller. Real-world POMDPs also tend to exhibit structural properties that can be exploited to mitigate the effect of large state spaces. To that effect, a value-directed compression (VDC) technique is also presented to reduce POMDP models to lower dimensional representations.

The Physical Object
Pagination144 leaves.
Number of Pages144
ID Numbers
Open LibraryOL19475491M
ISBN 100494027274


Share this book
You might also like
Non-teaching activities of commercial teachers in the state of Ohio

Non-teaching activities of commercial teachers in the state of Ohio

Seven stories about Jesus.

Seven stories about Jesus.

They passed this way

They passed this way

A systematic monograph of the flatfishes (Heterosomata)

A systematic monograph of the flatfishes (Heterosomata)

Car parking for shoppers.

Car parking for shoppers.

Introducing professional practice and knowledge.

Introducing professional practice and knowledge.

Curability of cataract with medicines

Curability of cataract with medicines

Teachers manual, Business accounting 1

Teachers manual, Business accounting 1

Federal income taxation

Federal income taxation

Analysis of the chipped stone artifacts of El Mirador, Guatemala

Analysis of the chipped stone artifacts of El Mirador, Guatemala

A research review of quality assessment for software

A research review of quality assessment for software

Maces

Maces

American socialism and the first world war.

American socialism and the first world war.

Exploiting structure to efficiently solve large scale partially observable Markov decision processes by Pascal Poupart Download PDF EPUB FB2