NDLI: Improving the Convergence of RTDP

Content Provider	IEEE Xplore Digital Library
Author	Tian Liang Jigui Sun Minghao Yin
Copyright Year	2007
Description	Author affiliation: Jilin Univ., Changchun (Tian Liang; Jigui Sun; Minghao Yin)
Abstract	Real-time dynamic programming (RTDP) is an outstanding real-time algorithm for solving non-deterministic planning problems with full observability. RTDP has two key advantages comparing with other DP algorithms: first, it obtain an optimal policy without computing the whole space, second, it has a good anytime behavior. However, RTDP's convergence is slow. In this paper, we introduce RTDP(k), an algorithm based on RTDP with a similar structure. RTDP(k) improves the convergence as well as holds real-time algorithm properties. RTDP(k) updates k extended states per iteration following a "bounded propagation strategy". In Markov decision processes (MDPs), especially in stochastic shortest-path problems (SSPs), we have proved two points: first, every RTDP(k) trial terminates in a finite number of steps ,second, RTDP(k) eventually converges to an optimal policy. From a practical point of view, We show that RTDP(k) produces better solutions in the first trial and converges faster than RTDP on benchmarks for real-time search.
Starting Page	613
Ending Page	617
File Size	226255
Page Count	5
File Format	PDF
ISBN	9780769528748
DOI	10.1109/FSKD.2007.360
Language	English
Publisher	Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Publisher Date	2007-08-24
Publisher Place	China
Access Restriction	Subscribed
Rights Holder	Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Subject Keyword	Stochastic processes Cost function Educational institutions Probability distribution Dynamic programming State-space methods Labeling Observability Sun Convergence
Content Type	Text
Resource Type	Article

Sl.	Authority	Responsibilities	Communication Details
1	Ministry of Education (GoI), Department of Higher Education	Sanctioning Authority	https://www.education.gov.in/ict-initiatives
2	Indian Institute of Technology Kharagpur	Host Institute of the Project: The host institute of the project is responsible for providing infrastructure support and hosting the project	https://www.iitkgp.ac.in
3	National Digital Library of India Office, Indian Institute of Technology Kharagpur	The administrative and infrastructural headquarters of the project	Dr. B. Sutradhar bsutra@ndl.gov.in
4	Project PI / Joint PI	Principal Investigator and Joint Principal Investigators of the project	Dr. B. Sutradhar bsutra@ndl.gov.in Prof. Saswat Chakrabarti will be added soon
5	Website/Portal (Helpdesk)	Queries regarding NDLI and its services	support@ndl.gov.in
6	Contents and Copyright Issues	Queries related to content curation and copyright issues	content@ndl.gov.in
7	National Digital Library of India Club (NDLI Club)	Queries related to NDLI Club formation, support, user awareness program, seminar/symposium, collaboration, social media, promotion, and outreach	clubsupport@ndl.gov.in
8	Digital Preservation Centre (DPC)	Assistance with digitizing and archiving copyright-free printed books	dpc@ndl.gov.in
9	IDR Setup or Support	Queries related to establishment and support of Institutional Digital Repository (IDR) and IDR workshops	idr@ndl.gov.in

Labeled RTDP: Improving the convergence of real-time dynamic programming

Labeled RTDP: Improving the convergence of real-time dynamic programming (2003)

Convergence of the Q-ae learning under deterministic MDPs and its efficiency under the stochastic environment

Markov decision processes with constrained stopping times

A Survey of Approximate Dynamic Programming

Certainty equivalence for imperfect information finite state-space stochastic games

Optimal Control of Stochastic Hybrid Systems Based on Locally Consistent Markov Decision Processes

Labeled RTDP: Improving the Convergence of Real-Time Dynamic Programming (2003)

Markov decision processes with risk-sensitive criteria: dynamic programming operators and discounted stochastic games

Improving the Convergence of RTDP

Similar Documents

Labeled RTDP: Improving the convergence of real-time dynamic programming

Labeled RTDP: Improving the convergence of real-time dynamic programming (2003)

Convergence of the Q-ae learning under deterministic MDPs and its efficiency under the stochastic environment

Markov decision processes with constrained stopping times

A Survey of Approximate Dynamic Programming

Certainty equivalence for imperfect information finite state-space stochastic games

Optimal Control of Stochastic Hybrid Systems Based on Locally Consistent Markov Decision Processes

Labeled RTDP: Improving the Convergence of Real-Time Dynamic Programming (2003)

Markov decision processes with risk-sensitive criteria: dynamic programming operators and discounted stochastic games

Improving the Convergence of RTDP