NDLI: A State-Cluster Based Q-Learning

Content Provider	IEEE Xplore Digital Library
Author	Zhao Jin WeiYi Liu Jian Jin
Copyright Year	2009
Abstract	When apply Q-learning to complex real-world problems, the learning process is long enough to make this method unpractical. The major cause is Q-learning requires the agent to visit every state-action transition infinitely often for making Q value convergent. We propose a State-Cluster based Q-learning method to accelerate convergence and shorten learning process. This method creates the State-Cluster for each state the agent reached according to the state trajectory that the agent wandered. By our algorithm, the State-Cluster of a state would hold these acyclic shortest state paths from other states to this state. When a state's Q value is refined in one step of the agent, the refined Q value can be propagated immediately back to all these states in its State-Cluster along the state paths between them, instead of requiring the agent to visit these states again. With the State-Cluster, more Q value can be refined in one step of the agent, which speeds up the convergence of Q value. The experiments compared with Q-learning demonstrate this method is extraordinarily more effective. This method is aimed Q-learning, but it is also applicable for most other reinforcement learning methods based value function iteration.
Starting Page	44
Ending Page	49
File Size	1041992
Page Count	6
File Format	PDF
ISBN	9780769537368
DOI	10.1109/ICNC.2009.405
Language	English
Publisher	Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Publisher Date	2009-08-14
Publisher Place	China
Access Restriction	Subscribed
Rights Holder	Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Subject Keyword	Learning Information science Costs Accelerated aging Autonomous agents State-space methods Space exploration Acceleration Intelligent agent Convergence
Content Type	Text
Resource Type	Article

Sl.	Authority	Responsibilities	Communication Details
1	Ministry of Education (GoI), Department of Higher Education	Sanctioning Authority	https://www.education.gov.in/ict-initiatives
2	Indian Institute of Technology Kharagpur	Host Institute of the Project: The host institute of the project is responsible for providing infrastructure support and hosting the project	https://www.iitkgp.ac.in
3	National Digital Library of India Office, Indian Institute of Technology Kharagpur	The administrative and infrastructural headquarters of the project	Dr. B. Sutradhar bsutra@ndl.gov.in
4	Project PI / Joint PI	Principal Investigator and Joint Principal Investigators of the project	Dr. B. Sutradhar bsutra@ndl.gov.in Prof. Saswat Chakrabarti will be added soon
5	Website/Portal (Helpdesk)	Queries regarding NDLI and its services	support@ndl.gov.in
6	Contents and Copyright Issues	Queries related to content curation and copyright issues	content@ndl.gov.in
7	National Digital Library of India Club (NDLI Club)	Queries related to NDLI Club formation, support, user awareness program, seminar/symposium, collaboration, social media, promotion, and outreach	clubsupport@ndl.gov.in
8	Digital Preservation Centre (DPC)	Assistance with digitizing and archiving copyright-free printed books	dpc@ndl.gov.in
9	IDR Setup or Support	Queries related to establishment and support of Institutional Digital Repository (IDR) and IDR workshops	idr@ndl.gov.in

Noise Tolerance in Reinforcement Learning Algorithms

Learning task decomposition and exploration shaping for reinforcement learning agents

Reinforcement learning accelerated by using state transition model with robotic applications

A reinforcement learning accelerated by state space reduction

State-Clusters shared cooperative multi-agent reinforcement learning

Accelerated Q-learning for fail state and action spaces

Task-oriented reinforcement learning for continuous tasks in dynamic environment

Active Exploration Planning in Reinforcement Learning for Inverted Pendulum System Control

Multi-Agent Cooperation by Q-Learning in Continuous Action Domain

A State-Cluster Based Q-Learning

Similar Documents

Noise Tolerance in Reinforcement Learning Algorithms

Learning task decomposition and exploration shaping for reinforcement learning agents

Reinforcement learning accelerated by using state transition model with robotic applications

A reinforcement learning accelerated by state space reduction

State-Clusters shared cooperative multi-agent reinforcement learning

Accelerated Q-learning for fail state and action spaces

Task-oriented reinforcement learning for continuous tasks in dynamic environment

Active Exploration Planning in Reinforcement Learning for Inverted Pendulum System Control

Multi-Agent Cooperation by Q-Learning in Continuous Action Domain

A State-Cluster Based Q-Learning