NDLI: XCSF with tile coding in discontinuous action-value landscapes

Please wait, while we are loading the content...

XCSF with tile coding in discontinuous action-value landscapes

Content Provider	Springer Nature Link
Author	Lanzi, Pier Luca Loiaco, Daniele
Copyright Year	2015
Abstract	Tile coding is an effective reinforcement learning method that uses a rather ingenious generalization mechanism based on (1) a carefully designed parameter setting and (2) the assumption that nearby states in the problem space will correspond to similar payoff predictions in the action-value function. Previously, we extended XCSF with tile coding prediction and compared it to tabular tile coding, showing that (1) XCSF performs as well as parameter-optimized tile coding, while also (2) evolving individualized parameter settings in each problem subspace. Our comparison was based on a set of well-known reinforcement learning environments (2D Gridworld and the Mountain Car) that involved no action-value discontinuities and so posed no challenge to tabular tile coding. In this paper, we go a step further and test XCSF with tile coding on a set of problems designed to challenge tile coding by introducing discontinuities in the action value landscape. The new testbed (called MazeWorld) extends 2D Gridworld with impenetrable obstacles, a conceptually simple modification that can dramatically increase the problem complexity for tabular tile coding. We compare four versions of XCSF with tile coding (each adapting a different set of parameters) to tabular tile coding on four problems of increasing complexity. We show that our system (1) needs fewer training problems than standard tile coding to reach an optimal policy; (2) it can evolve adequate coding parameters in each subspace without any previous knowledge; and that (3) even when XCSF is not allowed to evolve these parameters, the genetic algorithm will still adapt classifier conditions to properly decompose the problem into subspaces thus being much less sensitive to the parameter settings than tabular tile coding.
Starting Page	117
Ending Page	132
Page Count	16
File Format	PDF
ISSN	18645909
Journal	Evolutionary Intelligence
Volume Number	8
Issue Number	2-3
e-ISSN	18645917
Language	English
Publisher	Springer Berlin Heidelberg
Publisher Date	2015-04-11
Publisher Place	Berlin, Heidelberg
Access Restriction	One Nation One Subscription (ONOS)
Subject Keyword	Learning classifier systems Tile coding Reinforcement learning XCSF ApplicationMathematics/Computational Methods of Engineering Artificial Intelligence (incl. Robotics) Statistical Physics, Dynamical Systems and Complexity Control, Robotics, Mechatronics Bioinformatics Applications of Mathematics
Content Type	Text
Resource Type	Article
Subject	Artificial Intelligence Cognitive Neuroscience Mathematics Computer Vision and Pattern Recognition

Sl.	Authority	Responsibilities	Communication Details
1	Ministry of Education (GoI), Department of Higher Education	Sanctioning Authority	https://www.education.gov.in/ict-initiatives
2	Indian Institute of Technology Kharagpur	Host Institute of the Project: The host institute of the project is responsible for providing infrastructure support and hosting the project	https://www.iitkgp.ac.in
3	National Digital Library of India Office, Indian Institute of Technology Kharagpur	The administrative and infrastructural headquarters of the project	Dr. B. Sutradhar bsutra@ndl.gov.in
4	Project PI / Joint PI	Principal Investigator and Joint Principal Investigators of the project	Dr. B. Sutradhar bsutra@ndl.gov.in Prof. Saswat Chakrabarti will be added soon
5	Website/Portal (Helpdesk)	Queries regarding NDLI and its services	support@ndl.gov.in
6	Contents and Copyright Issues	Queries related to content curation and copyright issues	content@ndl.gov.in
7	National Digital Library of India Club (NDLI Club)	Queries related to NDLI Club formation, support, user awareness program, seminar/symposium, collaboration, social media, promotion, and outreach	clubsupport@ndl.gov.in
8	Digital Preservation Centre (DPC)	Assistance with digitizing and archiving copyright-free printed books	dpc@ndl.gov.in
9	IDR Setup or Support	Queries related to establishment and support of Institutional Digital Repository (IDR) and IDR workshops	idr@ndl.gov.in

Risk neutrality in learning classifier systems

Sequential problems that test generalization in learning classifier systems

Function approximation with LWPR and XCSF: a comparative study

Special issue on advances in learning classifier systems

Special issue on advances in Learning Classifier Systems

Adaptive artificial datasets through learning classifier systems for classification tasks

XCSF with local deletion: preventing detrimental forgetting

Learning classifier systems to evolve classification rules for systems of memory constrained components

A brief history of learning classifier systems: from CS-1 to XCS and its variants

XCSF with tile coding in discontinuous action-value landscapes

Similar Documents

Risk neutrality in learning classifier systems

Sequential problems that test generalization in learning classifier systems

Function approximation with LWPR and XCSF: a comparative study

Special issue on advances in learning classifier systems

Special issue on advances in Learning Classifier Systems

Adaptive artificial datasets through learning classifier systems for classification tasks

XCSF with local deletion: preventing detrimental forgetting

Learning classifier systems to evolve classification rules for systems of memory constrained components

A brief history of learning classifier systems: from CS-1 to XCS and its variants

XCSF with tile coding in discontinuous action-value landscapes