NDLI: Modeling reward functions for incomplete state representations via echo state networks

Content Provider	IEEE Xplore Digital Library
Author	Bush ,. K. Anderson, C.
Copyright Year	2005
Description	Author affiliation: Dept. of Comput. Sci., Colorado State Univ., Fort Collins, CO, USA (Bush , K.; Anderson, C.)
Abstract	This paper investigates an echo state network (ESN) (Jaeger, 2001 and Maass and Markram, 2002) architecture as the approximation of the Q-function for temporally dependent rewards embedded in a linear dynamical system, the mass-spring-damper (MSD). This problem has been solved utilizing feed-forward neural networks (FNN) when all state information necessary to specify the dynamics is provided as input (Kretchmar, 2000). Time-delayed neural networks (TDNN) solve this problem with finite-size windows of incomplete state information. Our research demonstrates that the ESN architecture represents the Q-function of the MSD system given incomplete state information as well as current feed forward neural networks given either perfect state or a temporally-windowed, incomplete state vector. The remainder of this paper is organized as follows. We introduce basic concepts of reinforcement learning and the echo state network architecture. The MSD system simulation is defined in section IV. Experimental results for learning state quality given incomplete state information are presented in section V. Results for learning estimates of all future state qualities for incomplete state information is presented in section VI. Section VII discusses the potential of the ESN for use in reinforcement learning and provides current and future directions of research.
Starting Page	2995
Ending Page	3000
File Size	373720
Page Count	6
File Format	PDF
ISBN	0780390482
DOI	10.1109/IJCNN.2005.1556402
Language	English
Publisher	Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Publisher Date	2005-07-31
Publisher Place	Canada
Access Restriction	Subscribed
Rights Holder	Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Subject Keyword	Learning Recurrent neural networks Neural networks Computer science Feedforward systems Control systems Feedforward neural networks Electronic mail Computer architecture Fuzzy control
Content Type	Text
Resource Type	Article

Sl.	Authority	Responsibilities	Communication Details
1	Ministry of Education (GoI), Department of Higher Education	Sanctioning Authority	https://www.education.gov.in/ict-initiatives
2	Indian Institute of Technology Kharagpur	Host Institute of the Project: The host institute of the project is responsible for providing infrastructure support and hosting the project	https://www.iitkgp.ac.in
3	National Digital Library of India Office, Indian Institute of Technology Kharagpur	The administrative and infrastructural headquarters of the project	Dr. B. Sutradhar bsutra@ndl.gov.in
4	Project PI / Joint PI	Principal Investigator and Joint Principal Investigators of the project	Dr. B. Sutradhar bsutra@ndl.gov.in Prof. Saswat Chakrabarti will be added soon
5	Website/Portal (Helpdesk)	Queries regarding NDLI and its services	support@ndl.gov.in
6	Contents and Copyright Issues	Queries related to content curation and copyright issues	content@ndl.gov.in
7	National Digital Library of India Club (NDLI Club)	Queries related to NDLI Club formation, support, user awareness program, seminar/symposium, collaboration, social media, promotion, and outreach	clubsupport@ndl.gov.in
8	Digital Preservation Centre (DPC)	Assistance with digitizing and archiving copyright-free printed books	dpc@ndl.gov.in
9	IDR Setup or Support	Queries related to establishment and support of Institutional Digital Repository (IDR) and IDR workshops	idr@ndl.gov.in

Generational versus steady-state evolution for optimizing neural network learning

Feed-forward echo state networks

An improved TSK-type recurrent fuzzy network for dynamic system identification

Reservoir riddles: suggestions for echo state network research

Second differentials in arbitrary feedforward neural networks

Research for a Dynamic Recurrent Fuzzy Wavelet Network

Recurrent algebraic fuzzy neural networks based on fuzzy numbers

FEN (fuzzy expert network) learning architecture; generalization of self adjusting fuzzy modeling on event-driven acyclic neural networks using expert networks backpropagation learning

ELM-based Multiple Classifier Systems

Modeling reward functions for incomplete state representations via echo state networks

Similar Documents

Generational versus steady-state evolution for optimizing neural network learning

Feed-forward echo state networks

An improved TSK-type recurrent fuzzy network for dynamic system identification

Reservoir riddles: suggestions for echo state network research

Second differentials in arbitrary feedforward neural networks

Research for a Dynamic Recurrent Fuzzy Wavelet Network

Recurrent algebraic fuzzy neural networks based on fuzzy numbers

FEN (fuzzy expert network) learning architecture; generalization of self adjusting fuzzy modeling on event-driven acyclic neural networks using expert networks backpropagation learning

ELM-based Multiple Classifier Systems

Modeling reward functions for incomplete state representations via echo state networks