NDLI: Totally model-free reinforcement learning by actor-critic Elman networks in non-Markovian domains

Content Provider	IEEE Xplore Digital Library
Author	Mizutani, E. Dreyfus, S.E.
Copyright Year	1998
Description	Author affiliation: Dept. of Ind. Eng. & Oper. Res., California Univ., Berkeley, CA, USA (Mizutani, E.)
Abstract	We describe how an actor-critic reinforcement learning agent in a non-Markovian domain finds an optimal sequence of actions in a totally model-free fashion; that is, the agent neither learns transitional probabilities and associated rewards, nor by how much the state space should be augmented so that the Markov property holds. In particular, we employ an Elman-type recurrent neural network to solve non-Markovian problems since an Elman-type network is able to implicitly and automatically render the process Markovian. A standard "actor-critic" neural network model has two separate components: the action (actor) network and the value (critic) network. In animal brains, however, those two presumably may not be distinct, but rather somehow entwined. We thus construct one Elman network with two output nodes: actor node and critic node, and a portion of the shared hidden layer is fed back as the context layer, which functions as a history memory to produce sensitivity to non-Markovian dependencies. The agent explores small-scale three and four-stage triangular path-networks to learn an optimal sequence of actions that maximizes total value (or reward) associated with its transition from vertex to vertex. The posed problem has deterministic transition and reward associated with each allowable action (although either could be stochastic) and is rendered non-Markovian by the reward being dependent on an earlier transition. Due to the nature of neural model-free learning, the agent needs many iterations to find the optimal actions even in small-scale path problems.
Starting Page	2016
Ending Page	2021
File Size	699700
Page Count	6
File Format	PDF
ISBN	0780348591
ISSN	10987576
DOI	10.1109/IJCNN.1998.687169
Language	English
Publisher	Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Publisher Date	1998-05-04
Publisher Place	USA
Access Restriction	Subscribed
Rights Holder	Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Subject Keyword	Learning Intelligent networks History State-space methods Biological neural networks Neural networks Industrial engineering Operations research Animals Stochastic processes
Content Type	Text
Resource Type	Article

Sl.	Authority	Responsibilities	Communication Details
1	Ministry of Education (GoI), Department of Higher Education	Sanctioning Authority	https://www.education.gov.in/ict-initiatives
2	Indian Institute of Technology Kharagpur	Host Institute of the Project: The host institute of the project is responsible for providing infrastructure support and hosting the project	https://www.iitkgp.ac.in
3	National Digital Library of India Office, Indian Institute of Technology Kharagpur	The administrative and infrastructural headquarters of the project	Dr. B. Sutradhar bsutra@ndl.gov.in
4	Project PI / Joint PI	Principal Investigator and Joint Principal Investigators of the project	Dr. B. Sutradhar bsutra@ndl.gov.in Prof. Saswat Chakrabarti will be added soon
5	Website/Portal (Helpdesk)	Queries regarding NDLI and its services	support@ndl.gov.in
6	Contents and Copyright Issues	Queries related to content curation and copyright issues	content@ndl.gov.in
7	National Digital Library of India Club (NDLI Club)	Queries related to NDLI Club formation, support, user awareness program, seminar/symposium, collaboration, social media, promotion, and outreach	clubsupport@ndl.gov.in
8	Digital Preservation Centre (DPC)	Assistance with digitizing and archiving copyright-free printed books	dpc@ndl.gov.in
9	IDR Setup or Support	Queries related to establishment and support of Institutional Digital Repository (IDR) and IDR workshops	idr@ndl.gov.in

Totally Model-Free Reinforcement Learning by Actor-Critic Elman Networks in Non-Markovian Domains (1998)

Totally Model-Free Reinforcement Learning by Actor-Critic Elman Networks in Non-Markovian Domains (1998)

On using discretized Cohen-Grossberg node dynamics for model-free actor-critic neural learning in non-Markovian domains

Actor-critic neural learning in non-markovian domains.

Two stochastic dynamic programming problems by model-free actor-critic recurrent-network learning in non-Markovian settings

Policy-Gradient Based Actor-Critic Algorithms

Trading off perception with internal state: reinforcement learning and analysis of Q-Elman networks in a Markovian task

Reinforcement learning by backpropagation through an LSTM model/critic

Trading off Perception with Internal State: Reinforcement Learning and Analysis of Q-Elman Networks in a Markovian Task (2000)

Totally model-free reinforcement learning by actor-critic Elman networks in non-Markovian domains

Similar Documents

Totally Model-Free Reinforcement Learning by Actor-Critic Elman Networks in Non-Markovian Domains (1998)

Totally Model-Free Reinforcement Learning by Actor-Critic Elman Networks in Non-Markovian Domains (1998)

On using discretized Cohen-Grossberg node dynamics for model-free actor-critic neural learning in non-Markovian domains

Actor-critic neural learning in non-markovian domains.

Two stochastic dynamic programming problems by model-free actor-critic recurrent-network learning in non-Markovian settings

Policy-Gradient Based Actor-Critic Algorithms

Trading off perception with internal state: reinforcement learning and analysis of Q-Elman networks in a Markovian task

Reinforcement learning by backpropagation through an LSTM model/critic

Trading off Perception with Internal State: Reinforcement Learning and Analysis of Q-Elman Networks in a Markovian Task (2000)

Totally model-free reinforcement learning by actor-critic Elman networks in non-Markovian domains