NDLI: Towards a Multiple-Lookahead-Levels agent reinforcement-learning technique and its implementation in integrated circuits

Content Provider	Springer Nature Link
Author	Al Dayaa, H. S. Megherbi, D. B.
Copyright Year	2012
Abstract	Reinforcement learning (RL) techniques have contributed and continue to tremendously contribute to the advancement of machine learning and its many related recent applications. As it is well known, some of the main limitations of existing RL techniques are, in general, their slow convergence and their computational complexity. The contributions of this paper are two-fold: (1) First, it introduces a technique for reinforcement learning using multiple lookahead levels that grants an autonomous agent more visibility in its environment and helps it learn faster. This technique extends the Watkins’s Q-Learning algorithm by using the Multiple-Lookahead-Levels (MLL) model equation that we develop and present here. An analysis of the convergence of the MLL equation and proof of its effectiveness are performed. A method to compute the improvement rate of the agent’s learning speed between different look-ahead levels is also proposed and implemented. Here, both the time and space complexities are examined. Results show that the number of steps, required to achieve the goal, per learning path exponentially decreases with the learning path number (time). Results also show that the number of steps per learning path, to some degree, is less at any time when the number of look-ahead levels is higher (space). Furthermore, we perform the analysis of the MLL system in the time domain and prove its temporal stability using Lyapunov theory. (2) Second, based on this Lyapunov stability analysis, we subsequently, and for the first time, propose a circuit architecture for the MLL technique’s software configurable hardware system design for real-time applications.
Starting Page	588
Ending Page	615
Page Count	28
File Format	PDF
ISSN	09208542
Journal	The Journal of Supercomputing
Volume Number	62
Issue Number	1
e-ISSN	15730484
Language	English
Publisher	Springer US
Publisher Date	2012-01-17
Publisher Place	Boston
Access Restriction	One Nation One Subscription (ONOS)
Subject Keyword	Reinforcement learning Q-Learning Machine learning Integrated circuits Lyapunov stability Robotics Computer Science Programming Languages, Compilers, Interpreters Processor Architectures
Content Type	Text
Resource Type	Article
Subject	Theoretical Computer Science Information Systems Hardware and Architecture Software

Sl.	Authority	Responsibilities	Communication Details
1	Ministry of Education (GoI), Department of Higher Education	Sanctioning Authority	https://www.education.gov.in/ict-initiatives
2	Indian Institute of Technology Kharagpur	Host Institute of the Project: The host institute of the project is responsible for providing infrastructure support and hosting the project	https://www.iitkgp.ac.in
3	National Digital Library of India Office, Indian Institute of Technology Kharagpur	The administrative and infrastructural headquarters of the project	Dr. B. Sutradhar bsutra@ndl.gov.in
4	Project PI / Joint PI	Principal Investigator and Joint Principal Investigators of the project	Dr. B. Sutradhar bsutra@ndl.gov.in Prof. Saswat Chakrabarti will be added soon
5	Website/Portal (Helpdesk)	Queries regarding NDLI and its services	support@ndl.gov.in
6	Contents and Copyright Issues	Queries related to content curation and copyright issues	content@ndl.gov.in
7	National Digital Library of India Club (NDLI Club)	Queries related to NDLI Club formation, support, user awareness program, seminar/symposium, collaboration, social media, promotion, and outreach	clubsupport@ndl.gov.in
8	Digital Preservation Centre (DPC)	Assistance with digitizing and archiving copyright-free printed books	dpc@ndl.gov.in
9	IDR Setup or Support	Queries related to establishment and support of Institutional Digital Repository (IDR) and IDR workshops	idr@ndl.gov.in

Reinforcement learning technique using agent state occurrence frequency with analysis of knowledge sharing on the agent’s learning process in multiagent environments

Computational awareness towards green environments

ELM-based spammer detection in social networks

Design of Processor Arrays for Reconfigurable Architectures

Oscillatory Metastability in Optical Network Synchronizer Circuits

The MAGNeT Toolkit: Design, Implementation and Evaluation

Computing with T.Node Parallel Architecture

Software Architectures and Coordination Models

Advances in multicore systems architectures

Towards a Multiple-Lookahead-Levels agent reinforcement-learning technique and its implementation in integrated circuits

Similar Documents

Reinforcement learning technique using agent state occurrence frequency with analysis of knowledge sharing on the agent’s learning process in multiagent environments

Computational awareness towards green environments

ELM-based spammer detection in social networks

Design of Processor Arrays for Reconfigurable Architectures

Oscillatory Metastability in Optical Network Synchronizer Circuits

The MAGNeT Toolkit: Design, Implementation and Evaluation

Computing with T.Node Parallel Architecture

Software Architectures and Coordination Models

Advances in multicore systems architectures

Towards a Multiple-Lookahead-Levels agent reinforcement-learning technique and its implementation in integrated circuits