NDLI: On incorporating the paradigms of discretization and Bayesian estimation to create a new family of pursuit learning automata

Content Provider	Springer Nature Link
Author	Zhang, Xuan Granmo, Ole Christoffer Oommen, B. John
Copyright Year	2013
Abstract	There are currently two fundamental paradigms that have been used to enhance the convergence speed of Learning Automata (LA). The first involves the concept of utilizing the estimates of the reward probabilities, while the second involves discretizing the probability space in which the LA operates. This paper demonstrates how both of these can be simultaneously utilized, and in particular, by using the family of Bayesian estimates that have been proven to have distinct advantages over their maximum likelihood counterparts. The success of LA-based estimator algorithms over the classical, Linear Reward-Inaction (L RI )-like schemes, can be explained by their ability to pursue the actions with the highest reward probability estimates. Without access to reward probability estimates, it makes sense for schemes like the L RI to first make large exploring steps, and then to gradually turn exploration into exploitation by making progressively smaller learning steps. However, this behavior becomes counter-intuitive when pursuing actions based on their estimated reward probabilities. Learning should then ideally proceed in progressively larger steps, as the reward probability estimates turn more accurate. This paper introduces a new estimator algorithm, the Discretized Bayesian Pursuit Algorithm (DBPA), that achieves this by incorporating both the above paradigms. The DBPA is implemented by linearly discretizing the action probability space of the Bayesian Pursuit Algorithm (BPA) (Zhang et al. in IEA-AIE 2011, Springer, New York, pp. 608–620, 2011). The key innovation of this paper is that the linear discrete updating rules mitigate the counter-intuitive behavior of the corresponding linear continuous updating rules, by augmenting them with the reward probability estimates. Extensive experimental results show the superiority of DBPA over previous estimator algorithms. Indeed, the DBPA is probably the fastest reported LA to date. Apart from the rigorous experimental demonstration of the strength of the DBPA, the paper also briefly records the proofs of why the BPA and the DBPA are ϵ-optimal in stationary environments.
Starting Page	782
Ending Page	792
Page Count	11
File Format	PDF
ISSN	0924669X
Journal	Applied Intelligence
Volume Number	39
Issue Number	4
e-ISSN	15737497
Language	English
Publisher	Springer US
Publisher Date	2013-02-02
Publisher Place	Boston
Access Restriction	One Nation One Subscription (ONOS)
Subject Keyword	Learning automata Pursuit schemes Bayesian reasoning Estimator algorithms Discretized learning ε-optimality Artificial Intelligence (incl. Robotics) Mechanical Engineering Manufacturing, Machines, Tools
Content Type	Text
Resource Type	Article
Subject	Artificial Intelligence

Sl.	Authority	Responsibilities	Communication Details
1	Ministry of Education (GoI), Department of Higher Education	Sanctioning Authority	https://www.education.gov.in/ict-initiatives
2	Indian Institute of Technology Kharagpur	Host Institute of the Project: The host institute of the project is responsible for providing infrastructure support and hosting the project	https://www.iitkgp.ac.in
3	National Digital Library of India Office, Indian Institute of Technology Kharagpur	The administrative and infrastructural headquarters of the project	Dr. B. Sutradhar bsutra@ndl.gov.in
4	Project PI / Joint PI	Principal Investigator and Joint Principal Investigators of the project	Dr. B. Sutradhar bsutra@ndl.gov.in Prof. Saswat Chakrabarti will be added soon
5	Website/Portal (Helpdesk)	Queries regarding NDLI and its services	support@ndl.gov.in
6	Contents and Copyright Issues	Queries related to content curation and copyright issues	content@ndl.gov.in
7	National Digital Library of India Club (NDLI Club)	Queries related to NDLI Club formation, support, user awareness program, seminar/symposium, collaboration, social media, promotion, and outreach	clubsupport@ndl.gov.in
8	Digital Preservation Centre (DPC)	Assistance with digitizing and archiving copyright-free printed books	dpc@ndl.gov.in
9	IDR Setup or Support	Queries related to establishment and support of Institutional Digital Repository (IDR) and IDR workshops	idr@ndl.gov.in

A formal proof of the 𝜖-optimality of discretized pursuit algorithms

A formal proof of the ε-optimality of absorbing continuous pursuit algorithms using the theory of regular functions

A novel estimator based learning automata algorithm

Transfer learning for temporal nodes Bayesian networks

Incorporating Ancestors' Influence in Genetic Algorithms

Diagnosis of Active Systems by Automata-Based Reasoning Techniques

Handling numeric attributes when comparing Bayesian network classifiers: does the discretization method matter?

Optimizing channel selection for cognitive radio networks using a distributed Bayesian learning automata-based approach

BNC-VLA: bayesian network structure learning using a team of variable-action set learning automata

On incorporating the paradigms of discretization and Bayesian estimation to create a new family of pursuit learning automata

Similar Documents

A formal proof of the 𝜖-optimality of discretized pursuit algorithms

A formal proof of the ε-optimality of absorbing continuous pursuit algorithms using the theory of regular functions

A novel estimator based learning automata algorithm

Transfer learning for temporal nodes Bayesian networks

Incorporating Ancestors' Influence in Genetic Algorithms

Diagnosis of Active Systems by Automata-Based Reasoning Techniques

Handling numeric attributes when comparing Bayesian network classifiers: does the discretization method matter?

Optimizing channel selection for cognitive radio networks using a distributed Bayesian learning automata-based approach

BNC-VLA: bayesian network structure learning using a team of variable-action set learning automata

On incorporating the paradigms of discretization and Bayesian estimation to create a new family of pursuit learning automata