NDLI: Output feedback reinforcement learning based optimal output synchronisation of heterogeneous discrete-time multi-agent systems

Content Provider	IET Digital Library
Author	Rizvi, Syed Ali Asad Lin, Zongli
Abstract	This study proposes a model-free distributed output feedback control scheme that achieves synchronisation of the outputs of the heterogeneous follower agents with that of the leader agent in a directed network. A distributed two degree of freedom approach is presented that separates the learning of the optimal output feedback and the feedforward terms of the local control law for each agent. The local feedback parameters are learned using the proposed off-policy Q-learning algorithm, whereas a gradient adaptive law is presented to learn the local feedforward control parameters to achieve asymptotic tracking of each agent. This learning scheme and the resulting distributed control laws neither require access to the local internal state of the agents nor do they need an additional distributed leader state observer. The proposed approach has the advantage over the previous state augmentation approaches as it circumvents the need of introducing a discounting factor in the local performance functions. It is shown that the proposed algorithm converges to the optimal solution of the algebraic Riccati equation and the output regulator equations without explicitly solving them as long as the leader agent is reachable directly or indirectly from all the follower agents. Simulation results validate the proposed scheme.
Starting Page	2866
Ending Page	2876
Page Count	11
ISSN	17518644
Volume Number	13
e-ISSN	17518652
Issue Number	Issue 17, Nov (2019)
Alternate Webpage(s)	https://digital-library.theiet.org/content/journals/iet-cta/13/17
Alternate Webpage(s)	https://digital-library.theiet.org/content/journals/10.1049/iet-cta.2018.6266
Journal	IET Control Theory & Applications
Publisher Date	2019-01-31
Access Restriction	Open
Rights Holder	© The Institution of Engineering and Technology
Subject Keyword	Adaptive Control Closed Loop System Combinatorial Mathematics Control System Analysis Control System Analysis Computing Control System Design And Analysis Control System Synthesis Directed Network Discrete Control System Discrete Time System Discrete-time Multiagent System Distributed Control Distributed Leader State Observer Distributed Two Degree of Freedom Approach Feedback Feedforward Feedforward Term Freedom Approach Gradient Adaptive Law Heterogeneous Follower Agent Knowledge Engineering Technique Leader Agent Learning Algorithm Learning in AI Learning Scheme Linear Algebra Local Control Law Local Feedback Parameter Local Feedforward Control Parameter Local Internal State Local Performance Function Model-free Distributed Output Feedback Control Scheme Multi-agent System Multivariable Control System Numerical Analysis Observers Optimal Control Optimal Output Feedback Optimal Output Synchronisation Optimal Solution Optimisation Technique Output Feedback Reinforcement Learning Output Regulator Equation Resulting Distributed Control Law Riccati Equation Self Adjusting Control System Stability in Control Theory State Augmentation Approach Synchronisation Synthesis Method
Content Type	Text
Resource Type	Article
Subject	Control and Optimization Control and Systems Engineering Human-Computer Interaction Electrical and Electronic Engineering Computer Science Applications

Sl.	Authority	Responsibilities	Communication Details
1	Ministry of Education (GoI), Department of Higher Education	Sanctioning Authority	https://www.education.gov.in/ict-initiatives
2	Indian Institute of Technology Kharagpur	Host Institute of the Project: The host institute of the project is responsible for providing infrastructure support and hosting the project	https://www.iitkgp.ac.in
3	National Digital Library of India Office, Indian Institute of Technology Kharagpur	The administrative and infrastructural headquarters of the project	Dr. B. Sutradhar bsutra@ndl.gov.in
4	Project PI / Joint PI	Principal Investigator and Joint Principal Investigators of the project	Dr. B. Sutradhar bsutra@ndl.gov.in Prof. Saswat Chakrabarti will be added soon
5	Website/Portal (Helpdesk)	Queries regarding NDLI and its services	support@ndl.gov.in
6	Contents and Copyright Issues	Queries related to content curation and copyright issues	content@ndl.gov.in
7	National Digital Library of India Club (NDLI Club)	Queries related to NDLI Club formation, support, user awareness program, seminar/symposium, collaboration, social media, promotion, and outreach	clubsupport@ndl.gov.in
8	Digital Preservation Centre (DPC)	Assistance with digitizing and archiving copyright-free printed books	dpc@ndl.gov.in
9	IDR Setup or Support	Queries related to establishment and support of Institutional Digital Repository (IDR) and IDR workshops	idr@ndl.gov.in

Optimal output anti-synchronisation of cooperative-competitive multi-agent systems via distributed observer

Adaptive optimal output feedback tracking control for unknown discrete-time linear systems using a combined reinforcement Q-learning and internal model method

Cooperative output regulation of discrete-time linear time-delay multi-agent systems

Robust containment control of heterogeneous non-linear multi-agent systems via power series approach

Distributed adaptive containment control of heterogeneous linear multi-agent systems: an output regulation approach

Resilient adaptive optimal control of distributed multi-agent systems using reinforcement learning

Global stabilisation of a class of feedforward systems with distributed delays

Distributed leaderless synchronisation control for non-linear MASs with both parameter and non-parameter uncertainty terms

Distributed neuro-adaptive control protocols for non-strict feedback non-linear MASs with input saturation

Output feedback reinforcement learning based optimal output synchronisation of heterogeneous discrete-time multi-agent systems

Similar Documents

Optimal output anti-synchronisation of cooperative-competitive multi-agent systems via distributed observer

Adaptive optimal output feedback tracking control for unknown discrete-time linear systems using a combined reinforcement Q-learning and internal model method

Cooperative output regulation of discrete-time linear time-delay multi-agent systems

Robust containment control of heterogeneous non-linear multi-agent systems via power series approach

Distributed adaptive containment control of heterogeneous linear multi-agent systems: an output regulation approach

Resilient adaptive optimal control of distributed multi-agent systems using reinforcement learning

Global stabilisation of a class of feedforward systems with distributed delays

Distributed leaderless synchronisation control for non-linear MASs with both parameter and non-parameter uncertainty terms

Distributed neuro-adaptive control protocols for non-strict feedback non-linear MASs with input saturation

Output feedback reinforcement learning based optimal output synchronisation of heterogeneous discrete-time multi-agent systems