NDLI: Control of Markov chains with safety bounds

Please wait, while we are loading the content...

Control of Markov chains with safety bounds

Content Provider	IEEE Xplore Digital Library
Author	Arapostathis, A. Kumar, R. Hsu, S.-P.
Copyright Year	2004
Abstract	In an earlier paper, the authors introduced the notion of safety control of stochastic discrete event systems (DESs), modeled as controlled Markov chains. Safety was specified as an upper bound on the components of the state probability distribution, and the class of irreducible and aperiodic Markov chains were analyzed relative to this safety criterion. Under the assumption of complete state observations: 1) the authors identified the set of all state-feedback controllers that enforce the safety specification, for all safe initial probability distributions and 2) for any given state-feedback controller, the authors constructed the maximal invariant safe set (MISS). In this paper, the authors extend the work in several ways: 1) safety is specified in terms of both upper and lower bounds; 2) we consider a larger class of Markov chains that includes reducible and periodic chains; 3) we present a more general iterative algorithm for computing the MISS, which is quite flexible in its initialization; 4) we obtain an explicit upper bound for the number of iterations needed for the algorithm to terminate. Note to Practitioners-The paper studies "safety" control of stochastic systems modeled as Markov chains. Safety is defined as a requirement that the probability distribution in each state remain bounded between an upper and a lower bound. For example, a financial investment policy should be such that the probability of ever being bankrupt is bounded below by a positive number. Prior works on control of Markov chains have addressed optimality but not safety. A condition is obtained under which a controlled Markov chain is guaranteed to be safe at all times. For those chains that do not satisfy such a condition, a maximal subset of the safe set of distributions is computed so that if the chain is initialized with a distribution in that maximal subset, it remains safe all the times. A condition is obtained under which such a maximal set is nonempty. The computation of such a maximal set is iterative and we provide a condition under which the computation terminates in a finite number of iterations. Manufacturing system examples are included to illustrate the results.
Sponsorship	IEEE Robotics and Automation Society
Starting Page	333
Ending Page	343
Page Count	11
File Size	461676
File Format	PDF
ISSN	15455955
Volume Number	2
Issue Number	4
Language	English
Publisher	Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Publisher Date	2005-10-01
Publisher Place	U.S.A.
Access Restriction	One Nation One Subscription (ONOS)
Rights Holder	Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Subject Keyword	Safety Probability distribution Stochastic systems Upper bound Iterative algorithms Discrete event systems Control system synthesis Investments Optimal control Distributed computing stochastic discrete event system (DES) Markov chain reliability safety specification
Content Type	Text
Resource Type	Article
Subject	Control and Systems Engineering Electrical and Electronic Engineering

Sl.	Authority	Responsibilities	Communication Details
1	Ministry of Education (GoI), Department of Higher Education	Sanctioning Authority	https://www.education.gov.in/ict-initiatives
2	Indian Institute of Technology Kharagpur	Host Institute of the Project: The host institute of the project is responsible for providing infrastructure support and hosting the project	https://www.iitkgp.ac.in
3	National Digital Library of India Office, Indian Institute of Technology Kharagpur	The administrative and infrastructural headquarters of the project	Dr. B. Sutradhar bsutra@ndl.gov.in
4	Project PI / Joint PI	Principal Investigator and Joint Principal Investigators of the project	Dr. B. Sutradhar bsutra@ndl.gov.in Prof. Saswat Chakrabarti will be added soon
5	Website/Portal (Helpdesk)	Queries regarding NDLI and its services	support@ndl.gov.in
6	Contents and Copyright Issues	Queries related to content curation and copyright issues	content@ndl.gov.in
7	National Digital Library of India Club (NDLI Club)	Queries related to NDLI Club formation, support, user awareness program, seminar/symposium, collaboration, social media, promotion, and outreach	clubsupport@ndl.gov.in
8	Digital Preservation Centre (DPC)	Assistance with digitizing and archiving copyright-free printed books	dpc@ndl.gov.in
9	IDR Setup or Support	Queries related to establishment and support of Institutional Digital Repository (IDR) and IDR workshops	idr@ndl.gov.in

Controlled Markov chains with safety upper bound

Stabilisation of Markov jump linear systems subject to both state and mode detection delays

Minimal Communication for Essential Transitions in a Distributed Discrete-Event System

Supervisor Localization: A Top-Down Approach to Distributed Control of Discrete-Event Systems

Polynomial Test for Stochastic Diagnosability of Discrete-Event Systems

Synthesis of Over-Approximating Inference-Based Decentralized Supervisors for Discrete Event Systems

Optimal Supervisory Control of Probabilistic Discrete Event Systems

Bounds on the reliability of distributed systems with unreliable nodes & links

Minimal-Cut Reliability Lower-Bound for Systems Containing Standby Modules

Control of Markov chains with safety bounds

Similar Documents

Controlled Markov chains with safety upper bound

Stabilisation of Markov jump linear systems subject to both state and mode detection delays

Minimal Communication for Essential Transitions in a Distributed Discrete-Event System

Supervisor Localization: A Top-Down Approach to Distributed Control of Discrete-Event Systems

Polynomial Test for Stochastic Diagnosability of Discrete-Event Systems

Synthesis of Over-Approximating Inference-Based Decentralized Supervisors for Discrete Event Systems

Optimal Supervisory Control of Probabilistic Discrete Event Systems

Bounds on the reliability of distributed systems with unreliable nodes & links

Minimal-Cut Reliability Lower-Bound for Systems Containing Standby Modules

Control of Markov chains with safety bounds