NDLI: Influence of modeling structure in probabilistic sequential decision problems

Please wait, while we are loading the content...

2019

2018

A new barrier for a class of semidefinite problems

Numerical solutions of the mass transfer problem

Influence of modeling structure in probabilistic sequential decision problems

On-line models and algorithms for max independent set

Economic assessment of the Champagne wine qualitative stock mecanism

Large neighborhood improvements for solving car sequencing problems

On semidefinite bounds for maximization of a non-convex quadratic objective over the l1 unit ball

A regularization method for ill-posed bilevel optimization problems

An analytic center cutting plane algorithm for finding equilibrium points

Convex quadratic underestimation and Branch and Bound for univariate global optimization with one nonconvex constraint

Bounds of graph parameters for global constraints

Branch and Cut based on the volume algorithm: Steiner trees in graphs and Max-cut

Analyzing discrete-time bulk-service Geo/Geob/m queue

Solution of a fractional combinatorial optimization problem by mixed integer programming

Combining constraint Propagation and meta-heuristics for searching a Maximum Weight Hamiltonian Chain

Editorial

An ex-post bound on the greedy heuristic for the uncapacitated facility location problem

Des explications pour reconnaître et exploiter les structures cachées d'un problème combinatoire

Editorial

The periodic Vehicle routing problem: classification and heuristic

Centralized VERSUS decentralized production planning

Influence of modeling structure in probabilistic sequential decision problems

Content Provider	EDP Sciences
Author	Florent Teichteil-Königsbuch Patrick Fabiani
Abstract	Markov Decision Processes (MDPs) are a classical framework for stochastic sequential decision problems, based on an enumerated state space representation. More compact and structured representations have been proposed: factorization techniques use state variables representations, while decomposition techniques are based on a partition of the state space into sub-regions and take advantage of the resulting structure of the state transition graph. We use a family of probabilistic exploration-like planning problems in order to study the influence of the modeling structure on the MDP solution. We first discuss the advantages and drawbacks of a graph based representation of the state space, then present our comparisons of two decomposition techniques, and propose to use a global approach combining both state space factorization and decomposition techniques. On the exploration problem instance, it is proposed to take advantage of the natural topological structure of the navigation space, which is partitioned into regions. A number of local policies are optimized within each region, that become the macro-actions of the global abstract MDP resulting from the decomposition. The regions are the corresponding macro-states in the abstract MDP. The global abstract MDP is obtained in a factored form, combining all the initial MDP state variables and one macro-state “region” variable standing for the different possible macro-states corresponding to the regions. Further research is presently conducted on efficient solution algorithms implementing the same hybrid approach for tackling large size MDPs.
Ending Page	234
Starting Page	195
Page Count	40
File Format	HTM / HTML PDF
ISSN	03990559
Alternate Webpage(s)	https://www.rairo-ro.org/articles/ro/abs/2006/02/ro0577/ro0577.html
e-ISSN	12903868
Issue Number	2
Journal	RAIRO - Operations Research
Volume Number	40
DOI	10.1051/ro:2006019
Language	English
Publisher	EDP Sciences
Publisher Date	2006-04-01
Access Restriction	Subscribed
Rights Holder	© EDP Sciences, 2006
Subject Keyword	Probabilistic planning dynamic programming Markov decision processes application to autonomous decision making. Probabilistic planning / dynamic programming / Markov decision processes / application to autonomous decision making. Markov and semi-Markov decision processes Reasoning under uncertainty Problem solving (heuristics, search strategies, etc.) Combinatorics Factorization Graph theory (including graph drawing)
Content Type	Text
Resource Type	Article
Subject	Management Science and Operations Research Theoretical Computer Science Computer Science Applications

Sl.	Authority	Responsibilities	Communication Details
1	Ministry of Education (GoI), Department of Higher Education	Sanctioning Authority	https://www.education.gov.in/ict-initiatives
2	Indian Institute of Technology Kharagpur	Host Institute of the Project: The host institute of the project is responsible for providing infrastructure support and hosting the project	https://www.iitkgp.ac.in
3	National Digital Library of India Office, Indian Institute of Technology Kharagpur	The administrative and infrastructural headquarters of the project	Dr. B. Sutradhar bsutra@ndl.gov.in
4	Project PI / Joint PI	Principal Investigator and Joint Principal Investigators of the project	Dr. B. Sutradhar bsutra@ndl.gov.in Prof. Saswat Chakrabarti will be added soon
5	Website/Portal (Helpdesk)	Queries regarding NDLI and its services	support@ndl.gov.in
6	Contents and Copyright Issues	Queries related to content curation and copyright issues	content@ndl.gov.in
7	National Digital Library of India Club (NDLI Club)	Queries related to NDLI Club formation, support, user awareness program, seminar/symposium, collaboration, social media, promotion, and outreach	clubsupport@ndl.gov.in
8	Digital Preservation Centre (DPC)	Assistance with digitizing and archiving copyright-free printed books	dpc@ndl.gov.in
9	IDR Setup or Support	Queries related to establishment and support of Institutional Digital Repository (IDR) and IDR workshops	idr@ndl.gov.in

Topological Order Value Iteration Algorithm for Solving Probabilistic Planning

Embedding a state space model into a Markov decision process

Solving path planning problems in urban environments based on a priori sensor availability and execution error propagation

Nested-Batch-Mode Learning and Stochastic Optimization with An Application to Sequential MultiStage Testing in Materials Science

From Infinite to Finite Programs: Explicit Error Bounds with Applications to Approximate Dynamic Programming

On solving optimal policies for event-based dynamic programming

A Versatile Queuing System For Sharing Economy Platform Operations

Optimal assignment of sellers in a store with a random number of clients via the Armed Bandit model

Sequential monte carlo in probabilistic planning reachability heuristics (2006).

Influence of modeling structure in probabilistic sequential decision problems

Similar Documents

Topological Order Value Iteration Algorithm for Solving Probabilistic Planning

Embedding a state space model into a Markov decision process

Solving path planning problems in urban environments based on a priori sensor availability and execution error propagation

Nested-Batch-Mode Learning and Stochastic Optimization with An Application to Sequential MultiStage Testing in Materials Science

From Infinite to Finite Programs: Explicit Error Bounds with Applications to Approximate Dynamic Programming

On solving optimal policies for event-based dynamic programming

A Versatile Queuing System For Sharing Economy Platform Operations

Optimal assignment of sellers in a store with a random number of clients via the Armed Bandit model

Sequential monte carlo in probabilistic planning reachability heuristics (2006).

Influence of modeling structure in probabilistic sequential decision problems