Application of Lemke's method to a class of Markov decision problems

Nobuo Matsubayashi, Hisakazu Nishino

Research output: Contribution to journalArticle

2 Citations (Scopus)

Abstract

This paper presents an application of Lemke's method to a class of Markov decision problems, appearing in the optimal stopping problems, and other well-known optimization problems. We consider a special case of the Markov decision problems with finitely many states, where the agent can choose one of the alternatives: getting a fixed reward immediately or paying the penalty for one term. We show that the problem can be reduced to a linear complementarity problem that can be solved by Lemke's method with the number of iterations less than the number of states. The reduced linear complementarity problem does not necessarily satisfy the copositive-plus condition. Nevertheless we show that the Lemke's method succeeds in solving the problem by proving that the problem satisfies a necessary and sufficient condition for the extended Lemke's method to compute a solution in the piecewise linear complementarity problem.

Original languageEnglish
Pages (from-to)584-590
Number of pages7
JournalEuropean Journal of Operational Research
Volume116
Issue number3
DOIs
Publication statusPublished - 1999 Aug 1
Externally publishedYes

Fingerprint

Decision problem
Linear Complementarity Problem
Optimal Stopping Problem
Reward
Piecewise Linear
Immediately
Penalty
Choose
Optimization Problem
Iteration
Necessary Conditions
Class
Sufficient Conditions
Alternatives
Term
Linear complementarity problem
reward
penalty

ASJC Scopus subject areas

  • Information Systems and Management
  • Management Science and Operations Research
  • Statistics, Probability and Uncertainty
  • Applied Mathematics
  • Modelling and Simulation
  • Transportation

Cite this

Application of Lemke's method to a class of Markov decision problems. / Matsubayashi, Nobuo; Nishino, Hisakazu.

In: European Journal of Operational Research, Vol. 116, No. 3, 01.08.1999, p. 584-590.

Research output: Contribution to journalArticle

@article{a77cfedb7ac54bc6bfcd47ad9ca976eb,
title = "Application of Lemke's method to a class of Markov decision problems",
abstract = "This paper presents an application of Lemke's method to a class of Markov decision problems, appearing in the optimal stopping problems, and other well-known optimization problems. We consider a special case of the Markov decision problems with finitely many states, where the agent can choose one of the alternatives: getting a fixed reward immediately or paying the penalty for one term. We show that the problem can be reduced to a linear complementarity problem that can be solved by Lemke's method with the number of iterations less than the number of states. The reduced linear complementarity problem does not necessarily satisfy the copositive-plus condition. Nevertheless we show that the Lemke's method succeeds in solving the problem by proving that the problem satisfies a necessary and sufficient condition for the extended Lemke's method to compute a solution in the piecewise linear complementarity problem.",
author = "Nobuo Matsubayashi and Hisakazu Nishino",
year = "1999",
month = "8",
day = "1",
doi = "10.1016/S0377-2217(98)00041-1",
language = "English",
volume = "116",
pages = "584--590",
journal = "European Journal of Operational Research",
issn = "0377-2217",
publisher = "Elsevier",
number = "3",

}

TY - JOUR

T1 - Application of Lemke's method to a class of Markov decision problems

AU - Matsubayashi, Nobuo

AU - Nishino, Hisakazu

PY - 1999/8/1

Y1 - 1999/8/1

N2 - This paper presents an application of Lemke's method to a class of Markov decision problems, appearing in the optimal stopping problems, and other well-known optimization problems. We consider a special case of the Markov decision problems with finitely many states, where the agent can choose one of the alternatives: getting a fixed reward immediately or paying the penalty for one term. We show that the problem can be reduced to a linear complementarity problem that can be solved by Lemke's method with the number of iterations less than the number of states. The reduced linear complementarity problem does not necessarily satisfy the copositive-plus condition. Nevertheless we show that the Lemke's method succeeds in solving the problem by proving that the problem satisfies a necessary and sufficient condition for the extended Lemke's method to compute a solution in the piecewise linear complementarity problem.

AB - This paper presents an application of Lemke's method to a class of Markov decision problems, appearing in the optimal stopping problems, and other well-known optimization problems. We consider a special case of the Markov decision problems with finitely many states, where the agent can choose one of the alternatives: getting a fixed reward immediately or paying the penalty for one term. We show that the problem can be reduced to a linear complementarity problem that can be solved by Lemke's method with the number of iterations less than the number of states. The reduced linear complementarity problem does not necessarily satisfy the copositive-plus condition. Nevertheless we show that the Lemke's method succeeds in solving the problem by proving that the problem satisfies a necessary and sufficient condition for the extended Lemke's method to compute a solution in the piecewise linear complementarity problem.

UR - http://www.scopus.com/inward/record.url?scp=0032648142&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=0032648142&partnerID=8YFLogxK

U2 - 10.1016/S0377-2217(98)00041-1

DO - 10.1016/S0377-2217(98)00041-1

M3 - Article

AN - SCOPUS:0032648142

VL - 116

SP - 584

EP - 590

JO - European Journal of Operational Research

JF - European Journal of Operational Research

SN - 0377-2217

IS - 3

ER -