Infinite-horizon deterministic dynamic programming in discrete time

a monotone convergence principle and a penalty method

Takashi Kamihigashi, Masayuki Yao

Research output: Contribution to journalArticle

Abstract

We consider infinite-horizon deterministic dynamic programming problems in discrete time. We show that the value function of such a problem is always a fixed point of a modified version of the Bellman operator. We also show that value iteration converges increasingly to the value function if the initial function is dominated by the value function, is mapped upward by the modified Bellman operator and satisfies a transversality-like condition. These results require no assumption except for the general framework of infinite-horizon deterministic dynamic programming. As an application, we show that the value function can be approximated by computing the value function of an unconstrained version of the problem with the constraint replaced by a penalty function.

Original languageEnglish
Pages (from-to)1-10
Number of pages10
JournalOptimization
DOIs
Publication statusAccepted/In press - 2016 Jun 10

Fingerprint

Monotone Convergence
Penalty Method
Infinite Horizon
Dynamic programming
Value Function
Dynamic Programming
Discrete-time
Value Iteration
Transversality
Penalty Function
Operator
Fixed point
Penalty method
Infinite horizon
Value function
Converge
Computing

Keywords

  • Bellman operator
  • Dynamic programming
  • fixed point
  • penalty method
  • value iteration

ASJC Scopus subject areas

  • Applied Mathematics
  • Control and Optimization
  • Management Science and Operations Research

Cite this

Infinite-horizon deterministic dynamic programming in discrete time : a monotone convergence principle and a penalty method. / Kamihigashi, Takashi; Yao, Masayuki.

In: Optimization, 10.06.2016, p. 1-10.

Research output: Contribution to journalArticle

@article{5b7401105f8a45cd8ca7510f89705230,
title = "Infinite-horizon deterministic dynamic programming in discrete time: a monotone convergence principle and a penalty method",
abstract = "We consider infinite-horizon deterministic dynamic programming problems in discrete time. We show that the value function of such a problem is always a fixed point of a modified version of the Bellman operator. We also show that value iteration converges increasingly to the value function if the initial function is dominated by the value function, is mapped upward by the modified Bellman operator and satisfies a transversality-like condition. These results require no assumption except for the general framework of infinite-horizon deterministic dynamic programming. As an application, we show that the value function can be approximated by computing the value function of an unconstrained version of the problem with the constraint replaced by a penalty function.",
keywords = "Bellman operator, Dynamic programming, fixed point, penalty method, value iteration",
author = "Takashi Kamihigashi and Masayuki Yao",
year = "2016",
month = "6",
day = "10",
doi = "10.1080/02331934.2016.1193737",
language = "English",
pages = "1--10",
journal = "Optimization",
issn = "0233-1934",
publisher = "Taylor and Francis Ltd.",

}

TY - JOUR

T1 - Infinite-horizon deterministic dynamic programming in discrete time

T2 - a monotone convergence principle and a penalty method

AU - Kamihigashi, Takashi

AU - Yao, Masayuki

PY - 2016/6/10

Y1 - 2016/6/10

N2 - We consider infinite-horizon deterministic dynamic programming problems in discrete time. We show that the value function of such a problem is always a fixed point of a modified version of the Bellman operator. We also show that value iteration converges increasingly to the value function if the initial function is dominated by the value function, is mapped upward by the modified Bellman operator and satisfies a transversality-like condition. These results require no assumption except for the general framework of infinite-horizon deterministic dynamic programming. As an application, we show that the value function can be approximated by computing the value function of an unconstrained version of the problem with the constraint replaced by a penalty function.

AB - We consider infinite-horizon deterministic dynamic programming problems in discrete time. We show that the value function of such a problem is always a fixed point of a modified version of the Bellman operator. We also show that value iteration converges increasingly to the value function if the initial function is dominated by the value function, is mapped upward by the modified Bellman operator and satisfies a transversality-like condition. These results require no assumption except for the general framework of infinite-horizon deterministic dynamic programming. As an application, we show that the value function can be approximated by computing the value function of an unconstrained version of the problem with the constraint replaced by a penalty function.

KW - Bellman operator

KW - Dynamic programming

KW - fixed point

KW - penalty method

KW - value iteration

UR - http://www.scopus.com/inward/record.url?scp=84974853448&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84974853448&partnerID=8YFLogxK

U2 - 10.1080/02331934.2016.1193737

DO - 10.1080/02331934.2016.1193737

M3 - Article

SP - 1

EP - 10

JO - Optimization

JF - Optimization

SN - 0233-1934

ER -