Controlled semi-Markov models under long-run average rewards

Rabindra N Bhattacharya, Mukul Majumdar

Research output: Contribution to journalArticle

18 Citations (Scopus)

Abstract

Let the state space S be a Borel subset of a complete separable metric space, the action space A compact metric. Existence of stationary optimal policies is proved and a dynamic programming equation derived for general semi-Markov models under the long-run average reward criterion, focusing on it as a limiting case of optimization under discounting as the discount factor goes to one. This extends many earlier results. An example of Reed (1974) on harvesting a natural resource provides an application not covered by earlier results.

Original languageEnglish (US)
Pages (from-to)223-242
Number of pages20
JournalJournal of Statistical Planning and Inference
Volume22
Issue number2
DOIs
StatePublished - 1989
Externally publishedYes

Fingerprint

Semi-Markov Model
Natural resources
Long-run
Dynamic programming
Reward
Optimal Stationary Policy
Discount Factor
Discounting
Natural Resources
Harvesting
Dynamic Programming
Metric space
State Space
Limiting
Metric
Subset
Optimization
Markov model
State space
Discount factor

Keywords

  • dynamic programming equation
  • equicontinuity
  • Stationary optimal policy
  • uniformity in weak convergence

ASJC Scopus subject areas

  • Statistics, Probability and Uncertainty
  • Applied Mathematics
  • Statistics and Probability

Cite this

Controlled semi-Markov models under long-run average rewards. / Bhattacharya, Rabindra N; Majumdar, Mukul.

In: Journal of Statistical Planning and Inference, Vol. 22, No. 2, 1989, p. 223-242.

Research output: Contribution to journalArticle

@article{2eacb347c4e64cb294f31f46e7fb5cb0,
title = "Controlled semi-Markov models under long-run average rewards",
abstract = "Let the state space S be a Borel subset of a complete separable metric space, the action space A compact metric. Existence of stationary optimal policies is proved and a dynamic programming equation derived for general semi-Markov models under the long-run average reward criterion, focusing on it as a limiting case of optimization under discounting as the discount factor goes to one. This extends many earlier results. An example of Reed (1974) on harvesting a natural resource provides an application not covered by earlier results.",
keywords = "dynamic programming equation, equicontinuity, Stationary optimal policy, uniformity in weak convergence",
author = "Bhattacharya, {Rabindra N} and Mukul Majumdar",
year = "1989",
doi = "10.1016/0378-3758(89)90113-4",
language = "English (US)",
volume = "22",
pages = "223--242",
journal = "Journal of Statistical Planning and Inference",
issn = "0378-3758",
publisher = "Elsevier",
number = "2",

}

TY - JOUR

T1 - Controlled semi-Markov models under long-run average rewards

AU - Bhattacharya, Rabindra N

AU - Majumdar, Mukul

PY - 1989

Y1 - 1989

N2 - Let the state space S be a Borel subset of a complete separable metric space, the action space A compact metric. Existence of stationary optimal policies is proved and a dynamic programming equation derived for general semi-Markov models under the long-run average reward criterion, focusing on it as a limiting case of optimization under discounting as the discount factor goes to one. This extends many earlier results. An example of Reed (1974) on harvesting a natural resource provides an application not covered by earlier results.

AB - Let the state space S be a Borel subset of a complete separable metric space, the action space A compact metric. Existence of stationary optimal policies is proved and a dynamic programming equation derived for general semi-Markov models under the long-run average reward criterion, focusing on it as a limiting case of optimization under discounting as the discount factor goes to one. This extends many earlier results. An example of Reed (1974) on harvesting a natural resource provides an application not covered by earlier results.

KW - dynamic programming equation

KW - equicontinuity

KW - Stationary optimal policy

KW - uniformity in weak convergence

UR - http://www.scopus.com/inward/record.url?scp=38249021963&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=38249021963&partnerID=8YFLogxK

U2 - 10.1016/0378-3758(89)90113-4

DO - 10.1016/0378-3758(89)90113-4

M3 - Article

AN - SCOPUS:38249021963

VL - 22

SP - 223

EP - 242

JO - Journal of Statistical Planning and Inference

JF - Journal of Statistical Planning and Inference

SN - 0378-3758

IS - 2

ER -