# Average cost Markov control processes with weighted norms: existence of canonical policies

Evgueni Gordienko; Onésimo Hernández-Lerma

Applicationes Mathematicae (1995)

- Volume: 23, Issue: 2, page 199-218
- ISSN: 1233-7234

## Access Full Article

top## Abstract

top## How to cite

topGordienko, Evgueni, and Hernández-Lerma, Onésimo. "Average cost Markov control processes with weighted norms: existence of canonical policies." Applicationes Mathematicae 23.2 (1995): 199-218. <http://eudml.org/doc/219126>.

@article{Gordienko1995,

abstract = {This paper considers discrete-time Markov control processes on Borel spaces, with possibly unbounded costs, and the long run average cost (AC) criterion. Under appropriate hypotheses on weighted norms for the cost function and the transition law, the existence of solutions to the average cost optimality inequality and the average cost optimality equation are shown, which in turn yield the existence of AC-optimal and AC-canonical policies respectively.},

author = {Gordienko, Evgueni, Hernández-Lerma, Onésimo},

journal = {Applicationes Mathematicae},

keywords = {discounted cost; average cost optimality equation; long run average cost; (discrete-time) Markov control processes; average cost optimality inequality; weighted norms; Markov control processes; average cost (in-)equality; discrete; average optimal cost},

language = {eng},

number = {2},

pages = {199-218},

title = {Average cost Markov control processes with weighted norms: existence of canonical policies},

url = {http://eudml.org/doc/219126},

volume = {23},

year = {1995},

}

TY - JOUR

AU - Gordienko, Evgueni

AU - Hernández-Lerma, Onésimo

TI - Average cost Markov control processes with weighted norms: existence of canonical policies

JO - Applicationes Mathematicae

PY - 1995

VL - 23

IS - 2

SP - 199

EP - 218

AB - This paper considers discrete-time Markov control processes on Borel spaces, with possibly unbounded costs, and the long run average cost (AC) criterion. Under appropriate hypotheses on weighted norms for the cost function and the transition law, the existence of solutions to the average cost optimality inequality and the average cost optimality equation are shown, which in turn yield the existence of AC-optimal and AC-canonical policies respectively.

LA - eng

KW - discounted cost; average cost optimality equation; long run average cost; (discrete-time) Markov control processes; average cost optimality inequality; weighted norms; Markov control processes; average cost (in-)equality; discrete; average optimal cost

UR - http://eudml.org/doc/219126

ER -

## References

top- [1] A. Arapostathis, V. S. Borkar, E. Fernández-Gaucherand, M. K. Ghosh and S. I. Marcus, Discrete-time controlled Markov processes with average cost criterion: a survey, SIAM J. Control Optim. 31 (1993), 282-344. Zbl0770.93064
- [2] D. P. Bersekas and S. E. Shreve, Stochastic Optimal Control: The Discrete Time Case, Academic Press, New York, 1978.
- [3] E. B. Dynkin and A. A. Yushkevich, Controlled Markov Processes, Springer, New York, 1979. Zbl0073.34801
- [4] E. I. Gordienko, Controlled Markov processes with slowly varying characteristics. The problem of adaptive control. I, Soviet J. Comput. Syst. Sci. 23 (1985), 87-95. Zbl0595.93069
- [5] E. I. Gordienko and O. Hernández-Lerma, Average cost Markov control processes with weighted norms: value iteration, this volume, 219-237. Zbl0829.93068
- [6] O. Hernández-Lerma, Adaptive Markov Control Processes, Springer, New York, 1989.
- [7] O. Hernández-Lerma, Average optimality in dynamic programming on Borel spaces-unbounded costs and controls, Systems Control Lett. 17 (1991), 237-242. Zbl0771.90098
- [8] O. Hernández-Lerma and J. B. Lasserre, Average cost optimal policies for Markov control processes with Borel state space and unbounded costs, ibid. 15 (1990), 349-356. Zbl0723.93080
- [9] O. Hernández-Lerma and J. B. Lasserre, Discrete-Time Markov Control Processes, book in preparation. Zbl0724.93087
- [10] O. Hernández-Lerma, R. Montes-de-Oca and R. Cavazos-Cadena, Recurrence conditions for Markov decision processes with Borel state space: a survey, Ann. Oper. Res. 28 (1991), 29-46.
- [11] K. Hinderer, Foundations of Non-Stationary Dynamic Programming with Discrete Time Parameter, Lecture Notes Oper. Res. 33, Springer, New York, 1970. Zbl0202.18401
- [12] N. V. Kartashov, Inequalities in theorems of ergodicity and stability of Markov chains with common phase space. I, Theory Probab. Appl. 30 (1985), 247-259. Zbl0657.60088
- [13] N. V. Kartashov, Inequalities in theorems of ergodicity and stability of Markov chains with common phase space. II, ibid. 30 (1985), 507-515. Zbl0619.60066
- [14] N. V. Kartashov, Strongly stable Markov chains, J. Soviet Math. 34 (1986), 1493-1498. Zbl0594.60069
- [15] V. K. Malinovskiĭ, Limit theorems for Harris Markov chains, I, Theory Probab. Appl. 31 (1986), 269-285.
- [16] R. Montes-de-Oca and O. Hernández-Lerma, Conditions for average optimality in Markov control processes with unbounded costs and controls, J. Math. Systems Estim. Control 4 (1994), 1-19. Zbl0812.93077
- [17] R. Montes-de-Oca and O. Hernández-Lerma, Value iteration in average cost Markov control processes on Borel spaces, Acta Appl. Math., to appear. Zbl0843.93093
- [18] E. Nummelin, General Irreducible Markov Chains and Non-Negative Operators, Cambridge University Press, Cambridge, 1984. Zbl0551.60066
- [19] E. Nummelin and P. Tuominen, Geometric ergodicity of Harris recurrent Markov chains with applications to renewal theory, Stochastic Process. Appl. 12 (1982), 187-202. Zbl0484.60056
- [20] S. Orey, Limit Theorems for Markov Chain Transition Probabilities, Van Nostrand Reinhold, London, 1971. Zbl0295.60054
- [21] U. Rieder, Measurable selection theorems for optimization problems, Manuscripta Math. 24 (1978), 115-131. Zbl0385.28005
- [22] H. L. Royden, Real Analysis, 2nd ed., Macmillan, New York, 1971. Zbl0197.03501
- [23] M. Schäl, Conditions for optimality and for the limit of n-stage optimal policies to be optimal, Z. Wahrsch. Verw. Gebiete 32 (1975), 179-196. Zbl0316.90080
- [24] M. Schäl, Average optimality in dynamic programming with general state space, Math. Oper. Res. 18 (1993), 163-172. Zbl0777.90079
- [25] R. Sznajder and J. A. Filar, Some comments on a theorem of Hardy and Littlewood, J. Optim. Theory Appl. 75 (1992), 201-209.

## Citations in EuDML Documents

top- Xiaolong Zou, Xianping Guo, Another set of verifiable conditions for average Markov decision processes with Borel spaces
- Fernando Luque-Vásquez, Onésimo Hernández-Lerma, Semi-Markov control models with average costs
- Evgueni Gordienko, Onésimo Hernández-Lerma, Average cost Markov control processes with weighted norms: value iteration
- Evgueni I. Gordienko, Francisco Salem-Silva, Estimates of stability of Markov control processes with unbounded costs
- J. Minjárez-Sosa, Nonparametric adaptive control for discrete-time Markov processes with unbounded costs under average criterion
- Fernando Luque-Vásquez, J. Adolfo Minjárez-Sosa, Empirical approximation in Markov games under unbounded payoff: discounted and average criteria
- Oscar Vega-Amaya, Fernando Luque-Vásquez, Sample-path average cost optimality for semi-Markov control processes on Borel spaces: unbounded costs and mean holding times
- Onésimo Hernández-Lerma, Oscar Vega-Amaya, Infinite-horizon Markov control processes with undiscounted cost criteria: from average to overtaking optimality

## NotesEmbed ?

topTo embed these notes on your page include the following JavaScript code on your page where you want the notes to appear.