Occupation laws for some time-nonhomogeneous Markov chains

Zach Dietz, Sunder Sethuraman

Research output: Contribution to journalArticle

4 Citations (Scopus)

Abstract

We consider finite-state time-nonhomogeneous Markov chains whose transition matrix at time n is I + G/nζ where G is a "generator" matrix, that is G(i, j) > 0 for i, j distinct, and G(i, i) = -∑k≠i G(i, k), and ζ > 0 is a strength parameter. In these chains, as time grows, the positions are less and less likely to change, and so form simple models of age-dependent time-reinforcing schemes. These chains, however, exhibit a trichotomy of occupation behaviors depending on parameters. We show that the average occupation or empirical distribution vector up to time n, when variously 0 < ζ < 1, ζ > 1 or ζ = 1, converges in probability to a unique "stationary" vector V G, converges in law to a nontrivial mixture of point measures, or converges in law to a distribution μG with no atoms and full support on a simplex respectively, as n ↑ ∞. This last type of limit can be interpreted as a sort of "spreading" between the cases 0 < ζ 1 and ζ > 1. In particular, when G is appropriately chosen, μG is a Dirichlet distribution, reminiscent of results in Pólya urns.

Original languageEnglish (US)
Pages (from-to)661-683
Number of pages23
JournalElectronic Journal of Probability
Volume12
StatePublished - May 15 2007
Externally publishedYes

Fingerprint

Markov chain
Converge
Dirichlet Distribution
Empirical Distribution
Transition Matrix
Sort
Likely
Generator
Distinct
Model

Keywords

  • Dirichlet distribution
  • Laws of large numbers
  • Markov
  • Nonhomogeneous
  • Occupation
  • Reinforcement

ASJC Scopus subject areas

  • Statistics and Probability

Cite this

Occupation laws for some time-nonhomogeneous Markov chains. / Dietz, Zach; Sethuraman, Sunder.

In: Electronic Journal of Probability, Vol. 12, 15.05.2007, p. 661-683.

Research output: Contribution to journalArticle

@article{c6da0000b16344c29c5966882f3ae93e,
title = "Occupation laws for some time-nonhomogeneous Markov chains",
abstract = "We consider finite-state time-nonhomogeneous Markov chains whose transition matrix at time n is I + G/nζ where G is a {"}generator{"} matrix, that is G(i, j) > 0 for i, j distinct, and G(i, i) = -∑k≠i G(i, k), and ζ > 0 is a strength parameter. In these chains, as time grows, the positions are less and less likely to change, and so form simple models of age-dependent time-reinforcing schemes. These chains, however, exhibit a trichotomy of occupation behaviors depending on parameters. We show that the average occupation or empirical distribution vector up to time n, when variously 0 < ζ < 1, ζ > 1 or ζ = 1, converges in probability to a unique {"}stationary{"} vector V G, converges in law to a nontrivial mixture of point measures, or converges in law to a distribution μG with no atoms and full support on a simplex respectively, as n ↑ ∞. This last type of limit can be interpreted as a sort of {"}spreading{"} between the cases 0 < ζ 1 and ζ > 1. In particular, when G is appropriately chosen, μG is a Dirichlet distribution, reminiscent of results in P{\'o}lya urns.",
keywords = "Dirichlet distribution, Laws of large numbers, Markov, Nonhomogeneous, Occupation, Reinforcement",
author = "Zach Dietz and Sunder Sethuraman",
year = "2007",
month = "5",
day = "15",
language = "English (US)",
volume = "12",
pages = "661--683",
journal = "Electronic Journal of Probability",
issn = "1083-6489",
publisher = "Institute of Mathematical Statistics",

}

TY - JOUR

T1 - Occupation laws for some time-nonhomogeneous Markov chains

AU - Dietz, Zach

AU - Sethuraman, Sunder

PY - 2007/5/15

Y1 - 2007/5/15

N2 - We consider finite-state time-nonhomogeneous Markov chains whose transition matrix at time n is I + G/nζ where G is a "generator" matrix, that is G(i, j) > 0 for i, j distinct, and G(i, i) = -∑k≠i G(i, k), and ζ > 0 is a strength parameter. In these chains, as time grows, the positions are less and less likely to change, and so form simple models of age-dependent time-reinforcing schemes. These chains, however, exhibit a trichotomy of occupation behaviors depending on parameters. We show that the average occupation or empirical distribution vector up to time n, when variously 0 < ζ < 1, ζ > 1 or ζ = 1, converges in probability to a unique "stationary" vector V G, converges in law to a nontrivial mixture of point measures, or converges in law to a distribution μG with no atoms and full support on a simplex respectively, as n ↑ ∞. This last type of limit can be interpreted as a sort of "spreading" between the cases 0 < ζ 1 and ζ > 1. In particular, when G is appropriately chosen, μG is a Dirichlet distribution, reminiscent of results in Pólya urns.

AB - We consider finite-state time-nonhomogeneous Markov chains whose transition matrix at time n is I + G/nζ where G is a "generator" matrix, that is G(i, j) > 0 for i, j distinct, and G(i, i) = -∑k≠i G(i, k), and ζ > 0 is a strength parameter. In these chains, as time grows, the positions are less and less likely to change, and so form simple models of age-dependent time-reinforcing schemes. These chains, however, exhibit a trichotomy of occupation behaviors depending on parameters. We show that the average occupation or empirical distribution vector up to time n, when variously 0 < ζ < 1, ζ > 1 or ζ = 1, converges in probability to a unique "stationary" vector V G, converges in law to a nontrivial mixture of point measures, or converges in law to a distribution μG with no atoms and full support on a simplex respectively, as n ↑ ∞. This last type of limit can be interpreted as a sort of "spreading" between the cases 0 < ζ 1 and ζ > 1. In particular, when G is appropriately chosen, μG is a Dirichlet distribution, reminiscent of results in Pólya urns.

KW - Dirichlet distribution

KW - Laws of large numbers

KW - Markov

KW - Nonhomogeneous

KW - Occupation

KW - Reinforcement

UR - http://www.scopus.com/inward/record.url?scp=34249030618&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=34249030618&partnerID=8YFLogxK

M3 - Article

AN - SCOPUS:34249030618

VL - 12

SP - 661

EP - 683

JO - Electronic Journal of Probability

JF - Electronic Journal of Probability

SN - 1083-6489

ER -