Open main menu

Markov chains on a measurable state space

A Markov chain on a measurable state space is a discrete-time-homogenous Markov chain with a measurable space as state space.

Contents

HistoryEdit

The definition of Markov chains has evolved during the 20th century. In 1953 the term Markov chain was used for stochastic processes with discrete or continuous index set, living on a countable or finite state space, see Doob.[1] or Chung.[2] Since the late 20th century it became more popular to consider a Markov chain as a stochastic process with discrete index set, living on a measurable state space.[3][4][5]

DefinitionEdit

Denote with   a measurable space and with   a Markov kernel with source and target  . A stochastic process   on   is called a time homogeneous Markov chain with Markov kernel   and start distribution   if

 

is satisfied for any  . One can construct for any Markov kernel and any probability measure an associated Markov chain.[4]

Remark about Markov kernel integrationEdit

For any measure   we denote for  -integrable function   the Lebesgue integral as  . For the measure   defined by   we used the following notation:

 

Basic propertiesEdit

Starting in a single pointEdit

If   is a Dirac measure in  , we denote for a Markov kernel   with starting distribution   the associated Markov chain as   on   and the expectation value

 

for a  -integrable function  . By definition, we have then  .

We have for any measurable function   the following relation:[4]

 

Family of Markov kernelsEdit

For a Markov kernel   with starting distribution   one can introduce a family of Markov kernels   by

 

for   and  . For the associated Markov chain   according to   and   one obtains

 .

Stationary measureEdit

A probability measure   is called stationary measure of a Markov kernel   if

 

holds for any  . If   on   denotes the Markov chain according to a Markov kernel   with stationary measure  , then all   have the same probability distribution, namely:

 

for any  .

ReversibilityEdit

A Markov kernel   is called reversible according to a probability measure   if

 

holds for any  . Replacing   shows that if   is reversible according to  , then   must be a stationary measure of  .

ReferencesEdit

  1. ^ Joseph L. Doob: Stochastic Processes. New York: John Wiley & Sons, 1953.
  2. ^ Kai L. Chung: Markov Chains with Stationary Transition Probabilities. Second edition. Berlin: Springer-Verlag, 1974.
  3. ^ Sean Meyn and Richard L. Tweedie: Markov Chains and Stochastic Stability. 2nd edition, 2009.
  4. ^ a b c Daniel Revuz: Markov Chains. 2nd edition, 1984.
  5. ^ Rick Durrett: Probability: Theory and Examples. Fourth edition, 2005.