Rahul Sharma (Editor)

Additive Markov chain

Updated on
Edit
Like
Comment
Share on FacebookTweet on TwitterShare on LinkedInShare on Reddit

In probability theory, an additive Markov chain is a Markov chain with an additive conditional probability function. Here the process is a discrete-time Markov chain of order m and the transition probability to a state at the next time is a sum of functions, each depending on the next state and one of the m previous states.

Contents

Definition

An additive Markov chain of order m is a sequence of random variables X1, X2, X3, ..., possessing the following property: the probability that a random variable Xn has a certain value xn under the condition that the values of all previous variables are fixed depends on the values of m previous variables only (Markov chain of order m), and the influence of previous variables on a generated one is additive,

Pr ( X n = x n | X n − 1 = x n − 1 , X n − 2 = x n − 2 , … , X n − m = x n − m ) = ∑ r = 1 m f ( x n , x n − r , r ) .

Binary case

A binary additive Markov chain is where the state space of the chain consists on two values only, Xn ∈ { x1, x2 }. For example, Xn ∈ { 0, 1 }. The conditional probability function of a binary additive Markov chain can be represented as

Pr ( X n = 1 | X n − 1 = x n − 1 , X n − 2 = x n − 2 , … ) = X ¯ + ∑ r = 1 m F ( r ) ( x n − r − X ¯ ) , Pr ( X n = 0 | X n − 1 = x n − 1 , X n − 2 = x n − 2 , … ) = 1 − Pr ( X n = 1 | X n − 1 = x n − 1 , X n − 2 = x n − 2 , … ) .

Here X ¯ is the probability to find Xn = 1 in the sequence and F(r) is referred to as the memory function. The value of X ¯ and the function F(r) contain all the information about correlation properties of the Markov chain.

Relation between the memory function and the correlation function

In the binary case, the correlation function between the variables X n and X k of the chain depends on the distance n − k only. It is defined as follows:

K ( r ) = ⟨ ( X n − X ¯ ) ( X n + r − X ¯ ) ⟩ = ⟨ X n X n + r ⟩ − X ¯ 2 ,

where the symbol ⟨ ⋯ ⟩ denotes averaging over all n. By definition,

K ( − r ) = K ( r ) , K ( 0 ) = X ¯ ( 1 − X ¯ ) .

There is a relation between the memory function and the correlation function of the binary additive Markov chain:

K ( r ) = ∑ s = 1 m K ( r − s ) F ( s ) , r = 1 , 2 , … .

References

Additive Markov chain Wikipedia
(Text) CC BY-SA