Docsity
Docsity

Prepare for your exams
Prepare for your exams

Study with the several resources on Docsity


Earn points to download
Earn points to download

Earn points by helping other students or get them with a premium plan


Guidelines and tips
Guidelines and tips

Markov Chains: Eigenanalysis and Absorbing States, Slides of Probability and Stochastic Processes

The concept of markov chains, focusing on eigenanalysis and absorbing states. Topics include finite state markov chains, analysis of aperiodic and periodic chains, n-step reachability, importance of n-step stochastic matrices, and asymptotic behavior using eigenvalues and eigenvectors.

Typology: Slides

2011/2012

Uploaded on 08/04/2012

saroj
saroj 🇮🇳

4.5

(2)

169 documents

1 / 19

Toggle sidebar

This page cannot be seen from the preview

Don't miss anything!

bg1
CS723 - Probability
and
Stochastic Processes
CS723 - Probability
and
Stochastic Processes
docsity.com
pf3
pf4
pf5
pf8
pf9
pfa
pfd
pfe
pff
pf12
pf13

Partial preview of the text

Download Markov Chains: Eigenanalysis and Absorbing States and more Slides Probability and Stochastic Processes in PDF only on Docsity!

CS723 - Probability

and

Stochastic Processes

CS723 - Probability

and

Stochastic Processes

  • Lecture No. 39Lecture No.

Markov Chains

P

n

n

If the Markov chain has a stationary probabilitydistribution

P

Hence,

is left-side eigenvector of P with an

Probability distribution at next indexeigenvalue of 1

Markov Chains

A recurrent Markov chain with a unique non-variantdistribution

   

   

  1. 0 1. 0 0
  2. 0 0 2. 0
  3. 0 0 6. 0 P

P

P

2

Markov Chains

DV

V

P

&

PV

V

D

DV

VP

x

x

x

x

x

x

x

x

x

0

0

0

0

0 0 P x x x

x

x

x

x

x

x

x x x P x x x

x

xP

1

1

3 c

(^3) b

3 a

2 c

(^2) b

2 a

1 c

1 b

1 a

c

b

a

3 c

2 c

1 c

(^3) b

(^2) b

1 b

3 a

2 a

1 a

3 a

2 a

1 a

a

3 a

2 a

1 a

   

       

   

   

   

Eigen-analysis using left sided row eigenvectors

Markov Chains

Finding P

n

=

P

n

using diagonalization of transition

probability matrix based on left sided eigenvectors

     

     

   

 

n N

n

n

n

n

n

n

D

V

D

V

V

D

V

P

V

D

V

P

DV

VP

2

1

1

1 1

Markov Chains

Eigen-analysis using right sided columneigenvectors

   

   

 

87 . 0

087 . 0

043 . 0

87 . 0

087 . 0

043 . 0

87 . 0

087 . 0

043 . 0

P

   

   

   

   

 

07 .

0

0

0

57 .

0

0

0

1

D

10 . 0

07 . 0

58 . 0

99 . 0

24 . 0

58 . 0

61 . 0

97 . 0

58 . 0

V

Markov Chains

Eigen-analysis using left sided roweigenvectors

P

   

   

   

   

558 .

795 .

237 .

644 .

113 .

76 .

99 .

099 .

05 .

V

07 .

0

0

0

57 .

0

0

0

1

D

Markov Chains

     

     

0

1

0

0

0

0

0

0

0

0

1

0

P

(^12)

(^12)

(^12)

1 2

    

    

 

(^5).

(^5).

(^5).

(^5).

(^5).

(^5).

(^5).

(^5).

31 .

63 .

63 .

(^31).

31 .

63 .

(^63).

(^31).

V

     

     

5 .

0

0

0

0

5 .

0

0

0

0

1

0

0

0

0

1

D

Markov Chains

P

1 2

(^12)

(^12)

1 2

     

     

      

     

1 6

(^13)

(^13)

(^16)

(^16)

(^13)

(^13)

(^16)

1 6

(^13)

(^13)

(^16)

(^16)

(^13)

(^13)

(^16)

k

(^16)

(^13)

(^13)

(^16)

(^16)

(^13)

(^13)

(^16)

(^16)

(^13)

(^13)

(^16)

(^16)

(^13)

(^13)

(^16)

k

) 1

(

P

Markov Chains

Random walk with weak reflection

    

    

0

5017 .

0

0

0

0

9967 . 0

0

0

0

0

1

D

     

     

2

10

(^99). 0

0

0

  1. 0 5. 0 0 5. 0 5. 0

0

1

0

P

Markov Chains

Random walk with weak reflection

     

      

2

10

(^99). 0

0

0

(^5).

0

(^5).

0

0

(^5).

0

(^5).

0

0

1

0

P

    

    

1681 . 0

3328 . 0

3328 . 0

1664 . 0

1681 . 0

3328 . 0

3328 . 0

1664 . 0

1681 . 0

3328 . 0

3328 . 0

1664 . 0

1681 . 0

3328 . 0

3328 . 0

1664 . 0

P

Markov Chains

Random walk with weak reflection

     

      

4

10

(^9999). 0

0

0

(^5).

0

(^5).

0

0

(^5).

0

(^5).

0

0

1

0

P

    

    

1667 . 0

3333 . 0

3333 . 0

1667 . 0

1667 . 0

3333 . 0

3333 . 0

1667 . 0

1667 . 0

3333 . 0

3333 . 0

1667 . 0

P