











Study with the several resources on Docsity
Earn points by helping other students or get them with a premium plan
Prepare for your exams
Study with the several resources on Docsity
Earn points to download
Earn points by helping other students or get them with a premium plan
Community
Ask the community for help and clear up your study doubts
Discover the best universities in your country according to Docsity users
Free resources
Download our free guides on studying techniques, anxiety management strategies, and thesis advice from Docsity tutors
The concept of markov chains, focusing on eigenanalysis and absorbing states. Topics include finite state markov chains, analysis of aperiodic and periodic chains, n-step reachability, importance of n-step stochastic matrices, and asymptotic behavior using eigenvalues and eigenvectors.
Typology: Slides
1 / 19
This page cannot be seen from the preview
Don't miss anything!
P
If the Markov chain has a stationary probabilitydistribution
P
Hence,
is left-side eigenvector of P with an
Probability distribution at next indexeigenvalue of 1
A recurrent Markov chain with a unique non-variantdistribution
2
DV
V
P
&
PV
V
D
DV
VP
x
x
x
x
x
x
x
x
x
0
0
0
0
0 0 P x x x
x
x
x
x
x
x
x x x P x x x
x
xP
1
1
3 c
(^3) b
3 a
2 c
(^2) b
2 a
1 c
1 b
1 a
c
b
a
3 c
2 c
1 c
(^3) b
(^2) b
1 b
3 a
2 a
1 a
3 a
2 a
1 a
a
3 a
2 a
1 a
Eigen-analysis using left sided row eigenvectors
Finding P
n
=
P
n
using diagonalization of transition
probability matrix based on left sided eigenvectors
n N
n
n
n
n
n
n
D
V
D
V
V
D
V
P
V
D
V
P
DV
VP
2
1
1
1 1
Eigen-analysis using right sided columneigenvectors
87 . 0
087 . 0
043 . 0
87 . 0
087 . 0
043 . 0
87 . 0
087 . 0
043 . 0
P
07 .
0
0
0
57 .
0
0
0
1
D
10 . 0
07 . 0
58 . 0
99 . 0
24 . 0
58 . 0
61 . 0
97 . 0
58 . 0
V
Eigen-analysis using left sided roweigenvectors
558 .
795 .
237 .
644 .
113 .
76 .
99 .
099 .
05 .
V
07 .
0
0
0
57 .
0
0
0
1
D
0
1
0
0
0
0
0
0
0
0
1
0
P
(^12)
(^12)
(^12)
1 2
(^5).
(^5).
(^5).
(^5).
(^5).
(^5).
(^5).
(^5).
31 .
63 .
63 .
(^31).
31 .
63 .
(^63).
(^31).
V
5 .
0
0
0
0
5 .
0
0
0
0
1
0
0
0
0
1
D
1 2
(^12)
(^12)
1 2
1 6
(^13)
(^13)
(^16)
(^16)
(^13)
(^13)
(^16)
1 6
(^13)
(^13)
(^16)
(^16)
(^13)
(^13)
(^16)
k
(^16)
(^13)
(^13)
(^16)
(^16)
(^13)
(^13)
(^16)
(^16)
(^13)
(^13)
(^16)
(^16)
(^13)
(^13)
(^16)
k
) 1
(
P
Random walk with weak reflection
0
5017 .
0
0
0
0
9967 . 0
0
0
0
0
1
D
2
10
(^99). 0
0
0
0
1
0
P
Random walk with weak reflection
2
10
(^99). 0
0
0
(^5).
0
(^5).
0
0
(^5).
0
(^5).
0
0
1
0
P
1681 . 0
3328 . 0
3328 . 0
1664 . 0
1681 . 0
3328 . 0
3328 . 0
1664 . 0
1681 . 0
3328 . 0
3328 . 0
1664 . 0
1681 . 0
3328 . 0
3328 . 0
1664 . 0
P
Random walk with weak reflection
4
10
(^9999). 0
0
0
(^5).
0
(^5).
0
0
(^5).
0
(^5).
0
0
1
0
P
1667 . 0
3333 . 0
3333 . 0
1667 . 0
1667 . 0
3333 . 0
3333 . 0
1667 . 0
1667 . 0
3333 . 0
3333 . 0
1667 . 0
P