CS723 - Probability
and
Stochastic Processes
CS723 - Probability
and
Stochastic Processes
docsity.com
Prepare for your exams
Study with the several resources on Docsity
Earn points to download
Earn points by helping other students or get them with a premium plan
Guidelines and tips
Prepare for your exams
Study with the several resources on Docsity
Earn points to download
Earn points by helping other students or get them with a premium plan
Community
Ask the community
Ask the community for help and clear up your study doubts
University Rankings
Discover the best universities in your country according to Docsity users
Free resources
Our save-the-student-ebooks!
Download our free guides on studying techniques, anxiety management strategies, and thesis advice from Docsity tutors
Markov Chains: Eigenanalysis and Absorbing States, Slides of Probability and Stochastic Processes
The concept of markov chains, focusing on eigenanalysis and absorbing states. Topics include finite state markov chains, analysis of aperiodic and periodic chains, n-step reachability, importance of n-step stochastic matrices, and asymptotic behavior using eigenvalues and eigenvectors.
Typology: Slides
2011/2012
1 / 19
This page cannot be seen from the preview
Don't miss anything!
Related documents
Partial preview of the text
Download Markov Chains: Eigenanalysis and Absorbing States and more Slides Probability and Stochastic Processes in PDF only on Docsity!
CS723 - Probability
and
Stochastic Processes
CS723 - Probability
and
Stochastic Processes
- Lecture No. 39Lecture No.
Markov Chains
P
n
n
If the Markov chain has a stationary probabilitydistribution
P
Hence,
is left-side eigenvector of P with an
Probability distribution at next indexeigenvalue of 1
Markov Chains
A recurrent Markov chain with a unique non-variantdistribution
- 0 1. 0 0
- 0 0 2. 0
- 0 0 6. 0 P
P
P
2
Markov Chains
DV
V
P
&
PV
V
D
DV
VP
x
x
x
x
x
x
x
x
x
0
0
0
0
0 0 P x x x
x
x
x
x
x
x
x x x P x x x
x
xP
1
1
3 c
(^3) b
3 a
2 c
(^2) b
2 a
1 c
1 b
1 a
c
b
a
3 c
2 c
1 c
(^3) b
(^2) b
1 b
3 a
2 a
1 a
3 a
2 a
1 a
a
3 a
2 a
1 a
Eigen-analysis using left sided row eigenvectors
Markov Chains
Finding P
n
=
P
n
using diagonalization of transition
probability matrix based on left sided eigenvectors
n N
n
n
n
n
n
n
D
V
D
V
V
D
V
P
V
D
V
P
DV
VP
2
1
1
1 1
Markov Chains
Eigen-analysis using right sided columneigenvectors
87 . 0
087 . 0
043 . 0
87 . 0
087 . 0
043 . 0
87 . 0
087 . 0
043 . 0
P
07 .
0
0
0
57 .
0
0
0
1
D
10 . 0
07 . 0
58 . 0
99 . 0
24 . 0
58 . 0
61 . 0
97 . 0
58 . 0
V
Markov Chains
Eigen-analysis using left sided roweigenvectors
P
558 .
795 .
237 .
644 .
113 .
76 .
99 .
099 .
05 .
V
07 .
0
0
0
57 .
0
0
0
1
D
Markov Chains
0
1
0
0
0
0
0
0
0
0
1
0
P
(^12)
(^12)
(^12)
1 2
(^5).
(^5).
(^5).
(^5).
(^5).
(^5).
(^5).
(^5).
31 .
63 .
63 .
(^31).
31 .
63 .
(^63).
(^31).
V
5 .
0
0
0
0
5 .
0
0
0
0
1
0
0
0
0
1
D
Markov Chains
P
1 2
(^12)
(^12)
1 2
1 6
(^13)
(^13)
(^16)
(^16)
(^13)
(^13)
(^16)
1 6
(^13)
(^13)
(^16)
(^16)
(^13)
(^13)
(^16)
k
(^16)
(^13)
(^13)
(^16)
(^16)
(^13)
(^13)
(^16)
(^16)
(^13)
(^13)
(^16)
(^16)
(^13)
(^13)
(^16)
k
) 1
(
P
Markov Chains
Random walk with weak reflection
0
5017 .
0
0
0
0
9967 . 0
0
0
0
0
1
D
2
10
(^99). 0
0
0
- 0 5. 0 0 5. 0 5. 0
0
1
0
P
Markov Chains
Random walk with weak reflection
2
10
(^99). 0
0
0
(^5).
0
(^5).
0
0
(^5).
0
(^5).
0
0
1
0
P
1681 . 0
3328 . 0
3328 . 0
1664 . 0
1681 . 0
3328 . 0
3328 . 0
1664 . 0
1681 . 0
3328 . 0
3328 . 0
1664 . 0
1681 . 0
3328 . 0
3328 . 0
1664 . 0
P
Markov Chains
Random walk with weak reflection
4
10
(^9999). 0
0
0
(^5).
0
(^5).
0
0
(^5).
0
(^5).
0
0
1
0
P
1667 . 0
3333 . 0
3333 . 0
1667 . 0
1667 . 0
3333 . 0
3333 . 0
1667 . 0
1667 . 0
3333 . 0
3333 . 0
1667 . 0
P