



Study with the several resources on Docsity
Earn points by helping other students or get them with a premium plan
Prepare for your exams
Study with the several resources on Docsity
Earn points to download
Earn points by helping other students or get them with a premium plan
Community
Ask the community for help and clear up your study doubts
Discover the best universities in your country according to Docsity users
Free resources
Download our free guides on studying techniques, anxiety management strategies, and thesis advice from Docsity tutors
Main points of this past exam are: Posterior Distribution, Conditional Probability, Algebraic Expression, Joint Distribution, Hidden States, Transition Diagram, State Transition, Hmm State, Probability Tables, Missing Entries
Typology: Exams
1 / 6
This page cannot be seen from the preview
Don't miss anything!
Unfortunately during spring due to illness and allergies, Billy is unable to distinguish the cause (X) of his symptoms which could be: coughing (C), sneezing (S), and temperature (T). If he is able to determine the cause with a reasonable accuracy, it would be beneficial for him to take either “Robitussin DM” or Benedryll. He has asked you to help him formulate his problem as a Bayesian Network. At any point in time, Billy can either be sick (X = sick), allergic (X = allergic), or well (X = well).
Figure 1: Bayes’ Net
(a) (2 points)List all independence and conditional independence relationships implied by this Bayes’ net.
(b) (2 points)Write the algebraic expression for P (C, S) in terms of the joint distribution P (X, C, T, S).
x
t
P (X = x, C, T = t, S)
(c) (2 points)What probability and conditional probability tables must we store for this Bayes’ net?
(d) (2 points)Derive an expression for the posterior distribution over X given whether Billy is coughing (C) and sneezing (S).
P P^ t^ P^ (X;C;T^ =t;S^ ) x
P t P^ (X^ =x;C;T^ =t;S^ ) It is also correct to rewrite the above using the fact P (X, C, T, S) = P (X)P (C X)P (T X)P (S C, T ).
(e) (2 points)Derive an expression for the likelihood of observing that he is coughing (C) and sneezing (S) given X.
P t P^ (X;C;T^ =t;S^ ) P (X )
(b) (2 points)Imagine you observe the sequence quiet, sneeze, sneeze. What is the probability that you were well all three days and observed these effects?
P (X 1 = w, X 2 = w, X 3 = w E 1 = q, E 2 = s, E 3 = s)
=
P (X 1 = w, X 2 = w, X 3 = w, E 1 = q, E 2 = s, E 3 = s) P (E 1 = q, E 2 = s, E 3 = s)
P (X 1 = w, X 2 = w, X 3 = w, E 1 = q, E 2 = s, E 3 = s) = 0. 60 0. 5 0. 5 0. 9 0. 1 0 .1 = 0. 00135
P (E 1 = q, E 2 = s, E 3 = s) =
x (^1)
x (^2)
x (^3)
P (X 1 = x 1 , X 2 = x 2 , X 3 = x 3 , E 1 = q, E 2 = s, E 3 = s)
P (X 1 = w, X 2 = w, X 3 = w E 1 = q, E 2 = s, E 3 = s) = 0. 00135 /P (E 1 = q, E 2 = s, E 3 = s)
(c) (2 points)What is the posterior distribution over your state on day 2 (X 2 ) if E 1 = quiet, E 2 = sneeze?
This is the filtering problem.
P (X 1 E 1 = q) P (X 1 )P (E 1 = q X 1 ) Running one step of filtering, we have (in the order of well, allergy, cold): P (X 1 E 1 = q) = [0. 885 0. 041 0 .074]
x 1 P^ (E^2 =^ s^ X^2 )P^ (X^2 X^1 =^ x^1 )P^ (X^1 =^ x^1 )P^ (E^1 =^ q^ X^1 =^ x^1 ) = P (E 2 = s X 2 )
x 1 P^ (X^2 X^1 =^ x^1 )P^ (X^1 =^ x^1 )P^ (E^1 =^ q^ X^1 =^ x^1 ) P (E 2 = s X 2 )
x 1 P^ (X^2 X^1 =^ x^1 )P^ (X^1 =^ x^1 E^1 =^ q) Using the equation above, after two steps of filtering, we have P (X 2 E 1 = q, E 2 = s) = [0. 104 0. 580 0 .316]
(d) (2 points)What is the posterior distribution over your state on day 3 (X 3 ) if E 1 = quiet, E 2 = sneeze, E 3 = sneeze? We simply need to compute another step of filtering, using our answer from part (c) above.
P (X 3 E 1 = q, E 2 = s, E 3 = s) P (E 3 = s X 3 )
x (^2)
P (X 3 X 2 = x 2 )P (X 2 E 1 = q, E 2 = s)
After normalizing, we have P (X 3 E 1 = q, E 2 = s, E 3 = s) = [0. 077 0. 652 0 .271]
Figure 4: MDP
Consider the above MDP, representing a robot on a circular wheel. The wheel is divided into eight states and the available actions are to move clockwise or counterclockwise. The robot has a poor sense of direction and will move with probability 0.6 in the intended direction and with probability 0.4 in the opposite direction. All states have reward zero, except the terminal states which have rewards 1 and +1 as shown. The discount factor is γ = 0.9.
(5 points)Compute the numeric values of the state-value function V (s) for states s1 through s6 (compute V (s1), V (s2), ...). Show your work below.
By symmetry, we know that V (s1) = V (s4), V (s2) = V (s5), and V (s3) = V (s6). To compute V (s1), V (s2), and V (s3), we could use value iteration. However, it is much easier to solve directly using the Bellman equations. It is clear that the optimal policy is to move toward the +1 reward.
V (s1) = 0. 4 γ ( 1) + 0. 6 γ V (s2) V (s2) = 0. 4 γ V (s1) + 0. 6 γ V (s3) V (s3) = 0. 4 γ V (s2) + 0. 6 γ (+1) Solving this system of equations yields
V (s1) = 0. 217 V (s2) = 0. 265 V (s3) = 0. 635