Joint Probability Distribution and Conditional Independence in Weather and Fun Scenarios, Study notes of Probability and Statistics

Download Joint Probability Distribution and Conditional Independence in Weather and Fun Scenarios and more Study notes Probability and Statistics in PDF only on Docsity!

Joint Probability Distribution

Sunny Cloudy Rainy Snowy

Yes 0.25 0.15 0.05 0.

No 0.05 0.1 0.25 0.

Weather

Have

Fun?

Distribution vs. Probability

discrete random variable

P(A) denotes a distribution

Probabilities for each value

P(A | B) denotes a different distribution across the

same values

P(A = a

2

) denotes the probability that A takes on the

value a

2

P(A = a

2

, B = b

3

) denotes the probability that A takes on

the value a

2

and B takes on the value b

3

Distribution

P(Weather | Fun=Yes)

P(Weather = Sunny | Fun = Yes) = 0.431+

0

Sunny Cloudy Rainy Snowy

Weather given Fun=Yes

Distribution Statements

P(A) = P(A | C)

vs

P(A) = P(A | C=c

5

Joint Probability Distribution

Sunny Cloudy Rainy Snowy

Yes 0.25 0.15 0.05 0.

No 0.05 0.1 0.25 0.

Weather

Have

Fun?

Now add RV for Time Of Day in 6hr

blocks

Conditional Independence

• Model the disease influencing the symptoms

• But no symptom interactions given the

disease

• Conditional independence:

P(A|B,C) = P(A|C)

“A is conditionally independent of B given C”

More Conditional Independence

If A and B are conditionally independent given C

C can still depend on both A and B

P(C|A)  P(C) and P(C|B)  P(C)

A and B are not necessarily independent:

P(B|A)  P(B) and P(A|B)  P(A)

But, Given C,

A and B do not influence each other

Knowledge of C separates A and B

Suppose we do NOT know C,

then A and B MAY influence each other

(think about this one until it’s intuitive

• what does “influence” mean here?)

Suppose that Symptoms are Conditionally

Independent given the Disease

• We know John has a cold
• Congestion, a sore throat, headache, rash

are more likely but not necessary

• We discover he does in fact have a sore throat
• Is he now more or less likely to also have a

headache?

Dentist Example

A

C

B

3 Boolean Random Variables:

C – Patient has a cavity

A – Patient reports a

toothache

B – Dentist’s probe catches

on tooth

C directly influences A and B

A and B are conditionally independent given C

CPT at each node specifies a probability distribution for each context

(configuration of parents)

How many numbers do we need for the full joint?

How many for the Bayesian net?

How Many Numbers

Joint:

3

• 1 = 7 (why?)

Bayes Net:

Distribution(s) at C

P(cavity), P(cavity)

Need one number

Distribution(s) at A

P(ache|cavity), P(ache|cavity)

P(ache| cavity), P(ache| cavity)

Distributions at B, like A

1 + 2 + 2 = 5 (a savings of 2!)

A

C

B

How Many Numbers?

• Suppose A,B,C,D,W are trinary

• How many numbers for

the joint?

• How many numbers

for the BN?

• Suppose A is binary,

B is trinary, C takes on 4 values

D takes on 5, W takes on 6?

A

C

B

D

W

5

• 1 = 242

W 5, D 4, C 3, A 4, B 8 = 24

Notation

• W, C, D, A, B are random variables
  • C (binary presence of cavity) ranges over values
  • C’s values might be Yes or No; 1 or 0; c 1

or c 2

• Probability distribution: P(C)
• Probability value: P(C=c 1

) or P(c

1

) or P(c)

• P(A,B,C,W,D) full joint probability distribution
• P(a,b,c,w,d) some particular entry in the joint