Docsity
Docsity

Prepare for your exams
Prepare for your exams

Study with the several resources on Docsity


Earn points to download
Earn points to download

Earn points by helping other students or get them with a premium plan


Guidelines and tips
Guidelines and tips

Understanding Mean with Examples & Union Bound: Linear Expectation & Random Variables, Lecture notes of Advanced Calculus

The concept of linear expectation in the context of discrete real-valued random variables. It covers the definition of expectation, the linearity of expectation theorem, and its applications to various distributions such as binomial and geometric distributions. Additionally, it introduces the union bound theorem, which is useful for upper bounding the probability of multiple events occurring. Examples and proofs to help students understand these concepts.

What you will learn

  • What is the linearity of expectation theorem and how is it used to compute the mean of complex random variables?
  • What is the Union Bound theorem and how is it used to upper bound the probability of multiple events occurring?
  • What is the definition of a discrete real-valued random variable?

Typology: Lecture notes

2021/2022

Uploaded on 09/12/2022

lakshmirnarman
lakshmirnarman 🇺🇸

5

(5)

221 documents

1 / 3

Toggle sidebar

This page cannot be seen from the preview

Don't miss anything!

bg1
LNMB: Randomized Algorithms
Handout: Linear of expectation Date: March 1, 2021
Instructor: Nikhil Bansal
We assume that the reader is already familiar with notions such as random variable, probability
space, events (our random variables and spaces will be discrete and finite). Let Xbe a discrete
real-valued random variable. The expectation or mean of Xis E[X] = PxxPr[X=x].
Example 1. If Xis an indicator random variable, which is 1 with probability pand 0 other-
wise. Then E[X] = Pr[X= 1] = p.
Example 2. Given a coin, with probability of heads p, let Xdenote the number of independent
coin tosses until the first head appears. Then Pr[X=i] = p(1 p)i1, for i= 1, . . .. Then
E[X] = X
i1
ip(1 p)i1= 1/p.
1 Linearity of Expectation
A very important (but deceptively simple) theorem is the following.
Theorem 1. Let X, Y be random variables. Then E[X+Y] = E[X] + E[Y].
Key point: This does not require any assumption on Xand Y(like they are independent). This
often allows us to compute the mean of very complex random variables in a much simpler way.
Proof.
E[X+Y] = X
x,y
(x+y) Pr[X=x, Y =y]
=X
x
x(X
y
Pr[X=x, Y =y]) + X
y
y(X
x
Pr[X=x, Y =y])
=X
x
xP r[X=x] + X
y
yPr[Y=y] = E[X] + E[Y].
1.1 Examples
1. Mean of binomial. Suppose you toss ncoins independently, and each has probability pon
turning up heads. Let Xdenote the number of heads. Then Xhas the binomial distribution
Bin(n, p) with Pr[X=k] = n
kpk(1 p)nk.
Now, one can compute E[X] by computing PkkPr[X=k] directly. By a much simpler
way would be the following. Let Xi= 1 if i-th coin is head and Xi= 0 otherwise. Then
X=PiXiand as E[Xi] = pfor each i, by linearity of expectation
E[X] = X
i
E[Xi] = np
1
pf3

Partial preview of the text

Download Understanding Mean with Examples & Union Bound: Linear Expectation & Random Variables and more Lecture notes Advanced Calculus in PDF only on Docsity!

LNMB: Randomized Algorithms Handout: Linear of expectation Date: March 1, 2021 Instructor: Nikhil Bansal

We assume that the reader is already familiar with notions such as random variable, probability space, events (our random variables and spaces will be discrete and finite). Let X be a discrete real-valued random variable. The expectation or mean of X is E[X] =

x x^ Pr[X^ =^ x].

Example 1. If X is an indicator random variable, which is 1 with probability p and 0 other- wise. Then E[X] = Pr[X = 1] = p.

Example 2. Given a coin, with probability of heads p, let X denote the number of independent coin tosses until the first head appears. Then Pr[X = i] = p(1 − p)i−^1 , for i = 1,.. .. Then

E[X] =

i≥ 1

ip(1 − p)i−^1 = 1/p.

1 Linearity of Expectation

A very important (but deceptively simple) theorem is the following.

Theorem 1. Let X, Y be random variables. Then E[X + Y ] = E[X] + E[Y ].

Key point: This does not require any assumption on X and Y (like they are independent). This often allows us to compute the mean of very complex random variables in a much simpler way.

Proof.

E[X + Y ] =

x,y

(x + y) Pr[X = x, Y = y]

x

x(

y

Pr[X = x, Y = y]) +

y

y(

x

Pr[X = x, Y = y])

x

xP r[X = x] +

y

y Pr[Y = y] = E[X] + E[Y ].

1.1 Examples

  1. Mean of binomial. Suppose you toss n coins independently, and each has probability p on turning up heads. Let X denote the number of heads. Then X has the binomial distribution Bin(n, p) with Pr[X = k] =

(n k

pk(1 − p)n−k. Now, one can compute E[X] by computing

k k^ Pr[X^ =^ k] directly.^ By a much simpler way would be the following. Let Xi = 1 if i-th coin is head and Xi = 0 otherwise. Then X =

i Xi^ and as^ E[Xi] =^ p^ for each^ i, by linearity of expectation

E[X] =

i

E[Xi] = np

  1. Consider the same setting above, but suppose that coin i has probability pi. Now, a direct computation will be much more messy, but using linearity of expectation we get

E[X] =

i

E[Xi] =

i

pi.

  1. Suppose there are n balls and n bins indexed 1,... , n. Pick a random permutation π (out of the n! ones) and assign ball i to bin π(i). Can a ball lucky if it is to a bin with the same index. E.g. if n = 2, the for the permutation 1 → 1 , 2 → 2, both balls are lucky, and for the permutation 1 → 2 , 2 → 1, no ball is lucky. What is the expected number of lucky balls? Attempt 1: Let L(π) denote the number of lucky balls in permutation π. A first attempt might be to try to compute Pr[L = i] for i = 0, 1 , 2 ,.. ., and evaluate Eπ[L] =

∑n i=0 i^ Pr[L^ =^ i]. But these probabilities get quite complicated and this becomes a mess. Attempt 2: Here is a much simpler way, using linearity of expectation. Let Xi(π) = 1 if ball i is lucky in π, and 0 otherwise. Then Xi is a random variable and for a random permutation π, we have Prπ[Xi = 1] = 1/n. This holds for all i. As X(π) =

i Xi(π) for any^ π, by linearity of expectation^ E[X] =^

i E[Xi] =^ n(1/n) = 1.

  1. Coupon collector. There are n different types of coupons, and at each time step I get a random coupon. How many times in expectation, until I see all the n types of coupons? Let Ti be the time when you see the i-th distinct coupon, and let Xi = Ti − Ti− 1 (and we use the convention T 0 = 0). In other words, Xi is the number of steps until you see a new coupon, after seeing i − 1 distinct coupons. Then we are interested in Tn = X 1 +... + Xn. After seeing i − 1 coupons, at each step there is (n − i + 1)/n probability of seeing a brand new coupon. So, Xi has a geometric distribution with mean p = (n − i + 1)/n (recall Example 1). So, E[Xi] = n/(n − i + 1) and hence by linearity of expectation

E[X] =

i

E[Xi] = n(

n

n − 1 +... + 1) = nHn ≈ n log n.

  1. (Tricky) Number of cycles in a random permutation. We view a permutation as a collection of cycles in the natural way. We are interested in the expected number of cycles in a random permutation. Again, calculating the probability of exactly k cycles, for k = 1, ..., n is quite hard. So we use linear of expectation is a smart way.. One way to count the number of cycles is if every vertex v pays 1/k(v), where k(v) is the length of the cycle containing v. This follows as the total payment per cycle is exactly 1. So for a vertex i, let Yi = 1/k if i lies in a cycle of length k. As discussed above, the number of cycles X(π) in any permutation π is equal to

∑n i=1 Yi(π), so^ E[X] =^

i E[Yi]. We now calculate E[Yi]. For any fixed element i, we claim that the probability that i lies in a cycle of length k in exactly 1/n, for each k = 1, 2 ,... , n (it is surprising this does not depend on k). To see this, imagine tracking the permutation starting from i. Chance it is of length