Download Lecture 10: Condensers in Complexity Theory and Pseudorandomness and more Schemes and Mind Maps Construction in PDF only on Docsity!
Lecture 10: Condensers
Topics in Complexity Theory and Pseudorandomness (Spring 2013) Rutgers University Swastik Kopparty Scribes: Ian Mertz, Yun Kuen Cheung
1 A Quick Recap
Last class we were studying randomness extractors, which convert impure sources of randomness into uniformly distributed random variables (which could be then be used to fuel our randomized algorithms, say). The impure sources from which we want to extract randomness must have some amount of randomness in them, and the measure of randomness that we used for this purpose was min-entropy.
Definition 1. The min-entropy of a distribution X, written as H∞(X) is defined by: H∞(X) ≤ k iff ∀x,Pr[X = x] ≤ 2 −k. The units of min-entropy are bits.
Note that if X is distrubuted over { 0 , 1 }n, then H∞(X) = n iff X = Un.
Definition 2. A function E : { 0 , 1 }n^ × { 0 , 1 }d^ → { 0 , 1 }m^ is a (k, ) extractor if for all distributions X on { 0 , 1 }n^ with H∞(X) ≥ k, E(X, Ud) is -close to Um in statistical distance.
Thus if we have a sample w ∈ { 0 , 1 }n^ which came from a k-bit min-entropy source, then by investing some d bits of pure randomness, we can get ourselves m bits of almost-pure randomness.
This would be interesting only if m > d, and indeed that is the case.
Theorem 3. Extractors exist with d = log(n − k) + 2 log( (^1) ) + O(1), m = k + d − 2 log( (^1) ).
Thus one can get back all d bits of randomness plus almost all of the k-bits of entropy that lie in the weak source!
This motivates the problem of constructing an extractor explicitly. Last lecture we constructed an extractor with m = k + d − 2 log (^1) (which is optimal), but which required d = 2n seed length. In this lecture we will construct an extractor with both seed length and output length optimal upto constant factors (i.e., d = O(log n) and m = Ω(k + d) ).
2 Extractor for High Min-Entropy Sources
We first construct an extractor which works when k = 0. 999 n.
This will be based on the following Chernoff-like bound for random walks on expanders.
Theorem 4. Let G be a C-regular, λ-absolute eigenvalue expander. Pick x 0 ∈ V uniformly. Let
x 1... xD be a random walk on G. Then Pr[ #i^ for which D xi^ ∈^ S> (^) ||SV || + ] ≤ e− (^2) D( 1 − (^) Cλ )/ 4 .
Since the first vertex x 0 of this walk is picked uniformly from V , it is easy to see that each xi is distributed uniformly over V. Thus the expected fraction of i ∈ [D] for which xi ∈ S equals (^) ||SV ||. The above theorem says that the xi sample with error bounds which are as nearly good as what one would get if they were sampled independently.
The way we get an extractor out of this is the following. We will first fix a large constant degree expander graph G. We identify { 0 , 1 }n^ with the space of random walks of length D on G. We will use the seed to index an integer i ∈ [D]. The output of the extractor E(x, i), given an input x ∈ { 0 , 1 }n^ and i ∈ [D], will be the ith vertex of the random walk corresponding to x.
Thus the bipartite graph associated to this extractor looks like the following:
Figure 1: The expander walk extractor
The total number of walks = 2n^ = |V |CD. The total number of vertices = 2m^ = |V |. Choose D = Θ(n) (so that the seed length d = log n + O(1)).
We now prove that this is (k, ) extractor with k = (1 − Ω(^2 )) · n. Let X be a distribution over { 0 , 1 }n^ with H∞(X) ≥ k. Without loss of generality, we assume that X is a flat distribution.
Suppose E(X, Ud) is not -close to Un. Then by the “distinguisher” characterization of statistical distance, there exists a set S ⊆ { 0 , 1 }m^ such that:
Pr[E(X, Ud) ∈ S] − Pr[Um ∈ S] ≥ .
Then we have that
Prx∈X,i∈Ud [E(x, i) ∈ S] ≥
|S|
|V |
This implies that most x ∈ X are bad:
Prx∈X
[
Pri∈Ud [E(x, i) ∈ S] ≥
|S|
|V |
]
(this kind of manipulation shows up in many many places and is very useful).
In other words, there exist lossless condensers that recovers all the original entropy of the weakly random variable, in addition to all the bits of randomness used.
Given a condenser, one can form the associated (2n, 2 m) bipartite graph. For a lossless condenser, the associated bipartite graph is an expander where each left vertex has degree D, and where sets of size k on the left have kD(1 − ) neighbors (such expanders are called lossless expanders). Furthermore, if the bipartite graph associated with a condenser is a lossless expander, the condenser is itself lossless. (Prove this!)
4 Simple Univariate Polynomial Condenser
Here we construct two condensers using finite fields and polynomials, the first weaker condenser motivating the second optimal one. Their analysis involves some very nice ideas from linear algebra.
4.1 Version 1: condensers from Reed-Solomon codes
Fix a prime power q and positive integer c. Let P be the set of polynomials with coefficients in Fq and degree at most c − 1, and identify it with { 0 , 1 }n; identify Fq with { 0 , 1 }d; identify F^2 q with { 0 , 1 }m. For any f ∈ P and α ∈ Fq, define C(f, α) = (α, f (α)).
We view the condenser C as a bipartite graph with left vertices from P and right vertices from F^2 q. For each f ∈ P, its neighborhood Γ(f ) is of size q. We want this graph be a good bipartite expander, i.e. for each S ⊂ P with |S| = K, Γ(S) ≥ AK for some constant A (the values of K and A determined later).
The analysis of the condenser will proceed as follows: we will show that for any T ⊂ F^2 q with |T | < AK, |{f ∈ P | Γ(f ) ⊂ T }| < K.
Lemma 8. If A + cK ≤ q, then for any T ⊂ F^2 q with |T | < AK, |{f ∈ P : Γ(f ) ⊂ T }| < K.
Proof. Take any set T ⊆ F^2 q with |T | < AK.
The crucial step: Interpolate a nonzero bivariate polynomial Q(X, Y ) =
∑A− 1
i=
∑K− 1
j=0 αij^ X iY j
such that Q(x, y) = 0 for all (x, y) ∈ T. Such a Q exists because (1) the space of all bivariate polynomials of those given degrees is a vector space of dimension AK (2) Each vanishing condi- tion Q(x, y) = 0 imposes one homogeneous linear constraint on the coefficients of the bivariate polynomial, and there are < AK such homogeneous constraints.
Suppose f (X) ∈ P is such that Γ(f ) ⊆ T. We then get that Q(α, f (α)) = 0 for all α ∈ Fq. Define the univariate polynomialH(X) = Q(X, f (X)). Note that H(X) is a polynomial in X of degree at most (A−1)+(c−1)(K −1), but H(α) = 0 for all α ∈ Fq. Since (A−1)+(c−1)(K −1) < A+cK ≤ q, H(X) is the zero polynomial.
Thus Q(X, f (X)) = H(X) ≡ 0, and so Y − f (X) divides Q(X, Y ) (this is the remainder theorem for polynomials). Since Y − f (X) are pairwise coprime for distinct f ’s, and the Y -degree of Q(X, Y ) = K − 1, there must be < K distinct f ’s with Γ(f ) ⊆ T.
Now we analyze the condenser. Without loss of generality, we just analyze the case where the weak
random source is a flat distribution. Let S be any subset of { 0 , 1 }n^ with size 2k, and let US be the uniform distribution over S. It suffices to show that C(US , Ud) is -close to having min-entropy ≥ k + d.
Let A = q − 2 kc. By Lemma 8, |Γ(S)| ≥ 2 kA. Thus C(US , Ud) is 1 − Aq = 2 k (^) c q -close to the uniform distribution over a set T ′^ with |T ′| = 2kq, which has min-entropy lg |T ′| = k + lg q = k + d.
Thus C(US , Ud) is 2 k (^) c q -close to a distribution with min-entropy^ k^ +^ d, and so^ C^ is a
k, 2 k (^) c q , k^ +^ d
condenser.
This is interesting only when k is very small compared to n. Here is one example of the kind of parameters we can get in this setting: Choose k = 10099 log q, and c = q 10001
. Then n = q 10001 log q,
d = log q = O(log n), k = 10099 log q = O(log n), m = 2 log q ≤ (1.01)·(k+d) and < q−^
1 (^1000) = (^) poly^1 (n).
4.2 Version 2: condensers from Parvaresh-Vardy codes
The previous construction was naturally limited by the fact that the output was very small (F^2 q ). Our plan now is to increase the output size, by evaluating not one polynomial, but many polyno- mials.
Let > 0 be an integer. We will first choose a special subset P′^ ⊂ P, and identify P′^ with { 0 , 1 }n. For any (f 1 , f 2 , · · · , f) ∈ P′^ and α ∈ Fq, let C((f 1 , f 2 , · · · , f), α) = (α, f 1 (α), f 2 (α), · · · , fj (α)). Thus the output space, F` q+1 , is identified with { 0 , 1 }m. Pictorially, P′^ is a collection of curves, and the condenser uses the weak random source to pick a curve from this collection, and then outputs a random point on this curve.
The key property we will need of P′^ is a bound on the number of curves from P′^ on which any given multivariate polynomial formally vanishes. We will later see how to choose a large P′^ with this property.
Property Z: If Q(X, Y 1 , Y 2 , · · · , Y) is a multivariate polynomial in Fq with individual degrees ≤ (A − 1 , h − 1 , h − 1 , · · · , h − 1), then ∣∣ {(f 1 , f 2 , · · · , f) ∈ P′^ | Q(X, f 1 (X), f 2 (X), · · · , f`(X)) ≡ 0 }
≤ h`^ − 1.
Lemma 9. Suppose P′^ has Property Z. Suppose h = K^1 /^ is an integer. If A + ch ≤ q, then for any T ⊆ Fj q+1 with |T | < AK: (^) ∣ ∣∣{ f~ ∈ P′ (^) | Γ( f~ ) ⊂ T }
∣∣ < K.
Remark: if K is not a perfect `-th power, this lemma is still true, but the argument involves one extra trick which we do not discuss here.
Proof. As argued in Lemma 8, there exists a multivariate polynomial Q(X, Y 1 , Y 2 , · · · , Y) vanishing on T with individual degrees at most (A − 1 , h − 1 , h − 1 , · · · , h − 1). Let f~ = (f 1 , f 2 , · · · , f) be such that Γ( f~ ) ⊆ T. Define H(X) = Q(X, f 1 (X), f 2 (X), · · · , f(X)). By definition of Q, H(α) = 0 for all α ∈ Fq. The degree of the polynomial H is at most (A − 1) + (c − 1)j(h − 1) < A + cjh ≤ q. Thus H(X) = Q(X, f 1 (X),... , f(X)) is the zero polynomial.
Proof. View Q as a polynomial in Y 1 ,... , Ywith coefficients in Fq[X]. If the coefficients have any common factors, we divide Q by them (this does not affect the zeroness of evaluations), and thus we may assume that all the coefficients have GCD 1. Consider the reduction Q(X, Y 1 ,... , Y) mod E(x), and call it Q˜(Y 1 , Y 2 , · · · , Y) ∈ (Fq[x]/E(x)) [Y 1 , Y 2 , · · · , Y]. Thus Q˜(Y 1 ,... , Y) is a non- zero polynomial with coefficients in the field Fqc (here we used the fact that E(X) is irreducible) and its degree is at most (h− 1 , h− 1 , · · · , h−1). Define R˜(Z) = Q˜(Z, Zh, Zh 2 , · · · , Zh− 1 ) ∈ Fqc [Z]. The degree of the polynomial R˜(Z) is at most (h − 1)(1 + h + h^2 + · · · + hj−^1 ) = hj^ − 1, and so R˜ has at most hj^ − 1 roots in Fqc.
Now the lemma follows from the observation that if f (X) is such that (f (X), f (X)h,... , f (X)h − 1 ) mod E(X) is a zero of Q(X, Y 1 ,... , Y), then f (X) mod E(X) is a zero of R˜(Z).
Unwinding Parameters. We just saw that C is a (log K, chq , log K + lg q)-condenser, where K = h. We now describe a setting of parameters that makes this compose nicely with the previously constructed extractor, thus giving our final extractor construction.
Let us choose h = q^0.^99 , c = q^0.^001 and ` = q^0.^0001. Note that the error < (^) qΩ(1)^1.
Since 2n^ = qc, we have that q ≤ nO(1), and so the seed length d = log q = O(log n). Thus the error < (^) poly^1 (n).
The output length, m, equals log(q+1) = ( + 1) log q. The input min-entropy, k, equals log K = log h^ = (0.99) · · log q ≥ 0. 98 m. Thus the output of the condenser is a random variable distributed over { 0 , 1 }m^ with entropy 0. 98 m, and this can be plugged into the earlier extractor we constructed, to yield a nearly-optimal extractor which works for all min-entropies.
