An Introduction to Randomness Extractors

An Introduction to Randomness Extractors Ronen Shaltiel University of Haifa Daddy, how do computers get random bits?

Randomized algorithms and protocols Randomized algorithms/protocols: • Receive stream of independent unbiased coin tosses. • Necessary for Crypto. deterministic algorithm Randomized input output • Provably help in distributed settings. • Randomized algorithms are often simpler and more efficient than known deterministic ones. (even though we conjecture that BPP=P).

Computers can sample from: Electro-magnetic noise (Intel) Key strokes of user (Unix) Timing of past events (Unix) These distributions are “somewhat random” but not “truly random”. Paradigm:randomness extractors Input:one sample from arbitrary “weak source of randomness”. Output: independent coin tosses. How do computers obtain random coin tosses? Coins may be biased and correlated Randomness Extractor “weak source of randomness” Randomized algorithm input output Extensively studied area, dates back to von-Neumann in 1951.

How do computers obtain random coin tosses? Computers can sample from: • Electro-magnetic noise (Intel) • Key strokes of user (Unix) • Timing of past events (Unix) These distributions are “somewhat random” but not “truly random”. Paradigm:randomness extractors Input:one sample from arbitrary “weak source of randomness”. Output: independent coin tosses. “weak source of randomness” Randomness Extractor Randomized algorithm input output Extensively studied area, dates back to von-Neumann in 1951.

Extractors have applications in: Randomized complexity theory. Cryptography. Network design. Ramsey theory. Coding theory. Combinatorics. Algorithm design. Data structures. Extractors have many applications “weak source of randomness” Randomness Extractor Randomized algorithm input output Often not directly related to randomness! Gives additional motivation to extractors (in addition to the initial motivation of extracting randomness for randomized algs).

Several notions of extractors • Deterministic extractors Restrict to specific families of “allowed sources”. • Multiple sources extractors Extractor gets samples from several independent sources. • Seeded extractors Allow extractor to get a seed of few truly random bits.

Deterministic extractors: Formal definition “weak source of randomness” • Dfn:Let C be a set of distributions over {0,1}n(family of “allowed sources”). • E:{0,1}n! {0,1}mis an extractor for C Distribution X from C Randomness Extractor ²-close to • if 8X2C, random variable E(X) is uniformover {0,1}m. • Two distributions Y,Z over the same domain are ²-close if 8 event A, |Pr[Y 2 A]-Pr[Z 2 A]| ≤ ². • Goal: Design efficiently computable extractors for interesting and general families of sources. Maximize number of extracted bits. Minimize error ².

Example: von-Neumann’s sources and extractor (1951!) Let 0<p≤½ be a parameter (e.g., p=1/10). A vN-source is a distribution X=(X1,..,Xn) s.t. • X1,..,Xn i.i.d. • p ≤ Pr[Xi=1] ≤ 1-p. vN extractor E(x) (extracts one bit): on input x2{0,1}n • Scan input bits from left to right. • If you see pair “01” stop and output “0”. • If you see pair “10” stop and output “1”. Observation: Pr[“01”] = Pr[“10”] (implies correctness). Subsequent work on extracting many bits [Elias72,Peres92]. X has entropy ≥ pn.

Impossibility of extraction from Santha-Vazirani sources Let 0<p≤½ be a parameter (e.g., p=1/10). A vN-source is a distribution X=(X1,..,Xn) s.t. • X1,..,Xn i.i.d. • p ≤ Pr[Xi=1] ≤ 1-p. An SV-source is a distribution X=(X1,..,Xn) s.t. • Source bits can be correlated. • Every next bit is somewhat unpredictable. • More formally, 8i, 8x1,..,xi-12{0,1}, p ≤ Pr[Xi=1|X1=x1,..,Xi-1=xi-1] ≤ 1-p. Thm:[SanthaVazirani86] No extractors for such sources. Historically => research on other notions of extractors. X has entropy ≥ pn.

Bit-fixing sources [ChorGoldreichFriedmanHastadRudichSmolensky85] k random bits Let k be a parameter. A k-bit-fixing source is a distribution X=(X1,..,Xn) s.t. • k bits are uniformly distributed. • remaining n-k bits are fixed to arbitrary values. Easy to extract one bit: E(X1,..,Xn)=parity(X1,..,Xn) Thm: [CGFHRS] Impossible to extract 2 bits with zero error for k<n/3. Probably not a good example for “extraction story”. Naturally arise in cryptographic scenarios.

public channel Alice Bob Eavesdropper (Non-interactive) Privacy amplification Use bit-fixing source extractor. • Alice and Bob share a uniformly chosen key Z2{0,1}n. • Can use random key to encrypt communication on public channel. • Eve somehow learns n-k bits of key. • Alice and Bob don’t know which bits. • Eve’s view: Z is a k-bit fixing-source. • Eve’s view: E(Z) is (close to) uniform. E(Z) is a new and secure key. • Motivates extractors: • Extract many bits (hopefully k bits). • Explicit (poly-time computable). Z2R{0,1}n Z2R{0,1}n E(Z) E(Z) From my point of view Z is distributed like: k random bits Extract m=(1-o(1))k bits [CGFHRS85] [CohenWigderson89] [KampZuckerman07] [GabizonRazShaltiel06] [Rao09]

Affine sources • Let F be a finite field (typically F2={0,1}). An affine source is a distribution that is uniform over some affine subspace with dimensionk of Fn. • Affine sources generalize bit-fixing sources. • Extractor is E:Fn!{0,1} is in particular “anti-linear”: non-constant on any affine subspace of dimension k. (In extractor jargon, this is called a “disperser”). • Exist for k=O(log n) by probabilistic method. • Explicit constructions: (poly-time computable) • Extractor : k=o(n) [Bourgain07]. • Disperser: k=no(1) (“anti-linear function”) [Shaltiel11].

Feasibly samplable sources [Blum86,TrevisanVadhan00]. • Sources defined by considering an allowed “sampling process”. • Source distribution = Sampler(uniform bits). • Restrictions on complexity of sampler induces family of sources. • Small space, Small circuits, Constant depth circuits… [TV00,KampRaoVadhanZuckerman06, KonigMaurer05,Shaltiel06,Viola11,DeWatson11]. • Orthogonal notion of “Feasibly recognizable sources” suggested in [Shaltiel09]. • Source uniform on {x:P-1(x)=1} for some procedure P. • Restrictions on complexity of procedures induce family.

n n Multiple sources extractors • No deterministic extractors for SV-sources. • Possible if you get samples from two independent sources! • Can allow a more general family than SV-sources. • C={distributions X with “high entropy”}. • Best we can hope for. X Y 2-source extractor

Dfn: (min-entropy) X has min-entropy ≥ k if ∀x: Pr[X=x] ≤ 2-k “Can hope to extract k random bits from X”. Seen examples of sources with min-entropy ≥ k. vN-sources. SV-sources. Bit-fixing sources. Affine sources. Another example:flat distributions: X uniformly distributed on a subset of size 2k of {0,1}n. subset flat distribution Measuring the entropy of the source distribution “weak source of randomness” A more stringent variant of Shannon entropy Distribution X over n bits {0,1}n size 2k X

n n Can be generalized to t>2 sources. Formal definition of Multiple sources extractors Definition: (emerged from [SanthaVazirani86]) A (k,ε)-2-source-extractor is a function E(x,y)s.t. for every two independent dist. X,Y over n bit strings each having min-entropy ≥ k, E(X,Y) is ε-close to uniform. Realistic model for generating random bits. Unfortunately, we don’t have good explicit constructions. X Y 2-source extractor

2-source extractor E(x,y)that outputs one bit is a matrix (w.l.o.g. symmetric) Property: Every XxY rectangle of size2k is balanced. Every XxX rectangle of size 2k is notmonochromatic. Adjac. matrix of a 2k -Ramsey graph: Graph with no 2k -clique or 2k -independent set. Explicitly constructing r-Ramsey graphs for small ris a longstanding open problem. X Y X Explicit 2-source extractors imply explicit Ramsey graphs 2n x y

2k-Ramsey graphs on 2n nodes Erdős 47: Probabilstic method achieves k≈log n Frankl and Wilson 81: Explicit construction k≈(n log n)½ [BKSSW05,BRSW06]: Explicit construction k=no(1) (Extractor techniques). Construct bipartite Ramsey graphs (stronger than Ramsey graphs but weaker than 2-source extractors). (k,ε)-2-source extractors Probabilstic method achieves k≈log n Chor and Goldreich 88: E(x,y)=<x,y>mod 2 works for k ≥ n/2. Bourgain 05: Explicit construction k=0.4999n. Progress on t-source extractors [BIW04,BKSSW05,Rao06]. Rao06: extract from log n/log k sources with min-entk. Explicit constructions of 2-source extractors and Ramsey graphs

seed Y Extractor random output Seeded extractors [NisanZuckerman92] • We allow an extractor to also receive an additional seed of (few) independent random bits. • Makes sense as long as: # bits extracted > seed length. • Handle all high min-entropy sources! source dist. X on n bits Randomness Definition: A (k,ε)-extractor is a function E(x,y)s.t. for every dist. X with min-entropy ≥ k, E(X,Y) is ε-close to uniform. Lower bounds[RadhakrishnanTaShma98]: seed length ≥ log(n-k) + 2log(1/ε) Probabilistic method: Exists optimal extractor which matches lower bound and extractsall the k random bits in the source distribution. Explicit constructions: E(x,y) can be computed in poly-time. uniformly distributed Current milestones in explicit constructions: [LuReingoldVadhanWigderson03, GuruswamiUmansVadhan07,DvirWigderson08,DvirKoppartySarafSudan09]. • “Optimal up to constants”: seed = O(log(n) + log(1/ε)) output(k)bits. • For constant error: seed = O(log(n)) output(1-o(1))∙k bits.

Randomness Extractor seed Simulating randomized algorithms using weak random sources Goal: Run rand algorithm with a weak random source of randomness. Where can we get a seed? Idea: Go over all seeds. • Given sample X from source. • ∀y computezy= E(X,y) • Compute Alg(input,zy) • Answer majority vote. seed=O(log n)=>poly-time. Explicit constructions. Unsuitable for crypto protocols. source dist. X on n bits random coins Randomized algorithm input output

Something about the tools used in explicit constructions • 2-wise independent hash functions [ImpagliazzoLevinLuby89,NisanZuckerman92]. • E(x,h)=h(x),h where h is chosen from small family of 2-wise independent hash functions. • Disadvantage: huge seed. • List decodable error correcting codes [Trevisan99]. • E(x,y)=Enc(x)y,ywhere Enc is a binary list decodable error correcting code. (also works vice-versa). • Rate ≥ 1/poly(n) => logarithmic seed. • Disadvantage: extract only one additional bit. • Can try and exploit properties of specific codes [TaShmaZuckeramanSafra01,ShaltielUmans01,GuruswamiUmansVadhan07]. • Various composition methods […]

correlated! Composing short seed extractor with long output extractor k bits of min-entropy short seed extractor long seed extractor Seeded Extractors are only guaranteed to work when the source and seed are independent. Short random output !long random output Nevertheless, many constructions make this “go through” by modifying initial extractors to have additional properties.

Extractor is a bipartite graph. Given extractor E(x,y) N=2n (# of inputs) M=2m (# of outputs) K=2k (# of source elements) D=2d (# of seeds) Connect x to E(x,1),..,E(x,D). Small seed length d ~ log n => small deg D ~ log N. E(x,1) .. E(x,D) Seeded extractors as graphswith “volume expansion”. N≈{0,1}n M≈{0,1}m D=2d edges x

Extractor property: ∀dist X of min-entropy≥k, E(X,Y) ε-close to uniform. =>“expansion” property: ∀set Xofsize K=2k , |Γ)X)| ≥ (1-ε)M. Such graph/function is called “Disperser”. Extractor graphs: volume expansion property N≈{0,1}n M≈{0,1}m X Γ(X) K=2k (1-ε)M

X Γ(X) Extractors and Expander graphs N≈{0,1}n N≈{0,1}n N≈{0,1}n M≈{0,1}m D=2d edges X Γ(X) K K=2k (1+δ)K (1-ε)M (1+δ)-Expander Extractor

X Γ(X) Extractors and Expander graphs N≈{0,1}n N≈{0,1}n N≈{0,1}n Size expansion: K -> (1+δ)K M≈{0,1}m X Γ(X) K K=2k (1+δ)K (1-ε)M Extractors produce better results in some applications of expanders Volume expansion: K -> (1-ε)M K/N -> (1-ε) (1+δ)-Expander Extractor

Expanders with expansion that beat the eigenvalue bound [WigdersonZuckerman93] Goal: Construct low deg expanders with huge expansion. Line up two low degree extractors. ∀set X ofsize K, (where K<<N) |Γ)X)| ≥ (1-ε)M > M/2. ∀sets X,X’ ofsize K X and X’ havecommon neighbour. • Contract middle layer. • Bipartite graph in which every set of size K sees N-K vertices. • Trivially degree ≥ (N-K)/K ≈ N/K. • Obtain low degree ND2/K. • Eigenvaluemethods cannot yield graphs with such parameters. N≈{0,1}n N≈{0,1}n X’ X

Random walk variables v1..vD behave like i.i.d: ∀A of size ½M Hitting property: Pr[∀i:vi∊A]≤δ= 2-Ω(D). Chernoff style property: Pr[#i : vi∊Afar from exp.] ≤ δ=2-Ω(D). # of random bits used for walk:m+O(D)=m+O(log(1/δ)) # of random bits for i.i.d. m∙D=m ∙O(log(1/δ)) Randomness efficient (oblivious) sampling using expanders [AjtaiKormlosSzemeredi87] Random walk on constant degree expander M≈{0,1}m v2 v3 v1 vD

Given parameters m,δ: Use E with k=m, n=m+log(1/δ) ε<½andsmall seed d. Choose random x: m+log(1/δ) random bits. Set vi=E(x,i) Expansion property⇒Hitting prop. ∀A of size ½M Call x badif ∀i: E(x,i) inside A. # of bad x’s < K=2k Pr[x is bad] < 2k/2n = δ E(x,1) .. E(x,D) Randomness efficient (oblivious) sampling using extractors [Sipser86,Zuckerman96] N≈{0,1}n M≈{0,1}m D edges x (1-ε)M A bad x’s

An (oblivious) sampling scheme uses a random n bit string x to generated D random variables. Thm: [Zuckerman06] if the scheme has sampling property then the derived graph is an extractor. Extractors oblvs Sampling E(x)1 .. E(x)D Every (oblivious) sampling scheme yields an extractor N≈{0,1}n M≈{0,1}m D=2d edges x

Extractors come in several flavors and have many applications in diverse fields. Goal: Explicitly construct extractors with parameters that match existential bounds. Many open problems. See article in proceedings for more details. Conclusion “weak source of randomness” Randomness Extractor Randomized algorithm input output

Thank You… Daddy, can you tell me that story again?

An Introduction to Randomness Extractors