1 / 21

expander codes and pseudorandom subspaces of R n

expander codes and pseudorandom subspaces of R n. James R. Lee. University of Washington. [joint with Venkatesan Guruswami (Washington) and Alexander Razborov (IAS/Steklov)]. Classical high-dimensional geometry [Kasin 77, Figiel-Lindenstrauss-Milman 77] :.

mendel
Download Presentation

expander codes and pseudorandom subspaces of R n

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. expander codes and pseudorandom subspaces of Rn James R. Lee University of Washington [joint with Venkatesan Guruswami (Washington) and Alexander Razborov (IAS/Steklov)]

  2. Classical high-dimensional geometry [Kasin 77, Figiel-Lindenstrauss-Milman 77]: random sections of the cross polytope For a random subspace X µRN with dim(X) = N/2, (e.g. choose X = span {v1, …, vN/2} where viare i.i.d. on the unit sphere) In other words, every x 2 X has its L2 mass very “spread” out: This holds not only for each vi, but every linear combination

  3. Classical high-dimensional geometry [Kasin 77, Figiel-Lindenstrauss-Milman 77]: random sections of the cross polytope For a random subspace X µRN with dim(X) = N/2, (e.g. choose X = span {v1, …, vN/2} where viare i.i.d. on the unit sphere)

  4. This is a prominent example of the (now ubiquitous) use of the probabilistic method in asymptotic convex geometry. an existential crisis Geometric functional analysts face a dilemma we know well: Almost every subspace satisfies this property, but we can’t pinpoint even one. [Szarek, ICM06; Milman, GAFA01; Johnson-Schechtman, handbook01]asked: Can we find an explicit subspace on which the L1 and L2norms are equivalent? Related questions about explicit, high-dim. constructions arose (concurrently) in CS: - explicit embeddings of L2into L1 for nearest-neighbor search (Indyk) - explicit compressed sensing matrices M : RNRn for n ¿ N (Devore) - explicit Johnson-Lindenstrauss (dimension reduction) transform (Ailon-Chazelle) Why do analytists / CSists care about explicit high-dimensional constructions?

  5. For a subspace X µRN, we define the distortion of Xby distortion By Cauchy-Schwarz, we always have N1/2¸(X) ¸ 1. Random construction: A random X µRN satisfies: dim(X) = (1-)Nand (X) = O(1). [Kasin 77] dim(X) = (N) and (X) ·1+. [Fiegel-Lindenstrauss-Milman 77] Example (Hadamard): Let X = ker(first N/2 rows of Hadamard), then (X) ¼ N1/4.

  6. distortion dimension applications Compressive sensing Coding in characteristic zero, Geometric functional analysis Nearest-neighbor search View as an embedding: O(1) distortion, (N) dimension 1+ distortion, small blowup in dimension (Milman believes impossible) Want a map A : RNRn with n ¿ N, such that any r-sparse signal x 2RN (vector with at most r non-zero entries) can be uniquely and efficiently recovered from Ax. Relation to distortion: [Kashin-Temlyakov] Can uniquely and efficiently recover any r-sparse signal for r · N/(ker(A))2. (Even tolerates additional “noise” in the “non-sparse” parts of the signal.)

  7. Want a map A : RNRn such that any r-sparse signal x 2RN (vector with at most r non-zero entries) can be uniquely and efficiently recovered from Ax. sensing and distortion Want to solve: Given compressed signal y, minimize ||x||0 subject to Ax = y. (P0) Highly non-convex optimization problem, NP-hard for general A. Basis Pursuit: Given compressed signal y, minimize ||x||1 subject to Ax = y. (P1) Can use linear programming! [Lots of work has been done here: Donoho et. al.; Candes-Tao-Romberg; etc.] [KT07]: If y = Av and v has at most N/[2 (ker(A))]2non-zero coordinates, then (P0) and (P1) give the same answer. let’s prove this

  8. [KT07]: If y = Av and v has at most N/[2 (ker(A))]2non-zero coordinates, then (P0) and (P1) give the same answer. sensing and distortion For x 2RN and S µ [N], let xS be x restricted to coordinates in S. If x 2 ker(A) and

  9. Sub-linear dimension: previous results: explicit Rudin’60 (and later LLR’94) achieve dim(X) ¼ N1/2 and (X) · 3 (X = span {4-wise independent vectors}) Indyk’00 achieves dim(X) ¼ exp((log N)1/2) and (X) = 1+o(1). Indyk’07 achieves dim(X) ¼ N/2(log log N)2 and (X) = 1+o(1). Our result: We construct an explicit subspace X µRN with dim(X) = (1-o(1))N and In our constructions, X = ker(explicit sign matrix).

  10. Partial derandomization: previous results: derandomization Let Ak, Nbe a random k £ N sign matrix (entries are ±1 i.i.d) Kashin’s technique shows that almost surely, (and dim(ker(Ak, N)) ¸ N – k) Can reduce to O(N log2 N) random bits [Indyk 00] Can reduce to O(N log N) random bits [Artstein-Milman 06] Can reduce to O(N) random bits [Lovett-Sodin 07] Our result: With No(1) random bits, we get (X) · polylog(N). With N random bits for any >0, we get (X) = O(1). [Guruswami-L-Wigderson]

  11. d j À n N G = ([N], [n], E) - bipartite graph, d-right-regular and L µRd a subspace. the expander code construction where xS2R|S| is x restricted to the coordinates in S µ [N] and (j) is the neighborhood of j. x1 x2 x3 Resembles construction of Gallager, Tanner (L is the “inner” code). Following Tanner and Sipser-Spielman, we will show that if L is “good” and G is an “expander” then X(G,L) is even better (in some parameters). xN

  12. Say that a subspace L µRd is (t, )-spread if every x 2 L satisfies some quantitative matters If L is ((d), )-spread, then Conversely, if L has (L) = O(1), then L is ((d),(1))-spread. For a bipartite graph G = ([N],[n],E), the expansion profile of Gis (This is expansion from left to right.)

  13. Setup: spread-boosting theorem G = ([N], [n], E) - bipartite graph, d-right-regular and left degree · D. L µRda(t, )-spread subspace. Conclusion: If X(G,L) is (T, )-spread, then X(G,L) is How to apply: Assume D=O(1) and G(q) = (q) 8q 2 [N] (impossible to achieve) X(G,L) is (½, 1)-spread )(t, )-spread )(t2, 2)-spread … )((N), logt(N))-spread )(X(G,L)) . (1/)logt(N)

  14. Setup: spread-boosting theorem G = ([N], [n], E) - bipartite graph, d-right-regular and left degree · D. L µRda(t, )-spread subspace. Conclusion: If X(G,L) is (T, )-spread, then X(G,L) is S should “leak” L2 mass outside (since L is spreading and G is an expander), unless most of the mass in S is concentrated on a small subset B (impossible by assumption) S B

  15. Let H be a (non-bipartite) d-regular graph with second eigenvalue  = O(d1/2). Let G be the edge-vertex incidence graph (an edge is connected to its endpoints) when L is random (explicit constructions exist by Margulis, Lubotsky-Phillips-Sarnak) Alon-Chung: nodes of H edges of H Random subspace L µRd is ((d), (1))-spread Letting d = N1/4, the spread-boosting thm gives X(G,L) is (T,)-spread )X(G,L) is Takes O(log log N) steps to reach (N)-sized sets )poly(log N) distortion.

  16. Spectral Lemma: explicit construction: ingredients for L Let A be any k £ d matrix whose columns a1, …, ad2Rkare unit vectors and such that for every i  j, |hai, aji|·. Then ker(A) is + Kerdock codes (aka Mutually Unbiased Bases) [Kerdock’72, Cameron-Seidel’73] ((d1/2), (1))-spread subspaces of dimension (1-)d for every eps>0

  17. Kerdock + Spectral Lemma gives ((d1/2), (1))-spread subspaces of dimension (1-)d for every eps>0 boosting L with sum-product expanders Problem: If G=Ramanujan construction and L=Kerdock, the spread-boosting theorem gives nothing. (Ramanujan loses d1/2 and Kerdock gains only d1/2) Solution: Produce L’ = X(G,L) where L=Kerdock and G=sum-product expander Sum-product theorems [Bourgain-Katz-Tao, …] For A µFp, with |A| · p0.99 we have

  18. Kerdock + Spectral Lemma gives ((d1/2), (1))-spread subspaces of dimension (1-)d for every eps>0 boosting L with sum-product expanders Problem: If G=Ramanujan construction and L=Kerdock, the spread-boosting theorem gives nothing. (Ramanujan loses d1/2 and Kerdock gains only d1/2) Solution: Produce L’ = X(G,L) where L=Kerdock and G=sum-product expander Using [Barak-Impagliazzo-Wigderson/BKSSW] and the spread-boosting theorem, L’ is (d1/2+c, (1))-spread for some c > 0.

  19. boosting L with sum-product expanders Now we can plug L’ into G=Ramanujan and get non-trivial boosting. (almost done…) Solution: Produce L’ = X(G,L) where L=Kerdock and G=sum-product expander Using [Barak-Impagliazzo-Wigderson/BKSSW] and the spread-boosting theorem, L’ is (d1/2+c, (1))-spread for some c > 0.

  20. Improve the current bounds: • First attempt would be O(1) distortion with sub-linear randomness. some open questions Improve dependence on the co-dimension (important for compressed sensing) If dim(X) ¸ (1-)N, we get distortion dependence (1/)O(log log N). Could hope for . • Stronger pseudorandom properties:Restricted Isometry Property • [T. Tao’s blog] Find an explicit collection of unit vectors v1, v2, …, vN2Rn with N À n so that every small enough sub-collection is “nearly orthogonal.” • Breaking the diameter bound: • Show that the kernel of a random {0,1} matrix with only 100 ones per • row has small distortion. Or prove that sparse matrices cannot work.

  21. Refuting random subspaces with high distortion • Give efficiently computable certificates for (X) small or Restricted Isometry • Property which exist almost surely for random X µRN. some open questions • Linear time expander decoding? • Are their recovery schemes that run faster than Basis Pursuit?

More Related