1 / 46

Lower Bounds for Depth Three Circuits with small bottom fanin

This research paper explores a lower bound theorem for representations of degree-d polynomials in the form of shallow circuits with small bottom fanin. It discusses the implications of lower bounds for low-depth arithmetic circuits and their relationship to VP and VNP. The paper also presents a common proof strategy and technical ingredients used in analyzing these circuits.

fbruce
Download Presentation

Lower Bounds for Depth Three Circuits with small bottom fanin

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Lower Bounds for Depth Three Circuits with small bottom fanin Neeraj Kayal ChandanSaha Indian Institute of Science

  2. A lower bound Theorem: Consider representations of a degree d polynomial of the form If the ’s have degree one and at most variables each then there is an explicit (family) of polynomials on variables such that s is at least .

  3. A lower bound Theorem: Consider representations of a degree d polynomial of the form If the ’s have degree one and at most variables each then there is an explicit (family) of polynomials on variables such that s is at least . • Remark: • For a generic , s must be at least • Any asymptotic improvement in the exponent will imply .

  4. A lower bound Corollary (via Kumar-Saraf): Consider representations of a degree d polynomial of the form If the ’s have degree one and at most variables each then there is an explicit (family) of polynomials on variables such for s is at least . • Remark: • Bad news. • Good news.

  5. Background/Motivation

  6. Arithmetic Circuits …

  7. Arithmetic Circuits … -1 …

  8. Arithmetic Circuits … … -1 …

  9. Arithmetic Circuits … … -1 …

  10. Arithmetic Circuits … … -1 …

  11. … … -1 …

  12. Size = Number of Edges … … -1 …

  13. Depth … … -1 …

  14. This talk. > Field is . > Gates have unbounded fanin > … … -1 …

  15. Two Fundamental Questions Can explicit polynomials be efficiently computed? Can computation be efficiently parallelized?

  16. Two Fundamental Questions Can explicit polynomials be efficiently computed? Does VP equal VNP? Can computation be efficiently parallelized? Can every efficient computation be also done by small, shallow circuits?

  17. Can computation be efficiently parallelized? Question:How efficiently can we simulate circuits of size by circuits of depth ?

  18. Can computation be efficiently parallelized? Theorem (Hyafil79, VSBR83, AV-08, Koiran-12, GKKS13, Tavenas-13, Wigderson/Tavenas): Any circuit of size s and degree can be simulated by a -depth circuit of size

  19. Can computation be efficiently parallelized? Theorem (Hyafil79, VSBR83, AV-08, Koiran-12, GKKS13, Tavenas-13, Wigderson/Tavenas): Any circuit of size s and degree can be simulated by a -depth circuit of size … also by a regular, homogeneous -depth circuit of size

  20. Can computation be efficiently parallelized? Theorem (Hyafil79, VSBR83, AV-08, Koiran-12, GKKS13, Tavenas-13, Wigderson/Tavenas): Any circuit of size s and degree can be simulated by a -depth circuit of size … also by a regular, homogeneous -depth circuit of size Question:Is this optimal?

  21. Can computation be efficiently parallelized? Theorem (Hyafil79, VSBR83, AV-08, Koiran-12, GKKS13, Tavenas-13, Wigderson/Tavenas): Any circuit of size s and degree can be simulated by a -depth circuit of size … also by a regular, homogeneous -depth circuit of size Question:Is this optimal? (KS15 + Ramprasad): For yes, but with caveats.

  22. Can computation be efficiently parallelized? Theorem (Hyafil79, VSBR83, AV-08, Koiran-12, GKKS13, Tavenas-13, Wigderson/Tavenas): Any circuit of size s and degree can be simulated by a -depth circuit of size … also by a regular, homogeneous -depth circuit of size Corollary:Strong enough lower bounds for low-depth circuits imply VP VNP.

  23. A possible way to approach VP vs VNP • Strong enough lower bounds for low-depth circuits imply VP VNP. Low Depth Circuit (’s simple) • Low-depth circuits are easy to analyze..

  24. A possible way to approach VP vs VNP • Strong enough lower bounds for low-depth circuits imply VP VNP. Low Depth Circuit (’s simple) • Low-depth circuits are easy to analyze.. • Lots of work on lower bounds for low depth arithmetic circuits in recent years – hope to discover general patterns and technical ingredients

  25. A possible way to approach VP vs VNP • Prove strong enough lower bounds for low depth circuits. Low Depth Circuit (’s simple) • Low-depth circuits are easy to analyze.. • Lots of work on low depth arithmetic circuits recently • This talk: • common pattern/proof strategy • Technical ingredients

  26. A common Proof Strategy and some technical ingredients

  27. Proof Strategy (’s simple ). Let . shallow circuit C • Find a geometric property GP of the ’s. • Express the property GP in terms of rank of a “big” matrix M: if T has the property than rank(M(T)) is “relatively small” • Show that rank(M()) is “relatively large”.

  28. Proof Strategy (’s simple ). Let . shallow circuit C • Find a geometric property GP of the ’s. • Express the property GP in terms of rank of a “big” matrix M: if T has the property than rank(M(T)) is “relatively small” • Show that rank(M()) is “relatively large”.

  29. Lower Bounding rank of large matrices • If a matrix M(f) has a large upper triangular submatrix, then it has large rank • (Alon): If the columns of M(f) are almostorthogonal then M(f) has large rank.

  30. When the ’s have low degree (’s have low degree). Let . shallow circuit C • Find a geometric property GP of the ’s. • Express the property GP in terms of rank of a “big” matrix M: if T has the property than rank(M(T)) is “relatively small” • Show that rank(M()) is “relatively large”.

  31. Finding a geometric property GP of T is a product of low degree polynomials V(T) is a union of low-degree hypersurfaces V(T) has lots of high-order singularities

  32. Finding a geometric property GP of T

  33. Finding a geometric property GP of T is a product of low degree polynomials V(T) is a union of low-degree hypersurfaces V(T) has lots of high-order singularities V( T) has lots of points

  34. When the ’s have low degree (’s have low degree). Let . shallow circuit C • Find a geometric property GP of the ’s. • Express the property GP in terms of rank of a “big” matrix M: if T has the property than rank(M(T)) is “relatively small” • Show that rank(M()) is “relatively large”.

  35. When the ’s have low degree (’s have low degree). Let . shallow circuit C • Find a geometric property GP of the ’s. • Express the property GP in terms of rank of a “big” matrix M: if T has the property than rank(M(T)) is “relatively small” • Show that rank(M()) is “relatively large”.

  36. Expressing largeness of a variety in terms of rank is a variety. Let = set of degree- polynomials.

  37. Expressing largeness of a variety in terms of rank is a variety. Let = set of degree- polynomials. Let = set of degree- polynomials which vanish at every point of V. Hilbert’s Theorem (Informal): If V is “large” then has small dimension.

  38. Expressing largeness of a variety in terms of rank is a variety. Let = set of degree- polynomials. Let = set of degree- polynomials which vanish at every point of V. Hilbert’s Theorem (Informal): If V is “large” then has small dimension. Let = { () of deg } . Hilbert’s Theorem (Formal): If V has dimension r then has asymptotic dimension .

  39. When the ’s have low degree (’s have low degree). Let . shallow circuit C • Find a geometric property GP of the ’s. • Express the property GP in terms of rank of a “big” matrix M: if T has the property than rank(M(T)) is “relatively small” • Show that rank(M()) is “relatively large”.

  40. Restrictions Example: is a function. f(0, 1, z) is a restriction of f. Geometrically, f(0, 1, z) is same as restricting f to the axis-parallel line W = V(x, y-1). Algebraically, f(0, 1, z) is same as mod More generally: Let be any ideal and a polynomial. Call mod as an algebraic restriction of f. Obs: Hilbert’s theorem can be generalized for algebraic restrictions of polynomials.

  41. Employing restrictions Lemma (KLSS14): If Q is a sparse polynomial then for a suitable random algebraic restriction mod is a low degree polynomial. • Yields lower bounds for homogeneous depth four (KLSS14 and KS14).

  42. Employing Restrictions Lemma (KLSS14): If T is a sparse polynomial a sum of product of low arity polynomials then for a suitable random algebraic restriction mod is a low degree polynomial.

  43. Employing Restrictions Lemma: If T is a sparse polynomial a sum of product of low arity polynomials then for a suitable random algebraic restriction mod is a low degree polynomial. • Yields lower bounds for homogeneous depth five with low bottom fanin (KS15 and BC15).

  44. A lemma by Shpilka and Wigderson Lemma (Shpilka-Wigderson): A depth three circuit C of size s can be converted to a homogeneous depth circuit C’ of size . Further if C has bottom fanin t then C’ also has bottom fanin t. • Yields lower bounds mentioned earlier.

  45. Conclusion • Proving lower bounds for low depth circuits is a potential way to prove lower bounds for more general circuits. • There is a meta-strategy common to many recent (and older) lower bounds. We don’t understand the power or limitations of this meta-strategy. • Open: lower bounds for homogeneous depth three circuits for polynomials of degree .

More Related