Co-NP problems on random inputs

1. Co-NP problems on random inputs Paul Beame University of Washington

2. Basic idea • NP is characterized by a simple property - having short certificates of membership • Show that co-NP doesn’t have this property • would separate P from NP so probably quite hard • Lots of nice, useful baby steps towards answering this question

3. Certifying language membership • Certificate of satisfiability • Satisfying truth assignment • Always short, SAT NP • Certificate of unsatisfiability • ????? • transcript of failed search for satisfying truth assignment • Frege-Hilbert proofs, resolution • Can they always be short? If so then NP=co-NP.

4. Proof systems • A proof system for L is a polynomial time algorithmA s.t. for all inputs x • x is in L iffthere exists a certificate P s.t. A accepts input (P,x) • Complexity of a proof system • How big |P| has to be in terms of |x| • NP = {L: L has polynomial-size proofs}

5. Propositional proof systems • A propositional proof system is a polynomial time algorithmA s.t. for all formulas F • F is unsatisfiable iff • there exists a certificate P s.t. A accepts input (P,F)

6. Sample propositional proof systems • Truth tables • Axiom/Inference systems, e.g. • modus ponens A, (A -> B) | B • excluded middle | (A v ~A) • Tableaux/Model Elimination systems • search through sub-formulas of input formula that might be true simultaneously • e.g. if ~(A -> B) is true A must be true and B must be false

7. Frege Systems • Finite # of axioms/inference rules • Proof of unsatisfiability of F - sequence F1, …, Fr of formulas s.t. • F1 = F • each Fjis an axiom or follows from previous ones via an inference rule • Fr = L trivial falsehood • All of equivalent complexity up to poly

8. Resolution • Frege-like system using CNF clauses only • Start with original input clauses of CNF F • Resolution rule • (A v x), (B v ~x) | (A v B) • Goal: derive empty clause L • Most-popular systems for practical theorem-proving

9. Davis-Putnam (DLL) Procedure • Both • a proof system • a collection of algorithms for finding proofs • As a proof system • a special case of resolution where the pattern of inferences forms a tree. • The most widely used family of complete algorithms for satisfiability

10. Simple Davis-Putnam Algorithm • Refute(F) • While (F contains a clause of size1) • set variable to make that clause true • simplify all clauses using this assignment • If F has no clauses then • output “F is satisfiable” and HALT • If F does not contain an empty clause then • Choose smallest-numbered unset variablex • Run Refute( ) • Run Refute( ) splitting rule

11. Hilbert’s Nullstellensatz • System of polynomialsQ1(x1,…,xn)=0,…,Qm(x1,…,xn)=0over fieldK hasnosolution in any extension field ofKiff there exist polynomials P1(x1,…,xn),…,Pm(x1,…,xn)inK[x1,…,xn]s.t.

12. Nullstellensatz proof system • Clause (x1 v ~x2 v x3) becomes equation(1-x1)x2(1-x3)=0 • Add equationsxi2-xi =0for each variable • Proof:polynomialsP1,…, Pm+n proving unsatisfiability

13. Polynomial Calculus • Similar to Nullstellensatz except: • Begin withQ1,…,Qm+nas before • Given polynomialsRand Scan infer • a R + b S for any a, b in K • xi R • Derive constant polynomial1 • Degree= maximum degree of polynomial appearing in the proof • Can find proof of degreedin timenO(d)using Groebner basis-like algorithm

14. Cutting Planes • Introduced to relate integer and linear programming: • Clause (x1 v ~x2 v x3) becomes inequalityx1+1-x2+x3 1 • Add xi 0 and 1-xi 0 • Derive0 1 using rules for adding inequalities andDivision Rule: • acx+bcydimplies ax+byd/c

15. Some Proof System Relationships ZFC P/poly-Frege Frege AC0-Frege Cutting Planes Polynomial Calculus Resolution Nullstellensatz Davis-Putnam Truth Tables

16. Random k-CNF formulas • Makemindependent choices of one of theclauses of lengthk • D = m/nis the clause-density of the formula • Distribution

17. Threshold behavior of random k-SAT

18. Contrast with ... • Theorem [CS]:For every constantD, randomk-CNF formulas almostcertainly require resolution proofs of size 2W(n) • What is the dependence onD ?

19. Width of resolution proofs • IfPis a resolution proof width(P)=length of longest clause inP • Theorem [BW]: Every Davis-Putnam (DLL) proof of size S can be converted to one of width log2S • Theorem [BW]:Every resolution proof of sizeScan be converted to one of width

20. Sub-critical Expansion • F- a set of clauses • s(F)-minimum size subset ofFthat isunsatisfiable • d F-boundary ofF- set of variables appearing in exactly one clause ofF • e(F)- sub-critical expansion ofF = max min { |d G|: GF, s/2< |G| s} s s(F)

21. L Width and expansion • Lemma[CS] : If P is a resolution proof of F then width(P)e(F). s(F) s/2 to s G contains d G

22. Consequences • Corollaries: • Any Davis-Putnam (DLL) proof ofFrequires size at least2e(F) • Any resolution proof of F requires size at least

23. s(F) and e(F) for random formulas • IfFis a random formula fromthen • s(F)isW (n/D1/(k-2)) almost certainly • e(F)isW (n/D2/(k-2)+e) almost certainly • Proved for Hypergraph expansion

24. Hypergraph Expansion • F- hypergraph • d F-boundary ofF- set of degree 1vertices ofF • sH(F)- minimum size subset ofFthat does not have a System of Distinct Representatives • eH(F)-sub-critical expansion ofF - max min { |d G|: GF, s/2< |G| s} s sH(F)

25. System of Distinct Representatives variables/nodes clauses/edges sH(F) s(F) so eH(F) e(F)

26. Density and SDR’s • The densityof a hypergraph is#(edges)/#(vertices) • Hall’s Theorem:A hypergraph F has a system of distinct representatives iff every subgraph has density at most 1.

27. Density and Boundary • A k-uniform hypergraph of density bounded below 2/k, say 2/k-e , has average degree bounded below2 • constant fraction of nodes are in the boundary

28. Density of random formulas • Fix setSof vertices/variables of sizer • Probabilitypthat a single edge/clause lands inSis at most(r/n)k • Probability thatScontainsat leastqedges is at most

29. s(F) for random formulas • Apply forq=r+1for all rup tos using union bound: • for s = O(n/D1/(k-2))

30. e(F) for random formulas • Apply forq=2r/kfor all r between s/2 ands using union bound: • for s = Q(n/D2/(k-2))

31. Hypergraph Expansion and Polynomial Calculus • Theorem [BI]:The degree of any polynomial calculus or Nullstellensatz proof of unsatisfiability of F is at least eH(F)/2 if the characteristic is not 2. • Groebner basis algorithm bound is only nO(eH(F))

32. k-CNFand parity equations • Clause(x1 v ~x2 v x3)is implied byx1+(x2+1)+x3 = 1 (mod 2)i.e.x1+x2+x3 = 0 (mod 2) • Derive contradiction 0 = 1 (mod 2) by adding collections of equations • # of variables in longest line is at least eH(F)

33. Parity equations and polynomial calculus • Given equations of form • x1+x2+x3 = 0 (mod 2) • Polynomial equationyi2-1=0for each variable • yi = 2xi-1 • Polynomial equationy1 y2 y3-1=0 • would bey1 y2 y3+1=0 if RHS were 1 • Imply the old Nullstellensatz equations ifchar(K)is not2

34. Lower bounds • For random k-CNF chosen from almost certainly for anye>0: • Any Davis-Putnam proof requires size • Any resolution proof requires size • Any polynomial calculus proof requires degree

35. Upper Bound • Theorem [BKPS]:For F chosen from and D above the threshold, the simple Davis-Putnam (DLL) algorithm almost certainly finds a refutation of size • and this is a tight bound...

36. y y x x Idea of proof • 2-clause digraph • (x v y) • Contradictory cycle: contains bothxandx • After settingO(n/D1/(k-2)) variables, > 1/2 the variables are almost certainly in contradictory cycles of the 2-clause digraph • a few splitting steps will pick one almost certainly • setting clauses of size 1 will finish things off

37. Implications • Random k-CNF formulas are provably hard for the most common proof search procedures. • This hardness extends well beyond the phase transition. • Even at clause ratio D=n1/3, current algorithms on random 3-CNF formulas have asymptotically the same running time as the best factoring algorithms.

38. Random graph k-colourability • Random graph G(n,p) where each edge occurs independently with probabilityp • Sharp threshold forwhether or not graph isk-colourable, e.g. p ~ 4.6/nfork=3 • What about proofs that the graph is not k-colourable?

39. Lower Bound • Theorem [BCM 99]:Non-k-colourabilityrequires exponentially large resolution proofs • Basic proof idea: • same outline as before • notion ofboundary of a sub-graph • set of vertices of degree< k • s(G) smallest non-k-colourable sub-graph

40. Challenges • Better bound fore(F)for randomF • Can it beQ(s(F))? • If so, the simple Davis-Putnam algorithm has asymptotically best possible exponent of any DP algorithm. • Extend lower bounds to other proof systems • must be based on something other than expansion since certain formulas with high expansion have small Cutting Planes proofs.

41. Challenges • Conjecture:Random k-CNF formulas are hard for Frege proofs • Extend to other random co-NP problems • Independent Set? • Best algorithms only get within factor of 2 of the largest independent set in a random graph

42. Sources • [Cook, Reckhow 79] • [Chvatal, Szemeredi 89] • [Mitchell, Selman, Levesque 93] • [Beame, Pitassi 97] • [Beame, Karp, Pitassi, Saks 98] • [Beame, Pitassi 98] • [Ben-Sasson, Wigderson 99] • [Ben-Sasson, Impagliazzo 99] • [Beame, Culberson, Mitchell 99]

43. Circuit Complexity • P/poly - polysize circuits • NC1- polysize formulas • CNF - polysize CNF formulas • AC0- constant-depth polysize circuits using and/or/not • AC0[m] - also = 0 mod m tests • TC0 - threshold instead

44. C-Frege Proofs • Given circuit complexity class C can define C-Frege proofs to be Frege-like proofs that manipulate circuits in Crather than formulas • Frege = NC1-Frege • Resolution = CNF-Frege • Extended-Frege = P/poly-Frege • AC0-Frege • AC0[m]-Frege • TC0-Frege