The Theory of NP-Completeness

1. The Theory of NP-Completeness

2. Cook’s Theorem (1971) • Prof. Cook • Toronto U. • Receiving Turing Award (1982) • Discussing difficult problems: worst case lower bound seems to be in the order of an exponential function • NP-complete (NPC) Problems

3. Finding lower bound by problem transformation • Problem A reduces to problem B (AB) • iff A can be solved by using any algorithm which solves B. • If AB, B is more difficult (B is at least as hard as A) • Since (A) (B) +T(tr1) + T(tr2), we have(B) (A) –(T(tr1) + T(tr2)) • We have (B) (A) if T(tr1) + T(tr2)  (A)

4. The lower bound of the convex hull problem • sorting  convex hull A B • an instance of A: (x1, x2,…, xn) ↓transformation an instance of B: {( x1, x12), ( x2, x22),…, ( xn, xn2)} assume: x1 < x2 < …< xn

5. The lower bound of the convex hull problem • If the convex hull problem can be solved, we can also solve the sorting problem, but not vice versa. • We have that the convex hull problem is harder than the sorting problem. • The lower bound of sorting problem is (n log n), so the lower bound of the convex hull problem is also (n log n).

6. NP NPC P P=NP ? NP: Non-deterministic Polynomial P: Polynomial NPC: Non-deterministic Polynomial Complete

7. Nondeterministic algorithms • A nondeterministic algorithm is an algorithm consisting of two phases: guessing and checking. • Furthermore, it is assumed that a nondeterministic algorithm always makes a correct guessing.

8. Nondeterministic algorithms • Machines for running nondeterministic algorithms do not exist and they would never exist in reality. (They can only be made by allowing unbounded parallelism in computation.) • Nondeterministic algorithms are useful only because they will help us define a class of problems: NP problems

9. NP algorithm • If the checking stage of a nondeterministic algorithm is of polynomial time-complexity, then this algorithm is called an NP (nondeterministic polynomial) algorithm.

10. NP problem • If a decision problem can be solved by a NP algorithm, this problem is called an NP (nondeterministic polynomial) problem. • NP problems : (must be decision problems)

11. Decision problems • The solution is simply “Yes” or “No”. • Optimization problem : harder Decision problem : easier • E.g. the traveling salesperson problem • Optimization version: Find the shortest tour • Decision version: Is there a tour whose total length is less than or equal to a constant C ?

12. Decision version of sorting • Given a1, a2,…, an and c, is there a permutation of ais ( a1, a2 , … ,an ) such that∣a2–a1∣+∣a3–a2∣+ … +∣an–an-1∣＜ C ?

13. Decision vs Original Version • We consider decision version problem D rather than the original problem O • because we are addressing the lower bound of a problem • and D ∝O

14. To express Nondeterministic Algorithm • Choice(S) : arbitrarily chooses one of the elements in set S • Failure : an unsuccessful completion • Success : a successful completion

15. Nondeterministic searching Algorithm : input: n elements and a target element x output: success if x is found among the n elements; failure, otherwise. j ← choice(1 : n)/* guess if A(j) = x then success/* check else failure • A nondeterministic algorithm terminates unsuccessfully iff there exist no set of choices leading to a success signal. • The time required for choice(1 : n) is O(1).

16. Relationship Between NP and P • It is known PNP. • However, it is not known whether P=NP or whether P is a proper subset of NP • It is believed NP is much larger than P • We cannot find a polynomial-time algorithm for many NP problems. • But, no NP problem is proved to have exponential lower bound. (No NP problem has been proved to be not in P.) • So, “does P = NP?” is still an open question! • Cook tried to answer the question by proposing NPC.

17. NP-complete (NPC) • A problem A is NP-complete (NPC) if A∈NP and every NP problem reduces to A.

18. SAT is NP-complete • Every NP problem can be solved by an NP algorithm • Every NP algorithm can be transformed in polynomial time to an SAT problem • Such that the SAT problem is satisfiable iff the answer for the original NP problem is “yes” • That is, every NP problem  SAT • SAT is NP-complete

19. Cook’s theorem (1971) • NP = P iff SAT  P • NP = P iff the satisfiability (SAT) problem is a P problem • SAT is NP-complete • It is the first NP-complete problem • Every NP problem reduces to SAT

20. Proof of NP-Completeness • To show that A is NP-complete (I)Prove that A is an NP problem (II)Prove that  B  NPC, B  A •  A  NPC • Why ? Transitive property of polynomial-time reduction

21. 0/1 Knapsack problem Given M (weight limit) and V, is there is a solution with value larger than V? • This is an NPC problem.

22. Traveling salesperson problem • Given: A set of n planar points and a value L Find: Is there a closed tour which includes all points exactly once such that its total length is less than L? • This is an NPC problem.

23. Partition problem • Given: A set of positive integers S Find: Is there a partition of S1 and S2 such that S1S2=, S1S2=S, iS1i=iS2 i (partition S into S1 and S2 such that element sum of S1 is equal to that of S2) • e.g. S={1, 7, 10, 9, 5, 8, 3, 13} • S1={1, 10, 9, 8} • S2={7, 5, 3, 13} • This problem is NP-complete.

24. Art gallery problem: *Given a constant C, is there a guard placement such that the number of guards is less than C and every wall is monitored?*This is an NPC problem.

25. Karp • R. Karp showed several NPC problems, such as 3-STA, node (vertex) cover, and Hamiltonian cycle, etc. • Karp received Turing Award in 1985

26. NP-Completeness Proof: Reduction All NP problems SAT Clique 3-SAT Vertex Cover Chromatic Number Dominating Set

27. NP-Completeness • “NP-complete problems”: the hardest problems in NP • Interesting property • If any oneNP-complete problem can be solved in polynomial time, then every problem in NP can also be solved in polynomial time (i.e., P=NP) • Many believe P≠NP

28. Importance of NP-Completeness • NP-complete problems: considered “intractable” • Important for algorithm designers & engineers • Suppose you have a problem to solve • Your colleagues have spent a lot of time to solve it exactly but in vain • See whether you can prove that it is NP-complete • If yes, then spend your time developing an approximation (heuristic) algorithm • Many natural problems can be NP-complete

29. Some concepts • Up to now, none of the NPC problems can be solved by a deterministic polynomial time algorithm in the worst case. • It does not seem to have any polynomial time algorithm to solve the NPC problems. • The lower bound of any NPC problem seems to be in the order of an exponential function. • The theory of NP-completeness always considers the worst case.

30. Caution ! • If a problem is NP-complete, its special cases may or may not be of exponential time-complexity. • We consider worst case lower bound in NP-complete.

31. Some concepts • Not all NP problems are difficult. (e.g. the MST problem is an NP problem.) (But NPC problem is difficult.) • If A, B  NPC, then A  B and B  A. • Theory of NP-completeness If any NPC problem can be solved in polynomial time, then all NP problems can be solved in polynomial time. (NP = P)

32. Undecidable Problems • They cannot be solved by guessing and checking. • They are even more difficult than NP problems. • E.G.: Halting problem: Given an arbitrary program with an arbitrary input data, will the program terminate or not? • It is not NP • It is NP-hard (SAT Halting problem)

33. NP: the class of decision problem which can be solved by a non-deterministic polynomial algorithm. • P: the class of problems which can be solved by a deterministic polynomial algorithm. • NP-hard: the class of problems to which every NP problem reduces. (It is “at least as hard as the hardest problems in NP.”) • NP-complete: the class of problems which are NP-hard and belong to NP.

34. The satisfiability (SAT) problem • Def : Given a Boolean formula, determine whether this formula is satisfiable or not. • A literal : xi or -xi • A clause : x1 v x2 v -x3 ci • A formula : conjunctive normal form C1& c2 & … & cm

35. The satisfiability (SAT) problem • The satisfiability problem • The logical formula : x1 v x2 v x3 & - x1 & - x2 the assignment : x1 ← F , x2 ← F , x3 ← T will make the above formula true (-x1, -x2 , x3) represents x1 ← F , x2 ← F , x3 ← T

36. The satisfiability problem • If there is at least one assignment which satisfies a formula, then we say that this formula is satisfiable; otherwise, it is unsatisfiable. • An unsatisfiable formula : x1 v x2 & x1 v -x2 & -x1 v x2 & -x1 v -x2

37. The satisfiability problem x1 cannot satisfyc1 and c2 at thesame time, so itis deleted.. • Resolution principle c1 : -x1 v -x2 v x3 c2 : x1 v x4  c3 :-x2 v x3 v x4 (resolvent) • If no new clauses can be deduced  satisfiable -x1 v -x2 v x3 (1) x1 (2) x2 (3) (1) & (2) -x2 v x3 (4) (4) & (3) x3 (5) (1) & (3) -x1 v x3 (6)

38. The satisfiability problem • If an empty clause is deduced  unsatisfiable - x1 v -x2 v x3 (1) x1 v -x2 (2) x2 (3) - x3 (4)  deduce (1) & (2) -x2 v x3 (5) (4) & (5) -x2 (6) (6) & (3) □ (7)

39. Nondeterministic SAT • Guessing for i = 1 to n do xi← choice( true, false ) if E(x1, x2, … ,xn) is true Checking then success else failure

40. Transforming the NP searching algorithm to the SAT problem • Does there exist a number in { x(1), x(2), …, x(n) }, which is equal to 7? • Assume n = 2

41. Transforming searching to SAT i=1 v i=2 & i=1 → i≠2 & i=2 → i≠1 & x(1)=7 & i=1 → SUCCESS & x(2)=7 & i=2 → SUCCESS & x(1)≠7 & i=1 → FAILURE & x(2)≠7 & i=2 → FAILURE & FAILURE → -SUCCESS & SUCCESS (Guarantees a successful termination) & x(1)=7 (Input Data) & x(2)≠7

42. Transforming searching to SAT • CNF (conjunctive normal form) : i=1 v i=2 (1) i≠1 v i≠2 (2) x(1)≠7 v i≠1 v SUCCESS (3) x(2)≠7 v i≠2 v SUCCESS (4) x(1)=7 v i≠1 v FAILURE (5) x(2)=7 v i≠2 v FAILURE (6) -FAILURE v -SUCCESS (7) SUCCESS (8) x(1)=7 (9) x(2)≠7 (10)

43. Transforming searching to SAT • Satisfiable at the following assignment : i=1 satisfying (1) i≠2 satisfying (2), (4) and (6) SUCCESS satisfying (3), (4) and (8) -FAILURE satisfying (7) x(1)=7 satisfying (5) and (9) x(2)≠7 satisfying (4) and (10)

44. Searching in CNF with inputs • Searching for 7, but x(1)7, x(2)7 CNF :

45. Searching in CNF with inputs • Apply resolution principle :

46. Searching in CNF with inputs

47. Q&A