CSC 413/513: Intro to Algorithms

1. CSC 413/513: Intro to Algorithms NP Completeness

2. Problems and Instances • Problem: Sort(A) • Problem instance: Sort([4,2,7,1,6,9]) • Input encoding • Bin(4) concat Bin(2) concat Bin(7) … • Size: n lg x • n: size of A • lg x: max. number of bits required to represent A[i], for any i • We say size = ϴ(n), or simply n

3. NP-Completeness • Some problems are intractable: as the input size increases, we are unable to solve them in reasonable time • What constitutes reasonable time? Standard working definition: polynomial time • On an input of size n the worst-case running time is O(nk) for some constantk • Polynomial time: O(n2), O(n3), O(1), O(n lg n) • Not in polynomial time: O(2n), O(nn), O(n!)

4. Polynomial-Time Problems • Are some problems solvable in polynomial time? • Of course: every algorithm we’ve studied provides polynomial-time solution to some problem • Except DP algorithm to 0-1 knapsack • We define P to be the class of problems solvable in polynomial time • That is, all instances of such problems are polynomially solvable (e.g., sorting) • These are the “easy” problems

5. Polynomial-Time Problems • Are all problems solvable in polynomial time? • No: Turing’s “Halting Problem” is not solvable by any computer, no matter how much time is given • But we often detect infinite loops in a program by inspection, and hence “solve” the halting problem • However, there are instances that are not solvable, even by our brains • Such problems are clearly intractable, not in P • Similarly, there are problems that can be solved, but we haven’t found algorithms that will take any instance of the problem and solve it in polynomial time

6. NP-Complete Problems • The NP-Complete problems are an interesting class of problems whose status is unknown • No polynomial-time algorithm has been discovered for an NP-Complete problem, for all instances of it • No suprapolynomial lower bound has been proved for some instance of any NP-Complete problem, either • We call this the P = NP question • The biggest open problem in CS

7. An NP-Complete Problem:Hamiltonian Cycles • An example of an NP-Complete problem: • A hamiltonian cycle of an undirected graph is a simple cycle that contains every vertex • The hamiltonian-cycle problem: given a graph G, does it have a hamiltonian cycle? • Describe a naïve algorithm for solving the hamiltonian-cycle problem. Running time?

8. P and NP • As mentioned, P is set of problems that can be solved in polynomial time • NP (nondeterministic polynomial time) is the set of problems that can be solved in polynomial time by a nondeterministic computer • What the hell is that?

9. Nondeterminism • Think of a non-deterministic computer as a computer that magically “guesses” a solution, and then has to verify that it is correct • If a solution exists, n.d. computer always guesses it • One way to imagine it: a parallel computer that can freely spawn an infinite number of processes • Have one processor work on each possible solution • All processors attempt to verify that their solution works, in parallel • If a processor finds it has a working solution • So: NP = problems verifiable in polynomial time • Note: PNP (think of verifying the output of sorting)

10. P and NP • Summary so far: • P = problems that can be solved in polynomial time • NP = problems for which a solution can be verified in polynomial time • PNP • Unknown whether P = NP (most suspect not) • Hamiltonian-cycle problem is in NP: • Don’t know how to solve in polynomial time • But, easy to verify solution in polynomial time (How?)

11. Intuition behind NP-Complete problems Does not have optimal substructure • Many problems exhibit structure that is exploited • E.g., recursive structure in sorting, optimal substructure in problems solved by DP, greedy etc. • But, some problems seem to have no exploitable structure • E.g., longest simple path in a graph • Contrast with shortest paths in graphs, which has the optimal substructure • Typically, these are the NP-Complete problems

12. What about 0-1 Knapsack? • Has optimal substructure • But input contains a number (W), and the optimal substructure depends on the magnitude of this number, not its size • Algorithm turns out to be exponential in the size of W (although polynomial in the magnitude of W) • Its DP Algorithm is called pseudo-polynomial • 0-1 knapsack is called weakly NP-complete

13. NP-Complete Problems • We will see that NP-Complete problems are the “hardest” problems in NP: • If any one NP-Complete problem can be solved in polynomial time… • …then every NP-Complete problem can be solved in polynomial time… • …and in fact every problem in NP can be solved in polynomial time (which would show P = NP) • Thus: solve hamiltonian-cycle in O(n100) time, you’ve proved that P = NP. Retire rich & famous.

14. A Simplification • Henceforth, we will only talk about decision problems, with boolean answers • Most problems of interest can be posed as decision problems • E.g., SHORTEST_PATH(G,u,v) • Its decision problem is PATH(G,u,v,k): is there a path between u and v involving no more than k edges? • Can solve SHORTEST_PATH by calling PATH repeatedly with decreasing k, until the answer is “No” • So, “solve” ≡ “decide”

15. How to Verify? • How do you verify the answer to a decision problem (in polynomial time)? • Need a certificate • E.g., certificate for PATH(G,u,v,k) would be the path: <u,v1,v2,…,v> • How do you verify the certificate? • Check length ≤ k • Check first vertex is u, last vertex is v • Check all path-edges actually exist • How much time?

16. Why only Decision Problems? • So that problems can be considered as languages • Then, “algorithm” ≡ Turing machine • Consider the binary encoding of a problem instance (i.e., the input) • Collect all input binary strings for which the answer/output is “Yes” • This set is a language • A machine that decides the language also solves the problem • So, “decision problem” ≡ “language”

17. Reduction • The crux of NP-Completeness is reducibility • Informally, a problem P can be reduced to another problem Q if any instance of P can be “easily rephrased” as an instance of Q, the solution to which provides a solution to the instance of P • What do you suppose “easily” means? • This rephrasing is called transformation • Intuitively: If P reduces to Q, P is “no harder to solve” than Q Polynomial time

18. Polynomial Reduction • Polynomial reduction can be used for polynomial time solution • If A polynomially reduces to B, and we know a polynomial algorithm to B, then we can solve A in polynomial time as well.

19. Reducibility • An example: • A: Given a set of Booleans, is at least one TRUE? • B: Given a set of integers, is their sum > 0? • Transformation: Given (x1, x2, …, xn) construct (y1, y2, …, yn) where yi = 1 if xi = TRUE, yi = 0 if xi = FALSE • Another example: • Solving linear equations is reducible to solving quadratic equations • How can we easily use a quadratic-equation solver to solve linear equations?

20. NP-Hard and NP-Complete • If P is polynomial-time reducible to Q, we denote this P p Q • Definition of NP-Hard and NP-Complete: • If all problems R  NP are reducible to P, then P is NP-Hard • We say P is NP-Complete if P is NP-Hard and P  NP • If P p Q and P is NP-Complete, Q is alsoNP- Complete

21. Why Prove NP-Completeness? • Though nobody has proven that P != NP, if you prove a problem NP-Complete, most people accept that it is probably intractable • Therefore it can be important to prove that a problem is NP-Complete • Don’t need to come up with an efficient algorithm • Can instead work on approximation algorithms

22. Proving NP-Completeness • What steps do we have to take to prove a problem Pis NP-Complete? • Pick a known NP-Complete problem Q • Reduce Q to P • Describe a transformation that maps an arbitrary instance of Q to some instance of P, s.t. “yes” for P = “yes” for Q • Prove the transformation works • Prove the transformation takes polynomial time • Oh yeah, prove P NP (What if you can’t?)

23. Coming Up • Given one NP-Complete problem, we can prove many interesting problems NP-Complete • Graph coloring (recall the wrestler-rivalry HW problem) • Hamiltonian cycle • Hamiltonian path • Knapsack problem • Traveling salesman • Job scheduling with penalties • Many, many more