CS E 024 : Design & Analysis of Algorithms

1. CSE024: Design & Analysis of Algorithms Chapter 9: NP Completeness Sedgewick Chp:40 David Luebke’s Course Notes / University of Virginia, Computer Science

2. Course Content • Introduction, Algorithmic Notation and Flowcharts (Brassard & Bratley, Chap. 3) • Efficiency of Algorithms (Brassard & Bratley, Chap. 2) • Basic Data Structures (Brassard & Bratley, Chap.5) • Sorting (Weiss, Chap. 7) • Searching (Brassard & Bratley, Chap.9) • Graph Algorithms (Weiss, Chap. 9) • Randomized Algorithms (Weiss, Chap. 10) • String Searching (Sedgewick, Chap. 19) • NP Completeness (Sedgewick, Chap. 40) CS 615 Design & Analysis of Algorithms

3. Lecture Content • Intractability • P & NP Problems • Hamiltonian Path Problem • Reduction • NP-Hard and NP-Complete Problems • The Traveling Salesman Problem • Conjunctive Normal Form • Clique & Vertex Cover • General Comments CS 615 Design & Analysis of Algorithms

4. Tractibility • Some problems are undecidable: • no computer can solve them • E.g., Turing’s “Halting Problem” • take an Abstract Machines & Languages class • Other problems are decidable, but intractable: • as they grow large, • we are unable to solve them • in reasonable time • What constitutes reasonable time? • Polynomial algorithms! CS 615 Design & Analysis of Algorithms

5. Efficiency of Algorithms:polynomial time / non-polynomialtime • O(1) • O(n logn) • O(n2) • O(n3) • O(n10) • O(nlog n) • O(2n) • O(n!) • O(222n) polynomial time O(nc) for some constant c non-polynomial time CS 615 Design & Analysis of Algorithms

6. Polynomial Time Algorithms • Are there problems • solvable in polynomial time? • Answer is yes: • every algorithm we’ve studied • provides polynomial-time solution to some problems • Define P: • the class of problems • solvable in polynomial time • Can we solve all problems • that are solvable in polynomial time? • No: Turing’s “Halting Problem” • is not solvable by any computer • no matter how much time is given • Such problems are clearly intractable, not in P CS 615 Design & Analysis of Algorithms

7. Turing’s Halting Problem • Definition of algorithm • a procedure always terminates • and gives ananswer. • Formal Definition of Algorithm • a Turing machine that alwayshalts • whether succeeds or fails • A problem is solvable • if there is an algorithm • solves every instance of the givenproblem • A problem is said to be unsolvable • if no algorithm exists • to solve the givenproblem • The Halting Problem for Turing machines: • “Does there exist an effective procedure • for a computable function e(M) • to decide forevery pair (e(M),x); • does M halt for x?” • The Halting problem for Turing machine is unsolvable! • If it were solvable • There is a Turing Machine that can solve the halting problem • There is another Turing Machine to solve the problem above • ... CS 615 Design & Analysis of Algorithms

8. P and NP • P(Polynomial Time) • the set of problems • can be solved in polynomial time • NP (Nondeterministic Polynomial Time) • provably decidable in polynomial time • on a nondeterministic computer • PNP • The big question: • Does P = NP? CS 615 Design & Analysis of Algorithms

9. Determinist-Nondeterminist Machines Deterministic Finite State Automata Example Nondeterministic Finite State Automata Example CS 615 Design & Analysis of Algorithms

10. Reduction • A problem P can be reduced • to another problem Q • if any instance of P can be rephrased • to an instance of Q, • This rephrasing is called a transformation • Intuitively: • If P reduces in polynomial time to Q, • P is “no harder to solve” than Q CS 615 Design & Analysis of Algorithms

11. Reducibility: Example • P: Given a set of Booleans, • is at least one TRUE? • Q: Given a set of integers, • is their sum positive? • Transformation: • (x1, x2, …, xn) = (y1, y2, …, yn) • where yi = 1 if xi = TRUE, • yi = 0 if xi = FALSE CS 615 Design & Analysis of Algorithms

12. NP-Hard and NP-Complete • P p Q • P is polynomial-time reducible to Q, • If all problems R  NP • are reducible to P, • then P is NP-Hard • We say P is NP-Complete • if P is NP-Hard and P  NP • If P p Q and P is NP-Complete, • Q is alsoNP- Complete CS 615 Design & Analysis of Algorithms

13. Using Reductions • If P is polynomial-time reducible to Q, • we denote this P p Q • Definition of NP-Complete: • If P is NP-Complete, • P  NP • and all problems R are reducible to P • Formally: R p P  R  NP • If P p Q and P is NP-Complete, • Q is also NP-Complete CS 615 Design & Analysis of Algorithms

14. Popular NP-Complete Problems • Given one NP-Complete problem, • we can prove many interesting problems • are NP-Complete • Popular Problems: • Boolean satisfiability problem (Sat.) • N-puzzle • Knapsack problem • Hamiltonian path problem • Travelling salesman problem • Subgraph isomorphism problem • Subset sum problem • Clique problem • Vertex cover problem • Independent set problem • Dominating set problem • Graph coloring problem CS 615 Design & Analysis of Algorithms

15. NP-Complete Problems • NP-Complete problems • are the “hardest” problems in NP: • If any NP-Complete problem • can be solved in polynomial time • then every NP-Complete problems • can be solved in polynomial time • To show every problem in NP • solve hamiltonian-cycle in O(n100) time • Then it is proved that P = NP CS 615 Design & Analysis of Algorithms

16. Hamiltonian Cycle & Graph • Given a graph G=(V,E) • Find a path • To visit all nodes • without visiting any nodes twice! • A Hamiltonian cycle, also: Hamiltonian circuit, vertex tour or graph cycle • a cycle that visits each vertex exactly once • except the vertex which is both the start and end, • and so is visited twice • A graph that contains a Hamiltonian cycle • is called a Hamiltonian graph. CS 615 Design & Analysis of Algorithms

17. The Traveling Salesman Problem • The well-known traveling salesman problem: • a salesman must travel to n cities, • visiting each city exactly once • and finishing where he begins • with minimum cost! • Model as complete graph with cost c(i,j) • to go from city i to city j • How would we turn this • into a decision problem? • Is there a simple cycle • over cities • with cost < k ? • ask if  a TSP with cost < k An optimal TSP tour through Germany’s 15 largest cities. It is the shortest among 43 589 145 600 possible tours visiting each city exactly once CS 615 Design & Analysis of Algorithms

18. Hamiltonian Cycle  TSP • How can we prove the TSP is NP-Complete? • A: Prove TSP  NP; • reduce the undirected hamiltonian cycle problem to TSP • TSP  NP: straightforward • Reduction: need to show that • if we can solve TSP • we can solve hamiltonian cycle problem CS 615 Design & Analysis of Algorithms

19. The Satisfiability (SAT) Problem • One of the first problems • to be proved NP-Complete • was satisfiability (SAT): • Given a Boolean expression on n variables, • can we assign values • such that the expression is TRUE? • Ex: ((x1x2)  ((x1 x3)  x4)) x2 • Cook’s Theorem: • The satisfiability problem is NP-Complete CS 615 Design & Analysis of Algorithms

20. Conjunctive Normal Form • A Boolean formula is • in conjunctive normal form, or CNF, • if it is an AND of clauses, • each of which is an OR of literals • Ex: (x1  x2)  (x1  x3  x4)  (x5) • 3-CNF: • each clause has exactly 3 distinct literals • Ex: (x1  x2  x3)  (x1  x3  x4)  (x5  x3  x4) • Result is true • if at least one literal in each clause is true CS 615 Design & Analysis of Algorithms

21. The 3-CNF Problem • Theorem 36.10: • Satisfiability of Boolean formulas in 3-CNF form • is NP-Complete • Proof: Nope • the 3-CNF problem is • it is relatively easy to reduce to others • Thus by proving 3-CNF NP-Complete • we can prove many seemingly unrelated problems • NP-Complete CS 615 Design & Analysis of Algorithms

22. 3-CNF  Clique • Cliqueof a graph G • a subset of vertices • fully connected to each other • i.e. a complete subgraph of G • The clique problem: • how large is the maximum-size clique in a graph? • k-clique problem • can we turn clique problem • into a decision problem? • Is the k-clique problem within NP? A graph with 23 1-vertex cliques (its vertices), 42 2-vertex cliques (its edges), 19 3-vertex cliques (the light blue triangles), 24-vertex cliques (dark blue). CS 615 Design & Analysis of Algorithms

23. Clique  Vertex Cover • A vertex cover for a graph G • is a set of vertices incident • to every edge in G • The vertex cover problem: • what is the minimum size vertex cover in G? • Restated as a decision problem: • does a vertex cover of size k exist in G? • Thm 36.12: • vertex cover is NP-Complete CS 615 Design & Analysis of Algorithms

24. Clique  Vertex Cover • First, show vertex cover in NP • Next, reduce k-clique to vertex cover • The complementGC of a graph G • contains exactly those edges not in G • Compute GC in polynomial time • G has a clique of size k • iff GC has a vertex cover of size |V| - k CS 615 Design & Analysis of Algorithms

25. Clique  Vertex Cover • Claim: • If G has a clique of size k, • GC has a vertex cover of size |V| - k • Let V’ be the k-clique • Then V - V’ is a vertex cover in GC • Let (u,v) be any edge in GC • Then u and v cannot both be in V’ • Why? • Thus at least one of u or v is in V-V’ • Why? • So edge (u, v) is covered by V-V’ • Since true for any edge in GC, • V-V’ is a vertex cover CS 615 Design & Analysis of Algorithms

26. Clique  Vertex Cover • Claim: • If GC has a vertex cover V’  V, • with |V’| = |V| - k, • then G has a clique of size k • For all u,v V, • if (u,v)  GC then u  V’ or v  V’ or both • Why? • Contrapositive: • if u  V’ and v  V’, • Then (u,v)  E • In other words, • all vertices in V-V’ • are connected by an edge, • thus V-V’ is a clique • Since |V| - |V’| = k, • the size of the clique is k CS 615 Design & Analysis of Algorithms

27. General Comments • Literally hundreds of problems • have been shown to be NP-Complete • Some reductions are profound, • some are comparatively easy, • many are easy once the key insight is given CS 615 Design & Analysis of Algorithms

28. Some NP-Complete Problems • Subset-sum: • Given a set of integers, • does there exist a subset • adds up to some target T? • 0-1 knapsack: • when weights not just integers • Hamiltonian path: Obvious • Graph coloring: • can a given graph be colored • with k colors • such that no adjacent vertices are the same color? • Etc… CS 615 Design & Analysis of Algorithms

29. Chapter 9 NP Completeness CS 615 Design & Analysis of Algorithms