1 / 58

# Design and Analysis of Computer Algorithm Lecture 10

Design and Analysis of Computer Algorithm Lecture 10. Pradondet Nilagupta Department of Computer Engineering. Acknowledgement. Download Presentation ## Design and Analysis of Computer Algorithm Lecture 10

E N D

### Presentation Transcript

1. Design and Analysis of Computer AlgorithmLecture 10 Pradondet Nilagupta Department of Computer Engineering

2. Acknowledgement • This lecture note has been summarized from lecture note on Data Structure and Algorithm, Design and Analysis of Computer Algorithm all over the world. I can’t remember where those slide come from. However, I’d like to thank all professors who create such a good work on those lecture notes. Without those lectures, this slide can’t be finished.

3. The theory of NP-completeness • Tractable and intractable problems • NP-complete problems

4. Classifying problems • Here: Classify problems as tractable or intractable. • Problem is tractable if there exists a polynomial bound algorithm that solves it. • An algorithm is polynomial bound if its worst case growth rate can be bound by a polynomial p(n) in the size n of the problem

5. What constitutes reasonable time? • Standard working definition: polynomial time • On an input of size n the worst-case running time is O(nk) for some constant k • Polynomial time: O(n2), O(n3), O(1), O(n lg n) • Not in polynomial time: O(2n), O(nn), O(n!)

6. Polynomial-Time Algorithms • Are some problems solvable in polynomial time? • Of course: every algorithm we’ve studied provides polynomial-time solution to some problem • We define P to be the class of problems solvable in polynomial time • Are all problems solvable in polynomial time? • No: Turing’s “Halting Problem” is not solvable by any computer, no matter how much time is given • Such problems are clearly intractable, not in P

7. Intractable problems • Problem is intractable if it is not tractable. • Any algorithm that solves it is not polynomial bound. • It has a worst case growth rate f(n) which cannot be bound by a polynomial p(n) in the size n of the problem. • For intractable problems the bounds are:

8. Why is this classification useful? • If problem is intractable, no point in trying to find an efficient algorithm • Any algorithm too slow for large inputs. • To solve use approximations, heuristics, etc. • Sometimes we need to solve only a restricted version of the problem. • If restricted problem tractable design an algorithm for restricted problem

9. Intractable problems • Turing showed some problems so hard that no algorithm can solve them (undecidable) • Other researchers showed some decidable problems from automata, mathematical logic, etc. are intractable • These problems are so hard that they cannot be solved in polynomial time by a “nondeterministic” computer

10. Hard practical problems • Many practical problems for which no one has yet found a polynomial bound algorithm. • Examples: traveling salesperson, knapsack, graph coloring, etc. • Most design automation problems such as testing and routing. • Many networks, database and graph problems.

11. How are they solved? • A variety of algorithms based on backtracking, branch and bound, etc. • None can be shown to be polynomial bound • Problems can be solved by a polynomial bound verification algorithm

12. The theory of NP completeness • The theory of NP-completeness enables showing that these problems are at least as hard as NP-complete problems • Practical implication of knowing problem is NP-complete is that it is probably intractable ( whether it is or not has not been proved yet) • So any algorithm that solves it will probably be very slow for large inputs

13. We need to define • Decision problems • The class P • Nondeterministic algorithms • The class NP • The concept of polynomial transformations • The class of NP-complete problems

14. The theory of NP-Completeness • Decision problems • Converting optimization problems into decision problems • The relationship between an optimization problem and its decision version • The class P • Verification algorithms • The class NP

15. Decision Problems • A decision problem answers yes or no for a given input • Examples: • Is there a path from s to t of length at most k? • Does graph G contain a Hamiltonian cycle?

16. A decision problem: HAMILTONIAN CYCLE • A hamiltonian cycle of a graph G is a cycle that includes each vertex of the graph exactly once. • Problem: Given a graph G, does G have a hamiltonian cycle?

17. Converting to decision problems • Optimization problems can be converted to decision problems (typically) by adding a bound B on the value to optimize, and asking the question: • Is there a solution whose value is at most B? (for a minimization problem) • Is there a solution whose value is at least B? (for a maximization problem)

18. An optimization problem: traveling salesman (TS) • Given: • A finite set C={c1,...,cm} of cities, • A distance function d(ci, cj) of nonnegative numbers. • Find the length of the minimum distance tour which includes every city exactly once

19. A decision problem: traveling salesman • Given a finite set C={c1,...,cm} of cities, a distance function d(ci, cj) of nonnegative numbers and a bound B • Is there a tour of all the cities (in which each city is visited exactly once) with total length at most B?

20. The relation between • If we have a solution to the optimization problem we can compare the solution to the bound and answer “yes” or “no”. • Therefore if the optimization problem is tractable so is the decision problem • If the decision problem is “hard” the optimization problems are also “hard”

21. Class of Problems: P and NP • Definition: The class P • P is the class of decision problems that are polynomially bounded. • there exist a deterministic algorithm • Definition: The class NP • NP is the class of decision problems for which there is a polynomially bounded non-deterministic algorithm. • The name NP comes from “Non-deterministic Polynomially bounded.” • there exist a non-deterministic algorithm • Theorem: P NP

22. The goal of verification algorithms • The goal of a verification algorithm is to verify a “yes” answer to a decision problem’s input. • The inputs to the verification algorithm are the original input and a certificate (possible solution)

23. Example • A verification algorithm for TS, verifies that a given TS tour has length at most B

24. A verification algorithm for PATH • Given the problem PATH (does there exist a path of length k or less in a graph G between vertices u and v?), and a certificate p. • It is simple to verify that the length of p is at most k (we have to also check that p is indeed a path from u to v).

25. Verification Algorithms • Other problems like HAMILTONIAN CYCLE are not known to have polynomial bound algorithms but given a hamiltonian cycle, it is easy to verify that the cycle is indeed hamiltonian in polynomial time. • A verification algorithm, takes a problem instance x and verifies it, if there exists a certificate y such that the answer for x with certificate y is “yes”

26. Polynomial bound verification algorithms • Given a decision problem d. • A verification algorithm for d is polynomial bound if given an input x to d, there exists a certificate y, such that |y|=O(|x|c) where c is a constant, and a polynomial bound algorithm A(x, y) that verifies an answer “yes” for d with input x

27. The class NP • NP is the class of decision problems for which there is a polynomial bounded verification algorithm • It can be shown that: • all decision problems in P, and • decision problems such as traveling salesman and knapsack are also in NP

28. A non-deterministic algorithm • The non-deterministic “guessing” phase. • Some completely arbitrary string s, “proposed solution” • each time the algorithm is run the string may differ • The deterministic “verifying” phase. • a deterministic algorithm takes the input of the problem and the proposed solution s, and • return value true or false • The output step. • If the verifying phase returned true, the algorithm outputs yes. Otherwise, there is no output.

29. P and NP • Summary so far: • P = problems that can be solved in polynomial time • NP = problems for which a solution can be verified in polynomial time • Unknown whether P = NP (most suspect not) • Hamiltonian-cycle problem is in NP: • Cannot solve in polynomial time • Easy to verify solution in polynomial time (How?)

30. A Problem Which is in NP • Can solve variant of TSP which is in form of a decision problem • TSP*: Given a complete directed graph G with cost for each edge, and an integer k. Return YES, if there is a tour with total distance  k; NO otherwise • Can be solved in polynomial time with nondeterministic computer • How? • Cannot be converted to polynomial time algorithm for regular computer • Why?

31. NP-Complete Problems • We will see that NP-Complete problems are the “hardest” problems in NP: • If any one NP-Complete problem can be solved in polynomial time… • …then every NP-Complete problem can be solved in polynomial time… • …and in fact every problem in NP can be solved in polynomial time (which would show P = NP) • Thus: solve hamiltonian-cycle in O(n100) time, you’ve proved that P = NP. Retire rich & famous.

32. The Class NP-Complete (1/2) • A problem Q is NP-complete • if it is in NP and • it is NP-hard. • A problem Q is NP-hard • if every problem in NP • is reducible to Q. Theorem: Let A be in NP-Complete, and B is in NP. If A PB, then B is also NP-complete.

33. The Class NP-Complete (2/2) • A problem P is reducible to a problem Q if • there exists a polynomial reduction function T such that • For every string x, • if x is a yes input for P, then T(x) is a yes input for Q • if x is a no input for P, then T(x) is a no input for Q. • T can be computed in polynomially bounded time.

34. Class P and Class Relationships • Problems that are solvable in polynomial time on a regular computer are said to be in class P • All problems in P are solvable in p-time on nondeterministic computer • Some problems in NP are NP-complete • e.g., Clique problem for undirected graphs • All problems solvable in exponential time is an even bigger class • Note that all problems solvable in p-time are certainly solvable in exponential time

35. Theoretician’s View of World Exponential time problems NP problems TOH NP-Complete problems TSP* P problems SORTING

36. Polynomial Reductions T(x) Algorithm for Q T Yes or no Answer x (an input For P) An input for Q • Problem P is reducible to Q • P p Q • Transforming inputs of P to inputs of Q • Reducibility relation is transitive.

37. Reduction • The crux of NP-Completeness is reducibility • Informally, a problem P can be reduced to another problem Q if any instance of P can be “easily rephrased” as an instance of Q, the solution to which provides a solution to the instance of P • What do you suppose “easily” means? • This rephrasing is called transformation • Intuitively: If P reduces to Q, P is “no harder to solve” than Q

38. Reducibility • An example: • P: Given a set of Booleans, is at least one TRUE? • Q: Given a set of integers, is their sum positive? • Transformation: (x1, x2, …, xn) = (y1, y2, …, yn) where yi = 1 if xi = TRUE, yi = 0 if xi = FALSE • Another example: • Solving linear equations is reducible to solving quadratic equations • How can we easily use a quadratic-equation solver to solve linear equations?

39. Using Reductions • If P is polynomial-time reducible to Q, we denote this P p Q • Definition of NP-Hard and NP-Complete: • If all problems R  NP are reducible to P, then P is NP-Hard • We say P is NP-Complete if P is NP-Hard and P  NP • If P p Q and P is NP-Complete, Q is alsoNP - Complete • This is the key idea you should take away today

40. Why Prove NP-Completeness? • Though nobody has proven that P != NP, if you prove a problem NP-Complete, most people accept that it is probably intractable • Therefore it can be important to prove that a problem is NP-Complete • Don’t need to come up with an efficient algorithm • Can instead work on approximation algorithms

41. Proving NP-Completeness • What steps do we have to take to prove a problem Pis NP-Complete? • Pick a known NP-Complete problem Q • Reduce Q to P • Describe a transformation that maps instances of Q to instances of P, s.t. “yes” for P = “yes” for Q • Prove the transformation works • Prove it runs in polynomial time • Oh yeah, prove P NP (What if you can’t?)

42. If you tell me that this graph is 3-colourable, it is very difficult for me to check whether you are right.

43. But if you tell me that this graph is 3-colorable and give me a solution, it is very easy for me to verify whether you are right. Loosely speaking, problems that are difficult to compute, but easy to verify are known as Non-deterministic Polynomial.

44. Cook’s Theorem Any NP problem can be converted to SATin polynomial time.

45. The SAT Problem • One of the first problems to be proved NP-Complete was satisfiability (SAT): • Given a Boolean expression on n variables, can we assign values such that the expression is TRUE? • Ex: ((x1x2)  ((x1 x3)  x4)) x2 • Cook’s Theorem: The satisfiability problem is NP-Complete • Note: Argue from first principles, not reduction • Proof: not here

46. Conjunctive Normal Form • Even if the form of the Boolean expression is simplified, the problem may be NP-Complete • Literal: an occurrence of a Boolean or its negation • A Boolean formula is in conjunctive normal form, or CNF, if it is an AND of clauses, each of which is an OR of literals • Ex: (x1  x2)  (x1  x3  x4)  (x5) • 3-CNF: each clause has exactly 3 distinct literals • Ex: (x1  x2  x3)  (x1  x3  x4)  (x5  x3  x4) • Notice: true if at least one literal in each clause is true

47. The 3-CNF Problem • Satisfiability of Boolean formulas in 3-CNF form (the 3-CNF Problem) is NP-Complete • Proof: Nope • The reason we care about the 3-CNF problem is that it is relatively easy to reduce to others • Thus by proving 3-CNF NP-Complete we can prove many seemingly unrelated problems NP-Complete

48. 3-CNF  Clique • What is acliqueof a graph G? • A: a subset of vertices fully connected to each other, i.e. a complete subgraph of G • The clique problem: how large is the maximum-size clique in a graph? • Can we turn this into a decision problem? • A: Yes, we call this the k-clique problem • Is the k-clique problem within NP? this graph contains a 4-clique

49. 3-CNF  Clique • What should the reduction do? • A: Transform a 3-CNF formula to a graph, for which a k-clique will exist (for some k) iff the 3-CNF formula is satisfiable

50. 3-CNF  Clique • The reduction: • Let B = C1  C2  …  Ck be a 3-CNF formula with k clauses, each of which has 3 distinct literals • For each clause put a triple of vertices in the graph, one for each literal • Put an edge between two vertices if they are in different triples and their literals are consistent, meaning not each other’s negation • Run an example: B = (x  y  z)  (x  y  z )  (x  y  z )

More Related