1 / 58

Design and Analysis of Computer Algorithm Lecture 10

Design and Analysis of Computer Algorithm Lecture 10. Pradondet Nilagupta Department of Computer Engineering. Acknowledgement.

shawn
Download Presentation

Design and Analysis of Computer Algorithm Lecture 10

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Design and Analysis of Computer AlgorithmLecture 10 Pradondet Nilagupta Department of Computer Engineering

  2. Acknowledgement • This lecture note has been summarized from lecture note on Data Structure and Algorithm, Design and Analysis of Computer Algorithm all over the world. I can’t remember where those slide come from. However, I’d like to thank all professors who create such a good work on those lecture notes. Without those lectures, this slide can’t be finished.

  3. The theory of NP-completeness • Tractable and intractable problems • NP-complete problems

  4. Classifying problems • Here: Classify problems as tractable or intractable. • Problem is tractable if there exists a polynomial bound algorithm that solves it. • An algorithm is polynomial bound if its worst case growth rate can be bound by a polynomial p(n) in the size n of the problem

  5. What constitutes reasonable time? • Standard working definition: polynomial time • On an input of size n the worst-case running time is O(nk) for some constant k • Polynomial time: O(n2), O(n3), O(1), O(n lg n) • Not in polynomial time: O(2n), O(nn), O(n!)

  6. Polynomial-Time Algorithms • Are some problems solvable in polynomial time? • Of course: every algorithm we’ve studied provides polynomial-time solution to some problem • We define P to be the class of problems solvable in polynomial time • Are all problems solvable in polynomial time? • No: Turing’s “Halting Problem” is not solvable by any computer, no matter how much time is given • Such problems are clearly intractable, not in P

  7. Intractable problems • Problem is intractable if it is not tractable. • Any algorithm that solves it is not polynomial bound. • It has a worst case growth rate f(n) which cannot be bound by a polynomial p(n) in the size n of the problem. • For intractable problems the bounds are:

  8. Why is this classification useful? • If problem is intractable, no point in trying to find an efficient algorithm • Any algorithm too slow for large inputs. • To solve use approximations, heuristics, etc. • Sometimes we need to solve only a restricted version of the problem. • If restricted problem tractable design an algorithm for restricted problem

  9. Intractable problems • Turing showed some problems so hard that no algorithm can solve them (undecidable) • Other researchers showed some decidable problems from automata, mathematical logic, etc. are intractable • These problems are so hard that they cannot be solved in polynomial time by a “nondeterministic” computer

  10. Hard practical problems • Many practical problems for which no one has yet found a polynomial bound algorithm. • Examples: traveling salesperson, knapsack, graph coloring, etc. • Most design automation problems such as testing and routing. • Many networks, database and graph problems.

  11. How are they solved? • A variety of algorithms based on backtracking, branch and bound, etc. • None can be shown to be polynomial bound • Problems can be solved by a polynomial bound verification algorithm

  12. The theory of NP completeness • The theory of NP-completeness enables showing that these problems are at least as hard as NP-complete problems • Practical implication of knowing problem is NP-complete is that it is probably intractable ( whether it is or not has not been proved yet) • So any algorithm that solves it will probably be very slow for large inputs

  13. We need to define • Decision problems • The class P • Nondeterministic algorithms • The class NP • The concept of polynomial transformations • The class of NP-complete problems

  14. The theory of NP-Completeness • Decision problems • Converting optimization problems into decision problems • The relationship between an optimization problem and its decision version • The class P • Verification algorithms • The class NP

  15. Decision Problems • A decision problem answers yes or no for a given input • Examples: • Is there a path from s to t of length at most k? • Does graph G contain a Hamiltonian cycle?

  16. A decision problem: HAMILTONIAN CYCLE • A hamiltonian cycle of a graph G is a cycle that includes each vertex of the graph exactly once. • Problem: Given a graph G, does G have a hamiltonian cycle?

  17. Converting to decision problems • Optimization problems can be converted to decision problems (typically) by adding a bound B on the value to optimize, and asking the question: • Is there a solution whose value is at most B? (for a minimization problem) • Is there a solution whose value is at least B? (for a maximization problem)

  18. An optimization problem: traveling salesman (TS) • Given: • A finite set C={c1,...,cm} of cities, • A distance function d(ci, cj) of nonnegative numbers. • Find the length of the minimum distance tour which includes every city exactly once

  19. A decision problem: traveling salesman • Given a finite set C={c1,...,cm} of cities, a distance function d(ci, cj) of nonnegative numbers and a bound B • Is there a tour of all the cities (in which each city is visited exactly once) with total length at most B?

  20. The relation between • If we have a solution to the optimization problem we can compare the solution to the bound and answer “yes” or “no”. • Therefore if the optimization problem is tractable so is the decision problem • If the decision problem is “hard” the optimization problems are also “hard”

  21. Class of Problems: P and NP • Definition: The class P • P is the class of decision problems that are polynomially bounded. • there exist a deterministic algorithm • Definition: The class NP • NP is the class of decision problems for which there is a polynomially bounded non-deterministic algorithm. • The name NP comes from “Non-deterministic Polynomially bounded.” • there exist a non-deterministic algorithm • Theorem: P NP

  22. The goal of verification algorithms • The goal of a verification algorithm is to verify a “yes” answer to a decision problem’s input. • The inputs to the verification algorithm are the original input and a certificate (possible solution)

  23. Example • A verification algorithm for TS, verifies that a given TS tour has length at most B

  24. A verification algorithm for PATH • Given the problem PATH (does there exist a path of length k or less in a graph G between vertices u and v?), and a certificate p. • It is simple to verify that the length of p is at most k (we have to also check that p is indeed a path from u to v).

  25. Verification Algorithms • Other problems like HAMILTONIAN CYCLE are not known to have polynomial bound algorithms but given a hamiltonian cycle, it is easy to verify that the cycle is indeed hamiltonian in polynomial time. • A verification algorithm, takes a problem instance x and verifies it, if there exists a certificate y such that the answer for x with certificate y is “yes”

  26. Polynomial bound verification algorithms • Given a decision problem d. • A verification algorithm for d is polynomial bound if given an input x to d, there exists a certificate y, such that |y|=O(|x|c) where c is a constant, and a polynomial bound algorithm A(x, y) that verifies an answer “yes” for d with input x

  27. The class NP • NP is the class of decision problems for which there is a polynomial bounded verification algorithm • It can be shown that: • all decision problems in P, and • decision problems such as traveling salesman and knapsack are also in NP

  28. A non-deterministic algorithm • The non-deterministic “guessing” phase. • Some completely arbitrary string s, “proposed solution” • each time the algorithm is run the string may differ • The deterministic “verifying” phase. • a deterministic algorithm takes the input of the problem and the proposed solution s, and • return value true or false • The output step. • If the verifying phase returned true, the algorithm outputs yes. Otherwise, there is no output.

  29. P and NP • Summary so far: • P = problems that can be solved in polynomial time • NP = problems for which a solution can be verified in polynomial time • Unknown whether P = NP (most suspect not) • Hamiltonian-cycle problem is in NP: • Cannot solve in polynomial time • Easy to verify solution in polynomial time (How?)

  30. A Problem Which is in NP • Can solve variant of TSP which is in form of a decision problem • TSP*: Given a complete directed graph G with cost for each edge, and an integer k. Return YES, if there is a tour with total distance  k; NO otherwise • Can be solved in polynomial time with nondeterministic computer • How? • Cannot be converted to polynomial time algorithm for regular computer • Why?

  31. NP-Complete Problems • We will see that NP-Complete problems are the “hardest” problems in NP: • If any one NP-Complete problem can be solved in polynomial time… • …then every NP-Complete problem can be solved in polynomial time… • …and in fact every problem in NP can be solved in polynomial time (which would show P = NP) • Thus: solve hamiltonian-cycle in O(n100) time, you’ve proved that P = NP. Retire rich & famous.

  32. The Class NP-Complete (1/2) • A problem Q is NP-complete • if it is in NP and • it is NP-hard. • A problem Q is NP-hard • if every problem in NP • is reducible to Q. Theorem: Let A be in NP-Complete, and B is in NP. If A PB, then B is also NP-complete.

  33. The Class NP-Complete (2/2) • A problem P is reducible to a problem Q if • there exists a polynomial reduction function T such that • For every string x, • if x is a yes input for P, then T(x) is a yes input for Q • if x is a no input for P, then T(x) is a no input for Q. • T can be computed in polynomially bounded time.

  34. Class P and Class Relationships • Problems that are solvable in polynomial time on a regular computer are said to be in class P • All problems in P are solvable in p-time on nondeterministic computer • Some problems in NP are NP-complete • e.g., Clique problem for undirected graphs • All problems solvable in exponential time is an even bigger class • Note that all problems solvable in p-time are certainly solvable in exponential time

  35. Theoretician’s View of World Exponential time problems NP problems TOH NP-Complete problems TSP* P problems SORTING

  36. Polynomial Reductions T(x) Algorithm for Q T Yes or no Answer x (an input For P) An input for Q • Problem P is reducible to Q • P p Q • Transforming inputs of P to inputs of Q • Reducibility relation is transitive.

  37. Reduction • The crux of NP-Completeness is reducibility • Informally, a problem P can be reduced to another problem Q if any instance of P can be “easily rephrased” as an instance of Q, the solution to which provides a solution to the instance of P • What do you suppose “easily” means? • This rephrasing is called transformation • Intuitively: If P reduces to Q, P is “no harder to solve” than Q

  38. Reducibility • An example: • P: Given a set of Booleans, is at least one TRUE? • Q: Given a set of integers, is their sum positive? • Transformation: (x1, x2, …, xn) = (y1, y2, …, yn) where yi = 1 if xi = TRUE, yi = 0 if xi = FALSE • Another example: • Solving linear equations is reducible to solving quadratic equations • How can we easily use a quadratic-equation solver to solve linear equations?

  39. Using Reductions • If P is polynomial-time reducible to Q, we denote this P p Q • Definition of NP-Hard and NP-Complete: • If all problems R  NP are reducible to P, then P is NP-Hard • We say P is NP-Complete if P is NP-Hard and P  NP • If P p Q and P is NP-Complete, Q is alsoNP - Complete • This is the key idea you should take away today

  40. Why Prove NP-Completeness? • Though nobody has proven that P != NP, if you prove a problem NP-Complete, most people accept that it is probably intractable • Therefore it can be important to prove that a problem is NP-Complete • Don’t need to come up with an efficient algorithm • Can instead work on approximation algorithms

  41. Proving NP-Completeness • What steps do we have to take to prove a problem Pis NP-Complete? • Pick a known NP-Complete problem Q • Reduce Q to P • Describe a transformation that maps instances of Q to instances of P, s.t. “yes” for P = “yes” for Q • Prove the transformation works • Prove it runs in polynomial time • Oh yeah, prove P NP (What if you can’t?)

  42. If you tell me that this graph is 3-colourable, it is very difficult for me to check whether you are right.

  43. But if you tell me that this graph is 3-colorable and give me a solution, it is very easy for me to verify whether you are right. Loosely speaking, problems that are difficult to compute, but easy to verify are known as Non-deterministic Polynomial.

  44. Cook’s Theorem Any NP problem can be converted to SATin polynomial time.

  45. The SAT Problem • One of the first problems to be proved NP-Complete was satisfiability (SAT): • Given a Boolean expression on n variables, can we assign values such that the expression is TRUE? • Ex: ((x1x2)  ((x1 x3)  x4)) x2 • Cook’s Theorem: The satisfiability problem is NP-Complete • Note: Argue from first principles, not reduction • Proof: not here

  46. Conjunctive Normal Form • Even if the form of the Boolean expression is simplified, the problem may be NP-Complete • Literal: an occurrence of a Boolean or its negation • A Boolean formula is in conjunctive normal form, or CNF, if it is an AND of clauses, each of which is an OR of literals • Ex: (x1  x2)  (x1  x3  x4)  (x5) • 3-CNF: each clause has exactly 3 distinct literals • Ex: (x1  x2  x3)  (x1  x3  x4)  (x5  x3  x4) • Notice: true if at least one literal in each clause is true

  47. The 3-CNF Problem • Satisfiability of Boolean formulas in 3-CNF form (the 3-CNF Problem) is NP-Complete • Proof: Nope • The reason we care about the 3-CNF problem is that it is relatively easy to reduce to others • Thus by proving 3-CNF NP-Complete we can prove many seemingly unrelated problems NP-Complete

  48. 3-CNF  Clique • What is acliqueof a graph G? • A: a subset of vertices fully connected to each other, i.e. a complete subgraph of G • The clique problem: how large is the maximum-size clique in a graph? • Can we turn this into a decision problem? • A: Yes, we call this the k-clique problem • Is the k-clique problem within NP? this graph contains a 4-clique

  49. 3-CNF  Clique • What should the reduction do? • A: Transform a 3-CNF formula to a graph, for which a k-clique will exist (for some k) iff the 3-CNF formula is satisfiable

  50. 3-CNF  Clique • The reduction: • Let B = C1  C2  …  Ck be a 3-CNF formula with k clauses, each of which has 3 distinct literals • For each clause put a triple of vertices in the graph, one for each literal • Put an edge between two vertices if they are in different triples and their literals are consistent, meaning not each other’s negation • Run an example: B = (x  y  z)  (x  y  z )  (x  y  z )

More Related