Create Presentation
Download Presentation

Download Presentation
## Design and Analysis of Computer Algorithm Lecture 10

- - - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - - -

**Design and Analysis of Computer AlgorithmLecture 10**Pradondet Nilagupta Department of Computer Engineering**Acknowledgement**• This lecture note has been summarized from lecture note on Data Structure and Algorithm, Design and Analysis of Computer Algorithm all over the world. I can’t remember where those slide come from. However, I’d like to thank all professors who create such a good work on those lecture notes. Without those lectures, this slide can’t be finished.**The theory of NP-completeness**• Tractable and intractable problems • NP-complete problems**Classifying problems**• Here: Classify problems as tractable or intractable. • Problem is tractable if there exists a polynomial bound algorithm that solves it. • An algorithm is polynomial bound if its worst case growth rate can be bound by a polynomial p(n) in the size n of the problem**What constitutes reasonable time?**• Standard working definition: polynomial time • On an input of size n the worst-case running time is O(nk) for some constant k • Polynomial time: O(n2), O(n3), O(1), O(n lg n) • Not in polynomial time: O(2n), O(nn), O(n!)**Polynomial-Time Algorithms**• Are some problems solvable in polynomial time? • Of course: every algorithm we’ve studied provides polynomial-time solution to some problem • We define P to be the class of problems solvable in polynomial time • Are all problems solvable in polynomial time? • No: Turing’s “Halting Problem” is not solvable by any computer, no matter how much time is given • Such problems are clearly intractable, not in P**Intractable problems**• Problem is intractable if it is not tractable. • Any algorithm that solves it is not polynomial bound. • It has a worst case growth rate f(n) which cannot be bound by a polynomial p(n) in the size n of the problem. • For intractable problems the bounds are:**Why is this classification useful?**• If problem is intractable, no point in trying to find an efficient algorithm • Any algorithm too slow for large inputs. • To solve use approximations, heuristics, etc. • Sometimes we need to solve only a restricted version of the problem. • If restricted problem tractable design an algorithm for restricted problem**Intractable problems**• Turing showed some problems so hard that no algorithm can solve them (undecidable) • Other researchers showed some decidable problems from automata, mathematical logic, etc. are intractable • These problems are so hard that they cannot be solved in polynomial time by a “nondeterministic” computer**Hard practical problems**• Many practical problems for which no one has yet found a polynomial bound algorithm. • Examples: traveling salesperson, knapsack, graph coloring, etc. • Most design automation problems such as testing and routing. • Many networks, database and graph problems.**How are they solved?**• A variety of algorithms based on backtracking, branch and bound, etc. • None can be shown to be polynomial bound • Problems can be solved by a polynomial bound verification algorithm**The theory of NP completeness**• The theory of NP-completeness enables showing that these problems are at least as hard as NP-complete problems • Practical implication of knowing problem is NP-complete is that it is probably intractable ( whether it is or not has not been proved yet) • So any algorithm that solves it will probably be very slow for large inputs**We need to define**• Decision problems • The class P • Nondeterministic algorithms • The class NP • The concept of polynomial transformations • The class of NP-complete problems**The theory of NP-Completeness**• Decision problems • Converting optimization problems into decision problems • The relationship between an optimization problem and its decision version • The class P • Verification algorithms • The class NP**Decision Problems**• A decision problem answers yes or no for a given input • Examples: • Is there a path from s to t of length at most k? • Does graph G contain a Hamiltonian cycle?**A decision problem: HAMILTONIAN CYCLE**• A hamiltonian cycle of a graph G is a cycle that includes each vertex of the graph exactly once. • Problem: Given a graph G, does G have a hamiltonian cycle?**Converting to decision problems**• Optimization problems can be converted to decision problems (typically) by adding a bound B on the value to optimize, and asking the question: • Is there a solution whose value is at most B? (for a minimization problem) • Is there a solution whose value is at least B? (for a maximization problem)**An optimization problem: traveling salesman (TS)**• Given: • A finite set C={c1,...,cm} of cities, • A distance function d(ci, cj) of nonnegative numbers. • Find the length of the minimum distance tour which includes every city exactly once**A decision problem: traveling salesman**• Given a finite set C={c1,...,cm} of cities, a distance function d(ci, cj) of nonnegative numbers and a bound B • Is there a tour of all the cities (in which each city is visited exactly once) with total length at most B?**The relation between**• If we have a solution to the optimization problem we can compare the solution to the bound and answer “yes” or “no”. • Therefore if the optimization problem is tractable so is the decision problem • If the decision problem is “hard” the optimization problems are also “hard”**Class of Problems: P and NP**• Definition: The class P • P is the class of decision problems that are polynomially bounded. • there exist a deterministic algorithm • Definition: The class NP • NP is the class of decision problems for which there is a polynomially bounded non-deterministic algorithm. • The name NP comes from “Non-deterministic Polynomially bounded.” • there exist a non-deterministic algorithm • Theorem: P NP**The goal of verification algorithms**• The goal of a verification algorithm is to verify a “yes” answer to a decision problem’s input. • The inputs to the verification algorithm are the original input and a certificate (possible solution)**Example**• A verification algorithm for TS, verifies that a given TS tour has length at most B**A verification algorithm for PATH**• Given the problem PATH (does there exist a path of length k or less in a graph G between vertices u and v?), and a certificate p. • It is simple to verify that the length of p is at most k (we have to also check that p is indeed a path from u to v).**Verification Algorithms**• Other problems like HAMILTONIAN CYCLE are not known to have polynomial bound algorithms but given a hamiltonian cycle, it is easy to verify that the cycle is indeed hamiltonian in polynomial time. • A verification algorithm, takes a problem instance x and verifies it, if there exists a certificate y such that the answer for x with certificate y is “yes”**Polynomial bound verification algorithms**• Given a decision problem d. • A verification algorithm for d is polynomial bound if given an input x to d, there exists a certificate y, such that |y|=O(|x|c) where c is a constant, and a polynomial bound algorithm A(x, y) that verifies an answer “yes” for d with input x**The class NP**• NP is the class of decision problems for which there is a polynomial bounded verification algorithm • It can be shown that: • all decision problems in P, and • decision problems such as traveling salesman and knapsack are also in NP**A non-deterministic algorithm**• The non-deterministic “guessing” phase. • Some completely arbitrary string s, “proposed solution” • each time the algorithm is run the string may differ • The deterministic “verifying” phase. • a deterministic algorithm takes the input of the problem and the proposed solution s, and • return value true or false • The output step. • If the verifying phase returned true, the algorithm outputs yes. Otherwise, there is no output.**P and NP**• Summary so far: • P = problems that can be solved in polynomial time • NP = problems for which a solution can be verified in polynomial time • Unknown whether P = NP (most suspect not) • Hamiltonian-cycle problem is in NP: • Cannot solve in polynomial time • Easy to verify solution in polynomial time (How?)**A Problem Which is in NP**• Can solve variant of TSP which is in form of a decision problem • TSP*: Given a complete directed graph G with cost for each edge, and an integer k. Return YES, if there is a tour with total distance k; NO otherwise • Can be solved in polynomial time with nondeterministic computer • How? • Cannot be converted to polynomial time algorithm for regular computer • Why?**NP-Complete Problems**• We will see that NP-Complete problems are the “hardest” problems in NP: • If any one NP-Complete problem can be solved in polynomial time… • …then every NP-Complete problem can be solved in polynomial time… • …and in fact every problem in NP can be solved in polynomial time (which would show P = NP) • Thus: solve hamiltonian-cycle in O(n100) time, you’ve proved that P = NP. Retire rich & famous.**The Class NP-Complete (1/2)**• A problem Q is NP-complete • if it is in NP and • it is NP-hard. • A problem Q is NP-hard • if every problem in NP • is reducible to Q. Theorem: Let A be in NP-Complete, and B is in NP. If A PB, then B is also NP-complete.**The Class NP-Complete (2/2)**• A problem P is reducible to a problem Q if • there exists a polynomial reduction function T such that • For every string x, • if x is a yes input for P, then T(x) is a yes input for Q • if x is a no input for P, then T(x) is a no input for Q. • T can be computed in polynomially bounded time.**Class P and Class Relationships**• Problems that are solvable in polynomial time on a regular computer are said to be in class P • All problems in P are solvable in p-time on nondeterministic computer • Some problems in NP are NP-complete • e.g., Clique problem for undirected graphs • All problems solvable in exponential time is an even bigger class • Note that all problems solvable in p-time are certainly solvable in exponential time**Theoretician’s View of World**Exponential time problems NP problems TOH NP-Complete problems TSP* P problems SORTING**Polynomial Reductions**T(x) Algorithm for Q T Yes or no Answer x (an input For P) An input for Q • Problem P is reducible to Q • P p Q • Transforming inputs of P to inputs of Q • Reducibility relation is transitive.**Reduction**• The crux of NP-Completeness is reducibility • Informally, a problem P can be reduced to another problem Q if any instance of P can be “easily rephrased” as an instance of Q, the solution to which provides a solution to the instance of P • What do you suppose “easily” means? • This rephrasing is called transformation • Intuitively: If P reduces to Q, P is “no harder to solve” than Q**Reducibility**• An example: • P: Given a set of Booleans, is at least one TRUE? • Q: Given a set of integers, is their sum positive? • Transformation: (x1, x2, …, xn) = (y1, y2, …, yn) where yi = 1 if xi = TRUE, yi = 0 if xi = FALSE • Another example: • Solving linear equations is reducible to solving quadratic equations • How can we easily use a quadratic-equation solver to solve linear equations?**Using Reductions**• If P is polynomial-time reducible to Q, we denote this P p Q • Definition of NP-Hard and NP-Complete: • If all problems R NP are reducible to P, then P is NP-Hard • We say P is NP-Complete if P is NP-Hard and P NP • If P p Q and P is NP-Complete, Q is alsoNP - Complete • This is the key idea you should take away today**Why Prove NP-Completeness?**• Though nobody has proven that P != NP, if you prove a problem NP-Complete, most people accept that it is probably intractable • Therefore it can be important to prove that a problem is NP-Complete • Don’t need to come up with an efficient algorithm • Can instead work on approximation algorithms**Proving NP-Completeness**• What steps do we have to take to prove a problem Pis NP-Complete? • Pick a known NP-Complete problem Q • Reduce Q to P • Describe a transformation that maps instances of Q to instances of P, s.t. “yes” for P = “yes” for Q • Prove the transformation works • Prove it runs in polynomial time • Oh yeah, prove P NP (What if you can’t?)**If you tell me that this graph is 3-colourable,**it is very difficult for me to check whether you are right.**But if you tell me that this graph is 3-colorable and**give me a solution, it is very easy for me to verify whether you are right. Loosely speaking, problems that are difficult to compute, but easy to verify are known as Non-deterministic Polynomial.**Cook’s Theorem**Any NP problem can be converted to SATin polynomial time.**The SAT Problem**• One of the first problems to be proved NP-Complete was satisfiability (SAT): • Given a Boolean expression on n variables, can we assign values such that the expression is TRUE? • Ex: ((x1x2) ((x1 x3) x4)) x2 • Cook’s Theorem: The satisfiability problem is NP-Complete • Note: Argue from first principles, not reduction • Proof: not here**Conjunctive Normal Form**• Even if the form of the Boolean expression is simplified, the problem may be NP-Complete • Literal: an occurrence of a Boolean or its negation • A Boolean formula is in conjunctive normal form, or CNF, if it is an AND of clauses, each of which is an OR of literals • Ex: (x1 x2) (x1 x3 x4) (x5) • 3-CNF: each clause has exactly 3 distinct literals • Ex: (x1 x2 x3) (x1 x3 x4) (x5 x3 x4) • Notice: true if at least one literal in each clause is true**The 3-CNF Problem**• Satisfiability of Boolean formulas in 3-CNF form (the 3-CNF Problem) is NP-Complete • Proof: Nope • The reason we care about the 3-CNF problem is that it is relatively easy to reduce to others • Thus by proving 3-CNF NP-Complete we can prove many seemingly unrelated problems NP-Complete**3-CNF Clique**• What is acliqueof a graph G? • A: a subset of vertices fully connected to each other, i.e. a complete subgraph of G • The clique problem: how large is the maximum-size clique in a graph? • Can we turn this into a decision problem? • A: Yes, we call this the k-clique problem • Is the k-clique problem within NP? this graph contains a 4-clique**3-CNF Clique**• What should the reduction do? • A: Transform a 3-CNF formula to a graph, for which a k-clique will exist (for some k) iff the 3-CNF formula is satisfiable**3-CNF Clique**• The reduction: • Let B = C1 C2 … Ck be a 3-CNF formula with k clauses, each of which has 3 distinct literals • For each clause put a triple of vertices in the graph, one for each literal • Put an edge between two vertices if they are in different triples and their literals are consistent, meaning not each other’s negation • Run an example: B = (x y z) (x y z ) (x y z )