1 / 27

The Complexity of Optimization Problems

The Complexity of Optimization Problems. Summary. Complexity of algorithms and problems Complexity classes: P and NP Reducibility Karp reducibility Turing reducibility NP-complete problems Complexity of optimization problems Classes PO and NPO NP-hard optimization problems.

giacomo-franklin
Download Presentation

The Complexity of Optimization Problems

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. The Complexity of Optimization Problems

  2. Summary • Complexity of algorithms and problems • Complexity classes: P and NP • Reducibility • Karp reducibility • Turing reducibility • NP-complete problems • Complexity of optimization problems • Classes PO and NPO • NP-hard optimization problems

  3. Uniform and logarithmic cost • Uniform cost: the overall number of instructions executed by the algorithm before halting • Logarithmic cost:each instruction has a cost depending on the number of bits of the operands • E.g. product of two n-bit integer costs O(nlogn) • Same for space measure (but we will talk only of time measure)

  4. Example: xy • Uniform cost: 2+3y • Logarithmic cost: aylogy+by2logx(logy+loglogx)+c begin r:=1; whiley  0 do begin r:=r*x; y:=y-1 end; return r end.

  5. Worst case analysis • Instances of the same size may result in different execution costs (e.g. sorting) • Cost of applying the algorithm on the worst case instance of a given size • Gives certainty that the algorithm will perform its task within the established time bound • It is easier to determine

  6. Input size • Size of input: number of bits needed to present the specific input • Existence of encoding scheme which is used to describe any problem instance • For any pair of natural encoding schemes and for any instance x, the resulting strings are polynomially related • I.e., |ei(x)|  pi,j(|ej(x)|) and |ej(x)|  pi,j(|ei(x)|) • Avoid unary base encoding

  7. Asymptotic Analysis • Let t(x) be the running time of algorithm A on input x. The worst case running time of A is given by t(n)=max(t(x) | x such that |x| n) • Upper bound: A has complexity O(f(n)) if t(n) is O(f(n)) (that is, we ignore constants) • Lower bound: A has complexity (f(n)) if t(n) is (f(n))

  8. Complexity of a problem • A problem P • has a complexity lower bound(f(n)) if any algorithm for P has complexity (f(n)) • has a complexity upper boundO(f(n)) if an algorithm for P exists with complexity O(f(n))

  9. Decision problems • Set of instances partitioned into a YES-subset and a NO-subset • Given an instance x, decide which subset x belongs to • A decision problem P is solved by an algorithm A if, for every instance, A halts and returns YES if and only if the instance belongs to the YES-subset

  10. Complexity Classes • For any function f(n), TIME(f(n)) is the set of decision problems which can be solved with a time complexity O(f(n)) • P = the union of TIME(nk) for all k • EXPTIME = the union of TIME (2nk) for all k • P is contained in EXPTIME • It is possible to prove (by diagonalization) that EXPTIME is not contained in P

  11. Examples • SATISFYING TRUTH ASSIGNMENT: given a CNF formula Fand a truth assignment f, does f satisfy F? • SATISFYING TRUTH ASSIGNMENT is in P • SATISFIABILITY (simply, SAT): given a CNF formula F, is F satisfiable? • SAT is in EXPTIME. • Open problem: SAT is in P?

  12. Class NP • A problem P is in class NP if there exist a polynomial p and a polynomial-time algorithm A such that, for any instance x, x is a YES-instance if and only if there exists a string y with |y| p(|x|) such that A(x,y) returns YES • y is said to be a certificate • Example: SAT is in NP (the certificate is a truth assignment that satisfies the formula) • P is contained in NP (the certificate is the computation of the polynomial-time algorithm)

  13. Non-deterministic algorithms: SAT begin for each variable v guess Boolean value f(v); iff satisfies Fthenreturn YES elsereturn NO end. (v1orv2or (notv3)) and ((notv1) or (notv2) orv3)

  14. Non-deterministic algorithms and NP begin guess string y with |y| p(|x|); ifA(x,y) returns YES thenreturn YES elsereturn NO end. Every problem in NP admits a polynomial-time non-deterministic algorithm Each computation path, which returns YES, is a certificate of polynomial length that can be checked in polynomial time Every problem that admits a polynomial-time non-deterministic algorithm is in NP

  15. Karp reducibility • A decision problem P1 is Karp reducible to a decision problem P2 (in short, P1 P2) if there exists a polynomial-time computable function R such that, for any x, x is a YES-instance of P1 if and only if R(x) is a YES-instance of P2 • If P1 P2 and P2 is in P, then P1 is in P

  16. Example: {0,1}-Linear programming • SAT {0,1}-LINEAR PROGRAMMING • For each Boolean variable v of a CNF Boolean formula F, we introduce a {0,1}-valued variable z • For each clause l1or l2or … or lk of F, we introduce the inequality z1+ z2 + … + zk 1, where zi= z if li= v and zi= (1-z) if li= not v • E.g. (v1orv2or (notv3)) and ((notv1) or (notv2) orv3) is transformed into the following two inequalities: z1+z2+(1-z3)  1 and(1-z1)+(1-z2) +z3  1 • If f is a truth assignment, let g be the natural corresponding {0,1}-value assignment (0=FALSE,1=TRUE) • f satisfies F if and only g satisfies all inequalities

  17. Turing reducibility • A decision problem P1 is Turing reducible to a decision problem P2 if there exists a polynomial-time algorithm R solving P1 such that R may access to an oracle algorithm solving P2 • If P1 P2 then P1 is Turing reducible to P2

  18. Example: Equivalent formulas • SAT is Turing reducible to EQUIVALENT FORMULAS • Given a CNF Boolean formula F, query the oracle with input F and xand (not x) • If the oracle answers YES, then F is not satisfiable, otherwise F is satisfiable • It is not known whether SAT is Karp reducible to EQUIVALENT FORMULAS

  19. NP-complete problems • A decision problem P is NP-complete if P is in NP and, for any decision problem P1 in NP, P1 P • If P is NP-complete and P is in P, then P=NP • NP-complete problems are the hardest in NP • P versus NP question can be solved by focusing on an NP-complete problem • Cook’s Theorem: SAT is NP-complete

  20. Optimization problem • Optimization problem P characterized by • Set of instances I • Function SOL that associates to any instance the set of feasible solutions • Measure function m that, for any feasible solution, provides its positive integer value • Goal, that is, either MAX or MIN • An optimal solution is a feasible solution y* such thatm(x,y*) = Goal{m(x,y) | yÎ SOL(x)} • For any instance x, m*(x) denotes optimal measure

  21. MINIMUM VERTEX COVER • INSTANCE: Graph G=(V,E) • SOLUTION: A subset U of V such that, for any edge (u,v), either u is in U or v is in U • MEASURE: Cardinality of U

  22. Three problems in one • Constructive problem: given an instance, compute an optimal solution and its value • We will study these problems • Evaluation problem: given an instance, compute the optimal value • Decision problem: given an instance and an integer k, decide whether the optimal value is at least (if Goal=MAX) or at most (if Goal=MIN) k

  23. Class NPO • Optimization problems such that • I is recognizable in polynomial time • Solutions are polynomially bounded and recognizable in polynomial time • m is computable in polynomial time • Example: MINIMUM VERTEX COVER • Theorem : If P is in NPO, then the corresponding decision problem is in NP

  24. Class PO • NPO problems solvable in polynomial time • Example: MINIMUM PATH • An optimization problem P is NP-hard if any problem in NP is Turing reducible to P • Theorem: If the decision problem corresponding to a NPO problem P is NP-complete, then P is NP-hard • Example: MINIMUM VERTEX COVER • Corollary: If P  NP then PO  NPO

  25. Evaluating versus constructing • Decision problem is Turing reducible to evaluation problem • Evaluation problem is Turing reducible to constructive problem • Evaluation problem is Turing reducible to decision problem • Binary search on space of possible measure values • Is constructive problem Turing reducible to evaluation problem?

  26. MAXIMUM SATISFIABILITY • INSTANCE: CNF Boolean formula, that is, set C of clauses over set of variables V • SOLUTION: A truth-assignment f to V • MEASURE: Number of satisfied clauses

  27. Evaluating versus constructing: MAX SAT begin for each variable v begin k := MAX SATeval(x); xTRUE:= formula obtained by setting v to TRUE in x; xFALSE:= formula obtained by setting v to FALSE in x; if MAX SATeval(xTRUE) = kthen begin f(v) := TRUE; x := xTRUE end else begin f(v) := FALSE; x := xFALSE end; return f end. Theorem: if the decision problem is NP-complete, then the constructive problem is Turing reducible to the decision problem

More Related