Hard Computational Problems

1. Hard Computational Problems • Some computational problems are hard • Despite a numerous attempts we do not know any efficient algorithms for these problems • We are also far away from the proof that these problems are indeed hard to solve, in other words • NP=P or NPP, this is a question … Complexity of Algorithms

2. Decision/Optimisation Problems • A decision problem (DP) is a computational problem for which the intended output is either yes or no • In an optimisation problem (OP) we rather try to maximise or minimisesome value • An OP can be turned into a DP if we add a parameter k, and then ask whether the optimal value in OP is at most or at least k • Note that if a DP is hard, then its related its optimisation version must be hard too Complexity of Algorithms

3. Decision/Optimisation Problems • Example: • Optimisation problem - Given graph G with integer weights on its edges. What is the weight of a minimum spanning tree (MST) in G? • Decision problem – Given graph G with integer weights on its edges, and an integer k. Does G have a minimum spanning tree of weight at most k? Complexity of Algorithms

4. Problems and Languages • We say that the algorithm Aaccepts an input string x if A outputs yes on input x • A decision problem can be viewed as a set L of (binary) strings – the strings that should be accepted by an algorithm that correctly solves the problem • We often refer to L as a language • We say that an algorithm Aaccepts a language L if A outputs yes for each x in L and outputs no otherwise Complexity of Algorithms

5. The Complexity Class P • The complexity class P is the set of all decision problems (or languages) L that can be solved in worst-case polynomial time • That is, there is an algorithm A that if x  L, then on input x, A outputs yes in time p(n), where n is the size (length) of x and p(n) is a polynomial Complexity of Algorithms

6. The Complement of a Language • The complement of a language L consists of all strings that are not in L • If we have a p(n) time algorithm A that acceptsL (i.e., Lis in P) we can construct a p(n) time algorithm B (based on A) that accepts the complement of L, i.e., • Run algorithm A on input string x for p(n) steps • If A outputs yes, then B outputs no • If A outputs no or give no output, then B outputs yes • If L is in P, the complement of L is in P too! Complexity of Algorithms

7. The Complexity Class NP • An algorithm that chooses (by a really good guess!) some number of non-deterministic bits during its execution is called a non-deterministic algorithm • We say that an algorithm Anon-deterministically accepts a string x if there exists a choice of non-deterministic bits that leads to the ultimate answer yes • The complexity class NP is the set of decision problems (or languages) L that can be non-deterministically accepted in polynomial time • Obviously P  NP Complexity of Algorithms

8. The Complement of L in NP • Note that the definition of class NP does not address the running time of rejection (which might be very long) • And indeed even knowing that we can choose an appropriate number of non-deterministic bits for all strings in L in NP we cannot assure that such a choice is feasible for the complement of L • In fact there is a class co-NP that consists of all languages whose complements are in NP • Many researchers believe that co-NP  NP Complexity of Algorithms

9. The P = NP Question • Computer scientists do not know for certain whether P = NP or not • We also do not know whether P = NP  co-NP • However there is a common believe that P is different then both NP and co-NP, as well as they intersection Complexity of Algorithms

10. Hamiltonian Cycle is NP • Hamiltonian-Cycle is the problem that takes a graph G as an input and asks whether there is a simple (Hamiltonian) cycle in G that visits every vertex of G exactly once • The non-deterministic algorithm chooses a cycle (represented by a sequence of non-deterministic bits) and then it checks deterministically whether this cycle is indeed a Hamiltonian cycle in G Complexity of Algorithms

11. Boolean Circuit • A Boolean circuit is a directed graph where each node, called a logic gate corresponds to a simple Boolean function AND, OR, or NOT • The incoming edges for a logic gate correspond to inputs for its Boolean function and the outgoing edges correspond to the outputs Complexity of Algorithms

12. Boolean Circuit (example) Complexity of Algorithms

13. Circuit-SAT is in NP • Circuit-Sat is the problem that takes an input a Boolean circuit with a single output node, and asks whether there is an assignment of values to the circuit’s inputs so that its output value is 1 • The non-deterministic algorithm chooses an assignment of input bits (represented by a sequence of non-deterministic bits) and then it checks deterministically whether this input generates output 1 Complexity of Algorithms

14. Vertex Cover • Given a graph G=(V,E), • a vertex cover for G is a subset CV, s.t., • for every edge (v,w) in E, vC or wC • The optimisation problem is to find as small a vertex cover as possible • Vertex-Cover is the decision problem that takes a graph G and an integer k as input, and asks whether there is a vertex cover for G containing at most k vertices Complexity of Algorithms

15. Vertex-Cover is in NP • Suppose we are given an integer k and a graph G • The non-deterministic algorithm chooses a subset of vertices C  V, s.t., ¦C¦  k, (represented by a sequence of non-deterministic bits) and then • it checks deterministically whether this subset C is an appropriate vertex cover Complexity of Algorithms

16. Polynomial-Time Reducibility • We say that a language L, defining some decision problem, is polynomial-time reducible to a language M, if • there is a function f computable in polynomial time, that • takes an input x to L, and transforms it to an input f(x) of M, s.t., • x  L if and only if f(x)  M • We use notation L poly M to signify that language L is polynomial-time reducible to language M Complexity of Algorithms

17. NP-hardness • We say that a language M, defining some decision problem, is NP-hard if every other language L in NP is polynomial-time reducible to M, i.e., • M is NP-hard, if for every L NP, L poly M • If a language M is NP-hard and it belongs to NP itself, then M is NP-complete • NP-complete problem is, in a very formal sense, one of the hardest problems in NP, as far as polynomial-time reducibility is concerned Complexity of Algorithms

18. The Cook-Levin Theorem • The Cook-Levin Theorem • Circuit-Sat is NP-complete • Proof [sketch]: A computation steps of any (reasonable) algorithm can be simulated by layers in appropriately constructed (in polynomial time and size) Boolean circuit Complexity of Algorithms

19. The Cook-Levin Theorem Complexity of Algorithms

20. Other NP-complete Problems • We have just noted that there is at least one NP-complete problem • Using polynomial-time reducibility we can show existence of other NP-complete problems according to • Lemma: • If L1poly L2 and L2poly L3 then L1poly L3 Complexity of Algorithms

21. Types of reduction • Let M be known NP-complete problem. The types of reductions are: • By restriction: noting that known NP-complete problem is a special case of our problem L • Local replacement: dividing instances of M and L into basic units, and then showing how each basic unit of M can be locally converted into a basic unit of L • Component design: building components for an instance of L that will enforce important structural functions for instances of M Complexity of Algorithms

22. Important NP-complete Problems Complexity of Algorithms

23. Conjunctive Normal Form • A Boolean formula is in conjunctive normal form (CNF) if it is formed as a collection of clauses combined using operator AND (·), where each clause is formed by literals (variables or their negations) combined using operator OR (+), e.g., Complexity of Algorithms

24. CNF-SAT & 3SAT • Problem CNF-SAT takes a Boolean formula in CNF form as input and asks if there is an assignment of Boolean values to its variables so that the formula evaluates to 1 (i.e., formula is satisfiable) • 3SAT is CNF-SAT in which each clause has exactly 3 literals • Fact: CNF-SAT and 3-SAT are NP-complete Complexity of Algorithms

25. Vertex-Cover is NP-complete • We can show that Vertex-Cover is NP-hard by reducing 3SAT problem to it in polynomial time • This reduction is an example of a reduction from a logic problem to a graph problem • It also illustrates an application of the component design proof technique Complexity of Algorithms

26. Vertex-Cover is NP-complete Complexity of Algorithms

27. Vertex-Cover is NP-complete Complexity of Algorithms

28. Vertex-Cover is NP-complete Complexity of Algorithms

29. Approximation Schemes • One way of dealing with NP-completeness for optimisation problems is to use an approximation algorithm • The goal of an approximation algorithm is to come as close to the optimum value as possible • Such an algorithm typically runs much faster than an algorithm that strives for exact solution Complexity of Algorithms

30. Approximation Schemes • Let c(S) be the value of a solution S delivered by an algorithm A to an optimisation problem P and OPT will be the optimal solution for P • We say that A is a -approximation algorithm for a minimisation problem P if • c(S)  ·OPT • And A is is a -approximation algorithm for a maximisation problem P if • c(S)  ·OPT Complexity of Algorithms

31. Polynomial-Time Approximation Scheme (PTAS) • There are some problems for which we can construct -approximation algorithms that run in polynomial time with =1+, for any fixed value  > 0 • The running time of such collection of algorithms depends both on n, the size of an input and also on a fixed value  • We refer to such collection of algorithms as a polynomial-time approximation scheme, or PTAS • If the running time is polynomial in both n and 1/ we have a fully polynomial-time approximation scheme Complexity of Algorithms