COMP108 Algorithmic Foundations Computational Complexity Theory

1. COMP108Algorithmic FoundationsComputational Complexity Theory Prudence Wong http://www.csc.liv.ac.uk/~pwong/teaching/comp108

2. Introduction to Computational Complexity Theory …

3. Hard Computational Problems An algorithm is efficient if its running time is bounded by a polynomial of its input size. Some computational problems seemshard to solve. Despite numerous attempts we do not know any efficient algorithms for these problems We are also far away from proving these problems are indeed hard to solve In more formal language, we don’t know whetherNP = P or NP ≠ P. This is an important and fundamental question in theoretical computer science!

4. Hard Computational Problems • Named as one of the seven "Millennium Problems" by theClayInstitute • can earn you $1 million for itssolution (and a place inmathematical and computer sciencehistory). • Check www.claymath.org/millennium/to read more about this (select the "P vs NP" link). An algorithm is efficient if its running time is bounded by a polynomial of its input size. Some computational problems seemshard to solve. Despite numerous attempts we do not know any efficient algorithms for these problems We are also far away from proving these problems are indeed hard to solve In more formal language, we don’t know whetherNP = P or NP ≠ P. This is an important and fundamental question in theoretical computer science!

5. Examples We have seen two examples of these difficult problems where no efficient (polynomial) algorithm is known Knapsack problem(lecture notes #2, slides #87-89) Hamiltonian circuit problem(lecture notes #4, slides #27) One more: Circuit-SAT All we know is to exhaust all possible solutions to find the best one

6. Hamiltonian Circuit Input: A connected graph G Question: Does G have a Hamiltonian circuit?i.e., does G have a circuit that passes through every vertex exactly once, except for the starting and ending vertex?

7. Knapsack Problem Input: Given n items with integer weights w1, w2, …, wn and integer values v1, v2, …, vn, and a knapsack with capacity W. Problem (optimisation version): Find a subset of items whose total weight does not exceed W and that maximises the total value. (taking fractional parts of items is NOT allowed) Known as 0/1 Knapsack Problem

8. Boolean Circuit In COMP109, you learned about propositional logic, and thebasic operations for combining truth values. A Boolean Circuit is a directed graph where each vertex, called alogic gate corresponds to a simple Boolean function, one ofAND, OR, or NOT. Incoming edges:inputs for itsBoolean functionOutgoing edges:outputs

9. Boolean Circuit - Example

10. Circuit-SAT Input:a Boolean Circuit with a single outputvertex Question:is there an assignment of values to the inputs so that theoutput value is 1? SAT means satisfiability

11. Circuit-SAT 0 1 1 0 0 1 0 1 1 1 0 0 1 1 0 1 0 1 1 1 1 0 1 1 0 1 0 0 4 inputs can give an answer 1

12. Decision/Optimisation problems A decision problem is a computational problem for which the output is either yes or no. In an optimisation problem, we try to maximise or minimise some value. An optimisation problem can be turned into a decision problem if we add a parameter k; and then ask whether the optimal value in the optimisation problem is at most or at least k. Note that if a decision problem is hard, then its related optimisation version must also be hard.

13. Example - MST Optimisation problem: Given a graph G with integer weights on its edges. What is the weight of a minimum spanning tree (MST) in G? Decision problem: Given a graph G with integer weights on its edges, and an integer k. Does G have a MST of weight at most k?

14. Example – Knapsack problem Input: Given n items with integer weights w1, w2, …, wn and integer values v1, v2, …, vn, a knapsack with capacity W and a value k. Optimisation problem: Find a subset of items whose total weight does not exceed W and that maximises the total value. Decision problem: For any integer k, is there a subset of items whose total weight does not exceed W and whose total value is at least k ?

15. Exercise State the decision version of the following problems • Given an array A[ ] of n numbers, find the maximum sum found in any contiguous subarray • Given a weighted graph G and a source vertex a, find the shortest paths from a to every other vertex

16. Exercise - Solution • Given an array A[ ] of n numbers and a value k, is there any contiguous subarray such that the sum is at least k ? • Given a weighted graph G, a source vertex a and a value k, are there shortest paths from a to every other vertex such that each path is of weight at most k ? How can solving a decision problem helps to find solution to the optimization problem?

17. Solving/Verifying a problem Solving a problem is different from verifying a problem • solving: we are given an input, and then we have to FIND the solution • verifying: in addition to the input, we are given a "certificate" and we verify whether the certificate is indeed a solution We may not know how to solve a problem efficiently, but we may know how to verify whether a candidate is actually a solution

18. Example – Hamiltonian circuit problem Suppose through some (unspecified) means (like good guessing), we find a candidate for a Hamiltonian circuit, i.e. a list of vertices and edges that might be aHamiltonian circuit in the input graph G. It is easy to check if this is indeed a Hamiltonian circuit. Check • that all the proposed edges exist in G, • that we indeed have a cycle, and • that we hit every vertex in Gonce. If the candidate solution is indeed a Hamiltonian circuit, then it is a certificateverifying that the answer to the decision problem is "Yes"

19. Example – 0/1 Knapsack Problem Consider an instance of the 0/1 Knapsackproblem (decision version) Suppose someone proposes a subset of items, it is easy to check • if those items have total weight at most W and • if the total value is at least k • If both conditions are true, then thesubset of items is a certificate for the decision problem • i.e., itverifies that the answer to the 0/1 knapsack decision problemis "Yes"

20. Example – Circuit-SAT Consider a Boolean Circuit Suppose someone proposes an assignment of truth values to the input, it is easy to check • if the input values lead to a final value of 1 in the output • this is done by checking every logic gate If the input truth values give a final value of 1, these values form a certificate for the decision problem

21. Complexity Classes P and NP The complexity class P is the set of all decision problems that can be solved in worst-case polynomial time. The complexity class NP is the set of all problems that can be verified in polynomial time. P stands for polynomial, and NP stands for non-deterministic polynomial.Non-deterministic machine is a model of computation, and can be used to model verification.

22. The Class P MST problem is in P • Given a weighted graph G and a value k, does there exists a MST with weight at most k? • run Kruskal’s algorithm (polynomial time) and if the MST found has weight at most k, then the answer is "Yes" Single-source-shortest-paths problem is in P • Given a weighted graph G, a source vertex s, and a value k, does there exist shortest paths from s to every other vertex whose path length is at most k? • run Dijkstra’s algorithm (polynomial time) and if the paths found have lengths at most k, then answer is "Yes"

23. The Class NP Hamiltonian circuit problem is in NP • we can check in polynomial time if a proposed circuit is a Hamiltonian circuit 0/1 Knapsack problem is in NP • we can check in polynomial time if a proposed subset of items whose weight is at most W and whose value is at least k Circuit-SAT is in NP • we can check in polynomial time if proposed values lead to a final output value of 1

24. P = NP ? Note that P NP The (million dollar) question is that mathematicians and computer scientists do not know whether P = NP or P ≠ NP However, there is a common belief that P is different from NP • i.e., there is some problem in NP that is not in P

25. Polynomial-time reduction Given any two decision problems A and B, we say that • A is polynomial time reducible to B, or • there is a polynomial time reduction from A to B if given any input  of A, we can construct in polynomial time an input  of B such that is yes if and only if is yes. We use the notation A ≤P B Intuitively, this means that problem Bis at least as difficult as problem A

26. NP-hardness / NP-completeness A problem M is said to be NP-hard if every other problem in NP is polynomial time reducible to M • intuitively, this means that M is at least as difficult as all problems in NP M is further said to be NP-complete if • M is in NP, and • M is NP-hard NP-complete problems are some of the hardest problems in NP

27. NP-Complete Problem The Cook-Levin Theorem states that Circuit-SAT isNP-complete Using polynomial time reducibility we can show existence ofother NP-complete problems A useful result to proveNP-completeness: Lemma If L1≤PL2 and L2 ≤PL3, then L1 ≤PL3

28. Other NP-Complete Problems We have seen these NP-Complete Problems • Hamiltonian Circuit Problem • 0/1 Knapsack Problem • Circuit-SAT Others • CNF-SAT and 3-SAT (conjunctive normal form satisfiability problem) • Vertex Cover

29. Conjunctive normal form (CNF) • a Boolean formula is in CNF if it is formed as a collection of clauses combined using the operator AND ( • ) and each clause is formed by literals (variables or their negations) combined using the operator OR ( + ) • example: (x1+x2+x4+x5) • (x2+x1+x4)

30. CNF-SAT and 3-SAT CNF-SAT • Input: a Boolean formula in CNF • Question: Is there an assignment of Boolean values to its variables so that the formula evaluates to true? (i.e., the formula is satisfiable) 3-SAT • Input: a Boolean formula in CNF in which each clause has exactly 3 literals CNF-SAT and 3-SAT are NP-complete

31. Vertex Cover Given a graph G = (V,E) A vertex cover is a subset C  V such that for every edge (v,w) in E, vC or wC some graphs and their vertex cover (shaded vertices)

32. Vertex Cover The optimisation problem is to find as small a vertex cover as possible Vertex Cover is the decision problem that takes a graph G and an integer k and asks whether there is a vertex cover for G containing at mostk vertices Vertex Cover is NP-complete

33. Prove: 3SAT is NP-hard optional Reduce CNF-SAT to 3SAT • Let E be an arbitrary instance of SAT. • We will replace each clause of E with several clauses, each has exactly three variables • Target: to prove that the clause of E is satisfiable if and only if all the transformed clauses are satisfied

34. Prove: 3SAT is NP-hard (2) optional Suppose C = (x1+x2+x3+…+xk) be a clause of E such that k4. We introduce new variables y1+y2+y3+…+yk-3 to form C’ C’ = (x1+x2+y1) • (x3+y1+y2) • (x4+y2+y3) • • • (xk-2+yk-4+yk-3) • (xk-1+xk+yk-3) Target: Prove C’ is satisfiable if and only if C is satisfiable

35. Prove: 3SAT is NP-hard (3) optional If C is satisfiable • one of the xi must be set to 1 • if x3=1, we set y1=1, y2=0, and the rest of yi=0 • y1=1 implies (x1+x2+y1) is 1; y2=0 & x3=1 implies (x3+y1+y2) is 1; rest of yi=0 implies all other clauses are 1 • if xi=1, we set y1,y2,…,yi-2 = 1, the rest = 0 • y1,y2,…,yi-2 = 1 implies the first (i-2) clauses are 1; xi=1 implies (xi+yi-2+yi-1) is 1; the rest = 0 implies the remaining clauses are 1 C = (x1+x2+x3+…+xk) C’ = (x1+x2+y1) • (x3+y1+y2) • (x4+y2+y3)• • • (xk-2+yk-4+yk-3) • (xk-1+xk+yk-3)

36. Prove: 3SAT is NP-hard (4) optional If C’ is satisfiable • Claim: at least one of xi must be 1 • If all xi are 0, C’ becomes (y1) • (y1+y2) • (y2+y3)• • • (yk-4+yk-3) • (yk-3) C’ is NOT satisfiable, contradiction • Therefore, C is satisfiable C = (x1+x2+x3+…+xk) C’ = (x1+x2+y1) • (x3+y1+y2) • (x4+y2+y3)• • • (xk-2+yk-4+yk-3) • (xk-1+xk+yk-3)

37. Prove: 3SAT is NP-hard (5) optional For clauses with one or two variables If C has two variables, C = (x1+x2), then we set C’ = (x1+x2+z) • (x1+x2+z) If C has one variable, C = (x1), then we set C’ = (x1+y+z) • (x1+y+z) • (x1+y+z) • (x1+y+z) We have transformed every clause C in E to a sequence of clause C’ with exactly three variables, such that C is satisfiable if and only if C’ is. This reduction can be done in polynomial time.