Lecture

1. Lecture

2. Today • Problem set 9 out (due next Thursday) • Topics: • Complexity Theory • Optimization versus Decision Problems • P and NP • Efficient Verification • NP- Completeness • Section: Network Flow

3. So Far... • Algorithmic Techniques : Divide and Conquer, Greedy, Dynamic Programming, DFS) • Data Structures: Trees, Heaps, Graphs • Practice: many problems even iftry every trick in the book, nothing seems to work • Some Problems:Intrinsic Difficulty -- Domain of complexity Theory -- 6.045

4. Complexity Theory • Halting problem: Given a computer program and an input, does the program halt on that input? -- Unsolvable • Today: Look at problems which are Solvable but not efficiently

5. The Clique Problem Def:Let G=(V,E) be an undirected graph. A clique of G is a subset V’ of the vertices s.t. for all u,v in V’, (u,v) is an edge. I.e clique is a complete sub-graph of G

6. Optimization Problems Optimization: minimize of maximize some value Optimization Problem for Clique: Given graph G, find size of the largest clique. How to solve this problem? • For j = |V| down-to 1 try all subsets of j vertices, if find clique, output j. Time: 2O(|V|)VERY SLOW! • Is there a “tractable” (efficient) algorithm? • What do you mean by tractable (efficient)?

7. Traveling Salesman problem • Def: Given n cities and cost cij for traveling between every two cities, what is the cost of the cheapest way to visit every city exactly once and return to starting city. • How to solve this problem? Try all possible orderings of cities, and evaluate each orderings cost, output the cheapest cost. Time: n! (for n cities ) Is there a “tractable” (efficient) algorithm?

8. Q:What do we mean by efficient:A: Polynomial Time Algorithms Defn: An algorithm runs in polynomial time if there exists k s.t. the running time is O(nk) on any problem instance of size n. Q1: What do we mean by “size n”? Q2: Why is this a reasonable choice for “efficient”

9. Encoding A: Any problem instance can be encoded as a sequence of 0’s and 1’s Assume all numbers in encoding are in binary? Size of problem instance = length of encoding Multiple arguments are coded into 1 string. Choice of the encoding doesn’t matter as long as can compute one from another in polynomial time

10. Example : Factoring Given number n, determine m that divides n • Suppose n is encoded in unary: 1111111…1 then check all numbers m=2,…,sqrt(n), takes O(sqrt(n) ) = O(sqrt(L)) divisions • Suppose n is encoded in binary, L=log n, same procedure takes O(2L/2) divisions L=n

11. Pitfalls • Some functions don’t look like polynomials but are bounded above by one. E.g n log n<O(n2) • Sometime what you think is polynomial isn’t. Example: Take as input <G,k> where G is a graph of n nodes, and k an integer. An algorithm of running time o(nk) for this input, is NOT polynomial. Why? Because k is an input to the problem --- The constant in the polynomial should be independent of the input size.

12. Justification of ‘Efficient = Polynomial Time” • Exponential time scales badly -- 2n for n=1000 is larger than number of atoms in universe • Want our theory of efficient algorithms to be model-independent (I.e hold for any reasonable model of computer). The polynomial-time definition of “efficiency” ensures this • Bulk of problems that are solvable in polynomial time that are of practical interest, are bounded by small polynomial: MST, shortest path, numerical operation.

13. Polynomial Time Algorithms Most problems so far have poly time algorithms Q: Does Clique have a poly time algorithm? Q: Does TSP have a poly time algorithm? A: No one knows. TOC community would be surprised if answer is positive.

14. Decision Problems: Y/N output Decision problem for clique: Given <G,k>, G=(V,E) is an undirected graph, k is an integer, does G have a clique of size k? For purely technical reason in complexity theory easier to deal with decision problems. Comment:Even though seems less natural, as complexity theory task is to show that certain problems cannot be solved efficiently, showing it for decision version implies it for corresponding optimization problem. Note: A polynomial time procedure for the clique decision problem should run in time poly(m,n,log k) where n=|V|, m=|E|

15. Theorem (for clique): optimization problem solvable in polynomial time iff decision problem solvable in polynomial time Proof (for clique): (opt dec) Find max clique size, compare with k (dec opt) Binary search. Natural Question: true for other problems? Yes…when the thing to be optimized has a polynomial # of possible values in the zize o fthe input instance

16. Search vs. Decision • Search problem for clique: Given <G,k>, either find a clique of size k, or say none exists. • Theorem:Search Problem is solvable in polynomial time iff Decision problem is solved in polynomial time • Proof: Suppose we have a magic-algorithm MB s.t MB(<G,k>) = yes iff there exists a clique of size k. Then, on <G,k> : • Run MB(<G,k>), if MB(<G,k>) = no, output no, • elseOrder edges of G from 1,…,m, initialize G’= G, I=1 1. run MB(<G’,k> ) where G’ = G- edge I 2. If MB(<G’,k>) = no, then G’=G + edge I, I= I+1, else I=I +1. 3. If I>n, output edge set of G’, else goto 1

17. Formal Languages • Def: a formal language is a set of binary strings. • Every decision problem can be viewed as a language CLIQUE = { <G,k>: G has a clique of size k} (I.e binary encodings of all graphs G that have clique of size k) TSP = {<G,edge costs,k>: G has a TS Tour of cost < k} • In general, for decision problem D, the corresponding language is L={x in Sn: D(x)=1} where Sn is the encoding alphabet. (e.g. for binary encoding S={0,1}

18. Example:decision problem /corresponding language PATH Decision Problem: Given <G,u,v,k> where G is a graph, u, v vertices, and integer k>0, is there a path from u to v in G of length < k? Corresponding language: PATH = {<G,u,v,k>: G is a graph, u,v vertices, k>=0 integer, and there is a path from u and v in G whose length is < k}

19. Algorithms and Decision Problems • An algorithm accepts x if A(x)=1 • An algorithm rejects x if A(x) =0 Might do neither, Language L is accepted by A: L={x, A(x)=1} Language L is decided by A: if A accepts L and A halts on all inputs

20. Class P P = { L in {0,1}*: there exists algorithm A s.t. L is accepted by A in polynomial time} P - class of languages accepted by a polynomial time algorithms Examples: shortest path, MST, sorting. Q:Clique ? TSP? A:No one knows, big open problem

21. Class NP NP: class of languages accepted by a non-deterministic algorithm in polynomial time What does this mean? Interpretation 1:A non-deterministic algorithm has many possible outcomes (like a probabilistic algorithm) but we require • if x is a YES instance, then Pr(A(x)=YES) >0 • if x is a NO instance, then Pr(A(x)=NO) = 1.

22. NP: Efficient Verification (version 2 informally ) NP : class of languages(problems) for which the YES instances have short proofs that the answer is YES. proof of polynomial size (in input size) that can be verified in polynomial time Example:CLIQUE in NP, since for all <G,k> in CLIQUE there exists a short proof: subset of vertices V’ of size >=k, size of proof = O(|V|) verifying Procedure: check that for all u,v in V’, (u,v) in E time = O(|V|2) Short Proof = Certificate

23. NP: Efficient Verification (formally) NP = {L s.t. there exists polynomial -time algorithm A, and integer k>0 s.t. x in L there exists y, |y|<|x|k, A(x,y) =YES x out L for all y, A(x,y) =NO} y= “short proof”= certificate Language verified by A, is L= {x, there exists y s.t. A(x,y)= YES} NP: class of polynomial time verifiable problems (EVEN IF they are hard to solve)

24. Theorem : P NP UI • Let L be in P, then there a polynomial time algorithm which decides L, and can be converted into a verification algorithm • HOW? BIG QUESTION: is P= NP • Open for 25 years • What we DO know: If we can solve CLIQUE in poly-time, then P=NP. • NP-completeness (next time)

25. Summarize • Decision Problems, Languages • P: polynomial time solvable decision problems ( languages) • Certificate:evidence that allows us to verify in polynomial time that an input is in a given language • NP: polynomial time verifiable languages