Theoretical Computer Science Algorithms and Complexity

1. Theoretical Computer ScienceAlgorithms and Complexity Chris Umans

2. Algorithms Systems and Software Design and Implementation Theoretical Computer Science Computability and Complexity Theory

3. Theoretical Computer Science • At the heart of computer programs lie algorithms • To study algorithms we must be able to speak mathematically about: • computational problems • computers • algorithms

4. Theoretical Computer Science • At the heart of computer programs lie algorithms • To study algorithms we must be able to speak mathematically about: • computational problems: functions on bitstrings • computers: Turing Machines • algorithms: step-by-step instructions

5. Algorithms (example) Input: n numbers Output: n numbers in ascending order Algorithms for this problem? How do you know which one is “better”?

6. Worst-case, asymptotic analysis • measure running time as a function of input length • f(n) is worst-case over all n-bit inputs 3n2 +100 2n/30 • “efficient,” “tractable” = polynomial • “prohibitive,” “intractable” = exponential

7. Algorithms • lots of non-obvious and beautiful ideas that lead to fast algorithms • Example: matrix multiplication • Standard method: O(n3) operations A X B = C

8. Strassen’s Algorithm a1, 1 b1, 1 c1, 1 a1, 2 b1, 2 c1, 2 • linear combos of A, B entries • 7 mults • linear combos of results X = a2, 1 b2, 1 c2, 1 a2, 2 b2, 2 c2, 2 P1= (a1,1 - a1,2) ×(b2,1 + b2,2) P2= (a1,1 + a2,2) × (b1,1 + b2,2) P3= (a1,1 - a2,1) ×(b1,1 + b1,2) P4 = (a1,1 + a1,2) ×(b2,2) P5 = (a1,1) ×(b1,2 + b2,2) P6 = (a2,2) ×(b2,1 - b1,1) P7 = (a2,1 - a2,2) ×(b1,1) (a1,1 - a1,2) (b2,1 + b2,2) (a1,1 + a2,2) (b1,1 + b2,2) (a1,1 - a2,1) (b1,1 + b1,2) (a1,1 + a1,2) (b2,2) (a1,1) (b1,2 + b2,2) (a2,2) (b2,1 - b1,1) (a2,1 - a2,2) (b1,1) c1,1 = P1 + P2 - P4 + P6 c1,2 = P4 + P5 c2,1 = P6 + P7 c2,2 = P2 - P3 + P5 - P7

9. Strassen’s Algorithm • T(n) = # operations to multiply n x n matrices T(n) ≤ 7T(n/2) + O(n2) T(n) ≤ O(nlog27) (huge difference for huge matrices) can multiply n x n matrices in time O(n2.81…)!

10. Algorithms • Many other beautiful examples … and general ideas: • greedy algorithms • dynamic programming • divide and conquer • linear programming • but also, hardness around every corner

11. Algorithms • In a perfect world… • for each problem we would have an algorithm • the algorithm would be the fastest possible • What would Computer Science • look like in this world?

12. Limitations on algorithms • Our world (fortunately) is not so perfect: • not all problems have algorithms • for many problems we know embarrassingly little about what the fastest algorithm is • multiplying two integers • factoring an integer into primes • determining shortest tour of given n cities • for certain problems, fast algorithms would change the world

13. Outline • Two puzzles to show that it’s often difficult to tell if a problem is “easy” or “hard” • Reductions • Central questions of Computational Complexity Theory • Glimpse of cryptography…

14. A puzzle • Find an efficient algorithm to solve the following problem: • Input: sequence of pairs of symbols e.g. (A, b), (E, D), (d, C), (b, a) • Goal: determine if it is possible to circle at least one symbol in each pair without circling upper and lower case of same symbol.

15. A puzzle • Find an efficient algorithm to solve the following problem: • Input: sequence of pairs of symbols e.g. (A, b), (E, D), (d, C), (b, a) • Goal: determine if it is possible to circle at least one symbol in each pair without circling upper and lower case of same symbol.

16. 2SAT • This is a disguised version of a problem called 2-CNF-SAT • CNF = “AND of ORs” • SAT = “satisfiability” • satisfying truth assignment = assignment of TRUE/FALSE to each variable so that whole formula is TRUE • (A, b), (E, D), (d, C), (b, a) • (x1 x2)(x5  x4)(x4  x3)(x2  x1)

17. Algorithm for 2SAT • Build a graph with separate nodes for each variable and negated variable. • add directed edge (x, y) if formula includes clause (x  y) or (y  x ) (equiv. to x  y) x4 x1 x4 x3 x2 x5 x3 x1 x2 x5 e.g. (x1 x2)(x5  x4)(x4  x3)(x2  x1)

18. Algorithm for 2SAT Claim: formula is unsatisfiableiff there is some variable x with a path from x to x and a path from x to x in derived graph. • Proof () • edges represent implication . By transitivity of , a path from x to x means x  x, and a path from x to x means x  x.

19. Algorithm for 2SAT • Proof () • to construct satisfying assignment (if no x with a path from x to x and a path from x to x): • pick unassigned literal s with no path from s to s • assign it TRUE, as well as all nodes reachable from it; assign negations of these literals FALSE • note: path from s to t and s to t implies path from t to s and t to s, implies path from s to s • note: path s to t (assigned FALSE) implies path from t (assigned TRUE) to s, so s already assigned at that point.

20. Algorithm for 2SAT • Algorithm: • build derived graph • for every pair x, x check if there is a path from x to x and from x to x in the graph • Running time of algorithm (input length n): • O(n) steps to build graph • O(n) steps to perform each check • O(n) checks • total number of steps: O(n2) = “efficient”

21. Another puzzle • Find an efficient algorithm to solve the following problem. • Input: sequence of triples of symbols e.g. (A, b, C), (E, D, b), (d, A, C), (c, b, a) • Goal: determine if it is possible to circle at least one symbol in each triple without circling upper and lower case of same symbol.

22. 3SAT • This is a disguised version of 3-CNF-SAT • observe: solvable in exponential time (2n) • (A, b, C), (E, D, b), (d, A, C) • (x1x2 x3)  (x5x4x2)  (x4x1x3) • $1,000,000 prize (!) • give an efficient algorithm for 3-CNF-SAT or a proof that none exists

23. Reductions • Reduction = efficient transformation of problem A into problem B so that solution to A can be derived from solution to B. • Amazing fact: for some problems A, can show that every problem with certain very general features reduces to A

24. Reductions • To reason about entire class of problems, only need one concrete problem. • Example: every problem solvable by exhaustive search is reducible to 3-SAT. • this is the class “NP” • this is why there is a $1,000,000 prize

25. Complexity Theory Classify problems according to the computational resources required • running time • storage space • parallelism • randomness • others… Attempt to answer: what is computationally feasible with limited resources?

26. The central questions • Is finding a solution as easy as recognizing one? P = NP? • Is every sequential algorithm parallelizable? P = NC? • Can every efficient algorithm be converted into one that uses a tinyamount of memory? P = L? • Are there small Boolean circuits for all problems that require exponential running time? EXP  P/poly? • Can every efficient randomized algorithm be converted into a efficient deterministic algorithm? P = BPP?

27. The central questions We think we know the answers to all of these questions … … but no one has been able to prove that even a small part of this “world-view” is correct. If we’re wrong on any one of these then computer science will change dramatically

28. Cryptography • Computational hardness is useful! • One way functions: easy to compute, hard to invert • e.g. (x, y) -> x¢y(easy) vs. x¢y -> (x, y) (hard) • from this, “digital envelope” • easy to insert text “a” • hard to observe “a” from “E(a)” • impossible to change “a” once it is inside • given key, easy to open and verify contents a E(a)

29. Cryptography • “digital envelope”: • yields an amazing and useful capability: a E(a) • can convince someone you know the solution to an exhaustive search (NP) problem without revealing anything about the solution

30. Zero Knowledge • Any exhaustive search (NP) problem “A” can be reduced to this problem: • given a graph, is it “3-colorable” not 3-colorable 3-colorable

31. Zero Knowledge • Any exhaustive search (NP) problem “A” can be reduced to this problem: • given a graph, is it “3-colorable” not 3-colorable 3-colorable • Can convince someone (“verifier”) that you have a solution to “A” by convincing them you know a 3-coloring.

32. Zero Knowledge How to do it without revealing the coloring? • randomly permute the colors of the 3-coloring • put the colors in digital envelopes, one for each vertex • verifier picks a random edge • you open the two envelopes

33. Zero Knowledge • randomly permute the colors of the 3-coloring • put the colors in digital envelopes • verifier picks a random edge • you open the two envelopes • valid coloring: always pass the test • invalid coloring: ¸1/#edges chance of failing • “Zero Knowledge”: verifier can simulate what he sees on his own!

34. Some big achievements • theory of NP-completeness • mathematically rigorous cryptography • approximation algorithms for hard optimization problems • “PCP Theorem”: NP-hardness of approximation

35. Some “hot” areas • Algorithmic game theory • Approximation algorithms via “semi-definite programming” and limits to this technique via “Unique Games Conjecture” • Quantum computation

36. Theory courses to consider • CS21 Decidability and Tractability (2nd term every year) • CS38 Algorithms(3rd term every year) • CS151 Complexity Theory (3rd term every even year) • CS150 Probability and Algorithms (2nd term every odd year) • CS153 Current Topics in Theoretical Computer Science (3rd term every year) • Ph 219/CS 219 Quantum Computation