CS221

1. Week 2 - Friday CS221

2. Last time • What did we talk about last time? • Running time • Big O notation

3. Questions?

4. Project 1

5. Assignment 1

6. Assignment 2

7. Back to complexity

8. Mathematical issues • What's the running time to factor a large number N? • How many edges are in a completely connected graph? • If you have a completely connected graph, how many possible tours are there (paths that start at a given node, visit all other nodes, and return to the beginning)? • How many different n-bit binary numbers are there?

9. Hierarchy of complexities • Here is a table of several different complexity measures, in ascending order, with their functions evaluated at n = 100

10. Logarithms • What is a logarithm? • Definition: • If bx = y • Then logby = x(for positive b values) • Think of it as a de-exponentiator • Examples: • log10(1,000,000) = • log3(81) = • log2(512) =

11. More on log • Add one to the logarithm in a base and you'll get the number of digits you need to represent that number in that base • In other words, the log of a number is related to its length • Even big numbers have small logs • If there's no subscript, log10 is assumed in math world, but log2 is assumed for CS • Also common is ln, the natural log, which is loge

12. Log math • logb(xy) = logb(x) + logb(y) • logb(x/y) = logb(x) - logb(y) • logb(xy) = ylogb(x) • Base conversion: • logb(x) = loga(x)/loga(b) • As a consequence: • log2(n) = log10(n)/c1= log100(n)/c2= logb(n)/c3 for b > 1 • log2n is O(log10n) and O(log100n)and O(logbn) for b > 1

13. Formal definition of Big Oh • Let f(n) and g(n) be two functions over integers • f(n) is O(g(n)) if and only if • f(n) ≤ c∙g(n) for all n > N • for some positive real numbers c and N • In other words, past some arbitrary point, with some arbitrary scaling factor, g(n) is always bigger

14. Big Oh, Big Omega, Big Theta

15. All three are useful measures • Oestablishes an upper bound • f(n) is O(g(n)) if there exist positive numbers c and N such that f(n) ≤ cg(n) for all n ≥ N • Ωestablishes a lower bound • f(n) is Ω(g(n)) if there exist positive numbers c and N such that f(n) ≥ cg(n) for all n ≥ N • Θestablishes a tight bound • f(n) is Θ(g(n)) if there exist positive numbers c1,c2 and N such that c1g(n) ≤ f(n) ≤ c2g(n) for all n ≥ N

16. Tight bounds • O and Ω have a one-to-many relationship with functions • 4n2+ 3 is O(n2) but it is also O(n3) and O(n4 log n) • 6n log nis Ω(n log n) but it is also Ω(n) • Θ is one-to-many as well, but it has a much tighter bound • Sometimes it is hard to find Θ • Upper bounding isn’t too hard, but lower bounding is difficult for real problems

17. Facts • If f(n) is O(g(n)) and g(n) is O(h(n)), then f(n) is O(h(n)) • If f(n) is O(h(n)) and g(n) is O(h(n)), then f(n) + g(n) is O(h(n)) • ank is O(nk) • nk is O(nk+j), for any positive j • If f(n) is cg(n), then f(n) is O(g(n)) • logan is O(logbn) for integers a and b > 1 • logan is O(nk) for integer a > 1 and real k> 0

18. Binary search example • How much time does a binary search take at most? • What about at least? • What about on average, assuming that the value is in the list?

19. Complexity practice • Give a tight bound for n1.1 + n log n • Give a tight bound for 2n + a where a is a constant • Give functions f1 and f2 such that f1(n) and f2(n) are O(g(n)) but f1(n) is not O(f2(n))

20. NP-Completeness

21. Traveling Salesman Problem:Hamiltonian Cycle Meets Shortest Path • Given a graph, find the shortest tour that visits every node and comes back to the starting point • Like a lazy traveling salesman who wants to visit all possible clients and then return to his home

22. What’s the Shortest Tour? • Find the shortest TSP tour: • Starts anywhere • Visits all nodes • Has the shortest length of such a tour 5 6 A 4 4 3 3 E 5 B 2 D 3 F 7 C

23. How Can We Always Find the Shortest Tour? • Strategies: • Always add the nearest neighbor • Randomized approaches • Try every possible combination • Why are we only listing strategies?

24. Greedy Doesn’t Work • We are tempted to always take the closest neighbor, but there are pitfalls Greedy Optimal

25. Brute Force is Brutal • In a completely connected graph, we can try any sequence of nodes • If there are n nodes, there are (n – 1)! tours • For 30 cities, 29! = 8841761993739701954543616000000 • If we can check 1,000,000,000 tours in one second, it will only take about 20,000 times the age of the universe to check them all • We will (eventually) get the best answer!

26. What’s the Best Time for TSP? • No one knows how to find the best solution to TSP in an efficient amount of time • For a general graph, no good approximation even exists • For a graph with the triangle inequality, there is an approximation that yields a tour no more than 3/2 the optimal • Some variations on the problem are easier than others

27. Hard Problems • Is TSP the hardest problem there is? • Are there other problems equally hard? • How do we compare the difficulty of one problem to another?

28. NP-Completeness • TSP is one of many problems which are called NP-complete • All of these problems share the characteristic that the only way we know to find best solution uses brute force • All NP-complete problems are reducible to all other NP-complete problems • An efficient solution to one would guarantee an efficient solution to all

29. Fabulous Cash Prizes • These NP-complete problems are very hard • Many of them are really useful • Clay Mathematics Institute has offered a $1,000,000 prize • You can do one of two things to collect it: • Find an efficient solution to any of the problems • Prove that one cannot have an efficient solution

30. Eclipse Debugger Tutorial

31. Upcoming

32. Next time… • Linked lists • Read Chapter 3

33. Reminders • Read Chapter 3 • Finish Assignment 1 • Due tonight by 11:59pm • Keep working on Project 1 • Due Friday, September 19 by 11:59pm • Start Assignment 2 • Due next Friday by 11:59pm