CS 221

1. CS 221 Analysis of Algorithms Instructor: Don McLaughlin

2. Theoretical Analysis of Algorithms • Recursive Algorithms • One strategy for algorithmic design is solve the problem using the same design but with a smaller problem • The algorithm must be able use itself to compute a solution but with subset of the problem • Must be one case that can be solved with evoking the whole algorithm again – Base Case • This is called recursion

3. Theoretical Analysis of Algorithms • Recursive Algorithms • Recursion is elegant • But is it efficient?

4. Theoretical Analysis of Algorithms • A recursive Max algorithm Algorithm recursiveMax(A,n) Input: Array A storing n>=1 integers Output: Maximum value in A if n = 1 then return A[0] return max{recursiveMax(A,n-1),A[n-1]

5. Theoretical Analysis of Algorithms • Efficiency – T(n) = {3 if n = 1, T(n-1)+7 if otherwise} or T(n) = 7(n-1)+3 = 7n-2 * We will come back to this

6. Theoretical Analysis of Algorithms • Best-case • Worst-case • Average-case • A number if issues here • Not as straight forward as you would think

7. Theoretical Analysis of Algorithms • Average-case • Sometimes Average Case is needed • Best or Worst case may not be the best representation of the typical case with “usual” input • Best or worst case may be rare • Consider • A sort that typically gets a

8. Theoretical Analysis of Algorithms • Average-case • Consider • A sort that typically gets a partially sorted list • A concatenation of sorted sublists • A search that must find a match in a list • …and the list is partially sorted

9. Theoretical Analysis of Algorithms • Average-case • Can you take the average of best-case and worst case?

10. Theoretical Analysis of Algorithms • Average-case • Average case must consider the probability of each case or a range of n • N may not be arbitrary • One strategy – • Divide range of n into classes • Determine the probability that any given n is from each respective class • Use n in the analysis based on probability distribution of each class

11. Theoretical Analysis of Algorithms • Asymptotic notation • Remember that our analysis is concerned with the efficiency of algorithms • so we determine a function that describes (to a reasonable degree) the run-time cost (T) of the algorithm • we are particularly interested in the growth of the algorithm’s cost as the size of the problem grows (n)

12. Theoretical Analysis of Algorithms • Asymptotic notation • Often we need to know that run-time cost (T) is broader terms • …within certain boundaries • We want to describe the efficiency of an algorithm in terms of its asymptotic behavior

13. Theoretical Analysis of Algorithms • Big 0 • Suppose we have a function of n • g(n) • that we suggest gives us an upper bound on the worst case behavior of our algorithm’s runtime – • which we have determined to be f(n) • then…

14. Theoretical Analysis of Algorithms • Big 0 • We describe the upper bound on the growth of our run-time function f(n) – • f(n) is O(g(n)) • f(n) is bounded from above by g(n) for all significant values of n • if • f(n) <= cg(n) for all n >= n0 • there exists some constant c such that for all values of n >= n0 • f(n) <= cg(n)

15. Theoretical Analysis of Algorithms • Big 0 from: Levitin, Anany, The Design and Analysis of Algorithms, Addison-Wesley, 2007

16. Theoretical Analysis of Algorithms • Big 0 • …but be careful • f(n) = O(g(n)) is incorrect • the proper term is • f(n) is O(g(n)) • to be absolutely correct • f(n) Є O(g(n))

17. Theoretical Analysis of Algorithms • Big Ω • Big Omega • our function is bounded from below by g(n) • that is, • f(n) is Ω(g(n)) • if there exists some positive constant c • such that f(n) >= cg(n), n >= n0 • what does this mean?

18. Theoretical Analysis of Algorithms • Big Ω from: Levitin, Anany, The Design and Analysis of Algorithms, Addison-Wesley, 2007

19. Theoretical Analysis of Algorithms • Big Θ • Big Theta • our function is bounded from above and below by g(n) • that is, • f(n) is Θ(g(n)) • if there exists two positive constants c1 and c2 • such that c2g(n) <= f(n) >= c1g(n) for all n >= n0 • what does this mean?

20. Theoretical Analysis of Algorithms • Big Θ

21. Theoretical Analysis of Algorithms • Or said in another way • O(g(n)): class of functions f(n) that grow no faster than g(n) • Θ (g(n)): class of functions f(n) that grow at same rate as g(n) • Ω(g(n)): class of functions f(n) that grow at least as fast as g(n)

22. Theoretical Analysis of Algorithms • Little o • o • f(n) is o(g(n)) if f(n) < cg(n) for some n >= n0 for all constants c >0 • what does this mean? • f(n) is asymptotically smaller than g(n)

23. Theoretical Analysis of Algorithms • Little ω • ω • f(n) is ω(g(n)) if f(n) < cg(n) for all constants c and n>n0 • what does this mean? • f(n) is asymptotically larger than g(n)

24. Theoretical Analysis of Algorithms • Simplifying things a bit • Usually only really interested in the order of growth of our algorithm’s run-time function

25. Theoretical Analysis of Algorithms • Suppose we have a run-time function like T(n) = 4n3 + 12n2 + 2n + 7 • So, to simplify our run-time function T, we can • eliminate constant terms that are not coefficients of n • 4n3 + 12n2 + 2n • eliminate lowest order terms • 4n3 + 12n2 • Maybe only keep the highest order term • 4n3 • …and drop the coefficent of that term • n3

26. Theoretical Analysis of Algorithms • So, T(n) = 4n3 + 12n2 + 2n + 7 • therefore • T(n) is O(n3) • or is it that • T(n) is O(n6) (true or false) • True • but it is considered bad form • why?

27. Classes of Algorithmic Efficiency based on: Levitin, Anany, The Design and Analysis of Algorithms, Addison-Wesley, 2007

28. Properties of Asymptotic Comparisons • Transitivity • f(n) is O(g(n)) and g(n) is O(h(n)) then f(n) is O(h(n)) • f(n) is Θ(g(n)) and g(n) is Θ(h(n)) then f(n) is Θ(h(n)) • f(n) is Ω(g(n)) and g(n) is Ω(h(n)) then f(n) is Ω(h(n)) • f(n) is o(g(n)) and g(n) is o(h(n)) then f(n) is o(h(n)) • f(n) is ω(g(n)) and g(n) is ω(h(n)) then f(n) is ω(h(n)) • Reflexivity • f(n) is Θ(f(n)) • f(n) is O(f(n)) • f(n) is Ω(f(n))

29. Properties of Asymptotic Comparisons • Symmetry • f(n) is Θ(g(n)) if and only if g(n) is Θ(f(n)) • Tranpose Symmetry • f(n) is O(g(n)) if and only if g(n) is Ω(f(n)) • f(n) is o(g(n)) if and only if g(n) is ω(f(n))

30. Some rules of Asymptotic Notation • d(n) is O(f(n)), then ad(n) is O(f(n)) for any constant a > 0 • d(n) is O(f(n)) and e(n) is O(g(n)), then d(n)+e(n) is O(f(n) + g(n)) • d(n) is O(f(n)) and e(n) is O(g(n)), then d(n)e(n) is O(f(n)g(n)) • d(n) is O(f(n)) and f(n) is O(g(n)), then d(n) is O(g(n))

31. Some rules of Asymptotic Notation • f(n) is polynomial of degree d, the f(n) is O(nd) • nx is O(an) for any fixed x > 0 and a > 1 • lognx is O(log n) for any x > 0 • logxn is O(ny) for any fixed constants x > 0 and y > 0

32. Homework The Scream, by Edvard Munch, http://www.ibiblio.org/wm/paint/auth/munch/

33. Homework – due Wednesday, Sept. 3rd • Assume • Various algorithms would run 100 primitive instructions per input element (n) (constant coefficient of 100) • …and it will be implemented on a processor that run 2 billion primitive instructions per second • then estimate the execution times for the following algorithmic efficiency classes

34. Homework give answer in largest meaningful time increment