Download Presentation
## Growth Rates

- - - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - - -

**Growth Rates**• Growth rates of functions: • Linear n • Quadratic n2 • Cubic n3 • In a log-log chart, the slope of the line corresponds to the growth rate of the function Analysis of Algorithms**Constant Factors**• The growth rate is not affected by • constant factors or • lower-order terms • Examples • 102n+105is a linear function • 105n2+ 108nis a quadratic function Analysis of Algorithms**Big-Oh Notation**• Given functions f(n) and g(n), we say that f(n) is O(g(n))if there are positive constantsc and n0 such that f(n)cg(n) for n n0 • Example: 2n+10 is O(n) • 2n+10cn • (c 2) n 10 • n 10/(c 2) • Pick c = 3 and n0 = 10 Analysis of Algorithms**Big-Oh Example**• Example: the function n2is not O(n) • n2cn • n c • The above inequality cannot be satisfied since c must be a constant Analysis of Algorithms**More Big-Oh Examples**• 7n-2 7n-2 is O(n) need c > 0 and n0 1 such that 7n-2 c•n for n n0 this is true for c = 7 and n0 = 1 • 3n3 + 20n2 + 5 3n3 + 20n2 + 5 is O(n3) need c > 0 and n0 1 such that 3n3 + 20n2 + 5 c•n3 for n n0 this is true for c = 4 and n0 = 21 • 3 log n + log log n 3 log n + log log n is O(log n) need c > 0 and n0 1 such that 3 log n + log log n c•log n for n n0 this is true for c = 4 and n0 = 2 Analysis of Algorithms**Big-Oh and Growth Rate**• The big-Oh notation gives an upper bound on the growth rate of a function • The statement “f(n) is O(g(n))” means that the growth rate of f(n) is no more than the growth rate of g(n) • We can use the big-Oh notation to rank functions according to their growth rate Analysis of Algorithms**Big-Oh Rules**• If is f(n) a polynomial of degree d, then f(n) is O(nd), i.e., • Drop lower-order terms • Drop constant factors • Use the smallest possible class of functions • Say “2n is O(n)”instead of “2n is O(n2)” • Use the simplest expression of the class • Say “3n+5 is O(n)”instead of “3n+5 is O(3n)” Analysis of Algorithms**Relatives of Big-Oh**• big-Omega • f(n) is (g(n)) if there is a constant c > 0 and an integer constant n0 1 such that f(n) c•g(n) for n n0 • big-Theta • f(n) is (g(n)) if there are constants c’ > 0 and c’’ > 0 and an integer constant n0 1 such that c’•g(n) f(n) c’’•g(n) for n n0 • little-oh • f(n) is o(g(n)) if, for any constant c > 0, there is an integer constant n0 > 0 such that f(n) < c•g(n) for n n0 • little-omega • f(n) is (g(n)) if, for any constant c > 0, there is an integer constant n0 > 0 such that f(n) > c•g(n) for n n0 Analysis of Algorithms**Intuition for Asymptotic Notation**Big-Oh • f(n) is O(g(n)) if f(n) is asymptotically less than or equal to g(n) big-Omega • f(n) is (g(n)) if f(n) is asymptotically greater than or equal to g(n) big-Theta • f(n) is (g(n)) if f(n) is asymptotically equal to g(n) little-oh • f(n) is o(g(n)) if f(n) is asymptotically strictly less than g(n) little-omega • f(n) is (g(n)) if is asymptotically strictly greater than g(n) Analysis of Algorithms**Example Uses of the Relatives of Big-Oh**• 5n2 is (n2) f(n) is (g(n)) if there is a constant c > 0 and an integer constant n0 1 such that f(n) c•g(n) for n n0 let c = 5 and n0 = 1 • 5n2 is (n) f(n) is (g(n)) if there is a constant c > 0 and an integer constant n0 1 such that f(n) c•g(n) for n n0 let c = 1 and n0 = 1 • 5n2 is (n) f(n) is (g(n)) if, for any constant c > 0, there is an integer constant n0 > 0 such that f(n) > c•g(n) for n n0 need 5n02 > c•n0 given c, the n0 that satifies this is n0 > c/5 > 0 Analysis of Algorithms**Divide-and conquer is a general algorithm design paradigm:**Divide: divide the input data S in two or more disjoint subsets S1, S2, … Recur: solve the subproblems recursively Conquer: combine the solutions for S1,S2, …, into a solution for S The base case for the recursion are subproblems of constant size Analysis can be done using recurrence equations Divide-and-Conquer Analysis of Algorithms**Merge-sort on an input sequence S with n elements consists**of three steps: Divide: partition S into two sequences S1and S2 of about n/2 elements each Recur: recursively sort S1and S2 Conquer: merge S1and S2 into a unique sorted sequence Merge-Sort Review AlgorithmmergeSort(S, C) Inputsequence S with n elements, comparator C Outputsequence S sorted • according to C ifS.size() > 1 (S1, S2)partition(S, n/2) mergeSort(S1, C) mergeSort(S2, C) Smerge(S1, S2) Analysis of Algorithms**Recurrence Equation Analysis**• The conquer step of merge-sort consists of merging two sorted sequences, each with n/2 elements and implemented by means of a doubly linked list, takes at most bn steps, for some constant b. • Likewise, the basis case (n< 2) will take at b most steps. • Therefore, if we let T(n) denote the running time of merge-sort: • We can therefore analyze the running time of merge-sort by finding a closed form solution to the above equation. • That is, a solution that has T(n) only on the left-hand side. Analysis of Algorithms**Iterative Substitution**• In the iterative substitution, or “plug-and-chug,” technique, we iteratively apply the recurrence equation to itself and see if we can find a pattern: • Note that base, T(n)=b, case occurs when 2i=n. That is, i = log n. • So, • Thus, T(n) is O(n log n). Analysis of Algorithms**The Recursion Tree**• Draw the recursion tree for the recurrence relation and look for a pattern: Total time = bn + bn log n (last level plus all previous levels) Analysis of Algorithms**Guess-and-Test Method**• In the guess-and-test method, we guess a closed form solution and then try to prove it is true by induction: • Guess: T(n) < cn log n. • Wrong: we cannot make this last line be less than cn log n Analysis of Algorithms**Guess-and-Test Method, Part 2**• Recall the recurrence equation: • Guess #2: T(n) < cn log2 n. • if c > b. • So, T(n) is O(n log2 n). • In general, to use this method, you need to have a good guess and you need to be good at induction proofs. Analysis of Algorithms**Master Method**• Many divide-and-conquer recurrence equations have the form: • The Master Theorem: Analysis of Algorithms**Master Method, Example 1**• The form: • The Master Theorem: • Example: Solution: logba=2, so case 1 says T(n) is O(n2). Analysis of Algorithms**Master Method, Example 2**• The form: • The Master Theorem: • Example: Solution: logba=1, so case 2 says T(n) is O(n log2 n). Analysis of Algorithms**Master Method, Example 3**• The form: • The Master Theorem: • Example: Solution: logba=0, so case 3 says T(n) is O(n logn). Analysis of Algorithms**Master Method, Example 4**• The form: • The Master Theorem: • Example: Solution: logba=3, so case 1 says T(n) is O(n3). Analysis of Algorithms**Master Method, Example 5**• The form: • The Master Theorem: • Example: Solution: logba=2, so case 3 says T(n) is O(n3). Analysis of Algorithms**Master Method, Example 6**• The form: • The Master Theorem: • Example: (binary search) Solution: logba=0, so case 2 says T(n) is O(log n). Analysis of Algorithms**Master Method, Example 7**• The form: • The Master Theorem: • Example: (heap construction) Solution: logba=1, so case 1 says T(n) is O(n). Analysis of Algorithms**Iterative “Proof” of the Master Theorem**• Using iterative substitution, let us see if we can find a pattern: • We then distinguish the three cases as • The first term is dominant • Each part of the summation is equally dominant • The summation is a geometric series Analysis of Algorithms**Integer Multiplication**• Algorithm: Multiply two n-bit integers I and J. • Divide step: Split I and J into high-order and low-order bits • We can then define I*J by multiplying the parts and adding: • So, T(n) = 4T(n/2) + n, which implies T(n) is O(n2). • But that is no better than the algorithm we learned in grade school. Analysis of Algorithms**An Improved Integer Multiplication Algorithm**• Algorithm: Multiply two n-bit integers I and J. • Divide step: Split I and J into high-order and low-order bits • Observe that there is a different way to multiply parts: • So, T(n) = 3T(n/2) + n, which implies T(n) is O(nlog23), by the Master Theorem. • Thus, T(n) is O(n1.585). Analysis of Algorithms