1 / 22

CS 253: Algorithms

CS 253: Algorithms. Chapter 3 Growth of Functions. Credit : Dr. George Bebis. Analysis of Algorithms. Goal : To analyze and compare algorithms in terms of running time and memory requirements (i.e. time and space complexity )

sook
Download Presentation

CS 253: Algorithms

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. CS 253: Algorithms Chapter 3 Growth of Functions Credit: Dr. George Bebis

  2. Analysis of Algorithms Goal: • To analyze and compare algorithms in terms of running timeand memory requirements (i.e. time and spacecomplexity) • In other words, how does the running time and space requirements change as we increase the input size n ? (sometimes we are also interested in the coding complexity) • Input size (number of elements in the input) • size of an array or a matrix • # of bits in the binary representation of the input • vertices and/or edges in a graph, etc.

  3. Types of Analysis • Worst case • Provides an upper bound on running time • An absolute guarantee that the algorithm would not run longer, no matter what the inputs are • Best case • Provides a lower bound on running time • Input is the one for which the algorithm runs the fastest • Average case • Provides a prediction about the running time • Assumes that the input is random Lower Bound ≤ Running Time ≤ Upper Bound

  4. Computing the Running Time • Measure the execution time ? Not a good idea ! It varies for different microprocessors! • Count the number of statements executed? Yes, but you need to be very careful! High-level programming languages have statements which require a large number of low-level machine language instructions to execute (a function of the input size n). For example, a subroutine call can not be counted as one statement; it needs to be analyzed separately • Associate a "cost" with each statement. Find the "total cost“by multiplying the cost with the total number of times each statement is executed.  (we have seen examples before)

  5. Example Algorithm X Cost   sum = 0; c1 for(i=0; i<N; i++) c2 for(j=0; j<N; j++) c3 sum += arrY[i][j]; c4 ------------ Total Cost = c1+ c2*(N+1) + c3*N * (N+1) + c4*N2

  6. Asymptotic Analysis • To compare two algorithms with running times f(n) and g(n), we need a rough measure that characterizes how fast each function grows with respect to n • In other words, we are interested in how they behave asymptotically (i.e. for large n) (called rate of growth) • Big O notation: asymptotic “less than” or “at most”: f(n)=O(g(n)) implies: f(n) “≤” g(n) •  notation: asymptotic “greater than” or “at least”: f(n)=  (g(n)) implies: f(n) “≥” g(n) •  notation: asymptotic “equality” or “exactly”: f(n)=  (g(n)) implies: f(n) “=” g(n)

  7. Big-O Notation • We say fA(n) = 7n+18is order n, or O (n)It is, at most, roughly proportional to n. fB(n) = 3n2+5n +4 is order n2, or O(n2). It is, at most, roughly proportional to n2. • In general, any O(n2) function is faster- growing than any O(n) function. fB(n) fA(n) Function value  Increasing n 

  8. More Examples … • n4+ 100n2 + 10n + 50O(n4) 10n3+ 2n2 O(n3) n3- n2 O(n3) • constants 10 is O(1) 1273 is O(1) • what is the rate of growth for Algorithm X studied earlier (in Big O notation)? Total Time = c1 + c2*(N+1) + c2* N*(N+1) + c3*N2 If c1, c2, c3 , and c4 are constants then Total Time = O(N2)

  9. Definition of Big O • O-notation

  10. cn=31n Big-O example, graphically • Note that 30n+8 is O(n). Can you find a c and n0 which can be used in the formal definition of Big O ? • You can easily see that 30n+8 isn’t less than nanywhere (n>0). • But it is less than31n everywhere tothe right of n=8. • So, one possible (c , n0) pair that can be used in the formal definition: c = 31, n0 = 8 30n+8 Function value  n n0=8 n  30n+8  O(n)

  11. Big-O Visualization O(g(n)) is the set of functions with smaller or same order of growth as g(n)

  12. No Uniqueness • There is no unique set of values for n0 and c in proving the asymptotic bounds • Prove that 100n + 5 = O(n2) (i) 100n + 5 ≤ 100n + n = 101n ≤ 101n2for all n ≥ 5 You may pick n0= 5 and c = 101to complete the proof. (ii) 100n + 5 ≤ 100n + 5n = 105n ≤ 105n2 for all n ≥ 1 You may pick n0= 1 and c = 105to complete the proof.

  13. Definition of  (g(n)) is the set of functions with larger or same order of growth as g(n)

  14. Examples • 5n2= (n)  c, n0such that: 0  cn  5n2 cn  5n2  c = 1 and n > n0=1 • 100n + 5 ≠(n2)  c, n0 such that: 0  cn2  100n + 5 since 100n + 5  100n + 5n  n  1 cn2  105n  n(cn – 105)  0 Since n is positive  (cn – 105)  0  n  105/c  contradiction: n cannot be smaller than a constant • n = (2n), n3= (n2), n = (logn)

  15. Definition of  • -notation • (g(n)) is the set of functions with the same order of growth as g(n)

  16. Examples • n2/2 –n/2 = (n2) • ½ n2 - ½ n ≤ ½ n2n ≥ 0  c2= ½ • ¼ n2≤ ½ n2 - ½ n n ≥ 2  c1= ¼ • n ≠ (n2): c1 n2≤ n ≤ c2 n2 only holds for: n ≤ 1/c1 • 6n3 ≠ (n2): c1 n2≤ 6n3 ≤ c2 n2  only holds for: n ≤ c2 /6 • n ≠ (logn): c1logn≤ n ≤ c2logn  c2 ≥ n/logn,  n≥ n0 – impossible

  17. Relations Between Different Sets • Subset relations between order-of-growth sets. RR ( f ) O( f ) • f ( f )

  18. Common orders of magnitude

  19. Common orders of magnitude

  20. Logarithms and properties • In algorithm analysis we often use the notation “log n” without specifying the base Binary logarithm Natural logarithm

  21. More Examples • For each of the following pairs of functions, either f(n) is O(g(n)), f(n) is Ω(g(n)), or f(n) = Θ(g(n)). Determine which relationship is correct. • f(n) = log n2; g(n) = log n + 5 • f(n) = n; g(n) = log n2 • f(n) = log log n; g(n) = log n • f(n) = n; g(n) = log2 n • f(n) = n log n + n; g(n) = log n • f(n) = 10; g(n) = log 10 • f(n) = 2n; g(n) = 10n2 • f(n) = 2n; g(n) = 3n f(n) = (g(n)) f(n) = (g(n)) f(n) = O(g(n)) f(n) = (g(n)) f(n) = (g(n)) f(n) = (g(n)) f(n) = (g(n)) f(n) = O(g(n))

  22. Properties Theorem: f(n) = (g(n))  f = O(g(n)) and f = (g(n)) • Transitivity: • f(n) = (g(n))andg(n) = (h(n))  f(n) = (h(n)) • Same for O and  • Reflexivity: • f(n) = (f(n)) • Same for O and  • Symmetry: • f(n) = (g(n)) if and only if g(n) = (f(n)) • Transpose symmetry: • f(n) = O(g(n)) if and only if g(n) = (f(n))

More Related