1 / 68

Asymptotic Notations in Computer Science

Explore O-notation and related concepts like upper bounds, macro substitution, and solving recurrences in computer science. Learn how to apply them effectively in theoretical analysis of algorithms.

jacquelynr
Download Presentation

Asymptotic Notations in Computer Science

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Asymptotic notation O-notation (upper bounds): We write f(n) = O(g(n)) if there exist constants c > 0, n0 > 0 such that 0 f(n) cg(n) for all nn0.

  2. Asymptotic notation O-notation (upper bounds): We write f(n) = O(g(n)) if there exist constants c > 0, n0 > 0 such that 0 f(n) cg(n) for all nn0. EXAMPLE:2n2 = O(n3) (c = 1, n0 = 2)

  3. Asymptotic notation O-notation (upper bounds): We write f(n) = O(g(n)) if there exist constants c > 0, n0 > 0 such that 0 f(n) cg(n) for all nn0. EXAMPLE:2n2 = O(n3) (c = 1, n0 = 2) functions, not values

  4. Asymptotic notation O-notation (upper bounds): We write f(n) = O(g(n)) if there exist constants c > 0, n0 > 0 such that 0 f(n) cg(n) for all nn0. EXAMPLE:2n2 = O(n3) (c = 1, n0 = 2) funny, “one-way” equality functions, not values

  5. Set definition of O-notation O(g(n))= { f(n) : there exist constants c > 0, n0 > 0 such that 0 f(n) cg(n) for all nn0}

  6. Set definition of O-notation O(g(n))= { f(n) : there exist constants c > 0, n0 > 0 such that 0 f(n) cg(n) for all nn0} EXAMPLE:2n2O(n3)

  7. Set definition of O-notation O(g(n))= { f(n) : there exist constants c > 0, n0 > 0 such that 0 f(n) cg(n) for all nn0} EXAMPLE:2n2O(n3) (Logicians:n.2n2O(n.n3), but it’s convenient to be sloppy, as long as we understand what’s really going on.)

  8. Macro substitution Convention: A set in a formula represents an anonymous function in the set.

  9. Macro substitution Convention: A set in a formula represents an anonymous function in the set. f(n) = n3 + O(n2) means f(n) = n3 + h(n) for some h(n) O(n2) . EXAMPLE:

  10. Macro substitution Convention: A set in a formula represents an anonymous function in the set. n2 + O(n) = O(n2) means for any f(n) O(n): n2 + f(n) = h(n) for some h(n) O(n2) . EXAMPLE:

  11. -notation (lower bounds) O-notation is an upper-bound notation. It makes no sense to say f(n) is at least O(n2).

  12. -notation (lower bounds) O-notation is an upper-bound notation. It makes no sense to say f(n) is at least O(n2). (g(n))= { f(n) : there exist constants c > 0, n0 > 0 such that 0 cg(n) f(n) for all nn0}

  13. -notation (lower bounds) O-notation is an upper-bound notation. It makes no sense to say f(n) is at least O(n2). (g(n))= { f(n) : there exist constants c > 0, n0 > 0 such that 0 cg(n) f(n) for all nn0} EXAMPLE: (c = 1, n0 = 16)

  14. -notation (tight bounds) (g(n)) = O(g(n))  (g(n))

  15. -notation (tight bounds) (g(n)) = O(g(n))  (g(n)) EXAMPLE:

  16. o-notation and -notation O-notation and -notation are like  and . o-notation and -notation are like < and >. o(g(n))= { f(n) : for any constant c > 0, there is a constant n0 > 0 such that 0 f(n) <cg(n) for all nn0} 2n2 = o(n3) EXAMPLE: (n0 = 2/c)

  17. o-notation and -notation O-notation and -notation are like  and . o-notation and -notation are like < and >. (g(n))= { f(n) : for any constant c > 0, there is a constant n0 > 0 such that 0 cg(n)<f(n) for all nn0} EXAMPLE: (n0 = 1+1/c)

  18. Solving recurrences • The analysis of merge sort from Lecture 1 required us to solve a recurrence. • Recurrences are like solving integrals, differential equations, etc. • Learn a few tricks. • Lecture 3: Applications of recurrences to divide-and-conquer algorithms.

  19. Substitution method The most general method: • Guess the form of the solution. • Verify by induction. • Solve for constants.

  20. CSE408Methods for solving recurrences Lecture #12

  21. EXAMPLE:T(n) = 4T(n/2) + n • [Assume that T(1) = Q(1).] • Guess O(n3) . (Prove O and W separately.) • Assume that T(k) £ck3 for k < n . • Prove T(n) £cn3 by induction. Substitution method The most general method: • Guess the form of the solution. • Verify by induction. • Solve for constants.

  22. Example of substitution desired–residual desired whenever (c/2)n3 – n³ 0, for example, if c ³ 2 and n ³ 1. residual

  23. Example (continued) • We must also handle the initial conditions, that is, ground the induction with base cases. • Base:T(n) = Q(1) for all n < n0, where n0 is a suitable constant. • For 1 £n < n0, we have “Q(1)”£cn3, if we pick c big enough.

  24. This bound is not tight! Example (continued) • We must also handle the initial conditions, that is, ground the induction with base cases. • Base:T(n) = Q(1) for all n < n0, where n0 is a suitable constant. • For 1 £n < n0, we have “Q(1)”£cn3, if we pick c big enough.

  25. A tighter upper bound? We shall prove that T(n) = O(n2).

  26. A tighter upper bound? We shall prove that T(n) = O(n2). Assume that T(k) £ck2 for k < n:

  27. A tighter upper bound? We shall prove that T(n) = O(n2). Assume that T(k) £ck2 for k < n: Wrong! We must prove the I.H.

  28. A tighter upper bound? We shall prove that T(n) = O(n2). Assume that T(k) £ck2 for k < n: Wrong! We must prove the I.H. [ desired–residual ] for no choice of c > 0. Lose!

  29. A tighter upper bound! • IDEA:Strengthen the inductive hypothesis. • Subtracta low-order term. Inductive hypothesis: T(k) £c1k2 – c2k for k < n.

  30. A tighter upper bound! • IDEA:Strengthen the inductive hypothesis. • Subtracta low-order term. Inductive hypothesis: T(k) £c1k2 – c2k for k < n. T(n) = 4T(n/2) + n = 4(c1(n/2)2 – c2(n/2)) + n = c1n2 – 2c2n + n = c1n2 – c2n – (c2n – n) c1n2 – c2nifc2 1.

  31. A tighter upper bound! • IDEA:Strengthen the inductive hypothesis. • Subtracta low-order term. Inductive hypothesis: T(k) £c1k2 – c2k for k < n. T(n) = 4T(n/2) + n = 4(c1(n/2)2 – c2(n/2)) + n = c1n2 – 2c2n + n = c1n2 – c2n – (c2n – n) c1n2 – c2nifc2 1. Pick c1 big enough to handle the initial conditions.

  32. Recursion-tree method • A recursion tree models the costs (time) of a recursive execution of an algorithm. • The recursion-tree method can be unreliable, just like any method that uses ellipses (…). • The recursion-tree method promotes intuition, however. • The recursion tree method is good for generating guesses for the substitution method.

  33. Example of recursion tree Solve T(n) = T(n/4) + T(n/2) + n2:

  34. Example of recursion tree Solve T(n) = T(n/4) + T(n/2) + n2: T(n)

  35. T(n/2) T(n/4) Example of recursion tree Solve T(n) = T(n/4) + T(n/2) + n2: n2

  36. Example of recursion tree Solve T(n) = T(n/4) + T(n/2) + n2: n2 (n/2)2 (n/4)2 T(n/8) T(n/4) T(n/16) T(n/8)

  37. Example of recursion tree Solve T(n) = T(n/4) + T(n/2) + n2: n2 (n/2)2 (n/4)2 (n/8)2 (n/4)2 (n/16)2 (n/8)2 … Q(1)

  38. Example of recursion tree Solve T(n) = T(n/4) + T(n/2) + n2: n2 (n/2)2 (n/4)2 (n/8)2 (n/4)2 (n/16)2 (n/8)2 … Q(1)

  39. Example of recursion tree Solve T(n) = T(n/4) + T(n/2) + n2: n2 (n/2)2 (n/4)2 (n/8)2 (n/4)2 (n/16)2 (n/8)2 … Q(1)

  40. Example of recursion tree Solve T(n) = T(n/4) + T(n/2) + n2: n2 (n/2)2 (n/4)2 (n/8)2 (n/4)2 (n/16)2 (n/8)2 … … Q(1)

  41. Recursion Tree

  42. Example of recursion tree Solve T(n) = T(n/4) + T(n/2) + n2: n2 (n/2)2 (n/4)2 (n/8)2 (n/4)2 (n/16)2 (n/8)2 … … Q(1) Total = = Q(n2) geometric series

More Related