한양대학교 정보보호 및 알고리즘 연구실 2008. 2. 12 이재준 담당교수님 : 박희진 교수님

1. Chapter 3. Growth of function Chapter 4. Recurrences 한양대학교 정보보호 및 알고리즘 연구실 2008. 2. 12 이재준 담당교수님 : 박희진 교수님

2. Contents of Table A. Growth of function 1. Technicalities for abbreviation 2. Recurrence solution methods The substitution method 2. Notation ofasymptotically larger/smaller o-notation ω-notation 3. Relationship between this notations B. Recurrence

3. 2. Asymptotically larger/smaller notation • o-notation Asymptotically smaller - Definition for alln, n ≥ n0 ,any positive constant c, c>0, 0 ≤ f(n)< cg(n),the function f(n) is smaller to g(n) to within constant factor. o(g(n)) f(n) is asymptotically smaller than g(n) f(n) n0

4. 2. Asymptotically larger/smaller notation • o-notation • - Definition • o(g(n)) = { f(n) : for any positive constant c>0, there exist a constant • n0 >0 such that 0 ≤ f(n) < cg(n)for all n ≥ n0 } this limit shows the function f(n) becomes insignificant relative to g(n) as n approaches infinity

5. 1. Asymptotic notation • o-notation Example Use the definition of o-notation to prove the following property. f(n) = o(n2)

6. 2. Notation ofasymptotically larger/smaller • o-notation Solution to example 2n = o(n2) If f(n) = o((g(n)) then

7. 2. Notation ofasymptotically larger/smaller • ω-notation Asymptotically larger - Definition for alln, n ≥ n0 ,any positive constant c, c>0, 0 ≤ cg(n) < f(n),the function f(n) is larger to g(n) to within constant factor. f(n) is asymptotically larger than g(n) f(n) ω(g(n))

8. 2. Notation ofasymptotically larger/smaller • ω-notation • - Definition • ω(g(n)) = { f(n) : for any positive constant c>0, there exist a constant • n0 >0 such that 0 ≤ cg(n) < f(n) for all n ≥ n0 } if the limit exists. That is, f(n) becomes arbitrarily large relative to g(n) as n approaches infinity.

9. 2. Notation ofasymptotically larger/smaller • ω-notation - We use ω-notation to denote a lower bound that is not asymptotically tight. - We use o-notation to denote an upper bound that is not asymptotically tight. - f(n) ∈ ω(g(n)) if and only if g(n) ∈ o(f(n)).

10. 2. Notation ofasymptotically larger/smaller • ω-notation Example Use the definition of o-notation to prove the following property. f(n) = ω(n)

11. 2. Notation ofasymptotically larger/smaller • ω-notation Solution to example =ω(n) If f(n) = ω((g(n)) then

12. 3. Relationship between this notations o(g(n)) O(g(n)) Ω(g(n)) O(g(n)) o(g(n)) ω(g(n)) θ(g(n)) f(n) Ω(g(n)) ω(g(n))

13. 3. Relationship between this notations • Transitivity • Transpose symmetry o(g(n)) • f(n) = Θ(g(n)) and g(n) = Θ(h(n)) imply f(n) = Θ(h(n)) , • f(n) = O(g(n)) and g(n) = O(h(n)) imply f(n) = O(h(n)) , • f(n) = Ω(g(n)) and g(n) = Ω(h(n)) imply f(n) = Ω(h(n)) , • f(n) = o(g(n)) and g(n) = o(h(n)) imply f(n) = o(h(n)) , • f(n) = ω(g(n)) and g(n) = ω(h(n)) imply f(n) = ω(h(n)). O(g(n)) f(n) Ω(g(n)) ω(g(n)) • f(n) = O(g(n)) if and only if g(n) = Ω(f(n)), • f(n) = o(g(n)) if and only if g(n) = ω(f(n)).

14. 3. Relationship between this notations • Reflexivity • f(n) = Θ(f(n)) • f(n) = O(f(n)) • f(n) = Ω(f(n)) • Symmetry • f(n) = Θ(g(n)) if and only if g(n) = Θ(f(n)). O(g(n)) Ω(g(n)) θ(g(n))

15. Recurrences • Recurrence When should we use the recurrence? When algorithm contains a recursive call to itself, Its running time can often be described by a recurrence.

16. Recurrences • Recurrence - Definition Equation or inequality that describes a function in terms of its value on smaller inputs.

17. Recurrences • RecurrenceExample Worst-case running time of T(n)could be described by the recurrence Merge-Sort procedure Show that the solution of T(n) = Θ(n log n) if n=1 if n>1

18. Recurrences • RecurrenceSolution method to example - The substitution method guess a bound and prove our guess correct for small input. - The recursion-tree method convert the recurrence into a tree whose nodes represent the costs incurred at various level of recursion - The master method determining asymptotic boundsfor many simple recurrences of the form ( a ≥ 1, b ≥ 1, and f(n) is given function )

19. 1. Technicalities for abbreviation • Technicalities - assumption of integer The running time T(n) of algorithm is only defined when n is an integer - floors / ceilings We often omit floors and ceilings - boundary condition The running time of an algorithm on a constant-sized input is a constant that we typically ignore.

20. 1. Technicalities for abbreviation if n=1 if n>1 • Technicalities Example The recurrence describing the worst-case running time of Merge-Sort if n=1 if n>1 if n=1 Omit assumption of integer if n>1 Omit floors and ceilings Omit boundary condition

21. 2. Recurrence solution methods • Substitution Method - Definition When recurrence is easy to guess form of the answer with the inductive hypothesis is applied to smaller values, substitution of the guessed answer for the function . - It can be used to establish either upper or lower bounds on a recurrence.

22. 2. Recurrence solution methods • Substitution Method Steps 1. Guess the form of the solution 2. Use mathematical induction to find the constant and show that the solution works.

23. 2. Recurrence solution methods • Substitution Method - Making a good guess prove loose upper and lower bounds on the recurrence and then reduce the range of uncertainty. - Guessing a solution takes : experience, occasionally, creativity - we can use recursion tree

24. 2. Recurrence solution methods • Substitution Method - Subtleties we can prove something stronger for given value by assuming something stronger for a smaller values. We guess that the solution is O(n), and show that T(n) ≤ cn Overcome difficulty by subtracting a lower-order term from our previous guess ( b≥1, c must be chosen large enough )

25. 2. Recurrence solution methods • Substitution Method - Avoid pitfalls In recurrence we can falsely prove T(n)= O(n) by guessing T(n) ≤ cn

26. 2. Recurrence solution methods • Substitution Method Example Let us determine an upper bound on the recurrence

27. 2. Recurrence solution methods • Substitution Method Solution to example We need to find constant candn0 - Assume that this bound holds for ⌊n/2⌋, that is, T (⌊n/2⌋) ≤ c ⌊n/2⌋ lg(⌊n/2⌋). T(n) ≤ 2(c ⌊n/2⌋lg(⌊n/2⌋)) + n ≤ cn lg(n/2) + n ( because, ⌊n/2⌋ < n/2 ) = cn lg n - cn lg 2 + n = cn lg n - cn + n ≤ cn lg n (as long as c ≥ 1) 1. guess that the solution is 2. prove that ( c > 0 )

28. 2. Recurrence solution methods • Substitution Method Solution to example • - Find the constant n0 - we can take advantage of asymptotic notation, we can replace base case T(1) ≤ cn lg n T(1) = 1 but, c1 lg1 = 0 - So, we only have to prove T (n) = cn lg n for n ≥ n0 for n (n0 =2) Replace T(1) byT(2) andT(3) as the base cases by letting n0 =2.

29. 2. Recurrence solution methods • Substitution Method Solution to example • - Find the constant c • - choosing c large enough T (2) = 4 and T (3) = 5. T (2) = 4 ≤ c ( 2 lg 2 ) T (3) = 5 ≤ c ( 3 lg 3 ). Any choice of c ≥ 2 suffices for base case of n=2 and n =3

30. 2. Recurrence solution methods • Substitution Method Example Let us determine an upper bound on the recurrence

31. 2. Recurrence solution methods • Substitution Method Solution to example - Simplify this recurrence bychanging variable Renaming m = lg n S(m) = T(2m) This new recurrence has same solution : S(m) = O(m log m)

32. 2. Recurrence solution methods • Substitution Method Solution to example • Simplify this recurrence bychanging variable This new recurrence has same solution : S(m) = O(m log m) Changing back from S(m) to T(n) T(n) = T(2m) = S(m) = O(m lg m) = O( lg n lg(lg n) )