Big-Oh Notation

1. Big-Oh Notation MSIT

2. Agenda • What is Big-Oh Notation? • Example • Guidelines • Theorems

3. Big-Oh Notation (O) • f(x) is O(g(x))iff there exists constants ‘c’and ‘k’ such that f(x)<=c.g(x) where x>k • Pronounced as f(x) is Big-Oh of g(x) • This gives the upper bound value of a function

4. Example • f(n) = 10n + 5 and g(n) = n • To show f(n) is O(g(n)) • we must show constants c and k such that f(n) <= cg(n) for all n >=k or 10n+5 <= cn for all n >= k • We are allowed to choose c and k to be integers we want as long as they are positive.

5. Contd.. • They can be as big as we want, but they can't be functions of n. • Try c = 15. Then we need to show: 10n + 5 <= 15n. Solving for n we get: 5 <= 5n or 1 <= n. So f(n) = 10+5 <= 15g(n) for all n >= 1. (c = 15, k = 1). Therefore we have shown f(n) is O(g(n)).

6. How do we calculate big-O? 1 Loops 2 Nested loops 3 Consecutive statements 4 If-then-else statements 5 Logarithmic complexity Five guidelines for finding out the time complexity of a piece of code

7. executed n times constant time Guideline 1: Loops The running time of a loop is, at most, the running time of the statements inside the loop (including tests) multiplied by the number of iterations. for (i=1; i<=n; i++) { m = m + 2; } Total time = a constant c * n = cn = O(N)

8. outer loop executed n times inner loop executed n times constant time Guideline 2: Nested loops Analyse inside out. Total running time is the product of the sizes of all the loops. for (i=1; i<=n; i++) { for (j=1; j<=n; j++) { k = k+1; } } Total time = c * n * n * = cn2 = O(N2)

9. constant time executed n times constant time inner loop executed n times outer loop executed n times constant time Guideline 3: Consecutive statements Add the time complexities of each statement. x = x +1; for (i=1; i<=n; i++) { m = m + 2; } for (i=1; i<=n; i++) { for (j=1; j<=n; j++) { k = k+1; } } Total time = c0 + c1n + c2n2 = O(N2)

10. test: constant then part: constant else part: (constant + constant) * n Guideline 4: If-then-else statements Worst-case running time: the test, plus either the then part or the else part (whichever is the larger). if (depth( ) != otherStack.depth( ) ) { return false; } else { for (int n = 0; n < depth( ); n++) { if (!list[n].equals(otherStack.list[n])) return false; } } another if : constant + constant (no else part) Total time = c0 + c1 + (c2 + c3) * n = O(N)

11. Guideline 5: Logarithmic complexity An algorithm is O(log N) if it takes a constant time to cut the problem size by a fraction (usually by ½) Example algorithm (binary search): finding a word in a dictionary of n pages • Look at the centre point in the dictionary • Is word to left or right of centre? • Repeat process with left or right part of dictionary until the word is found

12. Theorems • Let d(n),e(n),f(n) and g(n) be functions mapping non negative integers real. Then • If d(n) is O(f(n)),then ad(n) is O(f(n)),for any constant a>0 • If d(n) is O(f(n)) and e(n) is O(g(n)),then d(n)+e(n) is O(f(n)+g(n)) • If d(n) is O(f(n)) and e(n) is O(g(n)),then d(n).e(n) is O(f(n) g(n)) • If d(n) is O(f(n)) and f(n) is O(g(n)),then d(n) is O(g(n)).

13. Contd.. • If f(n) is a polynomial of degree d i.e, f(n)=(a0+a1n+…+ad nd) then f(n) is O(nd). • n x is O(a n) for any fixed x>0 and a>1 • log n x is O(log n) for any fixed x>0 • log x n is O(n y) for any fixed constants x>0 and y>0

14. Example • 2n3+4n2logn is O(n3) • Proof: log n is O(n) – Rule 8 4n2logn is O(4n3) – Rule 3 2n3+4n2log n is O(2n3+4 n3) – Rule 2 2n3+4n2log n is O(n3) – Rule 1 2n3+4n2log n is O(n3) – Rule 4 Hence,Proved.

15. Relatives of Big-Oh • big-Omega f(n) is Ω(g(n)) if there is a constant c > 0 and an integer constant n0 ≥ 1 such that f(n) ≥ c*g(n) for n ≥n0 • big-Theta f(n) is θ(g(n)) if there are constants c ’ > 0 and c’’ > 0 and an integer constant n0≥ 1 such that c’•g(n) > f(n) > c’’•g(n) for n ≥n0

16. Contd… • little-oh f(n) is o(g(n)) if, for any constant c > 0, there is an integer constant n0≥ 0 such that f(n) ≤ c•g(n) for n ≥n0 • little-omega f(n) is w(g(n)) if, for any constant c > 0, there is an integer constant n ≥0such that f(n) ≥c•g(n) for n ≥n0

17. Intuition for Asymptotic Notation • Big-Oh : f(n) is O(g(n)) if f(n) is asymptotically less than or equal to g(n) • big-Omega : f(n) is W(g(n)) if f(n) is asymptotically greater than or equal to g(n) • big-Theta :f(n) is Q(g(n)) if f(n) is asymptotically equal to g(n) • little-oh : f(n) is o(g(n)) if f(n) is asymptotically strictly less than g(n) • little-omega: f(n) is w(g(n)) if is asymptotically strictly greater than g(n)

18. Bubble Sort – O(n)2 main() { int a[6]={7,5,3,4,2,1}; int i=0,j=0; int n=6; for(j=n-1;j>0;j--) { for(i=0;i<j;i++) { if(a[i]>a[i+1]) { int temp=a[i]; //swap a[i]=a[i+1]; a[i+1]=temp; } } }}

19. Assignment • Find the complexity of the following Algorithm in Big-Oh – Binary Search for (i=1;i<10 ;i++ ) { for (j=0;j<i ;j++ ) { if (arr[j]>arr[i]) { temp=arr[j]; arr[j]=arr[i]; for (k=i;k>j ;k-- ) arr[k]=arr[k-1]; arr[k+1]=temp; } }}

20. References • Fundamentals of Computer Algorithms Ellis Horowitz,Sartaj Sahni,Sanguthevar Rajasekaran • Algorithm Design Micheal T. GoodRich,Robert Tamassia • Analysis of Algorithms Jeffrey J. McConnell

21. THANK YOU