1 / 74

Algorithm Efficiency Analysis

Learn about algorithm efficiency analysis, time and space efficiency, empirical and theoretical analysis, order of growth, asymptotic notation, big-O notation, and examples of insertion sort.

kesslerj
Download Presentation

Algorithm Efficiency Analysis

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Analysis of Algorithm Efficiency Dr. Yingwu Zhu p5-11, p16-29, p43-53, p93-96

  2. What is Algorithm? • Any well-defined computational procedure that: • takes input: some value or a set of values and • produces output: some value or a set of values • Example: sorting problems • Input: a sequence of numbers {a1, a2, …, an} • Output: a permutation {b1, b2, …, bn} of the input s.t. b1<=b2<=…<=bn

  3. Algorithms: Motivation • At the heart of programs lie algorithms • Algorithms as a key technology • Think about: • mapping/navigation • Google search • word processing (spelling correction, layout…) • content delivery and streaming video • games (graphics, rendering…) • big data (querying, learning…)

  4. Algorithms: Motivation • In a perfect world • for each problem we would have an algorithm • the algorithm would be the fastest possible What would CS look like in this world?

  5. Algorithms: Motivation • Our world (fortunately) is not so perfect: • for many problems we know embarrassingly little about what the fastest algorithm is • multiplying two integers or two matrices • factoring an integer into primes • determining shortest tour of given n cities • for many problems we suspect fast algorithms are impossible (NP-complete problems) • for some problems we have unexpected and clever algorithms (we will see many of these)

  6. Analyzing Algorithms • Given computing resources and input data • how fast does an algorithm run? • Time efficiency: amount of time required to accomplish the task • Our focus • How much memory is required? • Space efficiency: amount of memory required • Deals with the extra space the algorithm requires

  7. These vary from one platform to another Time Efficiency • Time efficiency depends on : • size of input • speed of machine • quality of source code • quality of compiler So, we cannot express time efficiency meaningfully in real time units such as seconds!

  8. Empirical analysis of time efficiency • Select a specific (typical) sample of inputs • Use physical unit of time (e.g., milliseconds) or Count actual number of basic operation’s executions • Analyze the empirical data • Limitation: results dependent on the particular computer and the sample of inputs

  9. Input size running time Number of times basic operation is executed execution time for basic operation Theoretical analysis of time efficiency Time efficiency is analyzed by determining the number of repetitions of the basic operationas a function of input size • Basic operation: the operation that contributes most towards the running time of the algorithm T(n) ≈copC(n)

  10. Input size and basic operation examples

  11. Time Efficiency • T(n) = (approximated by) number of times the basic operation is executed. • Not only depends on the input size n, but also depends on the arrangement of the input items • Best case: not informative • Average case: difficult to determine • Worst case: is used to measure an algorithm’s performance

  12. Example: Sequential Search • Best case T(n) • Worst case T(n) • Average case T(n) • Assume success search probability of p

  13. Order of Growth • Established framework for analyzing T(n) • Order of growth as n→∞ • Highest-order term is what counts • Remember, we are doing asymptotic analysis • As the input size grows larger it is the high order term that dominates • disregard lower-order terms in running time • disregard coefficient on highest order term • Asymptotic notations: , O, 

  14. Asymptotic Order of Growth A way of comparing functions that ignores constant factors and small input sizes • O(g(n)): class of functions f(n) that grow no faster than g(n) • Θ(g(n)): class of functions f(n) that grow at same rate as g(n) • Ω(g(n)): class of functions f(n) that grow at least as fast as g(n)

  15. Big-oh

  16. Big-omega

  17. Big-theta

  18. Upper Bound Notation • In general a function • f(n) is O(g(n)) if there exist positive constants c and n0such that f(n)  c  g(n) for all n  n0 • Order of growth of f(n) order of growth of g(n) (within constant multiple) • Formally • O(g(n)) = { f(n):  positive constants c and n0such that f(n)  c  g(n)  n  n0

  19. Big-Oh • Examples • 10n is O(n2) • 5n+20 is O(n)

  20. An Example: Insertion Sort InsertionSort(A, n) {for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key} }

  21. An Example: Insertion Sort i =  j =  key = A[j] =  A[j+1] =  30 10 40 20 InsertionSort(A, n) {for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key} } 1 2 3 4

  22. An Example: Insertion Sort i = 2 j = 1 key = 10A[j] = 30 A[j+1] = 10 30 10 40 20 InsertionSort(A, n) {for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key} } 1 2 3 4

  23. An Example: Insertion Sort i = 2 j = 1 key = 10A[j] = 30 A[j+1] = 30 30 30 40 20 InsertionSort(A, n) {for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key} } 1 2 3 4

  24. An Example: Insertion Sort i = 2 j = 1 key = 10A[j] = 30 A[j+1] = 30 30 30 40 20 InsertionSort(A, n) {for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key} } 1 2 3 4

  25. An Example: Insertion Sort i = 2 j = 0 key = 10A[j] =  A[j+1] = 30 30 30 40 20 InsertionSort(A, n) {for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key} } 1 2 3 4

  26. An Example: Insertion Sort i = 2 j = 0 key = 10A[j] =  A[j+1] = 30 30 30 40 20 InsertionSort(A, n) {for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key} } 1 2 3 4

  27. An Example: Insertion Sort i = 2 j = 0 key = 10A[j] =  A[j+1] = 10 10 30 40 20 InsertionSort(A, n) {for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key} } 1 2 3 4

  28. An Example: Insertion Sort i = 3 j = 0 key = 10A[j] =  A[j+1] = 10 10 30 40 20 InsertionSort(A, n) {for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key} } 1 2 3 4

  29. An Example: Insertion Sort i = 3 j = 0 key = 40A[j] =  A[j+1] = 10 10 30 40 20 InsertionSort(A, n) {for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key} } 1 2 3 4

  30. An Example: Insertion Sort i = 3 j = 0 key = 40A[j] =  A[j+1] = 10 10 30 40 20 InsertionSort(A, n) {for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key} } 1 2 3 4

  31. An Example: Insertion Sort i = 3 j = 2 key = 40A[j] = 30 A[j+1] = 40 10 30 40 20 InsertionSort(A, n) {for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key} } 1 2 3 4

  32. An Example: Insertion Sort i = 3 j = 2 key = 40A[j] = 30 A[j+1] = 40 10 30 40 20 InsertionSort(A, n) {for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key} } 1 2 3 4

  33. An Example: Insertion Sort i = 3 j = 2 key = 40A[j] = 30 A[j+1] = 40 10 30 40 20 InsertionSort(A, n) {for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key} } 1 2 3 4

  34. An Example: Insertion Sort i = 4 j = 2 key = 40A[j] = 30 A[j+1] = 40 10 30 40 20 InsertionSort(A, n) {for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key} } 1 2 3 4

  35. An Example: Insertion Sort i = 4 j = 2 key = 20A[j] = 30 A[j+1] = 40 10 30 40 20 InsertionSort(A, n) {for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key} } 1 2 3 4

  36. An Example: Insertion Sort i = 4 j = 2 key = 20A[j] = 30 A[j+1] = 40 10 30 40 20 InsertionSort(A, n) {for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key} } 1 2 3 4

  37. An Example: Insertion Sort i = 4 j = 3 key = 20A[j] = 40 A[j+1] = 20 10 30 40 20 InsertionSort(A, n) {for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key} } 1 2 3 4

  38. An Example: Insertion Sort i = 4 j = 3 key = 20A[j] = 40 A[j+1] = 20 10 30 40 20 InsertionSort(A, n) {for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key} } 1 2 3 4

  39. An Example: Insertion Sort i = 4 j = 3 key = 20A[j] = 40 A[j+1] = 40 10 30 40 40 InsertionSort(A, n) {for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key} } 1 2 3 4

  40. An Example: Insertion Sort i = 4 j = 3 key = 20A[j] = 40 A[j+1] = 40 10 30 40 40 InsertionSort(A, n) {for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key} } 1 2 3 4

  41. An Example: Insertion Sort i = 4 j = 3 key = 20A[j] = 40 A[j+1] = 40 10 30 40 40 InsertionSort(A, n) {for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key} } 1 2 3 4

  42. An Example: Insertion Sort i = 4 j = 2 key = 20A[j] = 30 A[j+1] = 40 10 30 40 40 InsertionSort(A, n) {for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key} } 1 2 3 4

  43. An Example: Insertion Sort i = 4 j = 2 key = 20A[j] = 30 A[j+1] = 40 10 30 40 40 InsertionSort(A, n) {for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key} } 1 2 3 4

  44. An Example: Insertion Sort i = 4 j = 2 key = 20A[j] = 30 A[j+1] = 30 10 30 30 40 InsertionSort(A, n) {for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key} } 1 2 3 4

  45. An Example: Insertion Sort i = 4 j = 2 key = 20A[j] = 30 A[j+1] = 30 10 30 30 40 InsertionSort(A, n) {for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key} } 1 2 3 4

  46. An Example: Insertion Sort i = 4 j = 1 key = 20A[j] = 10 A[j+1] = 30 10 30 30 40 InsertionSort(A, n) {for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key} } 1 2 3 4

  47. An Example: Insertion Sort i = 4 j = 1 key = 20A[j] = 10 A[j+1] = 30 10 30 30 40 InsertionSort(A, n) {for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key} } 1 2 3 4

  48. An Example: Insertion Sort i = 4 j = 1 key = 20A[j] = 10 A[j+1] = 20 10 20 30 40 InsertionSort(A, n) {for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key} } 1 2 3 4

  49. An Example: Insertion Sort i = 4 j = 1 key = 20A[j] = 10 A[j+1] = 20 10 20 30 40 InsertionSort(A, n) {for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key} } 1 2 3 4 Done!

  50. Insertion Sort InsertionSort(A, n) {for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key} } How many times will this loop execute?

More Related