1 / 55

ITCS6114: Algorithms and Data Structures

This course provides a rigorous introduction to the design and analysis of algorithms, focusing on proof by induction and asymptotic notation. It is not a programming or math course.

leonej
Download Presentation

ITCS6114: Algorithms and Data Structures

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. ITCS6114: Algorithms and Data Structures Introduction Proof by Induction Asymptotic Notation

  2. The Course • Purpose: a rigorous introduction to the design and analysis of algorithms • Not a lab or programming course • Not a math course, either • Suggested Textbooks: Introduction to The Design & Analysis of Algorithms, Anany Levitin Introduction to Algorithms, Cormen, Leiserson, Rivest, Stein

  3. The Course • Instructor: Zbigniew W. Ras • ras@uncc.edu • Office: Woodward Hall 430C • Office hours: Tuesday: 12:00-1:30pm • TA: Ayman Hajja • ahajja@uncc.edu Office: Woodward Hall 404 (Future Computing Lab) • Office hours: Tuesday, Thursday 11:30-1:30pm

  4. The Course • Grading policy: • Midterm: 30 points • Project: 30 points • Final: 30 points • Participation: 10 points • Grade A from 86 to 100 points, Grade B from 71 to 85 points, Grade C from 56 to 70.

  5. Review: Induction • Suppose • S(k) is true for fixed constant k • Often k = 0 • S(n)  S(n+1) for all n >= k • Then S(n) is true for all n >= k David Luebke 51/3/2020

  6. Proof By Induction • Claim: S(n) is true for all n >= k • Basis: • Show formula is true when n = k • Inductive hypothesis: • Assume formula is true for an arbitrary n • Step: • Show that formula is then true for n+1 David Luebke 61/3/2020

  7. Induction Example: Gaussian Closed Form • Prove 1 + 2 + 3 + … + n = n(n+1) / 2 • Basis: • If n = 0, then 0 = 0(0+1) / 2 • Inductive hypothesis: • Assume 1 + 2 + 3 + … + n = n(n+1) / 2 • Step (show true for n+1): 1 + 2 + … + n + n+1 = (1 + 2 + …+ n) + (n+1) = n(n+1)/2 + n+1 = [n(n+1) + 2(n+1)]/2 = (n+1)(n+2)/2 = (n+1)(n+1 + 1) / 2 David Luebke 71/3/2020

  8. Induction Example:Geometric Closed Form • Prove a0 + a1 + … + an = (an+1 - 1)/(a - 1) for all a  1 • Basis: show that a0 = (a0+1 - 1)/(a - 1) a0 = 1 = (a1 - 1)/(a - 1) • Inductive hypothesis: • Assume a0 + a1 + … + an = (an+1 - 1)/(a - 1) • Step (show true for n+1): a0 + a1 + … + an+1 = a0 + a1 + … + an + an+1 = (an+1 - 1)/(a - 1) + an+1 = (an+1+1 - 1)/(a - 1) David Luebke 81/3/2020

  9. Induction • We’ve been using weak induction • Strong induction also holds • Basis: show S(0) • Hypothesis: assume S(k) holds for arbitrary k <= n • Step: Show S(n+1) follows • Another variation: • Basis: show S(0), S(1) • Hypothesis: assume S(n) and S(n+1) are true • Step: show S(n+2) follows David Luebke 91/3/2020

  10. Asymptotic Performance • In this course, we care most about asymptotic performance • How does the algorithm behave as the problem size gets very large? • Running time • Memory/storage requirements David Luebke 101/3/2020

  11. Asymptotic Notation • By now you should have an intuitive feel for asymptotic (big-O) notation: • What does O(n) running time mean? O(n2)?O(n lg n)? • How does asymptotic running time relate to asymptotic memory usage? • Our first task is to define this notation more formally and completely David Luebke 111/3/2020

  12. Input Size • Time and space complexity • This is generally a function of the input size • E.g., sorting, multiplication • How we characterize input size depends: • Sorting: number of input items • Multiplication: total number of bits • Graph algorithms: number of nodes & edges • Etc David Luebke 121/3/2020

  13. Analysis • Worst case • Provides an upper bound on running time • An absolute guarantee • Average case • Provides the expected running time • Very useful, but treat with care: what is “average”? • Random (equally likely) inputs • Real-life inputs David Luebke 131/3/2020

  14. An Example: Insertion Sort InsertionSort(A, n) {for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key} } David Luebke 141/3/2020 David Luebke 141/3/2020

  15. An Example: Insertion Sort i =  j =  key = A[j] =  A[j+1] =  30 10 40 20 1 2 3 4 InsertionSort(A, n) {for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key} } David Luebke 151/3/2020 David Luebke 151/3/2020

  16. An Example: Insertion Sort i = 2 j = 1 key = 10A[j] = 30 A[j+1] = 10 30 10 40 20 1 2 3 4 InsertionSort(A, n) {for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key} } David Luebke 161/3/2020 David Luebke 161/3/2020

  17. An Example: Insertion Sort i = 2 j = 1 key = 10A[j] = 30 A[j+1] = 30 30 30 40 20 1 2 3 4 InsertionSort(A, n) {for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key} } David Luebke 171/3/2020 David Luebke 171/3/2020

  18. An Example: Insertion Sort i = 2 j = 1 key = 10A[j] = 30 A[j+1] = 30 30 30 40 20 1 2 3 4 InsertionSort(A, n) {for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key} } David Luebke 181/3/2020 David Luebke 181/3/2020

  19. An Example: Insertion Sort i = 2 j = 0 key = 10A[j] =  A[j+1] = 30 30 30 40 20 1 2 3 4 InsertionSort(A, n) {for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key} } David Luebke 191/3/2020 David Luebke 191/3/2020

  20. An Example: Insertion Sort i = 2 j = 0 key = 10A[j] =  A[j+1] = 30 30 30 40 20 1 2 3 4 InsertionSort(A, n) {for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key} } David Luebke 201/3/2020 David Luebke 201/3/2020

  21. An Example: Insertion Sort i = 2 j = 0 key = 10A[j] =  A[j+1] = 10 10 30 40 20 1 2 3 4 InsertionSort(A, n) {for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key} } David Luebke 211/3/2020 David Luebke 211/3/2020

  22. An Example: Insertion Sort i = 3 j = 0 key = 10A[j] =  A[j+1] = 10 10 30 40 20 1 2 3 4 InsertionSort(A, n) {for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key} } David Luebke 221/3/2020 David Luebke 221/3/2020

  23. An Example: Insertion Sort i = 3 j = 0 key = 40A[j] =  A[j+1] = 10 10 30 40 20 1 2 3 4 InsertionSort(A, n) {for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key} } David Luebke 231/3/2020 David Luebke 231/3/2020

  24. An Example: Insertion Sort i = 3 j = 0 key = 40A[j] =  A[j+1] = 10 10 30 40 20 1 2 3 4 InsertionSort(A, n) {for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key} } David Luebke 241/3/2020 David Luebke 241/3/2020

  25. An Example: Insertion Sort i = 3 j = 2 key = 40A[j] = 30 A[j+1] = 40 10 30 40 20 1 2 3 4 InsertionSort(A, n) {for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key} } David Luebke 251/3/2020 David Luebke 251/3/2020

  26. An Example: Insertion Sort i = 3 j = 2 key = 40A[j] = 30 A[j+1] = 40 10 30 40 20 1 2 3 4 InsertionSort(A, n) {for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key} } David Luebke 261/3/2020 David Luebke 261/3/2020

  27. An Example: Insertion Sort i = 3 j = 2 key = 40A[j] = 30 A[j+1] = 40 10 30 40 20 1 2 3 4 InsertionSort(A, n) {for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key} } David Luebke 271/3/2020 David Luebke 271/3/2020

  28. An Example: Insertion Sort i = 4 j = 2 key = 40A[j] = 30 A[j+1] = 40 10 30 40 20 1 2 3 4 InsertionSort(A, n) {for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key} } David Luebke 281/3/2020 David Luebke 281/3/2020

  29. An Example: Insertion Sort i = 4 j = 2 key = 20A[j] = 30 A[j+1] = 40 10 30 40 20 1 2 3 4 InsertionSort(A, n) {for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key} } David Luebke 291/3/2020 David Luebke 291/3/2020

  30. An Example: Insertion Sort i = 4 j = 2 key = 20A[j] = 30 A[j+1] = 40 10 30 40 20 1 2 3 4 InsertionSort(A, n) {for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key} } David Luebke 301/3/2020 David Luebke 301/3/2020

  31. An Example: Insertion Sort i = 4 j = 3 key = 20A[j] = 40 A[j+1] = 20 10 30 40 20 1 2 3 4 InsertionSort(A, n) {for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key} } David Luebke 311/3/2020 David Luebke 311/3/2020

  32. An Example: Insertion Sort i = 4 j = 3 key = 20A[j] = 40 A[j+1] = 20 10 30 40 20 1 2 3 4 InsertionSort(A, n) {for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key} } David Luebke 321/3/2020 David Luebke 321/3/2020

  33. An Example: Insertion Sort i = 4 j = 3 key = 20A[j] = 40 A[j+1] = 40 10 30 40 40 1 2 3 4 InsertionSort(A, n) {for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key} } David Luebke 331/3/2020 David Luebke 331/3/2020

  34. An Example: Insertion Sort i = 4 j = 3 key = 20A[j] = 40 A[j+1] = 40 10 30 40 40 1 2 3 4 InsertionSort(A, n) {for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key} } David Luebke 341/3/2020 David Luebke 341/3/2020

  35. An Example: Insertion Sort i = 4 j = 2 key = 20A[j] = 30 A[j+1] = 40 10 30 40 40 1 2 3 4 InsertionSort(A, n) {for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key} } David Luebke 351/3/2020 David Luebke 351/3/2020

  36. An Example: Insertion Sort i = 4 j = 2 key = 20A[j] = 30 A[j+1] = 40 10 30 40 40 1 2 3 4 InsertionSort(A, n) {for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key} } David Luebke 361/3/2020 David Luebke 361/3/2020

  37. An Example: Insertion Sort i = 4 j = 2 key = 20A[j] = 30 A[j+1] = 30 10 30 30 40 1 2 3 4 InsertionSort(A, n) {for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key} } David Luebke 371/3/2020 David Luebke 371/3/2020

  38. An Example: Insertion Sort i = 4 j = 2 key = 20A[j] = 30 A[j+1] = 30 10 30 30 40 1 2 3 4 InsertionSort(A, n) {for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key} } David Luebke 381/3/2020 David Luebke 381/3/2020

  39. An Example: Insertion Sort i = 4 j = 1 key = 20A[j] = 10 A[j+1] = 30 10 30 30 40 1 2 3 4 InsertionSort(A, n) {for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key} } David Luebke 391/3/2020 David Luebke 391/3/2020

  40. An Example: Insertion Sort i = 4 j = 1 key = 20A[j] = 10 A[j+1] = 30 10 30 30 40 1 2 3 4 InsertionSort(A, n) {for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key} } David Luebke 401/3/2020 David Luebke 401/3/2020

  41. An Example: Insertion Sort i = 4 j = 1 key = 20A[j] = 10 A[j+1] = 20 10 20 30 40 1 2 3 4 InsertionSort(A, n) {for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key} } David Luebke 411/3/2020 David Luebke 411/3/2020

  42. An Example: Insertion Sort i = 4 j = 1 key = 20A[j] = 10 A[j+1] = 20 10 20 30 40 1 2 3 4 InsertionSort(A, n) {for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key} } Done! David Luebke 421/3/2020 David Luebke 421/3/2020

  43. Animating Insertion Sort • Check out the Animator, a java applet at:http://www.sorting-algorithms.com/insertion-sort • Try it out with random, ascending, and descending inputs David Luebke 431/3/2020 David Luebke 431/3/2020

  44. Insertion Sort InsertionSort(A, n) {for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key} } How many times will this loop execute? David Luebke 441/3/2020 David Luebke 441/3/2020

  45. Analysis • Simplifications • Ignore actual and abstract statement costs • Order of growth is the interesting measure: • Highest-order term is what counts • Remember, we are doing asymptotic analysis • As the input size grows larger it is the high order term that dominates David Luebke 451/3/2020 David Luebke 451/3/2020

  46. Upper Bound Notation • We say InsertionSort’s run time is O(n2) • Properly we should say run time is in O(n2) • Read O as “Big-O” (you’ll also hear it as “order”) • In general a function • f(n) is O(g(n)) if there exist positive constants c and n0such that f(n)  c  g(n) for all n  n0 • Formally • O(g(n)) = { f(n):  positive constants c and n0such that f(n)  c  g(n)  n  n0 David Luebke 461/3/2020 David Luebke 461/3/2020

  47. Insertion Sort Is O(n2) • Proof • Suppose runtime is an2 + bn + c • If any of a, b, and c are less than 0 replace the constant with its absolute value • an2 + bn + c  (a + b + c)n2 + (a + b + c)n + (a + b + c) •  3(a + b + c)n2 for n  1 • Let c’ = 3(a + b + c) and let n0 = 1 • Question • Is InsertionSort O(n3)? • Is InsertionSort O(n)? David Luebke 471/3/2020 David Luebke 471/3/2020

  48. Lower Bound Notation • We say InsertionSort’s run time is (n) • In general a function • f(n) is (g(n)) if  positive constants c and n0such that 0  cg(n)  f(n)  n  n0 • Proof: • Suppose run time is an + b • Assume a and b are positive (what if b is negative?) • an  an + b David Luebke 481/3/2020 David Luebke 481/3/2020

  49. Asymptotic Tight Bound • A function f(n) is (g(n)) if  positive constants c1, c2, and n0 such that c1 g(n)  f(n)  c2 g(n)  n  n0 • Theorem • f(n) is (g(n)) iff f(n) is both O(g(n)) and (g(n)) David Luebke 491/3/2020 David Luebke 491/3/2020

  50. Practical Complexity David Luebke 501/3/2020 David Luebke 501/3/2020

More Related