1 / 37

Analysis of Algorithms

This article discusses the concept of running time in algorithm analysis and demonstrates how to describe algorithms using pseudo-code. It covers the counting of primitive operations and provides examples of nested loops.

richardlove
Download Presentation

Analysis of Algorithms

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Analysis of Algorithms CS 105 Introduction to Data Structures and Algorithms

  2. What is a good algorithm? • It must be correct • It must be efficient • Implementations of an algorithm must run as fast as possible • How is this measured? • Running time

  3. Running time of a program • Amount of time required for a program to execute for a given input • If measured experimentally, • Dependent on hardware, operating system and software • Answer will be in milliseconds, seconds, minutes, hours, …

  4. Running time of an algorithm • Answer will not be in seconds or minutes • Instead, we count the number of operations carried out • Result will be a formula in terms of some input size, n • Takes into account all possible inputs • Example statement on running time: Running time of algorithm A is T(n) = 2n + 3

  5. Describing algorithms • First, we need to agree on how to describe or specify an algorithm • Note: algorithms are intended for humans (programs are intended for computers) • Descriptions should be high-level explanations that combine natural language and familiar programming structures: Pseudo-code

  6. Pseudo-code example Algorithm arrayMax(A,n): Input: An array A storing n integers. Output: The maximum element in A. currentMax  A[0] for i  1 to n - 1 do if currentMax < A[i] then currentMax  A[i] return currentMax

  7. Pseudo-Code conventions • General Algorithm Structure • Statements • Expressions • Control Structures

  8. Algorithm Structure Algorithm heading Algorithm name(param1, param2,...): Input : input elements Output : output elements statements…

  9. Statements • Assignment: use  instead of = • Method or function call: • object.method(arguments) • method(arguments) • Return statement: • return expression • Control structures

  10. Control Structures • if ... then ... [else ...] • while ... do • repeat ... until ... • for ... do • decision structures • while loops • repeat loops • for loop

  11. Expressions • Standard math symbols + - * / ( ) • Relational operators = > < <= >= != • Boolean operators and or not • Assignment operator  • Array indexing A[i]

  12. General rules on pseudo-code • Should communicate high-level ideas and not implementation details • Syntax is not as tight (e.g., indentation and line-breaks take the place of ; and {} in Java) • Clear and informative

  13. Back to running time • Once an algorithm has been described in pseudo-code, we can now count the primitive operations carried out by the algorithm • Primitive operations: assignment, calls, arithmetic operations, comparisons, array accesses, return statements, …

  14. Back to arrayMax example • Suppose the input array isA = {42, 6, 88, 53}, (n = 4) • How many primitive operations are carried out? • Some notes • The for statement implies assignments, comparisons, subtractions, and increments • the statement currentMax  A[i] will sometimes not be carried out

  15. What to count currentMax  A[0]: assignment, array access for i  1 to n - 1 do: assignment,comparison,subtraction,increment (2 ops) if currentMax < A[i] then: comparison, access currentMax  A[i]: assignment, access return currentMax: return

  16. Counting operations currentMax  A[0] 2 for i  1 to n - 1 do 15 if currentMax < A[i] then 6 currentMax  A[i] 2 return currentMax 1 OPERATIONS CARRIED OUT 26

  17. Handling all possible inputs • In the example just carried out,running time = 26 for the given input • A more useful answer is to provide a formula that applies in general • Formula is in terms of n, the input or array size • Since there may be statements that do not always execute, depending on input values, there are two approaches • Assume worst-case • Provide a range

  18. Measuring running time comparison & subtraction increment currentMax  A[0] 2 for i  1 to n - 1 do 1+2n+2(n-1) if currentMax < A[i] then 2(n-1) currentMax  A[i] 0 .. 2(n-1) return currentMax 1 RUNNING TIME (worst-case) 8n - 2

  19. Nested loops // prints all pairs from the set {1,2,…n} for i  1 to n do for j  1 to n do print i, j How many times does the print statement execute?

  20. Nested loops // prints all pairs from the set {1,2,…n} for i  1 to n do for j  1 to n do print i, j n2 times How about the operations implied in the for statement headers?

  21. for statement revisited for i  1 to n do … 1 assignment i  1 n+1 comparisons i <= n for each value of i in {1, 2, …, n+1} n increments = n assignments + n adds => 3n+2 operations

  22. Nested loops running time for i  1 to n do 3n+2 for j  1 to n do n (3n+2) print i, j n2 TOTAL: 4n2 + 5n + 2

  23. Nested loops example 2 // prints all pairs from the set {1,2,…n} // excluding redundancies (i.e., 1,1 is not a // real pair, 1,2 and 2,1 are the same pair) for i  1 to n-1 do for j i+1 to n do print i, j How many times does the print statement execute?

  24. Nested loops example 2 for i  1 to n-1 do for j i+1 to n do print i, j when i = 1, n-1 prints when i = 2, n-2 prints … when i = n-2, 2 prints when i = n-1, 1 print

  25. Nested loops example 2 Number of executions of the print statement: 1 + 2 + 3 + … + n-1 = n(n-1)/2 = ½n2 – ½n

  26. Nested loops example 2 for i  1 to n-1 do for j i+1 to n do print i, j Take-home exercise: count the operations implied by the for statement headers

  27. Running time function considerations • In determining the running time of an algorithm, an exact formula is possible only when the operations to be counted are explicitly specified • Different answers will result if we choose not to count particular operations • Problem: variations in the answers appear arbitrary

  28. Classifying running time functions • We want to classify running times under particular function categories • Goal: some assessment of the running time of the algorithm that eliminates the arbitrariness of counting operations • Example: arrayMax runs in time proportional to n

  29. Some function categories • The constant function: f(n) = c • The linear function: f(n) = n • The quadratic function: f(n) = n2 • The cubic function: f(n) = n3 • The exponential function: f(n) = bn • The logarithm function: f(n) = log n • The n log n function: f(n) = n log n

  30. Big-Oh notation • Consider the following (running time) functions f1(n) = 2n + 3 f2(n) = 4n2 + 5n + 2 f3(n) = 5n • We place f1 and f3 under the same category (both are proportional to n) and f2 under a separate category (n2 ) • We say: • 2n+3 is O(n), and 5n is O(n), while 4n2 + 5n + 2 is O(n2) • 2n+3 and 5n are linear functions,while 4n2 + 5n + 2 is a quadratic function

  31. Significance of Big-Oh • Example for i  0 to n-1 do A[i]  0 • Running time is 2n, if we count the assignments & array accesses in the loop body (A[i]  0) 4n+1, if we include implicit loop assignments/adds 5n+2, if we include implicit loop comparisons • Regardless, running time is O(n)

  32. Prefix Averages Problem • Given an array X storing n numbers, compute prefix averages or running averages of the sequence of numbers • A[i] = average of X[0],X[1],…,X[i] • Example • Input: X = {1,3,8,2} • Output: A = {1,2,4,3.5} • Algorithm?

  33. Algorithm 1 Algorithm prefixAverages1(A,n): Input: An array X of n numbers. Output: An array A containing prefix averages for i  0 to n - 1 do sum  0 for j  0 to i do sum  sum + X[j] A[i]  sum/(i+1) return array A

  34. Running time of Algorithm 1 • Exercise: count primitive operations carried out by Algorithm 1 • Regardless of what you decide to count, the answer will be of the form:a n2 + b n + c • Observe that the answer is O(n2)

  35. Algorithm 2 Algorithm prefixAverages2(A,n): Input: An array X of n numbers. Output: An array A containing prefix averages sum  0 for i  0 to n - 1 do sum  sum + X[i] A[i]  sum/(i+1) return array A

  36. Running time of Algorithm 2 • Running time of Algorithm 2 will be of the form: a n + b • Observe that the answer is O(n) • Assessment: Algorithm 2 is more efficient than Algorithm 1, particularly for large input (observe the growth rate of n and n2)

  37. Summary • Algorithms are expressed in pseudo-code • The running time of an algorithm is the number of operations it carries out in terms of input size • Answer depends on which operation(s) we decide to count • Big-Oh notation allows us to categorize running times according to function categories • Next: formal definition of Big-Oh notation

More Related