1 / 97

Chapter 1

Chapter 1. Fundamentals of the Analysis of Algorithm Efficiency. Introduction – What is an Algorithm? An algorithm is a sequence of unambiguous instructions for solving a problem, i.e., for obtaining a required output for any legitimate input in a finite amount of time.

grace
Download Presentation

Chapter 1

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Chapter 1 Fundamentals of the Analysis of Algorithm Efficiency

  2. Introduction – What is an Algorithm? An algorithm is a sequence of unambiguous instructions for solving a problem, i.e., for obtaining a required output for anylegitimate input in a finiteamount of time.

  3. Algorithms: Efficiency, Analysis and Order • Developing efficient algorithms, • Regardless of how fast computers become or how cheap memory gets, Algorithms’ efficiency will always remain an important consideration. • For example, • An approach to finding a name in the phone book: • a modified binary search is faster than a sequential search. • How do you compare algorithms for the two approaches to show how much faster the binary search is. • Generating the Fibonacci sequence by: • a recursive or iterative algorithms, based on its definition. • How do we compare algorithms for the two approaches to show how much faster the iterative way is?

  4. 2. Complexity analysis of algorithms • Analyze the algorithm for determining how efficiently an algorithm solves a problem. • Analyze the efficiency of an algorithm in terms of time and space. • Both time and space efficiencies are measured as functions of the algorithm’s input size. • Time efficiency is measured by counting the number of times the algorithm’s basic operation is executed. • Time complexity: worst-case, average case, and best-case efficiencies. • Space efficiency is measured by counting the number of extra memory units consumed by the algorithm. • space (memory) complexity).

  5. 3. And finally, orders • Compute the complexity of an algorithm in terms of input size n. • This allows us to set the order of algorithms and classify the time-efficient algorithms in terms of • constant algorithms Ω(1), Ο(1), Θ(1), • linear-time algorithms Ω(n), Ο(n), Θ(n), • quadratic-time algorithms Ω(n2), Ο(n2), Θ(n2), • exponential-time algorithms Ω(2n), Ο(2n), Θ(2n), , etc. • This allows us to determine the order of growth of the algorithm’s running time (extra memory units consumed) as its input size goes toward infinity.

  6. 3. And finally, orders • This allows us to determine the order of growth of the algorithm’s running time (extra memory units consumed) as its input size goes toward infinity. c

  7. Important Problem Types to be considered • The following problems will be used to illustrate different algorithms design techniques and methods of algorithm analysis. • Sorting, • Searching • String processing • String matching • Graph problems • Graph traversal algorithm • Shortest-path algorithm • Topological sorting • Traveling salesman problem • Graph-color problem • Combinatorial problems • Traveling salesman problem • Graph-color problem • Geometric problems • Closet-pair problem • Convex-hull problem • Numerical problems

  8. Traveling salesman problem • Definition: A complete graph is a graph with vertices and an edge between every two vertices. • Definition: A weighted graph is a graph in which each edge is assigned a weight (representing the time, distance, or cost of traversing that edge). • Definition: A Hamilton circuit is a circuit that uses every vertex of a graph once. Or, • Definition: A Hamiltonian cycle is a cycle that includes every vertex exactly once (NP-complete problem) • The TRAVELING SALESMAN PROBLEM (TSP): Find a Hamiltonian cycle of minimum length in a given complete weighted graph with weights which could be represented by distance from a node to another node.

  9. The travelling salesman problem (TSP) asks the following question: "Given a list of cities and the distances between each pair of cities, what is the shortest possible route that visits each city exactly once and returns to the origin city?" It is an NP-hard problem in combinatorial optimization,   important in operations research and theoretical computer science.   *NP stands for "nondeterministic polynomial time. P NP. The P class problem: on input size n, their worst-case time is O(nk), for some constant k. An NP-complete problem is an NP problem that is at least as "tough“ (“hard” ) as any other problem in NP. An NP-hard problem is harder than NPC problems, at least as hard as the hardest problems in NP

  10. A problem? - is a question to which we seek an answer. • To produce a computer program that can solveallinstances of a problem, we must specify a step-by-step procedure (called an algorithm) for producing the solution for each instance. • We say that the algorithm solves the problem.

  11. Example: Here we list a list of problems to be solved. • Sort a list A of n numbers in nondecreasing order. • Determine whether the number x is in the list A of n numbers • Add all the numbers in an array A of n numbers. • Determine the product of two n x n matrices. • Determine the nth term in the Fibonacci sequence. Q: What are allinstances of a problem?

  12. Example: Here we list a list of problems to be solved. • Sort a list A of n numbers in nondecreasing order. The answer is the number in sorted sequence. (Insertion Sort, Exchange Sort, Merge Sort,…). • Determine whether the number x is in the list A of n numbers. The answer is yes if x is in A and no if it is not. (Sequential Search, Binary Search, … ). • Add all the numbers in an array A of n numbers. The answer is the sum of the numbers in S. (Add Array Numbers) • Determine the product of two n x n matrices. The answer is a two-dimensional array of numbers C, which has both its rows and columns indexed from 1 to n, containing the product of A and B. (Matrix Multiplication, Strassen’s Matrix Multiplication, … ). • Determine the nth term in the Fibonacci sequence. The answer is the nth term of the Fibonacci sequence (fib1(n), fib2(n), … )

  13. There are many ways to design SORTing algorithms. Insertion Sort Insertion sort uses an incremental approach: having sorted the subarray A[1 .. j-1], we insert the single element A[j] into its proper place, yielding the sorted subarray A[1 .. j]. Principle: Save the content of A[j] in Key; move the contents of A[j-1] into A[j], A[j - 2] to A[j - 1], … down to A[1] to A[2], if they are larger than the content of the Key (i.e., A[j]). The move would be stop as the content A[j – k] less than or equal to the content of Key. Then insert the content of Key into A[j - (k - 1)]. Key A[1] ≤ A[2] ≤ … ≤ A[j-k] < A[j-k+1] ≤ … ≤ A[j-1] | A[j] … A[n] smaller than or equal to A[j] greater than A[j]

  14. Algorithm Insertion-Sort(A) Input: A sequence of n numbers (a1, a2, ..., an). Output: A permutation (reordering) (a’1, a’2, …, a’n) of the input sequence such that a’1 ≤ a’2 ≤ …, ≤ a’n. for j ← 2 to length[A] do { // A[1], …, A[n], the pointer j goes from 2 to length[A] key ← A[j]; // save the content of A[j] as key. // Insert A[j] into the sorted sequence A[1 .. j-1]. i ← j – 1; // the pointer i goes from j -1 through 1. while (i > 0 and A[i] > key)do {//i from j-1 through 1 A[i+1] ← A[i]; // bring A[i] to the right, if A[i] > key i ← i – 1; }//end while-loop. A[i+1] ← key; } // if the current A[i] is less than key // then insert A[j] into A[i+1] // end for n2

  15. Example: • 1 2 4 5 6 | 3 j = 6, key ← A[6] = 3 • …. A[3] = key = 3 • 1 2 3 4 5 6 |

  16. Analysis of any Algorithm: • Whenever we have an algorithm, there are three questions we always ask about it: • Is it correct? • How much time does it take, as a function of n? • And can we do better? Will the algorithm stop? – Halting Problem. Given input and output specifications, will the algorithm function correctly with respect to input and output specifications? T(n) Worse case, best case and average case. Upper bound and lower bound? 3. In term of time efficiency.

  17. Analysis of insertion sort • For its correctness, we ask • whether the algorithm will halt for the given input n items in the given array? • Whether the output data meets the output specifications {Q}with respect to the input data which meets the input specification {P}. • How much time does the algorithm Insertion_Sort(A) take for arranging n number of items in an array in an nondecreasing order. • Finally, we would like to design an algorithm that could be run efficiently, better than existing one.

  18. Axiomatic Semantics: The Inference Rule • Axioms or inference rules are defined for each statement type in the language using logic expressions, called assertions. • An inference rule is used to infer the truth of one assertion on the basis of the values of other assertions • Let Si be an assertion. The general form of an inference rule is • This states that if S1, S2, S3, …, and Sn are true, then we can infer that S is also true {P} S; {Q} S1, S2, S3, …, Sn S Antecedent { P } Consequent { Q }

  19. Axiomatic Semantics: The Inference Rule AII: { A[1..i], key = A[j] 2 ≤ j ≤ n, 1 ≤ i| {A[1] ≤ A[2] ≤ … ≤ A[i] and A[i + 1]} implies {A[1.. i +1] | A[1] ≤ A[2] ≤ … ≤A[i] ≤ A[i + 1], i > 0 && A[i] > A[j]}} A[i+1] := A[i]; //replace i by i + 1 AII: { A[1..i-1], key = A[j] 2 ≤ j ≤ n, 1 ≤ i-1 | {A[1] ≤ A[2] ≤ … ≤ A[i-1] and A[i]} implies {A[1.. i] | A[1] ≤ A[2] ≤ … ≤A[i-1] ≤ A[i], i-1 > 0 && A[i-1] > A[j]}} i = i -1; //replace i by i - 1 AII: { A[1..i], key = A[j] 2 ≤ j ≤ n, 1 ≤ i| {A[1] ≤ A[2] ≤ … ≤ A[i] and A[i+1]} implies {A[1.. i+1] | A[1] ≤ A[2] ≤ … ≤A[i] ≤ A[i+1], i > 0 && A[i] > A[j]}} AIII:{ { A[1..i +1] | A[1] ≤ A[2] ≤ … ≤ A[i]} and A[i+1]} implies {A[1.. i+1] | A[1] ≤ A[2] ≤ … ≤ A[i+1], (i < 0) || (A[i] ≤ key) }

  20. Axiomatic Semantics: The Inference Rule • An inference rule for logical pretest loops where I is the loop invariant (the inductive hypothesis) • The axiomatic description of a while loop is written as {P} while B do S end {Q} • The complete axiomatic description of a while loop construct requires all of the following to be true. • P => I • {I and B} S {I} • {I and (not B)} => Q • The loop terminates.

  21. Loop invariants and the correctness of insertion sort Elements A[1 .. j-1] are the elements originally in positions 1 through j – 1, but now in sorted order. We state these properties of A[1 .. j-1] formally as a loop invariant: At the start of each iteration of the for loop, the subarray A[1 .. j-1] consists of the elements originally in A[1 .. j-1] but in sorted order. This can be rewritten as: The original A[1..j-1] yields { A[1..j-1] in sorted order A[1] ≤ A[2] ≤ … ≤ A[j-1] } where 2 ≤ j ≤ n + 1.

  22. Algorithm INSERTION-SORT(A) //Input: A sequence of n numbers (a1, a2, ..., an). //Output: A permutation (reordering) (a’1, a’2, …, a’n) of the // input sequence such that a’1 ≤ a’2 ≤ …, ≤ a’n. P: { The original A[1..j ≤ n] | not(A[1] ≤ A[2] ≤ … ≤ A[j ≤ n]), 1 ≤ j ≤ n) } AI: { A[1..j ≤ n ] | key = A[j] ˄ A[1] ≤ A[2] ≤ … ≤ A[j-1], 2 ≤ j ≤ n}. for j ← 2 to length[A] do { key ← A[j]; //Insert A[j] into the sorted sequence A[1 .. j-1]. i ← j – 1; AII: { A[1..i], key = A[j] 2 ≤ j ≤ n, 1 ≤ i| {A[1] ≤ A[2] ≤ … ≤ A[i] and A[i+1]} implies {A[1.. i+1] | A[1] ≤ A[2] ≤ … ≤A[i] ≤ A[i+1], i > 0 && A[i] > A[j]}} while i > 0 and A[i] > key do { A[i+1] ← A[i]; i ← i – 1; } //end while-loop. AIII:{ { A[1..i +1] | A[1] ≤ A[2] ≤ … ≤ A[i]} and A[i+1]} implies {A[1.. i+1] | A[1] ≤ A[2] ≤ … ≤ A[i+1], (i < 0) || (A[i] ≤ key) } A[i+1] ← key; j = j + 1;} //if the current A[i] is less than key //then insert A[j] into A[i+1] //end for Q: {A[1..n] |A[1] ≤ A[2] ≤ … ≤ A[n], 1 ≤ n < j + 1} Prove that {P} Alg {Q}

  23. Loop Invariants AI: { A[1..j ≤ n ] | key = A[j] ˄ A[1] ≤ A[2] ≤ … ≤ A[j-1], 2 ≤ j ≤ n} j := 2 j <= Length[A] AII: { A[1..i], key = A[j] 2 ≤ j ≤ n, 1 ≤ i| {A[1] ≤ A[2] ≤ … ≤ A[i] and A[i+1]} implies {A[1.. i+1] | A[1] ≤ A[2] ≤ … ≤ A[i] ≤ A[i+1], i > 0 && A[i] > A[j]}} key := A[j] i := j -1 i > 0 and A[i] > Key A[i+1] := A[i] A[i+1] := key i := i - 1 j := j + 1

  24. P: The original A[1..j ≤ n] | not(A[1] ≤ A[2] ≤ … ≤ A[j ≤ n]), 1 ≤ j ≤ n) } AI: { A[1..j ≤ n ] | key = A[j] ˄ A[1] ≤ A[2] ≤ … ≤ A[j-1], 2 ≤ j ≤ n} Loop Invariants j := 2 Q: {A[1..n] |A[1] ≤ A[2] ≤ … ≤ A[n], 1 ≤ n < j + 1} j <= Length[A] AII: { A[1..i], key = A[j] 2 ≤ j ≤ n, 1 ≤ i| {A[1] ≤ A[2] ≤ … ≤ A[i] and A[i+1]} implies {A[1.. i+1] | A[1] ≤ A[2] ≤ … ≤ A[i] ≤ A[i+1], i > 0 && A[i] > A[j]}} key := A[j] AIII:{ { A[1..i +1] | A[1] ≤ A[2] ≤ … ≤ A[i]} and A[i+1]} implies {A[1.. i+1] | A[1] ≤ A[2] ≤ … ≤ A[i+1], ( i < 0) || (A[i] ≤ key) } i := j -1 i > 0 and A[i] > Key A[i+1] := A[i] A[i+1] := key i := i - 1 j := j + 1

  25. To show the algorithm is correct, we must show three things about a loop invariant. Initialization: It is true prior to the first iteration of the loop. [i.e., after j := 2] Maintenance: If it is true before an iteration of the loop, it remains true before the next iteration. Termination: When the loop terminates, the invariant gives us a useful property that helps show that the algorithm is correct.

  26. Algorithm INSERTION-SORT(A) //Input: A sequence of n numbers (a1, a2, ..., an). //Output: A permutation (reordering) (a’1, a’2, …, a’n) of the // input sequence such that a’1 ≤ a’2 ≤ …, ≤ a’n. P: { The original A[1..j ≤ n] | not(A[1] ≤ A[2] ≤ … ≤ A[j ≤ n]), 1 ≤ j ≤ n) } AI: { A[1..j ≤ n ] | key = A[j] ˄ A[1] ≤ A[2] ≤ … ≤ A[j-1], 2 ≤ j ≤ n}. for j ← 2 to length[A] do { key ← A[j]; //Insert A[j] into the sorted sequence A[1 .. j-1]. i ← j – 1; AII: { A[1..i], key = A[j] 2 ≤ j ≤ n, 1 ≤ i| {A[1] ≤ A[2] ≤ … ≤ A[i] and A[i+1]} implies {A[1.. i+1] | A[1] ≤ A[2] ≤ … ≤A[i] ≤ A[i+1], i > 0 && A[i] > A[j]}} while i > 0 and A[i] > key do { A[i+1] ← A[i]; i ← i – 1; } //end while-loop. AIII:{ { A[1..i +1] | A[1] ≤ A[2] ≤ … ≤ A[i]} and A[i+1]} implies {A[1.. i+1] | A[1] ≤ A[2] ≤ … ≤ A[i+1], (i < 0) || (A[i] ≤ key) } A[i+1] ← key; j = j + 1;} //if the current A[i] is less than key //then insert A[j] into A[i+1] //end for Q: {A[1..n] |A[1] ≤ A[2] ≤ … ≤ A[n], 1 ≤ n < j + 1} Prove that {P} Alg {Q}

  27. Algorithm INSERTION-SORT(A) //Input: A sequence of n numbers (a1, a2, ..., an). //Output: A permutation (reordering) (a’1, a’2, …, a’n) of the // input sequence such that a’1 ≤ a’2 ≤ …, ≤ a’n. AI: { A[1..j ≤ n ] | key = A[j] ˄ A[1] ≤ A[2] ≤ … ≤ A[j-1], 2 ≤ j ≤ n}. for j ← 2 to length[A] do { key ← A[j]; //Insert A[j] into the sorted sequence A[1 .. j-1]. i ← j – 1; while i > 0 and A[i] > key do { A[i+1] ← A[i]; i ← i – 1; } //end while-loop. A[i+1] ← key; } //if the current A[i] is less than key //then insert A[j] into A[i+1] //end for

  28. Shown: Initialization: Need to show that the loop invariant holds before the first loop iteration, when j = 2. When j = 2, the subarray A[1, j] consists of the single element A[1], which is in fact the original element in A[1]. This subarray is sorted (trivially) which shows that the loop invariant holds prior to the first iteration of the loop. That is: AI: { A[1..j ≤ n ] | key = A[j] ˄ A[1] ≤ A[2] ≤ … ≤ A[j-1], 2 ≤ j ≤ n}.

  29. Maintenance: Need to show that each iteration maintains the loop invariant. Informally, the body of the outer for loop works by moving A[j-1], A[j-2], A[j-3], and so on by one position to the right until the proper position for A[j] is found, at which point the value of A[j] is inserted {the last statement}. A more formal treatment of the second property would require us to state and show a loop invariant for the inner while loop. At this point, however we prefer not to get bogged down in such formalism. That is, AII: { A[1..i], key = A[j] 2 ≤ j ≤ n, 1 ≤ i| {A[1] ≤ A[2] ≤ … ≤ A[i] and A[i+1]} implies {A[1.. i+1] | A[1] ≤ A[2] ≤ … ≤A[i] ≤ A[i+1], i > 0 && A[i] > A[j]}}

  30. {P: j = 10} j = j -1; {Q: j = 9} To show it is correct : j = 9 implies that j – 1 = 9. That is, j = 10. {P: A[1…i] && j ≤ n} A[i+1] ← A[i]; {R: A[1..i+1] && j ≤ n} j ← j +1; {Q: A[i+1] && j + 1 ≤ n} {P} A[i+1] ← A[i]; {R} {R} j ← j +1; {Q} implies that {P } A[i+1] ← A[i]; j ← j +1; {Q}

  31. Termination: Finally, we examine what happens when the loop terminates. For insertion sort, the outer for loop ends when j exceeds n, i.e., j = n+1. Substituting n+1 for j in the wording of loop invariant, we have that the subarray A[1..n] consists of the elements originally in A[1..n] but in sort order. But the subarray A[1..n] is the entire array! Hence, the entire array is sorted, which means that the algorithm is correct. That is, when j = n+1, then the original P: { The original A[1..j ≤ n] | not(A[1] ≤ A[2] ≤ … ≤ A[j ≤ n]), 1 ≤ j ≤ n) } yields Q: {A[1..n] |A[1] ≤ A[2] ≤ … ≤ A[n], 1 ≤ n < j + 1}. This is The original A[1..n] yields { A[1..n] in sorted order A[1] ≤ A[2] ≤ … ≤ A[n] }

  32. Determine the time efficiency for this algorithm Insert_Sort(A). • Consider the INSERTION_SORT procedure with • the time “cost” of each statement and • the number of times each statement is executed. • For each j = 2, 3, …, n = length[A], let • tj be the number of times the while loop test, “while (i > 0 and A[i] > key)” is executed for that value of j. • When a for or while loop exits in the usual way, • the test is executed one time more than the loop body. • Comments are not executable statements, and so they take no time.

  33. Let us analyze the efficiency of this algorithm in terms of time - by counting every instruction executed in number of times of execution with its estimated cost assumption.

  34. Algorithm Insertion-Sort(A) Input: A sequence of n numbers (a1, a2, ..., an). Output: A permutation (reordering) (a’1, a’2, …, a’n) of the input sequence such that a’1 ≤ a’2 ≤ …, ≤ a’n. Cost(steps/nsec) times for j ← 2 to length[A] do { c1 key ← A[j]; c2 n - 1 / / Insert A[j] into the sorted sequence A[1 .. j-1]. 0 n - 1??0 i ← j – 1; c4 n - 1 while (i > 0 and A[i] > key) do { c5 A[i+1] ← A[i]; c6 i ← i – 1; } //end while-loopc7 A[i+1] ← key; } // end for c8 n - 1

  35. Think - writing each statement in terms of a sequence of statements (steps) in assembler language The running time for a statement is ci *n if the statement takes ci steps (or nanoseconds) to execute and is executed n times. The running time of the algorithm is the sum of running times for each statement executed. The running time of INSERTION-SORT T(n) is T(n) = c1 * n + c2 *(n-1) + c4 *(n-1) + c5 * + c6 * + c7*+ c8 (n-1). Even for inputs of a given size, an algorithm’s running time may depend on which input (the property) of that size is given.

  36. For example, in INSERTION_SORT, • For the best case occurs if the array is already sorted. • For each j = 2, 3, …, n, i has its initial value of j-1; and • we find that A[i] ≤ key in “while (i > 0 and A[i] > key)”. • Thus tj = 1(the while loop test is executed once) for j = 2, 3, …, n, and • the best-caserunning time is • T(n) = c1 * n + c2*(n-1) + c4 *(n-1) + c5*(n-1) + c8 *(n-1) • = (c1 + c2 + c4+ c5 + c8) n - (c2 + c4+ c5 + c8) • = a n + b • which is a linear function of n, Ω(n) where a and b depend on the statement cost ci.

  37. Theworst caseresult if the array is in reverse sorted order – that is, in decreasing order. • We must compare each element A[j] with each element in the entire sorted subarray A[1..j-1]. • Thus, tj = j (the while loop test is executed j number of • times) for j = 2, 3, …, n. • In the worst case, the running time of INSERTION_SORT is • T(n) = c1 *n + c2 *(n-1) + c4 *(n-1) + c5 * + • c6* + c7*+ c8 * (n-1) • …

  38. Note that • = [n(n+1)/2] – 1 and also = n - 2 + 1 • and • ∑nj=2 (j – 1) = [n(n-1)/2] { = [n(n+1)/2] – 1 - = … - (n-1)} • In the worst case, we have the running time of INSERTION_SORT: • …

  39. In the worst case, we have the running time of INSERTION_SORT: • T(n) = c1 *n + c2 *(n-1) + c4 *(n-1) + c5 *+ c6 *+ • c7 * + c8 * (n-1) • = c1 *n + c2 *(n-1) + c4 *(n-1) + c5 * ( [n(n+1)/2] – 1 ) + • c6 *( [n(n-1)/2] ) + c7 *( [n(n-1)/2]) + c8 * (n-1) • = (c5 /2 + c6 /2 + c7 /2) *n2 + • (c1 + c2 + c4 + c5 /2 - c6 /2 - c7 /2 + c8)*n – (c2 + c4+ c5 + c8) • = a*n2 + b*n + c • which is a quadratic function of n, O(n) for constants a, b and c that again depend on the statement costs ci. • …

  40. The worst-case running time of an algorithm is an upper bound on the running time for any input. • Knowing the worst-case running time gives us a guarantee that the algorithm will never take any longer O(n).

  41. ********************************************************************************************************************************************************** Algorithm Insertion-Sort(A) Input: A sequence of n numbers (a1, a2, ..., an). Output: A permutation (reordering) (a’1, a’2, …, a’n) of the input sequence such that a’1 ≤ a’2 ≤ …, ≤ a’n. Cost time for j ← 2 to length[A] do { c1 n key ← A[j]; c2 n-1 / /Insert A[j] into the sorted sequence A[1 .. j-1]. 0 n-1 i ← j – 1; c4 n-1 while (i > 0 and A[i] > key) do { c5 A[i+1] ← A[i]; c6 i ← i – 1; } //end while-loopc7 A[i+1] ← key; } //end for c8 n-1 *****************************************************************************

  42. The average caseis often roughly as bad as the worst case. • Choose randomly n numbers as an input for the insertion sort. • How long does it take to determine where in subarray A[1 .. j-1] to insert A[j]? • On average, half the elements in A[1 ... j-1] are less than A[j], and half the elements are greater. • On average, we check half of the subarray A[1 ... j-1]. So = . • It turns out that the resulting average-case running time is a quadratic function of the input size, just like the worst case running time. • Often we assume that all inputs of a given size are equally likely. In practice, this assumption may be violated, but we can sometimes use a randomized algorithm, which makes random choices, to allow a probabilistic analysis and yield an expected running time. • So far, we have seen an incremental approach is used to design insertion sort.

  43. The Analysis Framework: Analyzing an Algorithm: • Time efficiency is measured by counting the number of times the algorithm’s basic operation is executed. • Space efficiencyis measured by counting the number of extra memory units consumed by the algorithm. • Measuring an Input’s size • Both time and space efficiency are measured as functions of the algorithm’s input size [see Lecture Note 0]. • Units for measuring running time • …

  44. Units for measuring running time • Identifying the basic operation of the algorithm, the most important operation contributing the most to the total running time, and compute the number of times the basic operation is executed. • Let cop be the execution time (such as nanosec/instruction)of an algorithm’s basic operation on a particular computer, and let C(n) be the number of times this operation needs to be executed for this algorithm. We can estimate the running time T(n) of a program implementing this algorithm on that computer by the formula • T(n) ≈ cop * C(n).

  45. ++++++++++++++++++++++++++++ • In general, we describe • the running time of a program as a function of the size of its input, (i.e, T(n) ∞ f(n)) because the time taken by an algorithm grows with the size of the input. • The natural measure of input size is the number of items in the input. For example, the input size for sorting is the number of items n in the array. • Some algorithms such as computing nth Fibonacci tem, n is the input but not the size of the input. A reasonable measure of the size of the input is the number of symbols used to encode n. Using binary representation, the input size will be the number of bits it takes to encode n, which is └ log2 n ┘ + 1. For example,the total number of bits for multiplyingtwo integers n, m. (i.e, bits(n) * bits(m)) • The running time of an algorithm on a particular input is the number of primitive operations or “steps” executed.

  46. ++++++++++++++++++++++++++++ • In general, we describe • … • The running time of an algorithm on a particular input is the number of primitive operations or “steps” executed. 101 110 000 101 101 11110 5*6=30

  47.  To determine how efficiently an algorithm solves a problem, we need to analyze the algorithm. The efficiency of an algorithm in terms of time, (1) we do not determine the actual number of CPU cycles because this depends on the particular computer on which the algorithm is run. (2) We do not want to count every instruction executed, because the number of instructions depends on the programming languages used to implement the algorithm and the way the programmer writes the program. Rather, (3) we want a measure that is independent of the computer, the programming language, the programmer, and all the complex details of the algorithm such as incrementing of loop indices, setting pointers, and so forth. In general, the running time of algorithm increases with the size of input, and the total running time is roughly proportional to how many times some basic operation (such as a comparison instruction) is done. Therefore, we analyze the algorithm’s efficiency by determining the number of times some basic operation is done as a function of the size of input. +++++++++++++++++++++++++++++++++

  48. Order of growth • Concentrates only on the count’s order of growth for large input sizes. • log2 n < n < n log2 n < n2 < n3 < 2n < n! • The framework’s primary interest lies in the order of growth of the algorithm’s running time (extra memory units consumed) as its input size goes to infinity. • Algorithms that require an exponential number (i.e., 2n ) of operations are practical for solving only problems of very small sizes.

  49. Order of growth • Concentrates only on the count’s order of growth for large input sizes. • log2 n < n < n log2 n < n2 < n3 < 2n < n! • The framework’s primary interest lies in the order of growth of the algorithm’s running time (extra memory units consumed) as its input size goes to infinity. • Algorithms that require an exponential number (i.e., 2n ) of operations are practical for solving only problems of very small sizes. Figure 1.0 Growth rates of common complexity functions.

  50. Worst-case, Best-case and Average-case Efficiencies • An algorithm’s efficiency can be measured as a function of a parameter indicating the size of algorithm’s input. (i.e, T(n) ∞ f(n)) • Many algorithms’srunning time depends not only on an input size but also on the specific of a particular input. • The efficiencies of these algorithms may differ significantly for inputs of the same size. • For such algorithms, we need to distinguish between the worst-case, average-case, and best-case efficiencies.

More Related