1 / 37

Chapter (2) - Algorithm Analysis Objectives:

Chapter (2) - Algorithm Analysis Objectives: How to estimating the time required for a program to solve a problem. How to reduce the running time. Observe the result of careless use of recursion. Introducing an efficient algorithm to raise a number to power.

wendymartin
Download Presentation

Chapter (2) - Algorithm Analysis Objectives:

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Chapter (2) - Algorithm Analysis • Objectives: • How to estimating the time required for a program to solve a problem. • How to reduce the running time. • Observe the result of careless use of recursion. • Introducing an efficient algorithm to raise a number to power. • Computing the greatest common divisor (GCD) of two numbers. • Questions: • How do we determine how good a program is without writing it? • What is the quantitative measure of the performance of a program? • An algorithm is a clearly set of instructions to be followed to solve a problem. An algorithm is consider to be poor if: • It takes a long time to solve the problem, and/or • It takes a large part of main memory for the calculations.

  2. Mathematical Background Definition: T(n) = O(f(n)) if there are constantsc and n0such that: Definition: If there are constantsc and n0such that : Definition: if and only if Definition: if

  3. Definition: T(n) = O(f(n)) if there are constantsc and n0such that: There is some point n0past which c.f(n) is always at least as large as T(n), so that if constant factors are ignored, f(n) is at least as big as T(n). Example: 1000n > n2 for small values of n: 1000 > 1, 2000 > 2, 10,000 > 100, … , 999,000 > 998001.

  4. How about for n = 1000? The answer is no. 1,000,000 is not larger than 1,000,000. This will tell us that n2grow much faster than 1000n. Going back to the definition, in our case: T(n) = 1,000n, f(n)=n2, n0=1,000, and c = 1, or we could use n0 = 10 and c = 100. Thus, we will say that 1,000n = O(n2) (order of n-square). This means that we are guaranteed that 1000n does not grow faster than n2. In other words, n2 is the upper bound on 1000n. Since this implies that f(n)=  (T(n)), we will say that T(n) is a lower bound on f(n). This notation O( ), is known as Big-Oh, and o( ) is as little-oh .

  5. Example: n3 grows faster than n2, so we can say: f(n) = n2 and g(n) = 2n2 grow at the same rate. Thus, the above relation is true: When two functions grow at the same rate, then the decision on whether or not to signify this with can depend on the particular context. Example: if g(n) = 2n2, then g(n) = O(n4), g(n)=O(n3), and g(n)=O(n2) are all technically correct. But, the last one is the most accurate and is the best answer. Note: writing, says that g(n) = O(n2) and as good (tight) answer as possible.

  6. Important Rules:

  7. Proof of Rule 1(a):

  8. Proof of Rule 1(b):

  9. Typical Growth Rates:

  10. Important Notes: • Do not include constants or lower terms inside a Big-Oh, i.e. • T(n) = O(2n2) or T(n)=O(n2+n) • are not correct. The correct form for both is: O(n2). • The relative growth rates of two functions f(n) and g(n) can be computed using: • known as the L’Hopital’s rule. • L’Hopital’s rule states that: • Where, f’(n) and g’(n) are the derivatives of f(n) and g(n), respectively.

  11. The limit can have 4 possible values: 1. The limit is 0: This means that f(n) = o(g(n)), i.e. g(n) is the upper-bound to f(n). 2. The limit isa not zero value: This means that Both are growing with the same rates. 3. The limit is : This means that g(n) = o(f(n)), i.e., g(n) is the lower-bound to f(n). 4. The limit oscillates: There is no relation (this will not happen in out case).

  12. Model • A model of computation is needed for the analysis of algorithms in a formal framework. Our model is a normal computer in which instructions are executed sequentially. This model will do the simple instructions such as +, -, /, =, …, but unlike real computers, it takes exactly one time unit to do anything (simple). • What do we analyze? • The most important thing is the running time. Several factors affect the running time: • Effect of compilers and computer used will not be discussed here. • Effect of algorithm used and the input to the algorithm will be discussed here. • Typically, the size of the input is the main consideration. • The worst-case running time, is the quantity required in the analysis.

  13. Maximum Subsequence Sum Problem: • Given (possibly negative) integers a1, a2, a3, …,an, find the maximum value of . (For convenience, the maximum subsequence sum is 0 if all the integers are negative). • Example: For input: -2, 11, -4, 13, -5, -2, the answer is 20: • (11+(-4)+13 = 20, a2through a4). • There are many algorithms to solve this problem. Here we will discuss 4 of these methods. • Running Time Calculations • There are several ways to estimate the running time of a program: • Based on empirical data: write the code, run and measure the time it takes to perform a particular calculation. • Based on the Analysis of the program: this method defines the bottlenecks. It gives an upper bound to the running time, i.e., the Big-Oh running time.

  14. Sample Analysis Count Line4: 1 Line 5: 1+ n+1{<=}+ n {i++} = 2n+2 Line 6: n times [3 {2 “s” and 1 “+”}] = 3n Line 7: 1 Total count = 5n+4 which gives the function O(n).

  15. General Rules: RULE 1: Loops: The running time of a loop is at most the running time of the statements inside the loop (including tests) times the number of iterations. Example: /* 1 */ for(int i = 0; i < N; i++) /* 2 */ c = a*a*a+2; inside we have 2 (*) and 1(+), i.e., running time of 3. Total = 3*N (N is number of iterations).

  16. RULE 2: Nested Loops: Analyze these inside out. The total running time of a statement inside a group of nested loops is the running time of the statement multiplied by the product of the sizes of all the loops. Example: /* 1 */ for(int i = 0; i < N; i++) /* 2 */ for(int j = 0; j < M; j++) /* 3 */ k++; k++ = k = k+1, i.e., running time of 1. Total = 1*M*N = M*N.

  17. RULE 3: Consecutive Statements: just add (the maximum is the one that counts highest among all, recall: Example: /* 1 */ for(int i = 0; i < N; i++) /* 2 */ a[i] = 0; /* 3 */ for(i = 0; i < N; i++) /* 4 */ for(int j = 0; j < N; j++) /* 5 */ A[i] += i + j; inside we have 2 (+) and 1(+), i.e., running time of 2. Total = 2*N (N is number of iterations). 2n+ 2 O(n) nested loop: 2(N*N) O(n2)

  18. RULE 4: If-Then-Else: The running time of an if-then-else statement is never more than the running time of the test plus the larger of the running times of S1 and S2. Example: /* 1 */ if ( cond) /* 2 */ S1; /* 3 */ else /* 4 */ S2; Of course, there are cases for which we over estimate, but we are sure we never under estimate.

  19. Factorial if(N == 0) … else return Fact(5-1)*5; unsigned long Fact(int N) { if(N ==0 ) return 1; else return Fact(N-1)*N; } 2 if(N == 0) … else return Fact(4-1)*4; 2 if(N == 0) … else return Fact(3-1)*3; O(n) if(N == 0) return 1; if(N == 0) … else return Fact(1-1)*1; if(N == 0) … else return Fact(2-1)*2; 2

  20. if(N < 0){ … else if(N == 0) … else return Fact(5-1)*5; unsigned long Fact(int N) { if(N < 0){ cout << “No Factorial of negative \n"; return 0; } else if(N ==0 ) return 1; else return Fact(N-1)*N; } O(n) if(N < 0){ … else if(N == 0) … else return Fact(4-1)*4; 3 if(N < 0){ … else if(N == 0) … else return Fact(1-1)*1; 3 if(N < 0){ … else if(N == 0) … else return Fact(3-1)*3; if(N < 0){ … else if(N == 0) … else return Fact(2-1)*2; if(N < 0){ … else if(N == 0) return 1; 2

  21. Example: (recursion) An inefficient approach to find Fibonacci numbers {1 1 2 3 5 8 13 21 ….} /* 1 */ // Compute Fibonacci numbers /* 2 */ unsigned long int /* 3 */ Fib( const unsigned int N) /* 4 */ { /* 5 */ if( N <= 1) /* 6 */ return 1; /* 7 */ else /* 8 */ return Fib(N-1) + Fib(N-2); /* 9 */ }

  22. If n = 0 or n =1, then from line 5: we will have T(0) = T(1) =1. For n = 2, we have line 5 and line 8. Line 8 consists of two function calls and one addition. Function calls are not easy to analyze and must be analyzed by themselves. Fib(N-1) requires T(N-1) units of time and Fib(N-2) requires T(N-2) units of time. The total time is: T(n) = T(n-1) + T(n-2) + 2 (2 = one for line 5 + one for the addition on line 8)

  23. Using induction prove that: Base: T(0) = F(0) = 1 T(1) = F(1) = 1 Inductive hypothesis: This means that: is also assumed to be true. Using the definition, we can write: F(n) = F(n-1) + F(n-2). From the hypothesis induction: In previous page we showed that: T(n) = T(n-1) + T(n-2) + 2. Thus, or

  24. Example: Replace with old data

  25. We showed that: and in a similar way we can show that : This means that the running time grows exponentially. This is about as bad as possible. By using an array and a for loop, the running time can be reduced substantially. This program is slow because of the amount of redundant work. We redo some of the terms. MAXIM: “DON’T COMPUTE ANYTHING MORE THAN ONCE”

  26. Maximum Subsequence Sum Problem /** ???? maximum contiguous subsequence sum algorithm. */ int maxSubSum1( const vector<int> & a ) { /* 1*/ int maxSum = 0; /* 2*/ for( int i = 0; i < a.size( ); i++ ) /* 3*/ for( int j = i; j < a.size( ); j++ ) { /* 4*/ int thisSum = 0; /* 5*/ for( int k = i; k <= j; k++ ) /* 6*/ thisSum += a[ k ]; /* 7*/ if( thisSum > maxSum ) /* 8*/ maxSum = thisSum; } /* 9*/ return maxSum; } Algorithm 1 Determine complexity: O(n3)

  27. /** ???? maximum contiguous subsequence sum algorithm. */ int maxSubSum2( const vector<int> & a ) { /* 1*/ int maxSum = 0; /* 2*/ for( int i = 0; i < a.size( ); i++ ) { /* 3*/ int thisSum = 0; /* 4*/ for( int j = i; j < a.size( ); j++ ) { /* 5*/ thisSum += a[ j ]; /* 6*/ if( thisSum > maxSum ) /* 7*/ maxSum = thisSum; } } /* 8*/ return maxSum; } Algorithm 2 Determine complexity: O(n3)

  28. /** Recursive maximum contiguous subsequence sum algorithm. * Finds maximum sum in subarray spanning a[left..right]. * Does not attempt to maintain actual best sequence. */ int maxSumRec( const vector<int> & a, int left, int right ) { /* 1*/ if( left == right ) // Base case /* 2*/ if( a[ left ] > 0 ) /* 3*/ return a[ left ]; else /* 4*/ return 0; /* 5*/ int center = ( left + right ) / 2; /* 6*/ int maxLeftSum = maxSumRec( a, left, center ); /* 7*/ int maxRightSum = maxSumRec( a, center + 1, right ); /* 8*/ int maxLeftBorderSum = 0, leftBorderSum = 0; /* 9*/ for( int i = center; i >= left; i-- ) { /*10*/ leftBorderSum += a[ i ]; /*11*/ if( leftBorderSum > maxLeftBorderSum ) /*12*/ maxLeftBorderSum = leftBorderSum; }

  29. /*13*/ int maxRightBorderSum = 0, rightBorderSum = 0; /*14*/ for( int j = center + 1; j <= right; j++ ) { /*15*/ rightBorderSum += a[ j ]; /*16*/ if( rightBorderSum > maxRightBorderSum ) /*17*/ maxRightBorderSum = rightBorderSum; } /*18*/ return max3( maxLeftSum, maxRightSum, /*19*/ maxLeftBorderSum + maxRightBorderSum ); } /** Driver for divide-and-conquer maximum contiguous subsequence sum algorithm.*/ int maxSubSum3( const vector<int> & a ) { return maxSumRec( a, 0, a.size( ) - 1 ); } Algorithm 3 Determine complexity: O(nlogn)

  30. /** * Linear-time maximum contiguous subsequence sum algorithm. */ int maxSubSum4( const vector<int> & a ) { /* 1*/ int maxSum = 0, thisSum = 0; /* 2*/ for( int j = 0; j < a.size( ); j++ ) { /* 3*/ thisSum += a[ j ]; /* 4*/ if( thisSum > maxSum ) /* 5*/ maxSum = thisSum; /* 6*/ else if( thisSum < 0 ) /* 7*/ thisSum = 0; } /* 8*/ return maxSum; } Algorithm 4 Determine complexity:

  31. Binary Search Given an integer X and integers A0, A1, …, AN-1, which are presorted and already in memory, find i such that Ai = X, or return i = -1 if X is not in the input. template <class Comparable> int binarySearch( const vector<Comparable> & a, const Comparable & x ) { /* 1*/ int low = 0, high = a.size( ) - 1; /* 2*/ while( low <= high ) { /* 3*/ int mid = ( low + high ) / 2; /* 4*/ if( a[ mid ] < x ) /* 5*/ low = mid + 1; /* 6*/ else if( a[ mid ] > x ) /* 7*/ high = mid - 1; else /* 8*/ return mid; // Found } /* 9*/ return NOT_FOUND; // NOT_FOUND is defined as -1 }

  32. Euclid’s Algorithm This algorithm is used to find the greatest common divisor. The gcd of two integers is the largest integer that divides both. gcd(50,15)=5. long gcd( long m, long n ) { /* 1*/ while( n != 0 ) { /* 2*/ long rem = m % n; /* 3*/ m = n; /* 4*/ n = rem; } /* 5*/ return m; } Can you prove it is O(logn)

  33. Exponentiation Raising an integer to the poser of another integer. bool isEven( int n ) { return n % 2 == 0; } long pow( long x, int n ) { /* 1*/ if( n == 0 ) /* 2*/ return 1; /* 3*/ if( n == 1 ) /* 4*/ return x; /* 5*/ if( isEven( n ) ) /* 6*/ return pow( x * x, n / 2 ); else /* 7*/ return pow( x * x, n / 2 ) * x; } Can you prove it is O(logn)

  34. Checking the Analysis One way is to write a program for the algorithm and try it for different size of input to see if produces the complexity. It is very difficult to differentiate linear programs from O(NlogN) programs purely based on empirical evidence. A trick that is commonly used is verify that a program is O(f(N) is to compute the values T(N)/f(N) for a range of N.

More Related