Performance Analysis and Measurement

1. Performance Analysis and Measurement Chien Chin Chen Department of Information Management National Taiwan University Reference: Ellis Horowitz, Sartaj Sahni, and Dinesh P. Mehta, “Fundamentals of Data Structures,” 2nd Edition Section 1.7 Performance Analysis and Measurement

2. Performance Analysis and Measurement (1/3) • In chapter 1, we inspect a lot of facts about good solutions. • The total cost it incurs over all phases of its life cycle is minimal. • The cost of a solution includes: • Difficulties encountered by users. • The effort of changes (bug fixes or spec modification). • Computer resources that the program consumes. • Time & space. • …

3. Performance Analysis and Measurement (2/3) • Space complexity: • The amount of memory needed by a program to run to completion. • Time complexity: • The amount of computing time needed by a program to run to completion. • Performance evaluation (in terms of space or time) of a program can be loosely divided into two major phases: • A priori estimates. • Performance Analysis: to obtain estimates of time and space of a program execution, independent of machine.

4. Performance Analysis and Measurement (3/3) • A posteriori testing. • Performance Measurement: to obtain the actual space and time requirements of a program. • Machine dependent. • For example, the running times of a program on different machine will be quite different. (even on the same machine).

5. Performance Analysis —Space Complexity (1/5) • The space needed by a program is seen to be the sum of the following components: • A fixed space requirement: • The required memory that is independent of the instance characteristics of the program. • e.g., instruction (code) space, simple variables, fixed-size structured variables and constants. • A variable space requirement: • Memory that is dependent upon the instance characteristics of the program. • e.g., the recursion stack space. Instance characteristics of a program: number and size of the program’s inputs and outputs, a specific problem. e.g., factorial(3)

6. Performance Analysis —Space Complexity (2/5) • The space requirement S(P) of a program P can be written as: • S(P) = c + SP(instance characteristics). • We usually concentrate on estimating Sp(instance characteristics). • For a given problem, we shall need to first determine which instance characteristics to use to measure the space requirements. constant (or fixed memory requirement) variable memory requirement, depends on particular instance characteristics.

7. Performance Analysis —Space Complexity (3/5) • The instances are characterized by • SAbc = float Abc(float a, float b, float c) { return a + b + b * c; } a, b, and c. 0, independent of the instance characteristics.

8. Performance Analysis —Space Complexity (4/5) • The instances are characterized by n. • The recursion stack space includes space for the formal parameters, the local variables, and the return address. • Almost identical to the box trace!! • Suppose each recursion stack space requires 4 bytes, sp would be 4(n+1) bytes. int fact(int n) { if (n == 0) return 1; else return n *fact(n-1); }

9. Performance Analysis —Space Complexity (5/5) • More complex than time complexity. • It strongly depends on data structures design. • It affects the computing time complexity. • In general-purpose computing platform: • Memory, cache, secondary storage. • In embedded system: • Limited memory (SDRAM, FLASH (ROM)). • No secondary storage.

10. Performance Analysis —Time Complexity (1/6) • The time, T(P) taken by a program P is the sum of the compile time and the execution time. • We assume that a compiled program will be run several times without recompilation. • Consequently, we shall concern ourselves with just the execution time of a program. • Denoted by tp(instance characteristics). • we shall need to first determine which instance characteristics to use to measure the time complexity. • If we knew the characteristics of the compiler to be used, we could obtain an expression for tp(n) of the form: Time need for an addition Number of additions

11. Performance Analysis —Time Complexity (2/6) • Obtaining such a formula is an impossible task!! • The time needed for an addition often depends on the actual numbers being added. • In a multiuser system, the execution time will depend on many other factors. • E.g., the number of other running programs. • We can go one step further and count only thenumber ofprogram steps.

12. Performance Analysis —Time Complexity (3/6) • A program step: • Is defined as a syntactically or semantically meaningful segment of a program. • Its execution time is independent of the instance characteristics. • Example: a = b + c;

13. Performance Analysis —Time Complexity (4/6) • Example 1: • count: initial 0, used to calculate the step count of a program. instance characteristics float Sum(float *a, const int n) { float s = 0; count++; // for assignment. for(int i=0; i<n; i++) { count++; // for for s += a[i]; count++; // for assignment. } count++; // for last time of for count++; // for return return s; } 1 2n 1 1 = 2n+3

14. Performance Analysis —Time Complexity (5/6) • Example 2: instance characteristics n > 1 n=0 or 1 void Fibonacci(int n) { if (n<=1) count << n << endl; // F0 = 0 and F1 = 1 else { // fn = fn_1 + fn_2 int fn; int fn_1 = 1; int fn_2 = 0; for (int i=2; i<=n; i++) { fn = fn_1 + fn_2; fn_2 = fn_1; fn_1 = fn; } // end for cout << fn << endl; } // end if } 1 1 1 2 n n-1 n-1 n-1 1 =4n+1 =2

15. Performance Analysis —Time Complexity (6/6) • The number of steps of a program is itself a function of the instance characteristics. • E.g., tSum(n) = 2n+3. • For many programs, the time complexity is not dependent solely on the number (or magnitude) of inputs or outputs. • Consider the function binarySearch(const int anArray[], int first, int last, int value) of Chapter 2: • The instance characteristics can be the size of anArray. • However, for the same array size, the step count varies with the position of value in anArray. • This case, we consider three kinds of step counts: • Best-case, worst-case, and average-case.

16. Performance Analysis —Asymptotic Notation (1/17) • With the help of step counts: • We can compare the time complexities of two programs. • Can predict the growth in execution times as the instance characteristics change. • Linear, polynomial, or exponential. • Determining the exact step count of a program is a very difficult task. • The notion of a step is itself inexact. • E.g., x = y and x = y + (x/y) + (x*y*z) all count as one step!! • tp(n)=100n+10 75n≦tp(n)≦120n.

17. Performance Analysis —Asymptotic Notation (2/17) • “bounds”– useful and adequate in many situations. • E.g., c1n2 ≤ tp(n) ≤ c2n2. • For two programs p and q of the same problem. • tp(n) ≦ c1n2. • tq(n) ≦ c2n. where c1 and c2 are all positive. • Then and

18. Performance Analysis —Asymptotic Notation (3/17) • Definition [Big “oh”]: • 3n+2 = O(n) as 3n+2 ≤ 4n for all n≧ 2. • 1000n2+100n-6 = O(n2) as 1000n2+100n-6 ≤ 1001n2 for n≧ 100. • 6*2n+n2 = O(2n) as 6*2n+n2 ≤ 7*2n for n≧ 4. read the symbol ‘=’ as ‘is’.

19. Performance Analysis —Asymptotic Notation (4/17) • Relation: • The statement f(n) = O(g(n)) states that g(n) is an upper bound on the value of f(n) for all n, n≧ n0. • But ... 4n=O(n) and 4n = O(n2). • To be informative, g(n) should be as small a function of nas possible for which the statement is true. • E.g., 4n=O(n), and never say 4n=O(n2).

20. Performance Analysis —Asymptotic Notation (5/17) • Theorem 1: If f(n) = amnm + … + a1n + a0, then f(n) = O(nm). • Proof: So f(n)=O(nm).

21. Performance Analysis —Asymptotic Notation (6/17) • Definition [Omega]: • 3n+2 = Ω(n) as 3n+2 ≧ 3n for all n≧ 1. • 10n2+4n+2 = Ω(n2) as 10n2+4n+2 ≧n2 for n≧ 1. • 6*2n+n2 = Ω(2n) as 6*2n+n2≧ 2n for n≧ 1. • The function g(n) is a lower bound on f(n). • To be informative, g(n) should be as large a function of n as possible for which the statement is true.

22. Performance Analysis —Asymptotic Notation (7/17) • Definition [Theta]: • 3n+2 = θ(n) as 3n+2≧3n for all n≧2 and 3n+2≦4n for all n≧2 . • The theta notation is more precise than both the “big oh” and omega notations. • Provides both an upper and lower bound on f(n).

23. Performance Analysis —Asymptotic Notation (8/17) • Theorem 2: • Theorem 3:

24. Performance Analysis —Asymptotic Notation (9/17) • Example 3: Matrix addition. • a, b, and c are matrix of mxn. • c = a + b, that is, cij = aij + bij. instance characteristics void add(int **a, int **b, int **c, int m, int n) { for (int i=0; i<m; i++) for( int j=0; j<n; j++) c[i][j] = a[i][j] + b[i][j]; } θ(m) θ(mn) θ(mn) θ(mn)

25. Performance Analysis —Asymptotic Notation (10/17) • Example 4: Magic square. • An nxn matrix of the integers 1 to n2. • The sum of every row, column, and diagonal is the same. 65 65

26. Performance Analysis —Asymptotic Notation (11/17) instance characteristics void Magic (const int n) { // n must be odd,Coxeter’s rule then can be applied. const int MaxSize = 51; // max square size int square[MaxSize][MaxSize], k, l; if( (n>MaxSize) || (n<1)) { cout << “Error!...n out of range” << endl; exit(1); } else if( !(n%2)) { cout << “Error!...n is even” << endl; exit(1); } θ(1)

27. Performance Analysis —Asymptotic Notation (12/17) for(int i=0; i<n; i++) // initial square to 0 for(int j=0; j<n; j++) square[i][j] = 0; // start creating the magic square // middle of first row square[0][(n-1)/2] = 1; // i and j are current position int key=2; i=0; int j=(n-1)/2; while (key <= n*n) { // move up and left if(i-1 < 0) k=n-1; else k=i-1; if(j-1 < 0) l=n-1; else l=j-1; if(square[k][l]) // square[k][j] occupied, move down i = (i+1)%n; else { // square[k][l] is unoccupied i = k; j = l; } square[i][j] = key; key++; } θ(n2) θ(1) θ(1) θ(n2)

28. Performance Analysis —Asymptotic Notation (13/17) // output the magic square cout << “magic square of size “ << n << endl; for(i=0; i<n; i++) { for(j=0; j<n; j++) cout << “[” << square[i][j] << “]”; count << endl; } } θ(1) θ(n2) So Magic is θ(n2).

29. Performance Analysis —Asymptotic Notation (14/17) instance characteristics n = last – first + 1, array size. • Example 5: Binary search (worst case). int binarySearch(int anArray[], int first, int last, int value) { while (first <= last) { int middle = (left + right)/2; if(value < anArray[middle]) right = middle-1; else if(value > anArray[middle]) left = middle+1; else return middle; } return -1; } Each iteration of the for loop takes θ(1)time. The loop is iterated log2n+1 or θ(logn) times in the worst-case. The worst-case complexity of binarySearch is θ(logn). log10n = log2n / log210 = c log2n

30. Performance Analysis —Asymptotic Notation (15/17) • Example 6: Permutation generation. • Given a set of n ≧ 1 elements, print all possible permutations of this set. instance characteristics k, m. void Perm(char a[], int k, int m) { // generate all the permutations of a[k] ... a[m]. if(k == m) { for(int i=0; i<=m; i++) cout << a[i] << “ ”; cout << endl; } else // a[k:m] has more than one permutation. // Generate these recursively. for(int i=k; i<=m; i++) { swap(a[k],a[i]); Perm(a, k+1, m); swap(a[k],a[i]); } }

31. Performance Analysis —Asymptotic Notation (16/17) • Assume that a is of size n, n≧1. • So, initially, k=0 and m=n-1, or tperm(0, n-1); • When k=n–1, we see that the time taken is θ(n). • When k<n–1, the second loop is entered((n–1)–k+1)=n–k times. • Each iteration of this loop takes θ(tperm(k+1,n–1)). • So tperm(k,n–1) = θ((n–k)*tperm(k+1,n–1)).

32. Performance Analysis —Asymptotic Notation (17/17) • Assume that a is of size n, n≧1. a[0] to a[n-1] tPerm(0, n-1) = θ(n * tPerm(1,n-1)) = θ(n!*n) tPerm(1, n-1) = θ((n-1) * tPerm(2,n-1)) = θ((n-1)!*n) … tPerm(n-3, n-1) = θ(3 * tPerm(n-2,n-1)) = θ(3*2*n) tPerm(n-2, n-1) = θ(2 * tPerm(n-1,n-1)) = θ(2*n) tPerm(n-1, n-1) = θ(n)

33. Performance Analysis —Practical Complexities (1/2) • The time complexity of a program can be used to determine how the time requirements vary as the instance characteristics change. • To get a feel for how the various functions grow with n. • O(1): means a constant computing time. • O(n): linear. • O(n2): quadratic. • O(n3): cubic. • O(2n): exponential. awful!! polynomial

34. Performance Analysis —Practical Complexities (2/2) • A computer performing 1 billion steps per second.

35. Performance Measurement (1/2) • To measure the computing time and memory usage of a program by running it on a computer. • Compared to “performance analysis” in which a program’s time and space complexity are assessed (i.e. estimated)). • In C or C++: • CLOCK() function. • The function returns an approximation of processor time used by the program. (man clock) • The number of clock ticks of elapsed processor time. • To get the number of seconds used, divide it by CLOCK_PER_SEC (a built-in constant tick/sec).

36. Performance Measurement (2/2) #include <iostream.h> #include <time.h> void main() { double sum=1.0; clock_t start, finish; double duration; start=clock(); test(); //在這裡可以放置你要測量的函式 finish=clock(); duration=double(finish-start)/CLOCKS_PER_SEC; cout<<"Total time is "<<duration<<"/n"; }

37. Homework 3 • Chapter 9, exercises 2 and 6. • Show that the following equalities are correct: • 5n2-6n= θ(n2) • n!=O(nn) i.e., find c, c1, c2, and n0 that satisfy the definitions of omega and theta. • (due date: 10/22)