Algorithm Analysis

1. Algorithm Analysis Joe Meehean

2. Efficiency of an Algorithm • Algorithm • set of simple instructions to solve a problem • efficiency determined by resource usage • Resource Usage • CPU • Memory • Disk • Network

3. Runtime Complexity • Measures how CPU usage scales with problem size • Measured in “basic operations” • arithmetic operation (+,-,etc…) • assignment (a = 7) • condition test (a == 0) • write a single primitive • Usually care about the worst-case • most operations that could occur for a given problem size

4. Runtime Complexity • Constant Time • complexity group • number of operations independent of problem size • e.g., vector::size() • takes the same amount of operations regardless of the vector’s size • one primitive write (return value)

5. Runtime Complexity • Linear time • # of operations directly proportional to problem size • e.g., sum all numbers in a list • we need to… • create a sum variable • add each of the numbers to the sum

6. Examples • Constant • get any single item from an array (e.g., array[7]) • add together two integers • return a member variable of a class • Linear • find the largest number in a list • find the average number in a list • copy all items from one array to another

7. Runtime Complexity int matches = 0; for(inti = 0; i < array1_size; i++){ for(int j = 0; j < array2_size; j++){ if( array1[i] == array2[j] ){ matches++ }}} • Quadratic time • # of operations increase faster than the problem size • find matching items between 2 arrays • arrays are the same size

8. Example int matches = 0; for(inti = 0; i < array1_size; i++){ for(int j = 0; j < array2_size; j++){ if( array1[i] == array2[j] ){ matches++ }}} • Assume array1 and array2 have N items • 2 operations inside inner loop • N times for inner loop • N times outer loop • 2 * N * N = 2N2

9. Big O Notation • Constant-Time: O(1) • Order 1 • Linear-Time: O(N) • Order N • Quadratic-Time: O(N2) • Ignore constants or low order terms • 4n + 1 => O(n) • 4n + n2 => O(n2)

10. Mathematic Definition Read about it in book Important if you don’t understand Big O at the end of this lecture

11. Determining Complexity • Sequence of statements • sum each statement’s complexity • simple statements => O(1) • if-then-else • what is the worst case? • slowest of all possible paths

12. Determining Complexity • Loops • multiply statement block complexity by # of times loop executes • if loop statement-block => O(1) and loop executes N-timesthen complexity => O(N)

13. Determining Complexity for(inti = 0; i < N; i++){ for(int j = 0; j < M; j++){ //simple sequential statements } } • Independent nested loops • N & M are not dependent on each other • inner loop: O(M) • outer loop: O(N) • O(N*M), if N == M, then O(N2)

14. Determining Complexity for(inti = 0; i < N; i++){ for(int j = i; j < N; j++){ //simple sequential statements } } • Dependent nested loops • # of times inner loop executes dependent on outer loop

15. Determining Complexity for(inti = 0; i < N; i++){ for(int j = i; j < N; j++){ //simple sequential statements } } Inner loop executesi = 0 Ni = 1 N – 1…i = N – 1 1 (N + 1) * N / 2 => O(N2)

16. Determining Complexity • Method call statements • count as complexity of the method call

17. What does all this mean? • 2 algorithms with same Big O may not run at same speed • recall we drop constants and low orders • f(N) = 4N + 5N2 => O(N2) • g(N) = 2 + N2 => O(N2) • g(N) is faster • Both scale at the same rate • if N doubles, the run time for g and f quadruples

18. What does all this mean? • For sufficiently large N, and • h(N) => O(N2) • k(N) => O(N) • k(N) will always be faster • not always true for small N

19. What does all this mean?

20. What does all this mean?

21. What does all this mean? • Sometimes the best or average case is more important • All depends on the intended use of an algorithm • If you will only execute • the best-case code in algo A • and the worst-case in algo B, • you must compare best A to worst B

22. Questions?