Recursion

1. Recursion

2. Recursion: Definition • Function that solves a problem by relying on itself to compute the correct solution for a smaller version of the problem • Requires terminating condition: Case for which recursion is no longer needed

3. Factorial Recursion • Factorial: n! = n * (n-1)! • Base Case => 0! = 1 • Smaller problem => Solving (n-1)! • Implementation: long factorial(long inputValue) { if (inputValue == 0) return 1; else return inputValue * factorial(inputValue - 1); }

4. Searching • We want to find whether or not an input value is in a sorted list: 8 in [1, 2, 8, 10, 15, 32, 63, 64]? 33 in [1, 2, 8, 10, 15, 32, 63, 64]? ------------------------------------------------------ int index = 0; while (index < listSize) { if (list[index] == input) return index; index++; } return –1;

5. Searching • Better method? • Use fact that we know the list is sorted • Cut what we have to search in half each time • Compare input to middle • If input greater than middle, our value has be in the elements on the right side of the middle element • If input less than middle, our value has to be in the elements on the left side of the middle element • If input equals middle, we found the element.

6. Searching: Binary Search for (int left = 0, right = n –1; left <= right;) { middle =(left + right) / 2; if (input == list[middle]) return middle; else if (input < list[middle]) right = middle – 1; else left = middle + 1; } return – 1;

7. Searching: Binary Search 8 in [1, 2, 8, 10, 15, 32, 63, 64]? 1st iteration: Left = 0, Right = 7, Middle = 3, List[Middle] = 10 Check 8 == 10 => No, 8 < 10 2nd iteration: Left = 0, Right = 2, Middle = 1, List[Middle] = 2 Check 8 == 2 => No, 8 > 2 3rd iteration: Left = 2, Right = 2, Middle = 2 Check 8 == 8 => Yes, Found It!

8. Searching: Binary Search • Binary Search Method: • Number of operations to find input if in the list: • Dependent on position in list • 1 operation if middle • Log2 n operations maximum • Number of operations to find input if not in the list: • Log2 n operations maximum

9. Recursive Binary Search • Two requirements for recursion: • Same algorithm, smaller problem • Termination condition • Binary search? • Search in half of previous array • Stop when down to one element

10. Recursive Binary Search int BinarySearch(int *list, const int input, const int left, const int right) { if (left < right) { middle =(left + right) / 2; if (input == list[middle]) return middle; else if (input < list[middle]) return BinarySearch(list, input, left, middle-1); else return BinarySearch(list,input,middle+1,right); } return – 1; }

11. Fibonacci Computation • Fibonacci Sequence: • 1, 1, 2, 3, 5, 8, 13, 21, 34, 55, 89, … • Simple definition: • Fib[0] = 1 • Fib[1] = 1 • Fib[N] = Fib(N-1) + Fib(N-2)

12. Recursive Fibonacci int fibonacci(int input) { if ((input == 0) || (input == 1)) return 1; else return (fibonacci(input-1) + fibonacci(input-2)); }

13. Iterative Fibonacci int fibonacci(int input) { int first = 1; int second = 1; int temp; for (int k = 0; k < input; k++) { temp = first; first = second; second = temp + second; } return first; }

14. Types of Recursion • Linear Recursion: • 1 recursive call per function • Factorial, Binary Search examples • Tree Recursion: • 2 or more recursive calls per function • Fibonacci Example

15. Efficiency of Recursion • Recursion can sometimes be slower than iterative code • Two main reasons: • Program stack usage • When using recursive functions, every recursive call is added to the stack and it grows fast. • Result generation

16. Efficiency of Recursion • Stack Usage: • When a function is called by a program, that function is placed on the program call stack: • Every stack entry maintains information about the function: • Where to return to when the function completes • Storage for local variables • Pointers or copies of arguments passed in readFile() Returns file data to be used in getData() getData() main() Returns formatted data to be printed in main()

17. Efficiency of Recursion • Another Reason for Slowdowns [Tree Recursion] • Traditional Recursion doesn’t save answers as it executes • Fib(5) = Fib(4) + Fib(3) = Fib(3) + Fib(2) + Fib(3) = Fib(2) + Fib(1) + Fib(2) + Fib(3) = Fib(1) + Fib(0) + Fib(1) + Fib(2) + Fib(3) = Fib(1) + Fib(0) + Fib(1) + Fib(1) + Fib(0) + Fib(3) = Fib(1) + Fib(0) + Fib(1) + Fib(1) + Fib(0) + Fib(2) + Fib(1) = Fib(1) + Fib(0) + Fib(1) + Fib(1) + Fib(0) + Fib(1) + Fib(0) + Fib(1) • Solution: Dynamic programming – saving answers as you go and reusing them

18. Dynamic Programming • Fibonacci Problem • We know the upper bound we are solving for • I.e. Fibonacci (60) = 60 different answers • Generate an array 60 long and initialize to –1 • Everytime we find a solution, fill it in in the array • Next time we look for a solution, if the value in the array for the factorial we need is not –1, use the value present.

19. Algorithmic Complexity

20. Algorithmic Complexity:Two Key Metrics • Space Efficiency: • The maximum amount of memory required to perform a computation and how that maximum amount is related to the size of the input • Time Efficiency: • The maximum number of computations required to perform a computation and how that maximum amount is related to the size of the input • We’ll look at these in terms of recursion first, then extend to more general discussion.

21. Determining Time Efficiency • Two ways to model: • Experimental: Adding in counters to measure the number of operations performed • Theoretical: Using mathematical model of a functions computational requirements • Recursion? Some problems have very nice mathematical functions - Let’s start with those first.

22. Recursion Time Efficiency: Recurrence Relations • Computational Requirements: Three Issues • Amount of work needed for current iteration • Cost of basic function requirements • Number of subproblems that have to be solved • Linear recurrence vs Tree recurrence • Size (in terms of input) of subproblems to be solved • How much smaller is each of the subproblems

23. Recursive Binary Search • Recursive Binary Search Requirements: • Amount of work needed for current iteration • 1 Comparison (inputValue versus middle index) • 1 Adjustment (Changing left or right parameters) • Number of subproblems that have to be solved • 1 Subproblem (the left or right side of the array) • Size (in terms of input) of subproblems to be solved • Subproblem is half the size of the original problem

24. Recurrence Relation • General Recurrence Relation: T(n) = aT(n/b) + cnk a = number of sub problems b = 1/size of subproblems f(n) = current iteration work = constant * nk

25. Recurrence: Master Theorem T(n) = aT(n/b) + f (n) where f (n) ≈nk • a < bk T(n) ~ nk • a = bk T(n) ~ nk lg n • a > bk T(n) ~ nlog b a

26. Recurrence: Master Theorem T1(n) = 7 T1(n/7)+n a = 7, b = 7, k = 1 a ? bk 7 == 7 • nklgn=> n lgn T2(n) = 7 T2(n/2)+n2 a = 7, b = 2, k = 2 a ? bk 7 > 22 => nlogba => nlog27 => n2.81 T3(n) = 7 T3(n/3)+n2 a = 7, b = 3, k = 2 a ? bk 7 < 32 => nk => n2

27. Recursive Binary Search T(n) = aT(n/b) + f(n) a = number of sub problems = 1 b = 1/size of subproblems = 1/(1/2) => 2 f(n) = current iteration work = 2n0 so k = 0 Compare a to bk:1 vs 2^0 = 1 vs 1 If they are equal, computational cost is: nk log n = 1 * log n => log n [Formula can be looked up for >, <, and ==]

28. Space Complexity • Why are we interested in space complexity? • Most people worry about time complexity. • In general, problems with high time complexity can still be solved – we just have to wait longer (albeit there are some problems that take so long they are not solvable in our lifetime/multiple lifetimes). • Problems with too large space complexity we may not be able to run at all. Our finiteness of memory is more much more evident than our finiteness of time.

29. Space Complexity • In general, two components: • Fixed requirements C • Instruction space (code) • Constants • Simple variables • Instance requirements Sp • Variables whose size is dependent on problem instance • Recursion stack space • S(P) = C + Sp • Ignore C as it is usually dominated by Sp

30. Space Complexity float abc(float a, float b, float c) { return a + b + b*c + (a+b-c) / (a+b) + 4.0; } Assume space required is 1 word for each float. No matter what a, b, c are entered, need same amount of memory to execute. Space required for this function is independent of inputs, so Sabc = 0

31. Space Complexity float sum(float *a, const int n) { float s = 0; for (int i =0; i < n; i++) s += a[i]; return s; } Summing over an entire array, problem size = array length [n]. Need four words - one word apiece for a, n, s, i. Again requirements for this function are independent of inputs, so Ssum = 0 • Note the difference between this function space requirements and program as a whole (somebody has to allocate the four words for the data in the array).

32. Space Complexity float rsum(float *a, const int n) { if (n <= 0) return 0; else return (rsum(a, n-1) + a[n-1]); } Summing over an entire array, problem size = array length [n]. Need four words - one word apiece for a, n, s, i. Every recursive call needs those four words, and depth of recursion is n + 1 (base = 0, 1..n) Requirements for this function are dependent on inputs, so Srsum = 4(n+1)

33. Time Complexity • T(P)= Compile time + Execution Time • Compile time is: • Not dependent on program inputs • Negligible in comparison to execution time for real world, extensively used programs • Execution time: • Difficult to accurately obtain • Hardware dependent, OS dependent, load dependent … • Estimate in terms of program steps

34. Time Complexity in C++ Context • Comments: • Not executed • Step count = 0 • Declarations: (int b; ) • Either not executable or lumped into cost of calling function • Step count = 0

35. C++ Time Complexity • Expressions: (for example: a == true) • Generally step count = 1 • If expression contains functions, sum of costs for function invocations (defined later) • Assignments: <variable> = <expression> • Generally, step count for evaluation of expression • If variable size dependent on instance, • step count = size of variable + expression step count • I.E. a = (2+b); a,b both lists, then: • step count = cost of evaluating expression (2+b) + cost of copying list b into list a

36. C++ Time Complexity • Iteration: • Consider loop control elements, not internal statements of the loop body • While <expr> do • Do… while <expr> • For (<init_stmt>; <expr1>; <expr2>) • while, do: Each iteration requires step count of <expr> • for: • Generally 1 for each iteration unless <init_stmt>, <expr1>, <expr2> dependent on size • 1st time (initialize, test) = <init_stmt> steps + <expr1> steps • All other times (update, test) = <expr1> steps + <expr2> steps • Generally have to count iterations + 1 (all true iterations where fall into body plus one where the condition fails)

37. C++ Time Complexity • Switch statement: • switch <expr> { case cond1: <statement1> case cond2: <statement2> default: <statement>} • Header: <expr> steps • Conditions: Cost of own check plus cost of all preceding checks • If/else: • If (<expr>) <statements1>; else <statements2> • <expr> = true: <expr> +<statements1> cost • <expr> = false: <expr>+ <statements2> cost

38. C++ Time Complexity • Function invocation: • Step cost of 1 generally • Pass by value parameters: If variable size dependent on problem instance, add step cost of size (to do copy) • If recursive, include all copies of local variables that are size dependent because must be generated in recursive call

39. C++ Time Complexity • Memory management: new/delete • Step cost 1 generally • If call constructor, destructor, compute similar to function invocation (take into account cost of pass by value parameters) • Jump statements: continue/ break/ goto/ return / return<expr> : • Step cost 1 for continue/break/goto/return • return <expr>: Cost of <expr>

40. Measuring Complexity • First approach: • Extend programs to incorporate step count statements • Add global variable count • Whenever a statement is executed, add an additional statement that increments count

41. Measuring Complexity float sum(float *a, const int n) { float s = 0; for (int i =0; i < n; i++) s += a[i]; return s; } float sum(float *a, const int n) { float s = 0; count++; // assignment for (int i =0; i < n; i++) { count++; s += a[i]; count++;} // 1 for for , 1 for += count ++; // last time checking for return s; count ++; // return statement }

42. Measuring Complexity • Strip out everything except count statements: float sum(float *a, const int n) { count++; // assignment for (int i =0; i < n; i++) { count = count + 2;} // 1 for for, 1 for += // last time checking for and return statement count = count + 2; }

43. Measuring Complexity: • Sum all count statements: 1 at beginning + (N iterations of loop) * (2 within a loop) + 2 at end => 2N + 3

44. Measuring Complexity float rsum(float *a, int n) { if (n <= 0) return 0; else return (rsum(a, n-1) + a[n-1]); } float rsum(float *a, int n) { count ++; // if conditional if (n <= 0) { count++; return 0;} // return else { count++; return (rsum(a,n-1) + a[n-1]); // return statement } How to handle recursion?

45. Measuring Complexity: Recursion float rsum(float *a, cont int n) { count ++; // if conditional if (n <= 0) { count++;} // return else { count++; return (rsum(a,n-1) + a[n-1]); } If (n <= 0) count = 2 Otherwise count = 2 + count(n-1)

46. Measuring Complexity: Recursion • 2 + count(n-1) Recurrence relation • Solve by repeatedly substituting count(n-1) with its recursive definition => 2+ count(n-1) = 2 + 2 + count(n-2) = 2 + 2 + 2 + count(n – 3) … = 2 * n + count(0) => 2n + 2

47. Time Complexity • Iterative Sum: 2n + 3 • Recursive Sum: 2n + 2 • Is recursive sum faster than iterative? Not necessarily – Each step of recursive sum may actually be more expensive than a step of iterative sum Not a problem - We are really interested in measurements in relation to input size, which we see are very similar.

48. Time Complexity • 2n + 2 => Run time is linear in N • If input grows by a factor of 10, execution time should grow by a factor of 10 • If input grows by a factor of 10,000, cost should grow by a factor of 10,000 Xn + C => Xn often dominates constant term so we often ignore C

49. Time Complexity • Matrix addition example: void add(matrix a, matrix b, matrix c, int m, int n) { for (int i = 0; i < m; i++) { for (int j = 0; j < n; j++) { c[i][j] = a[i][j] + b[i][j] } } }

50. Time Complexity void add(matrix a, matrix b, matrix c, int m, int n) { for (int i = 0; i < m; i++) { count++; // for loop for (int j = 0; j < n; j++) { count++; // for loop c[i][j] = a[i][j] + b[i][j] count++; // assignment } count ++; // last time through for loop on j } count++; // last time through for loop on i }