Recurrence relations
Download
1 / 34

Recurrence Relations - PowerPoint PPT Presentation


  • 423 Views
  • Updated On :

Recurrence Relations. Connection to recursive algorithms Techniques for solving them. Recursion and Mathematical Induction. In both, we have general and boundary conditions: The general conditions break the problem into smaller and smaller pieces.

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Recurrence Relations' - cora


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
Recurrence relations l.jpg
Recurrence Relations

  • Connection to recursive algorithms

  • Techniques for solving them


Recursion and mathematical induction l.jpg
Recursion and Mathematical Induction

  • In both, we have general and boundary conditions:

    • The general conditions break the problem into smaller and smaller pieces.

    • The initial or boundary condition terminate the recursion.

  • Both take a Divide and Conquer approach to solving mathematical problems.



What if we knew we could solve part of the problem l.jpg
What if we knew we could solve part of the problem?

Assume we can move k (in this case, 4) different rings




Where do recurrence relations come from l.jpg
Where do recurrence relations come from?

  • Analysis of a divide and conquer algorithm

    • Towers of Hanoi, Merge Sort, Binary Search

  • Analysis of a combinatorial object

    • up-down permutations

  • This is the key analysis step I want you to master

  • Use small cases to check correctness of your recurrence relation


Recurrence relations8 l.jpg
Recurrence Relations

Possible recurrence relations:

an = an-1 + 1; a1 = 1 => an = n (linear)

an = an-1+ 2n - 1; a1=1 => an = n2 (polynomial)

an = 2an-1; a1 = 1 => an = 2n (exponential)

an = n an-1; a1 = 1 => an = n! (others…)


Can recurrence relations be solved l.jpg
Can recurrence relations be solved?

  • No general procedure for solving recurrence relations is known, which is why it is an art.

  • However, linear, finite history, constant coefficient recurrences always can be solved

  • Example: an = 2an-1 + 2an-2 +1 ; a1 = 1 ; a2 = 1

    • degree = 1

    • history = 2

    • coefficients = 2, 2, and 1

  • In the end, what is the best way to solve this?

    • Software like Mathematica or Maple, but I still want you to be able to solve some on your own without such aid


Solution techniques l.jpg
Solution Techniques

  • Guess a solution by looking at a few small examples and prove by induction.

  • Try back-substituting until you know what is going on.

  • Draw a recursion tree.

  • Master Theorem

  • General Technique


First step l.jpg
First step

  • Using the base case and the recursive case, calculate small values

  • Use these values to help guess a solution

  • Use these values to help verify correctness of your closed form solution


Guessing solution and proving by induction l.jpg
Guessing solution and proving by induction

  • We can use mathematical induction to prove that a general function solves for a recursive one.

    • Tn = 2Tn-1 + 1 ; T0 = 0

    • n = 0 1 2 3 4 5 6 7 8

    • Tn =

  • Guess what the solution is?


Guessing solution and proving by induction ii l.jpg
Guessing solution and proving by induction II

  • Prove: Tn = 2n - 1 by induction:

  • 1. Show the base case is true: T0 = 20 - 1 = 0

  • 2. Now assume true for Tn-1

  • 3. Substitute in Tn-1 in recurrence for Tn

  • Tn = 2Tn-1 + 1

  • = 2 ( 2n-1 - 1 ) + 1

  • = 2n -1


Back substitution l.jpg
Back-substitution

Example: T(n) = 3T(n/4) + n, T(1) = 1

= 3(3T(n/16)+n/4) + n

= 9T(n/16) + 3n/4 + n

= 9(3T(n/64) +n/16) + 3n/4 + n

= 27T(n/64)+9n/16 + 3n/4 + n


Recursion trees l.jpg
Recursion Trees

T(n) = 2 T(n/2) + n2 , T(1) = 1


Example problem l.jpg
Example Problem

Use induction to prove that MergeSort is an O(n log n) algorithm.

Mergesort(array)

n = size(array)

if ( n == 1) return array

array1 = Mergesort(array[1 .. n/2])

array2 = Mergesort(array[n/2 .. n])

return Merge(array1, array2)


Induction proof l.jpg
Induction Proof

Example: Prove that T(n) = 2T( n/2 ) + n , T(1) = 1 is O(n log n).

Base case: n=2, c>4.

We need to prove that T(n) < c n log n , for all n greater than some value.

Inductive step: Assume T(n/2) ≤ c (n/2) log (n/2)

Show that T(n) ≤ c n log n .


Induction proof18 l.jpg
Induction Proof

Given : T(n/2) ≤ c (n/2) log (n/2)

T(n) = 2T( n/2 ) + n

≤ 2( c(n/2) log (n/2) ) + n

≤ 2( c(n/2) log (n/2) ) + n (dropping floors makes it bigger!)

= c n log(n/2) + n

= c n ( log(n) - log(2) ) + n

= c n log(n) - c n + n (log22 = 1)

= c n log(n) - (c - 1) n

< c n log(n) (c > 1)


Example problem 2 l.jpg
Example Problem 2

T(n) = T(n/3) + T(2n/3) + n

Show T(n) is (n log n) by appealing to the recursion tree.



Master theorem l.jpg
Master Theorem

  • T(n) = a T(n/b) + f(n)

    • Ignore floors and ceilings for n/b

    • constants a >= 1 and b>1

    • f(n) any function

  • If f(n) = O(nlog_ba-e) for constant e>0, T(n) = Q(nlog_b a)

  • If f(n) = Q(nlog_ba), T(n) = Q(nlog_b a lg n)

  • If f(n) = W(nlog_ba+e) for some constant e >0, and if af(n/b) <= c f(n) for some constant c < 1 and all sufficiently large n, T(n) = Q(f(n)).

  • Key idea: Compare nlog_b a with f(n)


Example l.jpg
Example

  • T(n) = 4 T(n/2) + n

  • a =

  • b =

  • nlog_ba =

  • f(n) =


Characteristic equation approach l.jpg
Characteristic Equation Approach

  • tn = 3tn-1 + 4tn-2 for n > 1

    • t0 = 0, t1 = 5

  • Rewrite recurrence

    • tn - 3tn-1 - 4tn-2 = 0

  • Properties

    • Homogeneous: no terms not involving tn

    • Linear: tn terms have no squares or worse

    • constant coefficients: 1, -3, -4


Characteristic equation l.jpg
Characteristic Equation

  • tn - 3tn-1 - 4tn-2 = 0

  • Rewrite assume solution of the form tn = xn

  • xn – 3xn-1 – 4xn-2 = 0

  • xn-2 (x2 – 3x – 4) = 0

  • Find roots of (x2 – 3x – 4)

    • (x+1)(x-4)  roots are -1 and 4

  • Solution is of form c1(-1)n + c24n


Solving for constants l.jpg
Solving for constants

  • tn = c1(-1)n + c24n

  • Use base cases to solve for constants

    • t0 = 0 = c1(-1)0 + c240 = c1 + c2

    • t1 = 5 = c1(-1)1 + c241 = -c1 + 4c2

    • 5c2 = 5  c2 = 1  c1 = -1

  • tn = (-1)n+1 + 4n

  • Always test solution on small values!


Repeated roots for char eqn l.jpg
Repeated roots for char. eqn

  • tn - 5tn-1 + 8tn-2 – 4tn-3= 0

    • boundary conditions: tn = n for n = 0, 1, 2

  • x3 - 5x2 + 8x – 4 = 0

  • (x-1)(x-2)2 roots are 1, 2, 2

  • Solution is of form c1(1)n + c22n + c3n2n

    • If root is repeated third time, then n22n term, and so on


Solving for constants27 l.jpg
Solving for constants

  • tn = c1(1)n + c22n + c3n2n

  • Use base cases to solve for constants

    • t0 = 0 = c1(1)0 + c220 + c30 20 = c1 + c2

    • t1 = 1 = c1(1)1 + c221 + c31 21 = c1 + 2c2 + 2c3

    • t2 = 2 = c1(1)2 + c222 + c32 22 = c1 + 4c2 + 8c3

    • c1 = -2, c2 = 2, c3 = -1/2

  • tn = 2n+1 – n2n-1 – 2

  • Test the solution on small values!


Inhomogeneous equation l.jpg
Inhomogeneous Equation

  • tn - 2tn-1 = 3n

    • base case value for t0only

  • (x – 2) (x-3) = 0

    • (x-2) term comes from homogeneous solution

    • If rhs is of form bn poly(n) of degree d

      • In this case, b = 3, poly (n) = 1 is of degree 0

    • Plug (x-b)d+1 into characteristic equation


Solving for constants29 l.jpg
Solving for constants

  • (x – 2) (x-3) = 0

  • tn = c12n + c23n

  • Solve for c1 and c2 with only t0 base case

  • This is only 1 equation and 2 unknowns

  • Use recurrence to generate extra equations

    • tn - 2tn-1 = 3n t1 = 2t0 + 3

  • Now we have two equations

    • t0 = c120 + c230 = c1 + c2

    • t1 = 2t0 + 3 = c121 + c231 = 2c1 + 3c2

    • c1 = t0 – 3 and c2 = 3

  • tn = (t0-3)2n + 3n+1


Changing variable l.jpg
Changing variable

  • tn – 3tn/2 = n if n is a power of 2

    • t1 = 1

  • Let n = 2i and si = tn

  • si – 3si-1 = 2i for i >= 1

    • s0 = 1

  • (x-3)(x-2) = 0

    • x-3 from characteristic equation

    • x-2 bnpoly(n) rhs


Solving for constants31 l.jpg
Solving for constants

  • (x-3)(x-2) = 0

  • si = c13i + c22i

  • Generating two equations

    • t1 = s0 = 1 = c130 + c220 = c1 + c2

    • t2 = s1 = 3t1+2 = 5 = c131 + c221 = 3c1 + 2c2

    • c1 = 3, c2 = -2

  • si = 3i+1 – 2i+1 for i >= 0

  • tn = 3nlg 3 – 2n for n a power of 2 >= 1


Example up down permutations l.jpg
Example: Up-down Permutations

  • An up-down permutation is a permutation of the numbers from 1 to n such that the numbers strictly increase to n and then decrease thereafter

  • Examples

    • 1376542, 7654321, 3476521

  • Let tn denote the number of up-down permutations

    • What is recurrence relation for tn?

    • What is solution?


Example towers of hanoi l.jpg
Example: Towers of Hanoi

  • Suppose we have two disks of each size.

  • Let n be the number of sizes of disks

    • What is recurrence relation?

    • What is solution?


Example merge sort l.jpg
Example: Merge Sort

  • Merge sort breaking array into 3 pieces

    • What is recurrence relation?

    • What is solution?

    • How does this compare to breaking into 2 pieces?


ad