Recurrence relations l.jpg
Sponsored Links
This presentation is the property of its rightful owner.
1 / 34

Recurrence Relations PowerPoint PPT Presentation


  • 317 Views
  • Uploaded on
  • Presentation posted in: General

Recurrence Relations. Connection to recursive algorithms Techniques for solving them. Recursion and Mathematical Induction. In both, we have general and boundary conditions: The general conditions break the problem into smaller and smaller pieces.

Download Presentation

Recurrence Relations

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Recurrence Relations

  • Connection to recursive algorithms

  • Techniques for solving them


Recursion and Mathematical Induction

  • In both, we have general and boundary conditions:

    • The general conditions break the problem into smaller and smaller pieces.

    • The initial or boundary condition terminate the recursion.

  • Both take a Divide and Conquer approach to solving mathematical problems.


The Towers of Hanoi


What if we knew we could solve part of the problem?

Assume we can move k (in this case, 4) different rings


Can we do one better?


Solved for one more!


Where do recurrence relations come from?

  • Analysis of a divide and conquer algorithm

    • Towers of Hanoi, Merge Sort, Binary Search

  • Analysis of a combinatorial object

    • up-down permutations

  • This is the key analysis step I want you to master

  • Use small cases to check correctness of your recurrence relation


Recurrence Relations

Possible recurrence relations:

an = an-1 + 1; a1 = 1 => an = n(linear)

an = an-1+ 2n - 1; a1=1 => an = n2(polynomial)

an = 2an-1; a1 = 1 => an = 2n(exponential)

an = n an-1; a1 = 1 => an = n!(others…)


Can recurrence relations be solved?

  • No general procedure for solving recurrence relations is known, which is why it is an art.

  • However, linear, finite history, constant coefficient recurrences always can be solved

  • Example: an = 2an-1 + 2an-2 +1 ; a1 = 1 ; a2 = 1

    • degree = 1

    • history = 2

    • coefficients = 2, 2, and 1

  • In the end, what is the best way to solve this?

    • Software like Mathematica or Maple, but I still want you to be able to solve some on your own without such aid


Solution Techniques

  • Guess a solution by looking at a few small examples and prove by induction.

  • Try back-substituting until you know what is going on.

  • Draw a recursion tree.

  • Master Theorem

  • General Technique


First step

  • Using the base case and the recursive case, calculate small values

  • Use these values to help guess a solution

  • Use these values to help verify correctness of your closed form solution


Guessing solution and proving by induction

  • We can use mathematical induction to prove that a general function solves for a recursive one.

    • Tn = 2Tn-1 + 1 ; T0 = 0

    • n= 0 1 2 3 4 5 6 7 8

    • Tn=

  • Guess what the solution is?


Guessing solution and proving by induction II

  • Prove: Tn = 2n - 1 by induction:

  • 1. Show the base case is true: T0 = 20 - 1 = 0

  • 2. Now assume true for Tn-1

  • 3. Substitute in Tn-1 in recurrence for Tn

  • Tn = 2Tn-1 + 1

  • = 2 ( 2n-1 - 1 ) + 1

  • = 2n -1


Back-substitution

Example: T(n) = 3T(n/4) + n, T(1) = 1

= 3(3T(n/16)+n/4) + n

= 9T(n/16) + 3n/4 + n

= 9(3T(n/64) +n/16) + 3n/4 + n

= 27T(n/64)+9n/16 + 3n/4 + n


Recursion Trees

T(n) = 2 T(n/2) + n2 , T(1) = 1


Example Problem

Use induction to prove that MergeSort is an O(n log n) algorithm.

Mergesort(array)

n = size(array)

if ( n == 1) return array

array1 = Mergesort(array[1 .. n/2])

array2 = Mergesort(array[n/2 .. n])

return Merge(array1, array2)


Induction Proof

Example: Prove that T(n) = 2T( n/2 ) + n , T(1) = 1 is O(n log n).

Base case: n=2, c>4.

We need to prove that T(n) < c n log n , for all n greater than some value.

Inductive step: Assume T(n/2) ≤ c (n/2) log (n/2)

Show that T(n) ≤ c n log n .


Induction Proof

Given : T(n/2) ≤ c (n/2) log (n/2)

T(n)= 2T( n/2 ) + n

≤ 2( c(n/2) log (n/2) ) + n

≤ 2( c(n/2) log (n/2) ) + n (dropping floors makes it bigger!)

= c n log(n/2) + n

= c n ( log(n) - log(2) ) + n

= c n log(n) - c n + n (log22 = 1)

= c n log(n) - (c - 1) n

< c n log(n) (c > 1)


Example Problem 2

T(n) = T(n/3) + T(2n/3) + n

Show T(n) is (n log n) by appealing to the recursion tree.


Recursion Tree


Master Theorem

  • T(n) = a T(n/b) + f(n)

    • Ignore floors and ceilings for n/b

    • constants a >= 1 and b>1

    • f(n) any function

  • If f(n) = O(nlog_ba-e) for constant e>0, T(n) = Q(nlog_b a)

  • If f(n) = Q(nlog_ba), T(n) = Q(nlog_b a lg n)

  • If f(n) = W(nlog_ba+e) for some constant e >0, and if af(n/b) <= c f(n) for some constant c < 1 and all sufficiently large n, T(n) = Q(f(n)).

  • Key idea: Compare nlog_b a with f(n)


Example

  • T(n) = 4 T(n/2) + n

  • a =

  • b =

  • nlog_ba =

  • f(n) =


Characteristic Equation Approach

  • tn = 3tn-1 + 4tn-2 for n > 1

    • t0 = 0, t1 = 5

  • Rewrite recurrence

    • tn - 3tn-1 - 4tn-2 = 0

  • Properties

    • Homogeneous: no terms not involving tn

    • Linear: tn terms have no squares or worse

    • constant coefficients: 1, -3, -4


Characteristic Equation

  • tn - 3tn-1 - 4tn-2 = 0

  • Rewrite assume solution of the form tn = xn

  • xn – 3xn-1 – 4xn-2 = 0

  • xn-2 (x2 – 3x – 4) = 0

  • Find roots of (x2 – 3x – 4)

    • (x+1)(x-4)  roots are -1 and 4

  • Solution is of form c1(-1)n + c24n


Solving for constants

  • tn = c1(-1)n + c24n

  • Use base cases to solve for constants

    • t0 = 0 = c1(-1)0 + c240 = c1 + c2

    • t1 = 5 = c1(-1)1 + c241 = -c1 + 4c2

    • 5c2 = 5  c2 = 1  c1 = -1

  • tn = (-1)n+1 + 4n

  • Always test solution on small values!


Repeated roots for char. eqn

  • tn - 5tn-1 + 8tn-2 – 4tn-3= 0

    • boundary conditions: tn = n for n = 0, 1, 2

  • x3 - 5x2 + 8x – 4 = 0

  • (x-1)(x-2)2 roots are 1, 2, 2

  • Solution is of form c1(1)n + c22n + c3n2n

    • If root is repeated third time, then n22n term, and so on


Solving for constants

  • tn = c1(1)n + c22n + c3n2n

  • Use base cases to solve for constants

    • t0 = 0 = c1(1)0 + c220 + c30 20 = c1 + c2

    • t1 = 1 = c1(1)1 + c221 + c31 21 = c1 + 2c2 + 2c3

    • t2 = 2 = c1(1)2 + c222 + c32 22 = c1 + 4c2 + 8c3

    • c1 = -2, c2 = 2, c3 = -1/2

  • tn = 2n+1 – n2n-1 – 2

  • Test the solution on small values!


Inhomogeneous Equation

  • tn - 2tn-1 = 3n

    • base case value for t0only

  • (x – 2) (x-3) = 0

    • (x-2) term comes from homogeneous solution

    • If rhs is of form bn poly(n) of degree d

      • In this case, b = 3, poly (n) = 1 is of degree 0

    • Plug (x-b)d+1 into characteristic equation


Solving for constants

  • (x – 2) (x-3) = 0

  • tn = c12n + c23n

  • Solve for c1 and c2 with only t0 base case

  • This is only 1 equation and 2 unknowns

  • Use recurrence to generate extra equations

    • tn - 2tn-1 = 3n t1 = 2t0 + 3

  • Now we have two equations

    • t0 = c120 + c230 = c1 + c2

    • t1 = 2t0 + 3 = c121 + c231 = 2c1 + 3c2

    • c1 = t0 – 3 and c2 = 3

  • tn = (t0-3)2n + 3n+1


Changing variable

  • tn – 3tn/2 = n if n is a power of 2

    • t1 = 1

  • Let n = 2i and si = tn

  • si – 3si-1 = 2i for i >= 1

    • s0 = 1

  • (x-3)(x-2) = 0

    • x-3 from characteristic equation

    • x-2 bnpoly(n) rhs


Solving for constants

  • (x-3)(x-2) = 0

  • si = c13i + c22i

  • Generating two equations

    • t1 = s0 = 1 = c130 + c220 = c1 + c2

    • t2 = s1 = 3t1+2 = 5 = c131 + c221 = 3c1 + 2c2

    • c1 = 3, c2 = -2

  • si = 3i+1 – 2i+1 for i >= 0

  • tn = 3nlg 3 – 2n for n a power of 2 >= 1


Example: Up-down Permutations

  • An up-down permutation is a permutation of the numbers from 1 to n such that the numbers strictly increase to n and then decrease thereafter

  • Examples

    • 1376542, 7654321, 3476521

  • Let tn denote the number of up-down permutations

    • What is recurrence relation for tn?

    • What is solution?


Example: Towers of Hanoi

  • Suppose we have two disks of each size.

  • Let n be the number of sizes of disks

    • What is recurrence relation?

    • What is solution?


Example: Merge Sort

  • Merge sort breaking array into 3 pieces

    • What is recurrence relation?

    • What is solution?

    • How does this compare to breaking into 2 pieces?


  • Login