Recurrence relations l.jpg
This presentation is the property of its rightful owner.
Sponsored Links
1 / 34

Recurrence Relations PowerPoint PPT Presentation


  • 300 Views
  • Uploaded on
  • Presentation posted in: General

Recurrence Relations. Connection to recursive algorithms Techniques for solving them. Recursion and Mathematical Induction. In both, we have general and boundary conditions: The general conditions break the problem into smaller and smaller pieces.

Download Presentation

Recurrence Relations

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Recurrence relations l.jpg

Recurrence Relations

  • Connection to recursive algorithms

  • Techniques for solving them


Recursion and mathematical induction l.jpg

Recursion and Mathematical Induction

  • In both, we have general and boundary conditions:

    • The general conditions break the problem into smaller and smaller pieces.

    • The initial or boundary condition terminate the recursion.

  • Both take a Divide and Conquer approach to solving mathematical problems.


The towers of hanoi l.jpg

The Towers of Hanoi


What if we knew we could solve part of the problem l.jpg

What if we knew we could solve part of the problem?

Assume we can move k (in this case, 4) different rings


Can we do one better l.jpg

Can we do one better?


Solved for one more l.jpg

Solved for one more!


Where do recurrence relations come from l.jpg

Where do recurrence relations come from?

  • Analysis of a divide and conquer algorithm

    • Towers of Hanoi, Merge Sort, Binary Search

  • Analysis of a combinatorial object

    • up-down permutations

  • This is the key analysis step I want you to master

  • Use small cases to check correctness of your recurrence relation


Recurrence relations8 l.jpg

Recurrence Relations

Possible recurrence relations:

an = an-1 + 1; a1 = 1 => an = n(linear)

an = an-1+ 2n - 1; a1=1 => an = n2(polynomial)

an = 2an-1; a1 = 1 => an = 2n(exponential)

an = n an-1; a1 = 1 => an = n!(others…)


Can recurrence relations be solved l.jpg

Can recurrence relations be solved?

  • No general procedure for solving recurrence relations is known, which is why it is an art.

  • However, linear, finite history, constant coefficient recurrences always can be solved

  • Example: an = 2an-1 + 2an-2 +1 ; a1 = 1 ; a2 = 1

    • degree = 1

    • history = 2

    • coefficients = 2, 2, and 1

  • In the end, what is the best way to solve this?

    • Software like Mathematica or Maple, but I still want you to be able to solve some on your own without such aid


Solution techniques l.jpg

Solution Techniques

  • Guess a solution by looking at a few small examples and prove by induction.

  • Try back-substituting until you know what is going on.

  • Draw a recursion tree.

  • Master Theorem

  • General Technique


First step l.jpg

First step

  • Using the base case and the recursive case, calculate small values

  • Use these values to help guess a solution

  • Use these values to help verify correctness of your closed form solution


Guessing solution and proving by induction l.jpg

Guessing solution and proving by induction

  • We can use mathematical induction to prove that a general function solves for a recursive one.

    • Tn = 2Tn-1 + 1 ; T0 = 0

    • n= 0 1 2 3 4 5 6 7 8

    • Tn=

  • Guess what the solution is?


Guessing solution and proving by induction ii l.jpg

Guessing solution and proving by induction II

  • Prove: Tn = 2n - 1 by induction:

  • 1. Show the base case is true: T0 = 20 - 1 = 0

  • 2. Now assume true for Tn-1

  • 3. Substitute in Tn-1 in recurrence for Tn

  • Tn = 2Tn-1 + 1

  • = 2 ( 2n-1 - 1 ) + 1

  • = 2n -1


Back substitution l.jpg

Back-substitution

Example: T(n) = 3T(n/4) + n, T(1) = 1

= 3(3T(n/16)+n/4) + n

= 9T(n/16) + 3n/4 + n

= 9(3T(n/64) +n/16) + 3n/4 + n

= 27T(n/64)+9n/16 + 3n/4 + n


Recursion trees l.jpg

Recursion Trees

T(n) = 2 T(n/2) + n2 , T(1) = 1


Example problem l.jpg

Example Problem

Use induction to prove that MergeSort is an O(n log n) algorithm.

Mergesort(array)

n = size(array)

if ( n == 1) return array

array1 = Mergesort(array[1 .. n/2])

array2 = Mergesort(array[n/2 .. n])

return Merge(array1, array2)


Induction proof l.jpg

Induction Proof

Example: Prove that T(n) = 2T( n/2 ) + n , T(1) = 1 is O(n log n).

Base case: n=2, c>4.

We need to prove that T(n) < c n log n , for all n greater than some value.

Inductive step: Assume T(n/2) ≤ c (n/2) log (n/2)

Show that T(n) ≤ c n log n .


Induction proof18 l.jpg

Induction Proof

Given : T(n/2) ≤ c (n/2) log (n/2)

T(n)= 2T( n/2 ) + n

≤ 2( c(n/2) log (n/2) ) + n

≤ 2( c(n/2) log (n/2) ) + n (dropping floors makes it bigger!)

= c n log(n/2) + n

= c n ( log(n) - log(2) ) + n

= c n log(n) - c n + n (log22 = 1)

= c n log(n) - (c - 1) n

< c n log(n) (c > 1)


Example problem 2 l.jpg

Example Problem 2

T(n) = T(n/3) + T(2n/3) + n

Show T(n) is (n log n) by appealing to the recursion tree.


Recursion tree l.jpg

Recursion Tree


Master theorem l.jpg

Master Theorem

  • T(n) = a T(n/b) + f(n)

    • Ignore floors and ceilings for n/b

    • constants a >= 1 and b>1

    • f(n) any function

  • If f(n) = O(nlog_ba-e) for constant e>0, T(n) = Q(nlog_b a)

  • If f(n) = Q(nlog_ba), T(n) = Q(nlog_b a lg n)

  • If f(n) = W(nlog_ba+e) for some constant e >0, and if af(n/b) <= c f(n) for some constant c < 1 and all sufficiently large n, T(n) = Q(f(n)).

  • Key idea: Compare nlog_b a with f(n)


Example l.jpg

Example

  • T(n) = 4 T(n/2) + n

  • a =

  • b =

  • nlog_ba =

  • f(n) =


Characteristic equation approach l.jpg

Characteristic Equation Approach

  • tn = 3tn-1 + 4tn-2 for n > 1

    • t0 = 0, t1 = 5

  • Rewrite recurrence

    • tn - 3tn-1 - 4tn-2 = 0

  • Properties

    • Homogeneous: no terms not involving tn

    • Linear: tn terms have no squares or worse

    • constant coefficients: 1, -3, -4


Characteristic equation l.jpg

Characteristic Equation

  • tn - 3tn-1 - 4tn-2 = 0

  • Rewrite assume solution of the form tn = xn

  • xn – 3xn-1 – 4xn-2 = 0

  • xn-2 (x2 – 3x – 4) = 0

  • Find roots of (x2 – 3x – 4)

    • (x+1)(x-4)  roots are -1 and 4

  • Solution is of form c1(-1)n + c24n


Solving for constants l.jpg

Solving for constants

  • tn = c1(-1)n + c24n

  • Use base cases to solve for constants

    • t0 = 0 = c1(-1)0 + c240 = c1 + c2

    • t1 = 5 = c1(-1)1 + c241 = -c1 + 4c2

    • 5c2 = 5  c2 = 1  c1 = -1

  • tn = (-1)n+1 + 4n

  • Always test solution on small values!


Repeated roots for char eqn l.jpg

Repeated roots for char. eqn

  • tn - 5tn-1 + 8tn-2 – 4tn-3= 0

    • boundary conditions: tn = n for n = 0, 1, 2

  • x3 - 5x2 + 8x – 4 = 0

  • (x-1)(x-2)2 roots are 1, 2, 2

  • Solution is of form c1(1)n + c22n + c3n2n

    • If root is repeated third time, then n22n term, and so on


Solving for constants27 l.jpg

Solving for constants

  • tn = c1(1)n + c22n + c3n2n

  • Use base cases to solve for constants

    • t0 = 0 = c1(1)0 + c220 + c30 20 = c1 + c2

    • t1 = 1 = c1(1)1 + c221 + c31 21 = c1 + 2c2 + 2c3

    • t2 = 2 = c1(1)2 + c222 + c32 22 = c1 + 4c2 + 8c3

    • c1 = -2, c2 = 2, c3 = -1/2

  • tn = 2n+1 – n2n-1 – 2

  • Test the solution on small values!


Inhomogeneous equation l.jpg

Inhomogeneous Equation

  • tn - 2tn-1 = 3n

    • base case value for t0only

  • (x – 2) (x-3) = 0

    • (x-2) term comes from homogeneous solution

    • If rhs is of form bn poly(n) of degree d

      • In this case, b = 3, poly (n) = 1 is of degree 0

    • Plug (x-b)d+1 into characteristic equation


Solving for constants29 l.jpg

Solving for constants

  • (x – 2) (x-3) = 0

  • tn = c12n + c23n

  • Solve for c1 and c2 with only t0 base case

  • This is only 1 equation and 2 unknowns

  • Use recurrence to generate extra equations

    • tn - 2tn-1 = 3n t1 = 2t0 + 3

  • Now we have two equations

    • t0 = c120 + c230 = c1 + c2

    • t1 = 2t0 + 3 = c121 + c231 = 2c1 + 3c2

    • c1 = t0 – 3 and c2 = 3

  • tn = (t0-3)2n + 3n+1


Changing variable l.jpg

Changing variable

  • tn – 3tn/2 = n if n is a power of 2

    • t1 = 1

  • Let n = 2i and si = tn

  • si – 3si-1 = 2i for i >= 1

    • s0 = 1

  • (x-3)(x-2) = 0

    • x-3 from characteristic equation

    • x-2 bnpoly(n) rhs


Solving for constants31 l.jpg

Solving for constants

  • (x-3)(x-2) = 0

  • si = c13i + c22i

  • Generating two equations

    • t1 = s0 = 1 = c130 + c220 = c1 + c2

    • t2 = s1 = 3t1+2 = 5 = c131 + c221 = 3c1 + 2c2

    • c1 = 3, c2 = -2

  • si = 3i+1 – 2i+1 for i >= 0

  • tn = 3nlg 3 – 2n for n a power of 2 >= 1


Example up down permutations l.jpg

Example: Up-down Permutations

  • An up-down permutation is a permutation of the numbers from 1 to n such that the numbers strictly increase to n and then decrease thereafter

  • Examples

    • 1376542, 7654321, 3476521

  • Let tn denote the number of up-down permutations

    • What is recurrence relation for tn?

    • What is solution?


Example towers of hanoi l.jpg

Example: Towers of Hanoi

  • Suppose we have two disks of each size.

  • Let n be the number of sizes of disks

    • What is recurrence relation?

    • What is solution?


Example merge sort l.jpg

Example: Merge Sort

  • Merge sort breaking array into 3 pieces

    • What is recurrence relation?

    • What is solution?

    • How does this compare to breaking into 2 pieces?


  • Login