Recurrence Relations

Recurrence Relations • An equation that allow us to compute the nth term of a sequence from preceding terms. • Example1: The selection sort • In the selection sort algorithm the total number of comparisons is given by 0 if n = 1 Cn = Cn-1 + n-1 for n > 1 Recurrence Relations

Example2: The subset recurrence • The subset recurrence: let Sn be the number of subsets of N = {1, 2, 3,….., n-1, n}, then Sncan be computed from Sn-1 (which is the number of subsets of {1, 2, …..,n-1} as follows: • Let N’ = {1, 2, …..,n-1}, Sn-1 is the number of subsets of N’. Now subsets of N are either contain N or does not. • The subsets of N that do not contain n are the subsets of N’. Their number is Sn-1. • The subsets of N that contain n are the subsets of N’ with n added to each subset. Their number is Sn-1. Recurrence Relations

Example2: The subset recurrence 2 * Sn-1 for n > 1 • Thus Sn = S0 = 1 for n= 0 • where S0 is the number of subsets of the empty set. • Can we find out the first 5 terms of Sn? Recurrence Relations

Example3: The Bijection Recurrence • What is the number of one-to-one onto functions from a set A = {1, 2,…., n} to a set T with n elements? • Does it depend on n? if so? • Let it be bn. We would like to find it. Recurrence Relations

Example3: The Bijection Recurrence • bn can defined recursively since • for a function f • f(n) has n choices let it be t & the numbers {1, 2,…., n-1} have to be mapped to the remaining n-1 elements of T. • The number of bijections is bn-1 • Thus by the product principle there are n*bn-1 choices. That is bn=n*bn-1 • Also notice that b1 = 1 Recurrence Relations

The order of the recurrence relation • In the previous examples the nth term is computed from the nth-1 term without using other previous terms such as the nth-2 term. • Such a recurrence relations is called a first order recurrence relation. • A recurrence relation is said to be of the rth order if computing the nth term requires computing the preceding r terms. • What is the order of the Fibonacci number? • f(n) = f(n-1) + f(n-2) and both f(0) & f(1) = 1 Recurrence Relations

Linear recurrence relations • A first order recurrence relation is called linear recurrence relations if the nth term can be computed as • an = b(n) * an-1 + d(n) for b and d are functions of n or constants. • In general the linear recurrence equation is of the form an= b1(n)*an-1+b2(n)*an-2 + …..b0(n)*a0+ d(n) Recurrence Relations

First order linear homogenous recurrence equation • A first order linear recurrence equation is called homogenous equation if d(n) = 0. • Examples: • an = n*an-1 • an = an-1+ an-2 • an = nan-1+ n2 Recurrence Relations

Constant coefficient recurrence • A first order linear recurrence relation is constant coefficient recurrence if b(n) is constant. • If d(n) = 0 then the relation becomes homogenous constant coefficient recurrence. • Example: the number of subsets example is a homogenous relation • Are there other examples? Recurrence Relations

Solutions of recurrence equations • A function f is a solution to a recurrence equation if substituting f for an gives a valid equation for all n. • Example: Cn = n(n-1)/2 is a solution to the recurrence relation in example1. • Notice C1 = 1*(1-1)/2 = 1*0/2 = 0 • Now Cn = Cn-1 + n-1 is also obtained since • Cn-1= (n-1)(n-1-1)/2 • Adding n-1 to both sides we get • Cn-1 + n-1 = Cn & • (n-1)(n-2)/2 + (n-1)=(n-1)(n-2)/2 + 2(n-1)/2 = (n2-n)/2 = n(n-1)/2 Recurrence Relations

Solving recurrence Relation • First order linear homogenous constant coefficient recurrence relation • Let an= b*an-1 for n > m and c for n=m or am = c, then an = c*bn-m for n > m. • Proof by induction: • Base step: the statement is true for n=m • Assume it is true for n = k-1, show it is true for n = k. • Now ak-1=c*bk-1-m implies when multiplying by b • b*ak-1 = ak & b*c*bk-1-m= c*bk-m • Thus ak = c*bk-m • What is the solution for relation in example2? • Sn = 2*Sn-1 and S0=1. Thus the solution is sn = 2n since c = 1, m = 0 and b = 2 Recurrence Relations

Solving recurrence Relation • Second order linear homogenous constant coefficient recurrence relation • Substitute in the given relation and simplify to get an equation in terms of r, called the Characteristic Equation of the recurrence relation. • Solve the characteristic equation for its roots let the roots be distinct say r, s. • The solution for the recurrence relation will be: an = b*rn + d*sn where b & d are constants depending on the initial conditions. Recurrence Relations

Example to proof the solution to second order recurrence relation • Solve the recurrence relation: an = c1*an-1+c2*an-2 • The characteristic equation is: tn =c1*tn-1 +c2*tn-2…...** • If the solution of ** is r & s, then the solution to an = c1*an-1+c2*an-2 is an = b*rn + d*sn • Proof: substitute for an, an-1, an-2 their values and verify its correctness Recurrence Relations

Using the characteristic equation for first order relation • an = 2an-1 for n > 0 & 1 for n = 0 • The characteristic equation: rn = 2rn-1 • Finding the roots are rn-1 (r – 2) = 0 • r = 0 and r = 2 • The solution is 2, thus an = c2n • c is constant obtained from the initial condition when n = 0, a0 = c*20, thus c=1 Recurrence Relations

Example on second order recurrence relation • Solve the relation an = an-1 + 2*an-2 for n > 1, 4 for n = 0, and 5 for n = 1. • The characteristic equation is: tn – tn-1 – 2*tn-2 = 0 • The solution for the characteristic equation are t = 2 or -1, thus the solution for the recurrence relation is: an = b*2n + c*(-1)n • If n = 0: 4 = b * 20 + c*(-1)0 • If n = 1: 5 = b * 21 + c*(-1)1 • Solve for b & c we get b = 3 & c = 1 Recurrence Relations

The sum of two solutions is a solution • If r & s are two solutions of a linear recurrence relation, then b*r + c*s is a solution of the relation where b & c are constants with values obtained from the initial conditions. • This can be verified by plug in the values of the solutions in the recurrence relation. Recurrence Relations

One more example on second order recurrence relation • Solve the relation an = 6*an-1 - 9*an-2. • The characteristic equation is: tn - 6tn-1 + 9*tn-2 = 0 Its solution is t = 3 • Is 3n a solution to the relation? Try it. • How about an = b*3n?Try it. How about n*3n is it a solution? How about b*n*3n? • The solution is an = b*3n + c*n*3n where b & c are constants depending on the initial conditions Recurrence Relations

Solving recurrence Relation • Solving general Linear Homogeneous Recurrence Relations with Constant coefficients where s are known constants, and there are K initial conditions where s are known constants. Recurrence Relations

Solving Linear Homogeneous Steps • Substitute in the given relation and simplify to get an equation in terms of r, called the Characteristic Equation of the recurrence relation. • Solve the characteristic equation for its roots. • Foreach different root r of multiplicity m, add (to the solution of an) a new expression of the form • Solve for the unknown constants Ai s from the given initial conditions. Recurrence Relations

Example • Find the solution of an = 3an−1 − 4an−3, n >2, a0 = 0, a1 = 6, a2 = 24 • First, find characteristic equation: 0 = [r3 − 3r2 + 4]*rn-3 • Find the roots =(r − 2)(r − 2)(r + 1) • We get the roots r1 & r2 = 2, r3 = −1 • The solution is: an = A02n+A1n2n+A2(-1)n • Now solve for n = 0, 1,& 2 we get the equations: 0 = A0+A2, 6 = 2A0+2A1-A2, 24 = 4A0+8A1+A2 • Solve for A0, A1, A2: 0, 3, 0. Recurrence Relations

Solution of first order non-constant recurrence • The bijection recurrence bn= n*bn-1 is non-constant recurrence. This relation is similar to the inductive step in the n! definition • With b1=1, we deduce that bn= n! In General • If an= f(n) an-1 for n> m & a1 = c, then an= c* • Proof by induction. Is the solution Recurrence Relations

Solution of first order non-Homogenous recurrence • If an = c*an-1 + f(n), then the solution: an= an(h) + an(p) • Depending on f(n), the particular an(p) solution can be found. The following table shows some cases for f(n) and the corresponding solution. Recurrence Relations

Example on non-Homogenous recurrence relation • Given the following recurrence relation an=2an-1 + 3 • So the particular solution is: an(p)= D • Substitute this back into the recurrence relation and solve for D. • Namely, D = 2D + 3. Thus D = -3. • The solution for the homogenous equation is 2 • If the particular solution is not identical to the homogenous then the solution is: an=c2n-3 Recurrence Relations

A second example • Given an=2an-1+ 2n2 what is its solution? • Let an(p)=c1n2+c2n+c3 be the particular solution, solve for c1, c2, &c3 by substituting the solution in the relation. • C1*n2+c2*n+c3-2*c1*(n-1)2-2*c2*(n-1)-2*c3-2*n2 = 0 • In order for the solution to be independent of n both coefficients of n2 & n must be Zero in addition the constant term should be Zero too this yield to the following set of equations Recurrence Relations

Continue the solution an=2an-1+2n2 • The equations are: • c1-2c1-2=0, • c2+4c1-2c1=0, and • c3-2c1+2c2-2c3 = 0 • Thus C1 = -2, c2 = -8, and c3 =-12 • The particular solution is: • an(p) = -2n2-8n -12 • The homogenous solution is: an(h)= c2n • Thus the solution is: an= an(p)+ an(h) Recurrence Relations

One more example • Let an=2an-1+3*2n. • an(h) = c2n, an(p)=d*2n, but 2 is a root for the characteristic equation corresponding to the homogenous equation, so an(p)shouldbe = d*n*2n. • Solve for d in the initial recurrence relation to obtain d*n*2n=2d(n-1)2(n-1)+32n =dn2n-d2n+32n Thus d = 3 & hence an(p)= 3n2n. • The solution is an= c2n + 3n2n Recurrence Relations

Converting to first order • Let an= c*an/2+n • Is the relation of first order? Second? • Can we solve it? If not can we convert it to some order we know? • Let n = 2k, then n/2 = 2k/2 = 2k-1. Thus • If we let an= tk, then an/2= tk-1 & hence • The relation can be rewritten as: tk = c*tk-1+2k a first order recurrence relation. Recurrence Relations

Recurrence Relations