15-251

1. 15-251 Great Theoretical Ideas in Computer Science

2. X1 • X2 • + • + • X3 Counting III: Generating Functions Lecture 8 (September 17, 2009)

3. Recap

4. The Binomial Formula (1+X)0 = 1 (1+X)1 = 1 + 1X (1+X)2 = 1 + 2X + 1X2 (1+X)3 = 1 + 3X + 3X2 + 1X3 (1+X)4 = 1 + 4X + 6X2 + 4X3 + 1X4 Pascal’s Triangle: kth row are coefficients of (1+X)k Inductive definition of kth entry of nth row:Pascal(n,0) = Pascal (n,n) = 1; Pascal(n,k) = Pascal(n-1,k-1) + Pascal(n-1,k)

5. Pascal’s Triangle “It is extraordinary how fertile in properties the triangle is. Everyone can try his hand” 1 1 1 1 2 1 1 3 3 1 1 4 6 4 1 1 5 10 10 5 1 1 6 15 20 15 6 1

6. n  i k i = k n+1 k+1 Summing on kth Avenue 1 1 1 1 2 1 1 3 3 1 1 4 6 4 1 1 5 10 10 5 1 1 6 15 20 15 6 1 =

7. Fibonacci Numbers 1 1 1 1 2 1 1 3 3 1 1 4 6 4 1 1 5 10 10 5 1 1 6 15 20 15 6 1 = 2 = 3 = 5 = 8 = 13

8. n 2  2n n n k k = 0 Sums of Squares = 1 1 1 1 2 1 1 3 3 1 1 4 6 4 1 1 5 10 10 5 1 1 6 15 20 15 6 1 2 2 2 2 2 2 2

9. All these properties can be proved inductively and algebraically. We gave combinatorial proofs using the Manhattan block walking representation of binomial coefficients

10. 10 5 How many shortest routes from A to B? A B

11. j+k k Manhattan 0 0 1 1 jth street kth avenue 2 2 3 3 4 4 There are shortest routes from (0,0) to (j,k)

12. n k Manhattan 0 0 1 Level n kth avenue 1 2 2 3 3 4 4 There are shortest routes from (0,0) to (n-k,k)

13. n k Manhattan 0 0 1 Level n kth avenue 1 2 2 3 3 4 4 shortest routes from (0,0) to There are level n and kth avenue

14. n-1 n-1 n k-1 k k 0 0 1 Level n kth avenue 1 2 2 3 3 4 4 = +

15. 2 n k 0 0 1 Level n kth avenue 1 2 2 3 3 4 4 How many ways to get to via ?

16. n 2  2n n n k k = 0 0 0 1 Level n kth avenue 1 2 2 3 3 4 4 =

17. 0 0 1 Level n kth avenue 1 2 2 3 3 4 4 How many ways to get to via ? i n+1 k+1 k

18. n n+1  i = k+1 k i = k 0 0 1 Level n kth avenue 1 2 2 3 3 4 4 n+1 k+1

19. Xn – 1 1 - Xn X - 1 1 - X 1 + X1 + X2 + X3 + … + Xn-2 + Xn-1 = = Recall the Geometric Series

20. 1 1 - X 1 + X1 + X2 + X3 + … + Xn + … = the Infinite Geometric Series Holds when |X| < 1 Also makes sense if we viewthe infinite sum on the left as a formal power series

21. 1 1 - X P(X) = 1 + X1 + X2 + X3 + … + Xn + … - X1 - X2 - X3 - … - Xn - Xn+1 - … -X * P(X) = (1- X) P(X) = 1  P(X) =

22.   P(X) = aiXi i = 0 Formal Power Series There are no worries about convergence issues. This is a purely syntactic object.

23.   P(X) = aiXi i = 0 Formal Power Series If you want, think of as the infinite vector V = < a0, a1, a2, ..., an, … > But, as you will see, thinking of as a “polynomial” is very natural.

24. Operations A(X) = a0 + a1 X + a2 X2 + … B(X) = b0 + b1 X + b2 X2 + … adding them together (A+B)(X) = (a0+b0) + (a1+b1) X + (a2+b2) X2 + … like adding the vectors position-wise <4,2,3,…> + <5,1,1,….> = <9,3,4,…>

25. Operations A(X) = a0 X0 + a1 X1 + a2 X2 + … multiplying by X X * A(X) = 0 X0 + a0 X1 + a1 X2 + a2 X3 + … like shifting the vector entries SHIFT<4,2,3,…> = <0,4,2,3,…>

26. Example Example: Store: V := <1,0,0,…>; Loop n times V := V + SHIFT(V); V = <1,0,0,0,…> V = <1,1,0,0,…> V = <1,2,1,0,…> V = <1,3,3,1,…> V = nth row of Pascal’s triangle

27. Example Example: V := <1,0,0,…>; Loop n times V := V + SHIFT(V); PV := 1; PV := PV(1+X); V = nth row of Pascal’s triangle

28. Example Example: V := <1,0,0,…>; Loop n times V := V + SHIFT(V); PV = (1+ X)n V = nth row of Pascal’s triangle As expected, the coefficients of PV give the nth row of Pascal’s triangle

29.   P(X) = aiXi i = 0 To summarize… Given a sequence V = < a0, a1, a2, ..., an, … > associate a formal power series with it This is the “generating function” for V

30. Fibonaccis F0 = 0, F1 = 1, Fn = Fn-1 + Fn-2 i.e., the sequence <0,1,1,2,3,5,8,13…> is represented by the power series 0 + 1X1 + 1X2 + 2 X3 + 3 X4 + 5 X5 + 8 X6 +…

31. Two Representations F0 = 0, F1 = 1, Fn = Fn-1 + Fn-2 A(X) = 0 + 1X1 + 1X2 + 2 X3 + 3 X4 + 5 X5 + 8 X6 +… Can we write A(X) more succinctly?

32. A(X) = F0 + F1 X1 + F2 X2 + F3 X3 + … + Fn Xn +… X A(X) = 0 + F0 X1 + F1 X2 + F2 X3 + … + Fn-1 Xn +… X2 A(X) = 0 + 0 X1 + F0 X2 + F1 X3 + … + Fn-2 Xn +… (1 – X – X2) A(X) (F0 – 0 – 0) = + (F1 – F0 – 0)X1 + 0 = X X A(X) = (1 – X – X2)

33. Fibonaccis F0 = 0, F1 = 1, Fn = Fn-1 + Fn-2 has the generating function X A(X) = (1 – X – X2) i.e., the coefficient of Xn in A(X) is Fn

34. X + X2 + 2X3 + 3X4 + 5X5 + 8X6 1 – X – X2 X -(X – X2 – X3) X2 + X3 -(X2 – X3 – X4) 2X3 + X4 -(2X3 – 2X4 – 2X5) 3X4 + 2X5 -(3X4 – 3X5 – 3X6) 5X5 + 3X6 8X6 + 5X7 -(5X5 – 5X6 – 5X7) -(8X6 – 8X7 – 8X8)

35. Two representations of the same thing… X F0 = 0, F1 = 1, Fn = Fn-1 + Fn-2 A(X) = (1 – X – X2)

36. where φ = 1 + √5 2 Closed form? X F0 = 0, F1 = 1, Fn = Fn-1 + Fn-2 A(X) = (1 – X – X2) let’s factor (1 – X – X2) (1 – X – X2) = (1 - φX)(1 – (-1/φ)X)

37. -1 1 √5 √5 Simplify, simplify… X F0 = 0, F1 = 1, Fn = Fn-1 + Fn-2 A(X) = (1 - φX)(1 – (-1/φ)X) some elementary algebra omitted…* 1 1 A(X) = + (1 – (-1/φ)X) (1 - φX) *you are not allowed to say this in your answers…

38. 1 1 - Y -1 1 √5 √5 1 1 A(X) = + (1 – (-1/φ)X) (1 - φX) 1 = 1 + φ X + φ2 X2 + … + φn Xn + … (1 - φX) = 1 + Y1 + Y2 + Y3 + … + Yn + … the Infinite Geometric Series

39. 1 -1 -1 1 √5 √5 √5 √5 1 1 A(X) = + (1 – (-1/φ)X) (1 - φX) 1 = 1 + φ X + φ2 X2 + … + φn Xn + … (1 - φX) 1 = 1 + (-1/φ) X + … + (-1/φ)n Xn + … (1 – (-1/φ)X)  the coefficient of Xn in A(X) is… φn (-1/φ)n +

40. 1 -1 where φ = 1 + √5 √5 √5 2 Closed form for Fibonaccis Fn = φn (-1/φ)n + “golden ratio” Abraham de Moivre (1730) Leonhard Euler (1765) J.P.M. Binet (1843)

41. 1 -1 1 √5 √5 √5 Closed form for Fibonaccis Fn = φn (-1/φ)n + Fn = φn closest integer to

43.   P(X) = aiXi i = 0 To recap… Given a sequence V = < a0, a1, a2, ..., an, … > associate a formal power series with it This is the “generating function” for V We just used this for the Fibonaccis…

44. Multiplication A(X) = a0 + a1 X + a2 X2 + … B(X) = b0 + b1 X + b2 X2 + … multiply them together (A*B)(X) = (a0*b0) + (a0b1 + a1b0) X + (a0b2 + a1b1 + a2b0) X2 + (a0b3 + a1b2 + a2b1 + a3b0 ) X3 + … seems a bit less natural in the vector representation (it’s called a “convolution” there)

45. Mult.: special case 1 = 1-X A(X) = a0 + a1 X + a2 X2 + … Special case: B(X) = 1 + X + X2 + … multiply them together (A*B)(X) = a0 + (a0 + a1) X + (a0 + a1 + a2) X2 + (a0 + a1 + a2 + a3) X3 + … it gives us partial sums!

46. 1 1 1 1 1 For example… = = * = = 1-X 1-X 1-X 1-X (1-X)2 Suppose A(X) = 1 + X + X2 + … and B(X) = 1 + X + X2 + … then (A*B)(X) = 1 + 2X + 3X2 + 4X3 + … Generating function for the sequence <0,1,2,3,4…>

47. What happens if we again take prefix sums? 1 1 = = (1-X)2 (1-X)3 Take 1 + 2X + 3X2 + 4X3 + … multiplying through by 1/(1-X) ¢1 + ¢2X1 + ¢3X2 + ¢4X3 +… Generating function for the triangular numbers!

48. What’s the pattern? 1 1 1 1 = = = = 1-X (1-X)n (1-X)2 (1-X)3 <1,1,1,1,…> <1,2,3,4,…> <¢1,¢2,¢3,¢4,…> ???

49. What’s the pattern? 1 1 1 1 = = = = 0 1-X (1-X)n (1-X)3 (1-X)2 0 1 , , , , … 0 2 <1,2,3,4,…> 0 3 <¢1,¢2,¢3,¢4,…> 0 ???

50. What’s the pattern? 1 1 1 1 = = = = 1 0 1-X (1-X)n (1-X)2 (1-X)3 1 0 1 2 , , , , … 1 0 2 3 , , , , … 1 0 3 4 <¢1,¢2,¢3,¢4,…> 0 1 ???