Networking Seminar ECE@OSU

1. Networking Seminar ECE@OSU Network Information Flow R. Ahlswede, N. Cai, S.-Y. R. Li, and R. W. Yeung. Network Information Flow. IEEE Transactions on Information Theory, 46(4):1204–1216, 2000. (Referred to as this paper throughout the presentation) Presented by Bo Ji (ji@ece.osu.edu) 11-28-2007

2. Outline • Introduction • Max-Flow Min-Cut Theorem • Main Result • Simple optimal codes • Multiple sources case • Discussion and Conclusion Some of the slides are from Prof. Chih-Chun Wang@Purdue & Communications and Multimedia Laboratory@NTU

3. Outline • Introduction • Max-Flow Min-Cut Theorem • Main Result • Simple optimal codes • Multiple sources case • Discussion and Conclusion

5. Introduction • In existing computer networks, each node functions as a switch either relays or replicates information received. (Store and forward) • However, from the information-theoretic point of view, there is no reason to restrict the function of a node to that of a switch. • Rather, a node can function as an encoder in the sense that it receives information from all the input links, encodes, and sends information to all the output links. (Store, code and forward) • We refer to coding at a node in a network as network coding. • Basic idea of network coding - Coding at intermediate nodes

6. Introduction • Contributions: • Introduce a new class of problems called network information flow. • Obtained a simple characterization of the admissible coding rate region for single-source problem. (Max-Flow Min-Cut Theorem for network information flow) • Show that • the traditional technique for multicasting in a computer network in general is not optimal. • Rather, we should think of information as being “diffused” through the network from the source to the sinks by means of network coding. • Show that very simple optimal codes do exist for certain networks. (Actually, Li, Yeung and Cai [1] have devised a systematic procedure to construct linear codes for acyclic networks. ) [1] Ref: S. Y. R. Li, R. W. Yeung, and N. Cai. Linear Network Coding. IEEE Transactions on Information Theory, 49(2):371 – 381, 2003.

7. Outline • Introduction • Max-Flow Min-Cut Theorem • Main Result • Simple optimal codes • Multiple sources case • Discussion and Conclusion

8. Max-Flow Problem

9. Max-Flow Problem Ford, Fulkerson

10. Notations

11. Properties of Flow

12. Max-Flow

13. The Capacity of A Cut

14. Min-Cut • The cut with the minimum value of capacity in G.

15. Example

16. Example

17. Example

18. Example

19. Max-Flow Min-Cut Theorem

20. Max-Flow Min-Cut Theorem • Unicast Case: • MinCut Bound, Menger’s Theorem and Path Packing. • Broadcast Case: • Edmond’s Theorem and Spanning Tree Packing. • Multicast Case: • Steiner Tree Packing and Non-Achievability of MinCut Bound. • Unlike the broadcast case, the upper bound may not be achievable by routing information through a set of edge-disjoint trees. Is there any other way being able to achieve the upper bound?

21. A single-level diversity coding system A single-level diversity coding system The graph representing the coding system Let X1 ,…, Xi be mutually independent information sources and the information rate of X1 ,…,Xi is denoted by h1 ,…, hi. We drop the subscripts of X1 and h1, since there is only one-single source. Let ri be the coding rate of encoder i. In order for a decoder to reconstruct X, it is necessary that the sum of the coding rates of the encoder accessible by this decoder is at least h.

22. Max-Flow in Multicast

23. Max-Flow Min-Cut Theorem • Conjecture 1: Let G = (V,E) be a graph with source s and sinks t1 ,……,tL, and the capacity of an edge (i, j) be denoted by Rij. Then (R,h,G) is admissible if and only if the values of a max-flow from s to tl , l = 1,……,Lare greater than or equal to h, the rate of the information source. • The spirit of this conjecture resembles that of the celebrated Max-flow Min-cut Theorem in graph theory[2]. • Illustrate Conjecture 1by a few examples [2] Ref : B. Bollobas, Graph Theory, An Introductory Course. New York: Springer-Verlag, 1979.

24. A one-source one-sink network(L=1) Capacity Max-Flow

25. A one-source two-sink network without coding (L=2) h ≤ min(5,6)

26. A one-source two-sink network with coding (L=2)

27. A one-source three-sink network(L=3) • Quantified advantage: • Save bandwidth: 10% • Increase throughput: 1/3

28. Outline • Introduction • Max-Flow Min-Cut Theorem • Main Result • Simple optimal codes • Multiple sources case • Discussion and Conclusion

29. Main Result • Let us consider a block code of length n. • We assume that x, the value assumed by X, is obtained by selecting an index from a set Ω with uniform distribution. • The elements in Ωare called messages. • For (i,j) єE, node i can send information to node j which depends only on the information previously received by node i. • The paper confines its discussion to a class of block codes, called the α-code.

30. α-Code • An on a graph G is defined by the following components:

31. Construction of α-Code • At the beginning of the coding session, the value of X is available to node s. • In the coding session, there are K transactions which take place in chronological order, where each transaction refers to a node sending information to another node. • In the kth transaction, node u(k) encodes according to fk and sends an index in Ak to node v(k) . The domain of fk is the information received by node u(k) so far, and we distinguish two cases: • If u(k) = s , the domain of fk is Ω. • If u(k) ≠ s , Qk gives the indices of all previous transactions for which information was sent to node u(k) , so the domain of fk is ∏k’є QkAk’ • The set Tij gives the indices of all transactions for which information is sent from node i to node j, so ηij is the number of possible index-tuples that can be sent from node i to node j during the coding session. • Finally, Wlgives the indices of all transactions for which information is sent to tl, and gl is the decoding function at tl.

32. Theorem 1 • Theorem 1:

33. Outline • Introduction • Max-Flow Min-Cut Theorem • Main Result • Simple optimal codes • Multiple sources case • Discussion and Conclusion

34. An Example Show that very simple optimal codes do exist for certain cyclic networks. (A kind of convolutional code)

35. A Simple Optimal Coding Scheme • Show a coding scheme that can multicast {x0(k), x1(k), x2(k)} from the source to the sinks. • xl(k) = 0 for k ≤ 0. At time k ≥ 1, information transactions occur in the following order:

36. Outline • Introduction • Max-Flow Min-Cut Theorem • Main Result • Simple optimal codes • Multiple sources case • Discussion and Conclusion

37. Multiple Sources The multisource problem is not a trivial extension of the single-source problem, and it is extremely difficult in general. A multilevel diversity coding system The graph G representing the coding system

38. Multiple Sources • Assume the sources X1 and X2 are independent, and h1=h2=1. • For any admissible coding rate triple (r1,r2,r3), for i = 1,2,3, we can write where are the subrates associated with sources X1 and X2 respectively. • X1 is multicast to all the decoders, and X2 is multicast to Decoder 2,3,4. So we have following constraints:

39. Multiple Sources • [3] shows that the rate triple (1,1,1) is admissible, but it cannot be decomposed into two sets of subrates as prescribed above. Therefore, coding by superposition is not optimal in general, even when the two information sources are generated at the same node. • How to characterize multilevel diversity coding systems for which coding by superposition is always optimal is still an open problem. • Although the multisource problem in general is extremely difficult, there exist special cases which can be readily solved by the results for the single-source problem (Such as video-conferencing scenario). [3] Ref: R.W. Yeung, “Multilevel diversity coding with distortion,” IEEE Trans. Inform. Theory, vol. 41, pp. 412–422, Mar. 1995.

40. Outline • Introduction • Max-Flow Min-Cut Theorem • Main Result • Simple optimal codes • Multiple sources case • Discussion and Conclusion

41. Conclusion • have characterized the admissible coding rate region of the single-source problem. • result can be regarded as the Max-flow Min-cut Theorem for network information flow. • the discussion is based on a class of block codes called α-codes. Therefore, it is possible that the result can be enhanced by considering more general coding schemes. • prove that probabilistic coding does not improve performance. • The problem becomes more complicated when there are more than one source.

42. Conclusion • The most important contribution of this paper is to show that • the traditional technique for multicasting in a computer network in general is not optimal. • Rather, we should think of information as being “diffused” through the network from the source to the sinks by means of network coding. • This is a new concept in multicasting in a point-to-point network which may have significant impact on future design of switching systems.

43. Interesting Problems • by imposing the constraint that network coding is not allowed, i.e., each node functions as a switch in existing computer networks, whether a rate tuple R is admissible? • under what condition can optimality be achieved without network coding? • Besides, the problem of representing codes in graphs has received much attention. • ……

44. Discussion • Question: • How to identify when network coding solution is needed and feasible?

45. Two Simple Unicast Sessions Goal: X1: S1  t1 X2: S2  t2 When can we send X1 and X2 simutaneously? (2-2 case) Ref: C. Wang and N B. Shroff, " Beyond the Butterfly - A Graph-Theoretic Characterization of the Feasibility of Network Coding with Two Simple Unicast Sessions, " IEEE International Symposium on Information Theory, Nice, France, June 2007.