Scattercast: Taming Internet Multicast

1. Scattercast: Taming Internet Multicast 02/08/2001

2. The Problem • Can the Internet today support large-scale Internet broadcasting?  NO • Traditional unicast model does not scale • IP Multicast is not the right solution “Madonna’s London gig broadcast live on the Internet… But as she burst into her first song on Tuesday night, many fans were still queueing up outside the virtual venue, struggling to connect to the live feed.” —CNN News (Nov 29, 2000)

3. Our Solution: 10,000 Foot View • Explicit infrastructure support • Push complexity up the protocol stack • Application-layer broadcasting • Keep network layer simple and robust • Broadcast customization on per-application basis • Allow applications to optimize the broadcasting framework Scattercast: Infrastructure-service-based broadcasting—as opposed to global IP multicast

4. Outline • Motivation • Introduction: The Scattercast Architecture • Gossamer: Application-level Multicast • Semantic Transport: Application-aware customization • Summing up

5. The Problem • Traditional unicast model does not scale • Millions of clients server and network meltdown

6. Traditional solution: IP Multicast • Global broadcast distribution primitive • Source sends single stream • Routers split stream towards all clients • IP Multicast to the rescue

7. Problems with IP Multicast • Complex network protocol • Scaling issues: state explosion and inter-domain routing • No access control • Difficult to manage and debug • Heterogeneity • Single stream cannot satisfy all clients/networks • Different applications have different requirements • Reliable multicast • Much harder than TCP • No scalable loss recovery, congestion control • End-to-end argument fails for multicast • network layer is no longer simple and robust

8. Scattercast: Broadcasting as an Infrastructure Service ScatterCast proXies (SCXs) Application-aware computation Unicast connections Locally scoped multicast groups Infrastructure proxies (SCXs) provide the broadcast service Application-level broadcasting Application-specific customization

9. Benefits of this approach • Localize hard multicast problems • Bandwidth allocation, congestion control, loss recovery are tractable • Simplify network layer via intelligent infrastructure • No inter-domain multicast routing required • Impose access restrictions within SCXs • Leverage well-understood wide-area unicast protocols • Incorporate app-specific semantics within SCXs to address heterogeneity • App-specific reliability and data scheduling • On-the-fly content and bandwidth adaptation

10. New challenges • How do you distribute data efficiently across the infrastructure? • Gossamer: Application Level Multicast • How do you incorporate application-specific intelligence into the distribution infrastructure? • Application-customizable scattercast transport • How do you manage the service and ensure fault tolerance, availability, and scalability of SCXs? • Cluster-based SCX implementation • How do you distribute data efficiently across the infrastructure? • Gossamer: Application Level Multicast • How do you incorporate application-specific intelligence into the distribution infrastructure? • Application-customizable scattercast transport • How do you manage the service and ensure fault tolerance, availability, and scalability of SCXs? • Cluster-based SCX implementation

11. Outline • Motivation • Introduction: The Scattercast Architecture • Gossamer: Application-level Multicast • Semantic Transport: Application-aware customization • Summing up

12. Overview Unicast connections ScatterCast proXies (SCXs) • Source injects data into a session via its local SCX • SCXs dynamically construct overlay network of unicast connections: the mesh • Run DVMRP-style routing on top of this network to construct distribution trees

13. Goals • Minimize latency from source to all receivers • SCXs are not routers  Overlay tree not as optimal as IP multicast • Optimize overlay to “reflect” underlying Internet topology • Limit number of duplicate packets traversing any physical Internet link • Each SCX transmits to handful of nearby SCXs  Restrict degree of each SCX based on its bandwidth capabilities

14. SCX Discovery • Bootstrap using list of well-known rendezvous SCXs • Gossip-style discovery • Pick random SCX Xj; send it our membership list • Xj merges this into its own list • Xj responds with part of its own list • Gradually all SCXs discover each other Summary: well-known rendezvous + gossip to disseminate session membership

15. Mesh Construction • Set up connections with up to k other SCXs • k = degree restriction • Periodically probe a random SCX, Xj • Measure unicast distance to Xj • Use local optimization algorithm to determine suitability for picking as a neighbor • If Xj has better route towards source than a current neighbor, then replace that neighbor with Xj Summary: Local optimization based on unicast distances to choose mesh neighbors

16. Application-level Routing • Variant of distance vector routing • shortest path routing protocol • routing table entries only for source SCXs • to detect loops, store entire path in routing table • Build distribution trees from routing tables • source-rooted trees • reverse shortest path • forward data using reverse path forwarding Summary: Shortest path routing to build source-rooted trees

17. Evaluation • Simulate the Gossamer control protocol • 1000 node topology, approx. 4000 edges • Constructed using gt-itm topology generator • Measure: • Average latency compared to multicast: • Cost Ratio = (avg latency with Gossamer) (avg latency with multicast) • Time to construct stable overlay: • Time for changes in overlay structure to stop • Packet duplication overhead • Number of physical Internet links with multiple copies of same packet

18. Variation of Cost Ratio with Session Size Cost ratio remains low (< 1.9) even as session size increases

19. Time to Stability Most mesh changes occur early on in the protocol

20. Packet Duplication Overhead Most heavily loaded link for Gossamer: 14 copies Most heavily loaded link for unicast: 99 copies Load on physical links is lower for Gossamer than for vanilla unicast

21. Gossamer Summary • Application-level multicast is feasible • Mesh + routing approach results in stable overlay distribution structure • Gossamer is but one approach for application-level multicast

22. Outline • Motivation • Introduction: The Scattercast Architecture • Gossamer: Application-level Multicast • Semantic Transport: Application-aware customization • Summing up

23. Overview Application-aware computation • Different applications have different transport requirements • Reliability, bandwidth management, congestion control, etc. • One-size-fits-all solution will not work • Single data stream cannot satisfy all heterogeneous clients • Our solution: Application-awareTransport Framework • Expose underlying overlay topology to applications • Allow applications to define their own forwarding policies • Delivery of “information” rather than the “representation” of the information

24. An Example Application • Online Presentation • Distribute web pages for on-line presentations • Requires eventual reliability • High-bandwidth image data • Four levels of customization • Customizable data forwarding • Customizable data reliability • Transmission scheduling • Data transformation

25. Customizable Data Forwarding • Expose underlying overlay topology to transport layer • Local view of the distribution tree: upstream link towards source + list of downstream links • Allows applications to build custom transport protocols for reliability, congestion control, etc. • Transmit nacks/acks upstream towards source • Transmit data/retransmissions towards receivers

26. Customizable Data Reliability ignore losses unordered reliable HTML container Image container • Reliability constraints vary • Ordered/unordered delivery, best effort, periodically updating data • Different types of reliability within the same app • Apps define their own reliability policies • Application Data Units (ADUs) • Group related ADUs into containers • e.g. html in one container, images in another • Assign reliability policies to containers • e.g. ignore losses in image containerallow out-of-order delivery of ADUs

27. Customizable Transmission Scheduling • Customized bandwidth management • Buffer data to avoid congestion • Notify upstream SCX to slow down • Prioritize “important” ADUs over others HTML high priority images low priority

28. Customizable Data Transformation • Transform ADUs on the fly • Bandwidth adaptation: discard redundant information • Format conversion:adapt content to suit client devices • Feedback from scheduler drives transformation decisions • e.g. convert images to P-JPEG;prioritize base scan • limit P-JPEG size based on available bandwidth

29. Real Applications Scattercast overlay network Broadcast server • Electronic whiteboard • Shared drawing space • Adaptive reliability • Whiteboard for PalmPilot • Extreme client heterogeneity • Split application: PalmPilot app is simple; smarts in SCX • Streaming MP3 broadcast server • Radio over the Internet • Interface to standard clients e.g. WinAmp

30. Outline • Motivation • Introduction: The Scattercast Architecture • Gossamer: Application-level Multicast • Semantic Transport: Application-aware customization • Summing up

31. Scattercast: Broadcasting as an Infrastructure Service • End-to-end is not the right answer • Use intelligent infrastructure to simplify the network layer • Divide-and-conquer localizes hard problems • Use multicast only in local area where it is tractable; robust unicast across wide-area • Application-level intelligence is crucial • Adapt to heterogeneity by leveraging application hints in transport protocol

32. The Longer Term:Evolving Scattercast Scattercast network Yet-another- broadcast-network IP-multicast network Broadcast Inter-network • Flat scattercast overlay cannot scale • We may have many independent broadcast networks • Solution: build a broadcast inter-network across collections of broadcast networks

33. Come work with us! • AT&T Research: Menlo Park • http://www.research.att.com/labs/mp/irg/ • We are looking for summer interns as well as full time hires