Scalable group communications and systematic group modeling
This presentation is the property of its rightful owner.
Sponsored Links
1 / 51

Scalable Group Communications and Systematic Group Modeling PowerPoint PPT Presentation


  • 76 Views
  • Uploaded on
  • Presentation posted in: General

Scalable Group Communications and Systematic Group Modeling. Jun-Hong Cui University of Connecticut [email protected] http://www.cse.uconn.edu/~jcui. Cool Application 1 : Teleconferencing. Cool Application 2 : Telemedicine. Cool Application 3 : Net Gaming. Multicast: What and How?.

Download Presentation

Scalable Group Communications and Systematic Group Modeling

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Scalable group communications and systematic group modeling

Scalable Group Communications and Systematic Group Modeling

Jun-Hong Cui

University of Connecticut

[email protected]

http://www.cse.uconn.edu/~jcui


Cool application 1 teleconferencing

Cool Application 1 : Teleconferencing


Scalable group communications and systematic group modeling

Cool Application 2 : Telemedicine


Scalable group communications and systematic group modeling

Cool Application 3 : Net Gaming


Multicast what and how

Multicast: What and How?

  • Multicast:

    • One to many or many to many communications (group communications)

  • To achieve multicast:

    • Multiple unicast (one to one)

    • Network multicast---IP multicast

    • Overlay multicast (using proxies)

    • Application layer multicast (end host)


Outline of this talk

Outline of this talk

  • Scalable Group Communications

    --- Aggregated Multicast

  • Systematic Group Modeling

    --- GEM Model

  • Research Directions


Ip multicast

group

NHop

g1

Ab, A3

Domain B

A2

B1

Domain A

Ab

Aa

A3

A1

Domain C

X1

Y1

C1

Domain X

Domain Y

D1

Customer Networks, Domain D

IP Multicast

  • Group: IP D class address

  • Use Tree delivery structure

  • Routers: keep forwarding entries per-group/source (multicast state)

  • IP multicast

    • Resource efficient

    • Scalable to group size


The problem not scalable to the number of groups

group

NHop

Domain B

g1

Ab, A3

A2

B1

g2

Domain A

Ab, A3

Ab

Aa

A3

A1

X1

Domain C

Y1

C1

Domain X

Domain Y

D1

Customer Networks, Domain D

The Problem: Not Scalable to the Number of Groups

  • More groups  more trees

    • More forwarding entries

    • More tree maintenance overhead

  • IP multicast NOT scalable to the number of groups

    • State Scalability problem

    • Serious in transit domains

  • Our solution

    • Aggregated multicast to improve state scalability


Key insight

group

NHop

Domain B

g1

Ab, A3

A2

B1

g2

Domain A

Ab, A3

Ab

Aa

A3

A1

X1

Domain C

Y1

C1

Domain X

Domain Y

D1

Customer Networks, Domain D

Key Insight

  • There are many overlaps among multicast trees in transit domains


Aggregated multicast

Tree

NHop

Domain B

T1

Ab, A3

A2

B1

Domain A

Ab

Aa

A3

A1

X1

Domain C

Y1

C1

Domain X

Domain Y

D1

Customer Networks, Domain D

Aggregated Multicast

  • Key idea:

    • Force multiple groups share a single delivery tree (aggregated tree)

  • Benefits:

    • Reduce state at core routers

    • Reduce tree maintenance overhead

    • Push complexity to edge

  • Target:

    • Multicast provisioning in transit domains


Aggregated multicast cont

Tree

NHop

Domain B

T1

Ab, A3

A2

B1

Domain A

De-aggregation

Ab

Aa

A3

De-aggregation

Aggregation

A1

X1

Domain C

Y1

C1

Domain X

Domain Y

D1

Customer Networks, Domain D

Aggregated Multicast (cont.)

  • Core routers:

    • Keep state per-tree

  • Edge routers:

    • Keep group state

  • Groups:

    • Aggregate at incoming edge router

    • De-aggregate at outgoing edge routers


Perfect match vs leaky match

Tree

NHop

Domain B

T1

Ab, A3

A2

B1

Domain A

Ab

DiscardPackets

Aa

A3

A1

X1

Domain C

Y1

C1

Domain X

Domain Y

D1

Customer Networks, Domain D

Perfect Match vs. Leaky Match

  • Group-Tree match

    • Perfect match

    • Leaky match

  • Bandwidth waste in leaky match

    • Data delivery to non-member nodes


Aggregation control

Aggregation Control

  • Leaky match

    • Good for tree aggregation

    • But waste bandwidth

  • There is a trade-off

  • Static group-tree matching: NP hard

  • A dynamic group-tree matching algorithm to control the trade-off

    • Under a given bandwidth waste threshold


Group tree matching

Group-Tree Matching

Domain B

Domain E

E1

A2

B1

Domain A

A4

Ab

Aa

A3

A1

X1

Domain C

Y1

C1

Domain X

Domain Y

D1

Customer Networks, Domain D


Group tree matching1

Domain B

Domain E

E1

A2

B1

Domain A

A4

Ab

Aa

A3

A1

X1

Domain C

Y1

C1

Domain X

Domain Y

D1

Customer Networks, Domain D

Group-Tree Matching


Implementation issues

Implementation Issues

  • Multiplex multiple groups over a shared tree

    • IP encapsulation

    • MPLS (Multi-Protocol Label Switching)

  • Tree management and group-tree matching

    • Tree Manager (need to know group membership)

    • Distributed or centralized solutions

  • Have designed and implemented protocols:

    • ASSM for source specific multicast (SSM)

    • BEAM for shared tree multicast (ASM)

    • AQoSM for QoS multicast provisioning


Extend to overlay and adhoc net

Extend to Overlay and Adhoc Net

  • Overlay multicast

    • Implement multicast in overlay net

      • A collection of proxies (or gateways)

    • Processing power, memory & bandwidth more critical

    • Aggregated multicast reduces management overhead

  • Wireless multicast

    • Implement multicast in wireless adhoc net

      • No infrastructure, self-organized

    • Energy, memory, bandwidth, resilience very critical

    • Aggregated trees help to improve performance


Overlay network

Overlay Network


Adhoc network

Adhoc Network


Outline of this talk1

Outline of this talk

  • Scalable Group Communications

    --- Aggregated Multicast

  • Systematic Group Modeling

    --- GEM Model

  • Research Directions


The problem group modeling

The Problem: Group Modeling

  • The locations of the group members

    • Given a graph, where should we place them?

  • Current assumptions: uniform random model (unproven)

    • All members uniformly distributed

    • Not realistic for many applications


Group modeling is critical

Group Modeling is Critical

  • Some studies have shown the locations of members have significant effects on

    • Scaling properties of multicast trees

    • Aggregatability of multicast state

    • Performance of state reduction schemes

  • Realistic group models

    • Improve effectiveness of simulation

    • Guide the design of protocols


Our contributions

Our Contributions

  • Measure real group membership properties

    • MBONE (IETF/NASA) and Netgames (Quake)

  • Design a model to generate realistic membership

    • GEneralized Membership Model (GEM)

    • Use Maximum Enthropy: a statistical method


Roadmap

Roadmap

  • Membership Characteristics

  • Measurement and Analysis Results

  • Model Design and Validation


Beyond uniform random model

Beyond Uniform Random Model

  • How close are the members in a group?

  • Are all the members same in group participation?

  • What are the correlations between members in group participation?


An illustration teleconference

Member Router

Edge Router

An Illustration (Teleconference)

Seattle

0.7

Boston

0.5

Internet

1.0

Atlanta

0.4

Los Angeles

0.5


Membership characteristics

Membership Characteristics

  • Member clustering

    • Capture proximity of group members

    • Use network-aware clustering method

  • Group participation probability

    • Show difference among members/clusters

  • Pairwise correlation in group participation

    • Capture joint probability of two members/clusters

    • Use correlation coefficient (normalized covariance)


Measure membership properties

Measure Membership Properties

  • MBONE applications (from UCSB)

    • IETF-43 (Audio and Video, Dec. 1998)

    • NASA Shuttle Launch (Feb. 1999)

    • Cumulative data sets on MBONE (1997-1999)

  • Net Games (using QStat)

    • Quake I (query master server)

    • Choose 10 most popular servers (May. 2002)

  • Examine three membership properties


Member clustering

Member Clustering

MBONE cumulative data sets

(3, 0.64)

MBONE real data sets

Net game data sets

CDF of cluster size for MBONE and net games


Group participation probability

Group Participation Probability

CDF of participation probability for Net Game data sets


Group participation probability1

Group Participation Probability

CDF of participation probability for MBONE applications


Pairwise correlation in group participation

Pairwise Correlation in Group Participation

CDF of correlation coefficient for Net Game data sets


Pairwise correlation in group participation1

Pairwise Correlation in Group Participation

CDF of correlation coefficient for MBONE applications


Generalized membership model gem an overview

Generalized Membership Model--- GEM (An Overview)

Network topology

Cluster method

Group behavior

Distr. of participation prob.

Distr. of pairwise correlation

Distr. of member cluster size

Inputs

1. Create clusters in given topology

2. Select clusters as member clusters

According to input distributions

3. Choose nodes for each member clusters

GEM

Desired number of multicast groups

that follow the given distributions

Outputs


Member distribution generation

Member Distribution Generation

  • Definition:

    K clusters: C1 , C2 , … , Ci , … , CK

    Prob. pi for any i in [1, K]

    Joint prob. pi,jfor any i, j in [1, K]

    X=(X1 ,X2 , … , Xi , … , Xk): Xiis a binary indicator

    Xi = 1 if Ci is in the group

    Xi = 0 if Ci is not in the group

  • Objective:

    Generate vectors x=(x1 , x2 , … , xk) satisfying

    P(Xi = 1) = pi and P(Xi = 1 , Xj = 1) = pi,j


Maximum entropy method

Maximum Entropy Method

  • To calculate the distribution of (X1,X2, …, Xk) requires O(2K) constraints

  • But we only know O(K+K2) constraints

  • We use Maximum Entropy Method

    • Entropy is a measure of randomness

    • We construct a maximum entropy distr. p*(x)

      • Satisfy constraints in specified dimensions

      • Keep as random as possible in unconstrained dimensions

      • i.e. maximize entropy while match given constraints


Three cases

Three Cases

Considering P(Xi=1)= pi and P(Xi=1, Xj=1)=pi,j

  • Uniform distr. without correlation (easy)

    pi,j = pi * pj , and pi = pj

  • Non-uniform distr. without correlation (easy)

    pi,j = pi * pj , but pi = pj not necessary

  • Non-uniform distr. with pairwise correlation

    Neither pi,j = pi * pjnor pi = pj necessary

    Need to calculate the maximum entropy distr. p*(x)

    Entropy decreases from case 1 to case 3


Experimental validation

Experimental Validation

  • Consider all membership properties

  • Consider three cases

  • Figures omitted …

  • Our experiments show

    • GEM can regenerate groups satisfying given distributions (from real measurement)


Summary

Summary

  • Uniform random model

    • Can capture net games approximately

    • But not realistic for MBONE applications

  • GEM: a generalized membership model

    • Three cases (case 1: uniform random model)

    • Realistic membership can be regenerated

  • Beyond multicast

    • Peer-to-peer network modeling

  • Beyond wired network

    • Wireless adhoc networks, sensor networks …


Outline of this talk2

Outline of this talk

  • Scalable Group Communications

    --- Aggregated Multicast

  • Systematic Group Modeling

    --- GEM Model

  • Research Directions


Networking expanding visions

Networking: Expanding Visions

(from Jim Kurose)


Peer to peer networking

Peer-to-Peer Networking


Peer to peer networking1

Peer-to-Peer Networking

Focus at the application level


Applications challenges

Applications & Challenges

  • Applications

    • P2P file sharing (Napster, Gnutella, Freenet, etc.)

    • Application-layer multicast

  • Characteristics

    • each node potentially same responsibility, functionality

    • logical connectivity rather than physical connectivity

  • Why P2P?

    • High resource utilization (bandwidth, memory, CPU)

  • Challenges

    • Self-organized and large scale (routing)

    • Reliability and security


Research directions

Research Directions

  • Overlay multicast

    • Scalability, QoS, security, pricing, …

  • Multicast modeling

    • Systematic multicast evaluation

  • Peer-to-peer networks

    • measurement & modeling, complex queries

  • Wireless adhoc networks

    • Mobility modeling, scalable multicast

  • Sensor networks

    • Sensor deployment and security

    • Very large scale sensor network design


Scalable group communications and systematic group modeling

Questions?

[email protected]

http://www.cse.uconn.edu/~jcui


Scalable group communications and systematic group modeling

THANKS!!!


Network characteristics

Network Characteristics

  • No fixed infrastructure, instantly deployable

  • Node portability, mobility

  • Error-prone channel

  • Limited resources

    • bandwidth, energy supply, memory and CPU.

  • Heterogeneous nodes

    • big/small; fast/slow etc

  • Heterogeneous traffic

    • voice, image, video, data

  • Wireless multihop connection

    • to save power, overcome obstacles, enhance spatial spectrum reuse, etc


Calculate the maximum entropy distribution

The maximum entropy distr. p*(x) is the solution for:

Subject to

and

and

Calculate the Maximum Entropy Distribution

Use lagrange multipliers and numerical method to construct p*(x), Then Gibbs Sampler to sample it


Group participation probability2

Group Participation Probability

Participation probability distribution for IETF43-Video


Pairwise correlation in group participation2

Pairwise Correlation in Group Participation

Joint probability distribution for IETF43-Video


  • Login