consensus
Download
Skip this Video
Download Presentation
Consensus

Loading in 2 Seconds...

play fullscreen
1 / 59

Consensus - PowerPoint PPT Presentation


  • 90 Views
  • Uploaded on

Consensus. or approximate majority quantile summaries selection problem …. Milan Vojnovic Microsoft Research. Workshop on Performance and Control of Large-Scale Networks Eindhoven, Netherlands, June 30-July 2, 2014. A retro spective talk …. …. Approximate majority. 0. 1. 1. 0. 0.

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about ' Consensus' - jerry-wolfe


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
consensus

Consensus

or

approximate majority

quantile summaries

selection problem

Milan Vojnovic

Microsoft Research

Workshop on Performance and Control of Large-Scale Networks Eindhoven, Netherlands, June 30-July 2, 2014

approximate majority
Approximate majority

0

1

1

0

0

1

1

0

1

Input: each node holds a binary value, either 0 or 1

Output: each node to report the majority vote (with high probability)

Requirement: limited memory per node and pairwise communication between nodes

slide4

0

1

0

0

0

1

0

1

1

slide5

0

1

0

0

0

1

0

1

1

our notation
Our notation

0

1

0

0

0

1

0

1

1

approximate majority algorithms
Approximate majority algorithms

2 states

  • States: 0, 1
  • Convergence time =
  • Probability of error =

3 states

  • States: 0, e, 1
  • Convergence time =
  • Probability of error =

4 states

  • States: 0, e0, e1, 1
  • Convergence time =
  • Probability of error =

= number of nodes, = voting margin

questions of i nterest
Questions of interest

Correctness: probability that each node identifies the initial majority state?

Convergence time: time to reach consensus?

Dependence on the number of nodes voting margin , network structure?

desiderata
Desiderata
  • Reach correct consensus – initial majority
  • Fast convergence
  • Small communication overhead
  • Small processing per node
  • Decentralized
outline
Outline

Related work

3-state algorithm

4-state algorithm

Conclusion

some related work
Some related work

More references in this slide deck

classical voter model hassin peleg 01
Classical voter model[Hassin-Peleg-01]

0

1

1

0

0

1

0

1

  • 0 initially held by nodes, 1 initially held by nodes
  • Complete graph node interactions
  • Probability of incorrect consensus

Node takes over the state of the contacted node

Binary state per node & binary signaling

statistical tests with limited memory information theory 70 s
Statistical tests with limited memory[Information Theory 70’s]

000110111110100011

S

i. i. d. mean

  • How many states S needs to identify the correct hypothesis with probability with the number of observations?
  • m+1necessary and sufficient [Koplowitz, IEEE Trans IT ’75]
quantile summaries greenwald knanna 2004
Quantilesummaries[Greenwald- Knanna-2004]
  • Approximate quantile computation: Input: rank rel. acc. par. Output: element of rank
  • Quantile summaries: max number of data elements communicated by any node

Coordinator

elements

outline1
Outline

Related work

3-state algorithm

4-state algorithm

Conclusion

3 state algorithm
3-state algorithm

0

0

e

1

e

1

0

0

e

e

0

1

  • Both processing and signaling take one of three states
    • 0 or 1 or e
    • e = “indecisive” state
assumptions
Assumptions
  • Interactions: asynchronous continuous-time, complete graphEach node samples another node uniformly at random at instances of a Poisson process with intensity 1
3 state algorithm state evolution
3-state algorithm: state evolution
  • Markov process:

= number of nodes in state 0

= number of nodes in state 1

= total number of nodes

ternary protocol probability of error
Ternary protocol: probability of error
  • = initial point,

Theorem – probability of error:

probability of error c ont d
Probability of error (cont’d)

Corollary – For initial state such that , for , we have, large

Exponentially decreasing in

Correctness with high probability if

proof main ideas
Proof main ideas

First-step analysis:where with the boundary conditions: for for

proof main ideas cont d
Proof main ideas (cont’d)
  • i.e. is the error probability for
  • Lemma – solution of

with the boundary conditions: for , , for

proof main ideas cont d1
Proof main ideas (cont’d)

# of paths from to not intersecting

-- Ballot theorem

convergence time
Convergence time
  • The limit ODE
  • Def: = smallest time such that and are of order given that and

Proof:

convergence time l ower b ound
Convergence time lower bound
  • Lower bound:
  • Example: pathreduction to classical voter model

U

V

1

1

1

1

0

0

0

0

. . .

. . .

convergence time l ower bound cont d
Convergence time lower bound (cont’d)
  • Ternary protocol on a path corresponds to a classical voter model dynamics

1

1

1

0

0

0

0

1/2

1/2

1

1

e

0

0

0

0

1/2

1

1

0

0

0

0

0

extension to plurality problem jung kim v 2012
Extension to plurality problem[Jung-Kim-V.-2012]
  • alternatives
    • Binary consensus as special case:
  • Output: each node to correctly identify a state that is initially a plurality winner
plurality algorithm
Plurality algorithm

observer

m alternatives

2m states: weak strong

state evolution
State evolution

Markov process:

the l imit ode
The limit ODE

For every and

convergence t ime
- convergence time

Given , defined as follows

limit points
Limit points
  • Theorem – Suppose that for and ThenMoreover, we have
limit points cont d
Limit points (cont’d)

The last theorem follows as a corollary of the following claims:

rate of convergence
Rate of convergence

For every non-plurality state

Exponential diminishing of non-plurality states

convergence time1
Convergence time

Theorem: For such that and , there exists a constant such that

Corollary:

Convergence time linear in the number of alternatives*

Logarithmic in the voting margin

* Up to poly-log factors

convergence time lower b ounds cont d
Convergence time lower bounds (cont’d)
  • Theorem: For every there exists an initial state with gap and constant such that for and small enough

Take:

probability of error babace draief 2013
Probability of Error[Babace-Draief-2013]
  • Theorem - suppose that for ,Then
polling algorithm cruise ganesh 2013
Polling algorithm[Cruise-Ganesh-2013]

do:

  • Sample node uniformly at random
  • Sample of m nodes from the population with replacement
  • number of nodes in in state 1
  • If
  • Else if

= number of nodes in in state 1

1

1

1

1

1

sample of nodes

1

polling algorithm cont d cruise ganesh 2013
Polling algorithm (cont’d)[Cruise-Ganesh-2013]
  • Probability of error:
  • Expected convergence time:
outline2
Outline

Related work

3-state algorithm

4-state algorithm

Conclusion

quaternary p rotocol
Quaternary protocol
  • Four states
  • Update rules: swap or annihilate

0

e0

e1

1

0

e0

0

e1

0

1

e0

e1

e0

1

e1

1

e0

0

e0

0

e1

e0

e1

e0

1

e1

1

e1

correctness benezit thiran vetterli 2010
Correctness[Benezit-Thiran-Vetterli-2010]

Corollary - For any given connected graph, the binary interval consensus converges to the correct state with probability 1.

convergence time2
Convergence time

Each edge activated at instances of a Poisson point process of intensity

Contract rate matrix:

Family of matrices: for every non-empty subset of nodes , defined by

eigenvalue gap
Eigenvalue gap

For any finite graph , there exists such that every eigenvalue of matrix satisfies

convergence time3
Convergence time
  • Two phases
    • Phase 1: time until depletion of state 1
    • Phase 2: time until depletion of state 2
  • Theorem:
state evolution in phase 1
State evolution in Phase 1

1 if node i in state 1

1 if node i in state 0

Phase 1

state evolution in phase 1 cont d
State evolution in Phase 1 (cont’d)
  • Probability that a node is in state 1 evolves as
  • System of linear ODEs:, = set of nodes in state 0
  • Bounds on the expected convergence time follow using a spectral bound
complete graph
Complete graph
  • Each edge activate at rate
  • , for
complete graph upper bound is tight
Complete graph: upper bound is tight
  • By direct analysis:where is the -th harmonic number
  • 0 and 1 states annihilate after a random time with exponential distribution with parameter
slide52
Star
  • Each edge activate at rate

, for

  • Tight: by direct analysis
erdos renyi graph
Erdos-Renyi graph
  • Edge (u,v) activated at rate , , for
  • If w.h.p.where is the inverse function of

1

1

conclusion
Conclusion
  • 3-state algorithm
    • Complete graph: correctness with high probability (exponentially decreasing error probability in ), fast convergence
    • Extensions to plurality problem
  • 4-state algorithm
    • Arbitrary connected graph: guaranteed correctness, expected convergence time upper bounds
    • Complete graph: expected convergence time
some open p roblems
Some open problems
  • Lower bounds? - given memory and communication constraints and a probability of error budget, lower bounds for the expected convergence time?
  • Better upper bounds?
  • Tradeoff accuracy-convergence time: dependence on the memory and communication constraints and the network structure?
references
References
  • J. Koplowitz, Necessary and Sufficient Memory Size for m-hypothesis Testing, IEEE Trans. on Information Theory, Vol 21, No 1, 1975
  • M. Greenwald and S. Khanna, Space-efficient Online Computation of Quantile Summaries, ACM SIGMOD 2001
  • Y. Hassin and D. Peleg, Distributed Probabilistic Polling and Applications to Proportionate Agreement, Information and Computation, 171, 2001
  • D. Kempe, J. Kleinberg and E. Tardos, Maximizing Influence through a Social Network, ACM KDD 2003
  • M. Greenwald and S. Khanna, Power-conserving Computation of Order-Statistics over Sensor Networks, ACM PODS 2004
  • T. M. Liggett, Interacting Particle Systems, Springer, 2006
  • S. Boyd, A. Ghosh, B. Prabhakar and D. Shah, Randomized gossip algorithms, IEEE Trans. on Information Theory, Vol 52, No 6, 2006
  • D. Angluin, J. Aspnes, D. Eisenstat, A Simple Population Protocol for Fast Robust Approximate Majority, DISC, 2007
  • F. Kuhn, T. Locher, R. Wattenhofer, Tight Bounds for Distributed Selection, ACM SPAA 2007
references cont d
References (cont’d)
  • W. P. Tay, J. N. Tsitsiklis and M. Z. Win, On the Subexponential Decay of Detection Error Probabilities in Long Tandems, IEEE Trans. on Info. The., Vol 54, No 10, 2008
  • A. Nedic, A. Olshevsky, A. Ozdaglar and J. N. Tsitsiklis, Distributed Averaging Algorithms and Quantization Effects, IEEE Conf. on Decision and Control, 2008
  • F. Benzit, P. Thiran and M. Vetterli, Interval Consensus: From Quantized Gossip to Voting, IEEE Int’l Conf. on Acoustics, Speech, and Signal Processing, 2009
  • E. Perron, D. Vasudevan, M. V., Using Three States for Binary Consensus on Complete Graphs, IEEE Infocom 2009
  • J. Cruise and A. Ganesh, Probabilistic Consensus via Polling and Majority Rules, Proc. of Allerton Conference, 2010
  • D. Acemoglu, M. A. Dahleh, I. Lobel and A. Ozdaglar, Bayesian Learning in Social Networks, forthcoming Review of Economic Studies, 2011
  • F. Benezit, P. Thiran and M. Vetterli, The Distributed Multiple Voting Problem, IEEE Journal on Selected Topics in Signal Processing, Vol 5, No. 4, 2011
  • M. Draief and M. V., Convergence Speed of Binary Interval Consensus, SIAM J. Control Optim., vol 50, pp 1087-1109
references cont d1
References (cont’d)
  • E. Mossel, J. Neeman and O. Tamuz, Majority Dynamics and Aggregation of Information in Social Networks, 2012
  • F. Chierichetti and J. Kleinberg, Voting with Limited Information and Many Alternatives, ACM SODA 2012
  • M. A. Abdullah and M. Draief, Majority Consensus on Random Graphs of a Given Degree Sequence, ArXiv, 2012
  • K. Jung, B. Y. Kim, M. V., Distributed Ranking in Networks with Limited Memory and Communication, IEEE ISIT 2012
  • S. Shang, P. W. Cuff, S. R. Kulkarniand P. Hui, An Upper Bound on the Convergence Time for Distributed Binary Consensus, 15th Int’l Conf. on Information Fusion, 2012
  • Z. Huang, K. Yi, and Q. Zhang, Randomized Algorithms for Tracking Distributed Count, Frequencies and Ranks, ACM PODS 2012
  • A. Babaee and M. Draief, Distributed Multivalued Consensus, Computer and Information Sciences III, 2013
references cont d2
References (cont’d)
  • G. B. Mertzios, S. E. Nikoletseas, C. L. Raptopoulos, P. G. Spirakis, Determining Majority in Networks with Local Interactions and very Small Local Memory, ICALP 2014
  • M. Feldman, N. Immorlica, B. Lucier, S. M. Weinberg, Information Aggregation in Social Networks, working paper, 2014
ad