Consensus
This presentation is the property of its rightful owner.
Sponsored Links
1 / 59

Consensus PowerPoint PPT Presentation


  • 71 Views
  • Uploaded on
  • Presentation posted in: General

Consensus. or approximate majority quantile summaries selection problem …. Milan Vojnovic Microsoft Research. Workshop on Performance and Control of Large-Scale Networks Eindhoven, Netherlands, June 30-July 2, 2014. A retro spective talk …. …. Approximate majority. 0. 1. 1. 0. 0.

Download Presentation

Consensus

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Consensus

Consensus

or

approximate majority

quantile summaries

selection problem

Milan Vojnovic

Microsoft Research

Workshop on Performance and Control of Large-Scale Networks Eindhoven, Netherlands, June 30-July 2, 2014


A retro spective talk

A retrospective talk …


Approximate majority

Approximate majority

0

1

1

0

0

1

1

0

1

Input: each node holds a binary value, either 0 or 1

Output: each node to report the majority vote (with high probability)

Requirement: limited memory per node and pairwise communication between nodes


Consensus

0

1

0

0

0

1

0

1

1


Consensus

0

1

0

0

0

1

0

1

1


Our notation

Our notation

0

1

0

0

0

1

0

1

1


Approximate majority algorithms

Approximate majority algorithms

2 states

  • States: 0, 1

  • Convergence time =

  • Probability of error =

3 states

  • States: 0, e, 1

  • Convergence time =

  • Probability of error =

4 states

  • States: 0, e0, e1, 1

  • Convergence time =

  • Probability of error =

= number of nodes, = voting margin


Questions of i nterest

Questions of interest

Correctness: probability that each node identifies the initial majority state?

Convergence time: time to reach consensus?

Dependence on the number of nodes voting margin , network structure?


Desiderata

Desiderata

  • Reach correct consensus – initial majority

  • Fast convergence

  • Small communication overhead

  • Small processing per node

  • Decentralized


Outline

Outline

Related work

3-state algorithm

4-state algorithm

Conclusion


Some related work

Some related work

More references in this slide deck


Classical voter model hassin peleg 01

Classical voter model[Hassin-Peleg-01]

0

1

1

0

0

1

0

1

  • 0 initially held by nodes, 1 initially held by nodes

  • Complete graph node interactions

  • Probability of incorrect consensus

Node takes over the state of the contacted node

Binary state per node & binary signaling


Statistical tests with limited memory information theory 70 s

Statistical tests with limited memory[Information Theory 70’s]

000110111110100011

S

i. i. d. mean

  • How many states S needs to identify the correct hypothesis with probability with the number of observations?

  • m+1necessary and sufficient [Koplowitz, IEEE Trans IT ’75]


Quantile summaries greenwald knanna 2004

Quantilesummaries[Greenwald- Knanna-2004]

  • Approximate quantile computation: Input: rank rel. acc. par. Output: element of rank

  • Quantile summaries: max number of data elements communicated by any node

Coordinator

elements


Outline1

Outline

Related work

3-state algorithm

4-state algorithm

Conclusion


3 state algorithm

3-state algorithm

0

0

e

1

e

1

0

0

e

e

0

1

  • Both processing and signaling take one of three states

    • 0 or 1 or e

    • e = “indecisive” state


Assumptions

Assumptions

  • Interactions: asynchronous continuous-time, complete graphEach node samples another node uniformly at random at instances of a Poisson process with intensity 1


3 state algorithm state evolution

3-state algorithm: state evolution

  • Markov process:

= number of nodes in state 0

= number of nodes in state 1

= total number of nodes


Ternary protocol probability of error

Ternary protocol: probability of error

  • = initial point,

Theorem – probability of error:


Probability of error c ont d

Probability of error (cont’d)

Corollary – For initial state such that , for , we have, large

Exponentially decreasing in

Correctness with high probability if


Proof main ideas

Proof main ideas

First-step analysis:where with the boundary conditions: for for


Proof main ideas cont d

Proof main ideas (cont’d)

  • i.e. is the error probability for

  • Lemma – solution of

    with the boundary conditions: for , , for


Proof main ideas cont d1

Proof main ideas (cont’d)

# of paths from to not intersecting

-- Ballot theorem


Convergence time

Convergence time

  • The limit ODE

  • Def: = smallest time such that and are of order given that and

Proof:


Convergence time l ower b ound

Convergence time lower bound

  • Lower bound:

  • Example: pathreduction to classical voter model

U

V

1

1

1

1

0

0

0

0

. . .

. . .


Convergence time l ower bound cont d

Convergence time lower bound (cont’d)

  • Ternary protocol on a path corresponds to a classical voter model dynamics

1

1

1

0

0

0

0

1/2

1/2

1

1

e

0

0

0

0

1/2

1

1

0

0

0

0

0


Extension to plurality problem jung kim v 2012

Extension to plurality problem[Jung-Kim-V.-2012]

  • alternatives

    • Binary consensus as special case:

  • Output: each node to correctly identify a state that is initially a plurality winner


Plurality algorithm

Plurality algorithm

observer

m alternatives

2m states: weak strong


State evolution

State evolution

Markov process:


The l imit ode

The limit ODE

For every and


Convergence t ime

- convergence time

Given , defined as follows


Limit points

Limit points

  • Theorem – Suppose that for and ThenMoreover, we have


Limit points cont d

Limit points (cont’d)

The last theorem follows as a corollary of the following claims:


Rate of convergence

Rate of convergence

For every non-plurality state

Exponential diminishing of non-plurality states


Convergence time1

Convergence time

Theorem: For such that and , there exists a constant such that

Corollary:

Convergence time linear in the number of alternatives*

Logarithmic in the voting margin

* Up to poly-log factors


Convergence lower b ounds

Convergence lower bounds

Theorem: For


Convergence time lower b ounds cont d

Convergence time lower bounds (cont’d)

  • Theorem: For every there exists an initial state with gap and constant such that for and small enough

Take:


Probability of error babace draief 2013

Probability of Error[Babace-Draief-2013]

  • Theorem - suppose that for ,Then


Polling algorithm cruise ganesh 2013

Polling algorithm[Cruise-Ganesh-2013]

do:

  • Sample node uniformly at random

  • Sample of m nodes from the population with replacement

  • number of nodes in in state 1

  • If

  • Else if

= number of nodes in in state 1

1

1

1

1

1

sample of nodes

1


Polling algorithm cont d cruise ganesh 2013

Polling algorithm (cont’d)[Cruise-Ganesh-2013]

  • Probability of error:

  • Expected convergence time:


Outline2

Outline

Related work

3-state algorithm

4-state algorithm

Conclusion


Quaternary p rotocol

Quaternary protocol

  • Four states

  • Update rules: swap or annihilate

0

e0

e1

1

0

e0

0

e1

0

1

e0

e1

e0

1

e1

1

e0

0

e0

0

e1

e0

e1

e0

1

e1

1

e1


Correctness benezit thiran vetterli 2010

Correctness[Benezit-Thiran-Vetterli-2010]

Corollary - For any given connected graph, the binary interval consensus converges to the correct state with probability 1.


Convergence time2

Convergence time

Each edge activated at instances of a Poisson point process of intensity

Contract rate matrix:

Family of matrices: for every non-empty subset of nodes , defined by


Eigenvalue gap

Eigenvalue gap

For any finite graph , there exists such that every eigenvalue of matrix satisfies


Convergence time3

Convergence time

  • Two phases

    • Phase 1: time until depletion of state 1

    • Phase 2: time until depletion of state 2

  • Theorem:


State evolution in phase 1

State evolution in Phase 1

1 if node i in state 1

1 if node i in state 0

Phase 1


State evolution in phase 1 cont d

State evolution in Phase 1 (cont’d)

  • Probability that a node is in state 1 evolves as

  • System of linear ODEs:, = set of nodes in state 0

  • Bounds on the expected convergence time follow using a spectral bound


Complete graph

Complete graph

  • Each edge activate at rate

  • , for


Complete graph upper bound is tight

Complete graph: upper bound is tight

  • By direct analysis:where is the -th harmonic number

  • 0 and 1 states annihilate after a random time with exponential distribution with parameter


Consensus

Star

  • Each edge activate at rate

    , for

  • Tight: by direct analysis


Erdos renyi graph

Erdos-Renyi graph

  • Edge (u,v) activated at rate , , for

  • If w.h.p.where is the inverse function of

1

1


Conclusion

Conclusion

  • 3-state algorithm

    • Complete graph: correctness with high probability (exponentially decreasing error probability in ), fast convergence

    • Extensions to plurality problem

  • 4-state algorithm

    • Arbitrary connected graph: guaranteed correctness, expected convergence time upper bounds

    • Complete graph: expected convergence time


Some open p roblems

Some open problems

  • Lower bounds? - given memory and communication constraints and a probability of error budget, lower bounds for the expected convergence time?

  • Better upper bounds?

  • Tradeoff accuracy-convergence time: dependence on the memory and communication constraints and the network structure?


References

References

  • J. Koplowitz, Necessary and Sufficient Memory Size for m-hypothesis Testing, IEEE Trans. on Information Theory, Vol 21, No 1, 1975

  • M. Greenwald and S. Khanna, Space-efficient Online Computation of Quantile Summaries, ACM SIGMOD 2001

  • Y. Hassin and D. Peleg, Distributed Probabilistic Polling and Applications to Proportionate Agreement, Information and Computation, 171, 2001

  • D. Kempe, J. Kleinberg and E. Tardos, Maximizing Influence through a Social Network, ACM KDD 2003

  • M. Greenwald and S. Khanna, Power-conserving Computation of Order-Statistics over Sensor Networks, ACM PODS 2004

  • T. M. Liggett, Interacting Particle Systems, Springer, 2006

  • S. Boyd, A. Ghosh, B. Prabhakar and D. Shah, Randomized gossip algorithms, IEEE Trans. on Information Theory, Vol 52, No 6, 2006

  • D. Angluin, J. Aspnes, D. Eisenstat, A Simple Population Protocol for Fast Robust Approximate Majority, DISC, 2007

  • F. Kuhn, T. Locher, R. Wattenhofer, Tight Bounds for Distributed Selection, ACM SPAA 2007


References cont d

References (cont’d)

  • W. P. Tay, J. N. Tsitsiklis and M. Z. Win, On the Subexponential Decay of Detection Error Probabilities in Long Tandems, IEEE Trans. on Info. The., Vol 54, No 10, 2008

  • A. Nedic, A. Olshevsky, A. Ozdaglar and J. N. Tsitsiklis, Distributed Averaging Algorithms and Quantization Effects, IEEE Conf. on Decision and Control, 2008

  • F. Benzit, P. Thiran and M. Vetterli, Interval Consensus: From Quantized Gossip to Voting, IEEE Int’l Conf. on Acoustics, Speech, and Signal Processing, 2009

  • E. Perron, D. Vasudevan, M. V., Using Three States for Binary Consensus on Complete Graphs, IEEE Infocom 2009

  • J. Cruise and A. Ganesh, Probabilistic Consensus via Polling and Majority Rules, Proc. of Allerton Conference, 2010

  • D. Acemoglu, M. A. Dahleh, I. Lobel and A. Ozdaglar, Bayesian Learning in Social Networks, forthcoming Review of Economic Studies, 2011

  • F. Benezit, P. Thiran and M. Vetterli, The Distributed Multiple Voting Problem, IEEE Journal on Selected Topics in Signal Processing, Vol 5, No. 4, 2011

  • M. Draief and M. V., Convergence Speed of Binary Interval Consensus, SIAM J. Control Optim., vol 50, pp 1087-1109


References cont d1

References (cont’d)

  • E. Mossel, J. Neeman and O. Tamuz, Majority Dynamics and Aggregation of Information in Social Networks, 2012

  • F. Chierichetti and J. Kleinberg, Voting with Limited Information and Many Alternatives, ACM SODA 2012

  • M. A. Abdullah and M. Draief, Majority Consensus on Random Graphs of a Given Degree Sequence, ArXiv, 2012

  • K. Jung, B. Y. Kim, M. V., Distributed Ranking in Networks with Limited Memory and Communication, IEEE ISIT 2012

  • S. Shang, P. W. Cuff, S. R. Kulkarniand P. Hui, An Upper Bound on the Convergence Time for Distributed Binary Consensus, 15th Int’l Conf. on Information Fusion, 2012

  • Z. Huang, K. Yi, and Q. Zhang, Randomized Algorithms for Tracking Distributed Count, Frequencies and Ranks, ACM PODS 2012

  • A. Babaee and M. Draief, Distributed Multivalued Consensus, Computer and Information Sciences III, 2013


References cont d2

References (cont’d)

  • G. B. Mertzios, S. E. Nikoletseas, C. L. Raptopoulos, P. G. Spirakis, Determining Majority in Networks with Local Interactions and very Small Local Memory, ICALP 2014

  • M. Feldman, N. Immorlica, B. Lucier, S. M. Weinberg, Information Aggregation in Social Networks, working paper, 2014


  • Login