1 / 24

An Efficient Decentralized Algorithm for the Distributed Trigger Counting (DTC) Problem

An Efficient Decentralized Algorithm for the Distributed Trigger Counting (DTC) Problem. Venkatesan T. Chakravarthy (IBM Research-India) Anamitra Roy Choudhury (IBM Research-India) Vijay Garg (University of Texas at Austin) Yogish Sabharwal (IBM Research-India).

alexia
Download Presentation

An Efficient Decentralized Algorithm for the Distributed Trigger Counting (DTC) Problem

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. An Efficient Decentralized Algorithm for the Distributed Trigger Counting (DTC) Problem Venkatesan T. Chakravarthy (IBM Research-India) Anamitra Roy Choudhury (IBM Research-India) Vijay Garg (University of Texas at Austin) Yogish Sabharwal (IBM Research-India)

  2. Distributed Trigger Counting (DTC) Problem • Distributed system with n processors • Each processor receives some triggers from an external source • Report to the user when the number of triggers received equals w. (In general w>>n)

  3. Applications of DTC Problem • Distributed monitoring • traffic volume : raise an alarm if #vehicles on a highway exceeds some threshold • wildlife behavior: #sightings of a particular species in a wildlife region exceeds a value. • Global Snapshots: • the distributed system must determine if all in-transit messages have been received to declare snapshot valid. This problem reduces to DTC Problem [Garg, Garg, Sabharwal 2006]

  4. Assumptions: • complete graph model, i.e., any processor can communicate with any other processor • no shared clock and no shared memory • processors communicate using messages • reliable message delivery • no faults in the processors.

  5. Measure of any DTC Algorithm: • Low message complexity • Low MaxRcvLoad, the maximum number of messages received by any processor in the system. • Low MsgLoad the maximum number of messages communicated by any processor in the system.

  6. Trivial Algorithm • Fix one node to be Master Node • Total Deficit (w) is maintained by the Master Node • Any processor that receives a trigger informs the Master Node • The master node decrements the deficit • Finish when deficit reaches zero • Total messages = O(w) • MaxRcvLoad and MsgLoad also O(w)

  7. Previous Work: Any deterministic algorithm has message complexity Ω(n log(w/n)) [Garg et al] Centralized algorithm message complexity O(n log w). MaxRcvLoad can be as high as O(n log w). Tree-based algorithm message complexity O(n log n log w). more decentralized in a heuristic sense. MaxRcvLoad can be as high as O(n log n log w), in the worst case.

  8. Modifications to the trivial algorithm • Any processor sends message (count of triggers received) to the master only after it receives B triggers. • Works in multiple rounds. • w’: deficit at beginning of a round. (initially w’ = w) • Master keeps count of the triggers reported by other processors and the triggers received by itself. • End-of-round declared when count reaches w’/2 • System never enters a dead state • Unreported triggers for each processor < B • Count of triggers at master < w’/2 • Message complexity O(n log w) • log w rounds • w’/2B = n messages exchanged in every round.

  9. Main Result: LayeredRand, a decentralized randomized algorithm. • Theorem: For any trigger pattern, the message complexity of the LayeredRand algorithm is O(n log n log w). Also, there exists constants c,d > 1 such that Pr[MaxRcvLoad > c log n log w] < 1/nd

  10. LayeredRand Algorithm • n = (2L -1) processors arranged in L layers • lth layer has 2l processors, l=0 to L-1 • Algorithm proceeds in multiple rounds. • w’: initial value of a round (number of triggers yet to be received) • Threshold for lth layer defined as • C(x): sum of triggers received by x and some processors in layers below.

  11. LayeredRand Algorithm (Contd.) • For non-root processor x at layer l • If a trigger is received: C(x)++ ; • If C(x)>= τ(l) • pick a processor y from level l-1 at random and send a coin to y. • C(x) := C(x) -τ(l); • If a coin is received from level l+1:C(x) :=C(x)+ τ(l+1). • Root r • maintains C(r) just like others. • If C(r) > , initiate end-of-round procedure • gets total number of triggers received in this round • broadcasts new value of w’ for next round.

  12. Example w’ = 96 G End of round w’ for next round = 96- 53 = 43 6 49 45 2 τ(1) = 4 E F 5 3 1 4 3 1 1 2 τ(2) = 2 2 1 2 1 1 1 1 D A B C

  13. Analysis • System does not stall in the middle of a round, when all the triggers have been delivered. • Message complexity to O(n log n log w) • MaxRcvLoad bounded to O(log n log w) with high probability

  14. Correctness • Consider the state of the system in the middle of any round. • x: any non-root processor at layer l • Dead state thus implies C(r)>3w’/4, leading to contradiction.

  15. Message Complexity O(n log n log w) • log w rounds • Every coin sent from layer l to l-1 means that at least τ(l) triggers have been received at layer l in this round. • #coins sent from layer l to the layer l-1 is at most w’/ τ(l) • #coins sent in a particular round • O(n) message exchanges for every end-of-round procedure.

  16. MaxRcvLoad O(log n log w) w.h.p. Prob[MaxRcvLoad of some processor exceedsc log n log w] < n-(c-1) , for any constant c>=48 • In any given round, #coins received by layer l < w’/ τ(l+1) < 4.2l+1.log n • Each coin sent uniformly and independently at random to one of the 2lprocessors occupying layer l. • Mx: r.v. denoting the number of coins received by x • E[Mx] = 8 log n log w • Prob[Mx > 8a. log n log w] < 2 -8a. log n log w < n -8a , for a>=6 • Above result follows by applying union bound.

  17. Concurrency • We assumed that the triggers are delivered one at a time • all the processing required for handling a trigger is completed before the next trigger arrives. • Relax on that assumption End of Round! ΣC(x) = 53 instead of 55 G w’ = 96 49 48 τ(1) = 4 E 1 1 F 2 1 1 1 τ(2) = 2 C D A B

  18. Handling Concurrency • Triggers and coins received during a round placed in queue and processed one at a time. • Additional features for handling end-of-round. • Default queue and Priority queue • Unprocessed triggers and coins placed in default queue • End-of-round messages in priority queue • Default queue serviced only when priority queue empty • Counters C(x), D(x) and RoundNum • D(x): triggers processed by x since the begin • C(x): reset after every round

  19. Thank You!!

  20. Backup

  21. End-of-round procedure • Processors arranged in a tree. • Four phases. • First Phase: root initiates RoundReset message • A processor x on receiving RoundReset • suspends processing of the default queue until end of round i.e., D(x) value not modified further till new round • Non-leaf processor forwards it to its children; • Leaf processor initiates the second phase End of Round! w’=96 G 49 48 E 1 F 1 2 1 1 1 C D D(x)=2 A B

  22. Second Phase • Leaf processor initiates Reduce message containing its D(x) value • A processor x on receiving Reduce from its children • Non-root processor adds its D(x) value to the sum and forwards it to its parent • Root processor computes w’ – termination or next round. End of Round! ΣD(x) = 55 New w’=41 w’=96 G 49 48 E 1 F 1 2 1 1 1 C D D(x)=2 A B

  23. Third Phase • Root broadcasts the new w’ by Inform message. • Every non-leaf processor forwards it to its children; • Leaf processors on receiving Inform message initiate the fourth phase. End of Round! New w’=41 G E τ(1) = 2 F 2 1 C D A B

  24. Fourth Phase • Processor in this phase perform the following • RoundNum incremented. • signifies new round i.e., processor does not process any coin from the previous rounds. • C(x) reset to zero. • InformAck message sent to its parent. • Processing of the default queue resumed. • System (all processors) enters next round when root receives InformAck End of Round! Next Round w’=41 G E F Discard this coin 2 1 C D A B

More Related