Create Presentation
Download Presentation

Download Presentation

Achieving Consensus with Unknown Participants

Download Presentation
## Achieving Consensus with Unknown Participants

- - - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - - -

**Achieving Consensus with Unknown Participants**Jichiang Tsai (蔡智強) 國立中興大學電機工程學系**Outline**• Introduction • Consensus with Unknown Participants (CUP) • Participant Detectors (PDs) • One Sink Reducibility (OSR) • The CUP Algorithm for OSR • d-times Removing to One Sink Reducibility (d-ROSR) • The CUP Algorithm for d-ROSR • Conclusions • Future Work**Consensus**• Every process pi proposes a value vi and all processes decide on some unique value v, which needs to be proposed by some process • Validity: If a process decides v, then v is the value proposed by a certain process • Agreement: No two correct processes decide differently • Termination: Every correct process eventually decides some value • A fundamental building block for solving fault-tolerant distributed problems that require agreement among a set of processes, like software based replication**Self-Organized Network**• Without any statically deployed infrastructure and central administrative authority • Allows participating processes to access services independently of their locations • Mobile Ad-Hoc Networks (MANETs) and Wireless Sensor Networks (WSNs) • Can benefit tremendously from consensus to achieve self-organization and robustness • When several entities enter an empty geographic region, they can uniformly agree on which node shall provide which service by executing consensus**System Model**• Consider a finite set of processes = {p1, p2, , pn} • Processes in do not necessarily know each other • No central authority initializing each process with certain context information • Processes in communicate by passing messages through reliable channels • No message creation, corruption and duplication • If processes of two ends are correct, a message sent is eventually delivered to its destination exactly once • There is no bound neither on the transmission delay of messages nor on relative speeds of processes • An asynchronous system**System Model (cont.)**• The communication is asymmetric • pi can send a message to process pj only if pi knows the existence of pj, and vice versa • After pj receives a message from pi, pj knows pi from there on and thus pj can send a message to pi • pi can collect more information • pj does not benefit from this new knowledge • A process does not know whether and when it will receive a message from an unknown process • We cannot force pj to determine that it has to wait a message from pi before accomplishing its objective • The tricky part to solve a problem in an asynchronous system with unknown participants**Consensus with Unknown Participants (CUP)**• Is similar to that of the classical consensus problem • Processes in have to achieve consensus in a situation that they are only acquainted with parts of participants in the computation • For classical consensus, processes know each other but may crash • The failures of processes make the problem hard to solve • For CUP, processes will not crash • Its difficulty lies on that every process has to achieve consensus with the ignorance of several other processes**Participant Detector (PD)**• If every process does not know any other process, no process can send a message to another process • Consensus cannot be achieved • Offers a process some hints about processes participating in the computation • The PDi of pi provides to pi a subset of processes in • Such a subset of processes is called the neighbors of pi • The information returned by PDi is non-decreasing over time • PDi does not return a process not belonging to • If some process queries its PD more than once, all processes may not have a consistent system view • The PD oracle is queried by its process exactly once**Terminology**• The knowledge connectivity graph (KC graph) is a digraph Gdi(V, Edi), where V = and (pi, pj) Edi iff pj PDi, i.e. pi knows pj • The knowledge connectivity graph is directed because information given by PDs is not necessarily symmetric • We may have pj PDi but pi PDj • The undirected knowledge connectivity graph (UKC graph) is an undirected graph G(V, E), where V = and (pi, pj) Eiff pj PDi or pi PDj**Example**P6 P9 P2 P1 P8 P7 P4 P5 P3 P10 P11**Terminology**• The knowledge connectivity graph (KC graph) is a digraph Gdi(V, Edi), where V = and (pi, pj) Edi iff pj PDi, i.e. pi knows pj • The knowledge connectivity graph is directed because information given by PDs is not necessarily symmetric • We may have pj PDi but pi PDj • The undirected knowledge connectivity graph (UKC graph) is an undirected graph G(V, E), where V = and (pi, pj) Eiff pj PDi or pi PDj**Example**P6 P9 P2 P1 P8 P7 P4 P5 P3 P10 P11**One Sink Reducibility (OSR)**• The UKC graph induced by it is connected • Two processes that cannot communicate with each other may decide independently • The DAG (Directed Acyclic Graph) obtained by reducing the KC graph induced by it to the SCCs (Strongly Connected Components) has a unique sink • The OSR PD is sufficient to solve the CUP problem**Example**A B P6 P9 P2 P1 P8 P7 P4 P5 P3 P10 C P11**Participant-Finding algorithm**• Allows a process to collect more information about system participants • Every process can obtain the maximal set of processes in KC graph reachable from it • This set of processes is called the participants of the process • Example: • p2 obtains all processes in the KC graph • p3 only obtains all processes in component C**A**B P6 P9 P2 P1 P8 P7 P4 P5 P3 P10 C P11 Example**Participant-Finding algorithm**• Allows a process to collect more information about system participants • Every process can obtain the maximal set of processes in KC graph reachable from it • This set of processes is called the participants of the process • Example: • p2 obtains all processes in the KC graph • p3 only obtains all processes in component C**A**B P6 P9 P2 P1 P8 P7 P4 P5 P3 P10 C P11 Example**Participant-Finding algorithm (cont.)**• Like a breadth-first search algorithm • A process asks the known processes for its neighbors • Asks any newly known processes for its neighbors • Repeat the previous step until no new process can be known**A**B P6 P9 P2 P1 P8 P7 P4 P5 P3 P10 C P11 Example**Participant-Finding algorithm (cont.)**• Like a breadth-first search algorithm • A process asks the known processes for its neighbors • Continuously asks any newly known processes for its neighbors • Repeat the previous step until no new process can be known**A**B P6 P9 P2 P1 P8 P7 P4 P5 P3 P10 C P11 Example**Participant-Finding algorithm (cont.)**• Like a breadth-first search algorithm • A process asks its neighbors for the known processes • Continuously asks any newly known processes for its neighbors • Repeat the previous step until no new process can be known**A**B P6 P9 P2 P1 P8 P7 P4 P5 P3 P10 C P11 Example**A**B P6 P9 P2 P1 P8 P7 P4 P5 P3 P10 C P11 Example**A**B P6 P9 P2 P1 P8 P7 P4 P5 P3 P10 C P11 Example**CUP Algorithm for OSR**• Every process chooses a leader from its participant set, particularly, a process with the lowest identifier • Every process queries its leader the decision value • Every leader asks all its participants if it is the leader • Only the leader of the sink SCC receives all positive replies • The sink leader makes the final decision • A non-sink leader queries the sink leader the decision value • Every leader informs the decision value to a process having queried it**A**B P6 P9 P2 P1 P8 P7 P4 P5 P3 P10 C P11 Example**CUP Algorithm for OSR**• Every process chooses a leader from its participant set, particularly, a process with the lowest identifier • Every process queries its leader the decision value • Every leader asks all its participants if it is the leader • Only the leader of the sink SCC receives all positive replies • The sink leader makes the final decision • A non-sink leader queries the sink leader the decision value • Every leader informs the decision value to a process having queried it**A**B P6 P9 P2 P1 P8 P7 P4 P5 P3 P10 C P11 Example**CUP Algorithm for OSR**• Every process chooses a leader from its participant set, particularly, a process with the lowest identifier • Every process queries its leader the decision value • Every leader asks all its participants if it is the leader • Only the leader of the sink SCC receives all positive replies • The sink leader makes the final decision • A non-sink leader queries the sink leader the decision value • Every leader informs the decision value to a process having queried it**A**B P6 P9 P2 P1 P8 P7 P4 P5 P3 P10 C P11 Example**CUP Algorithm for OSR**• Every process chooses a leader from its participant set, particularly, a process with the lowest identifier • Every process queries its leader the decision value • Every leader asks all its participants if it is the leader • Only the leader of the sink SCC receives all positive replies • The sink leader makes the final decision • A non-sink leader queries the sink leader the decision value • Every leader informs the decision value to a process having queried it**A**B P6 P9 P2 P1 P8 P7 P4 P5 P3 P10 C P11 Example**CUP Algorithm for OSR**• Every process chooses a leader from its participant set, particularly, a process with the lowest identifier • Every process queries its leader the decision value • Every leader asks all its participants if it is the leader • Only the leader of the sink SCC receives all positive replies • The sink leader makes the final decision • A non-sink leader queries the sink leader the decision value • Every leader informs the decision value to a process having queried it**A**B P6 P9 P2 P1 P8 P7 P4 P5 P3 P10 C P11 Example**A**B P6 P9 P2 P1 P8 P7 P4 P5 P3 P10 C P11 Example**Issue**• The OSR PD is claimed to be necessary for CUP • For a KC graph with two or more sinks, the decided values of some two sinks may be different • A sink has no outgoing edges to other SCCs, so processes in it do not know other processes outside the sink • Processes in a sink may decide merely based on the values proposed by the processes in the same sink • The agreement property of consensus is violated • Consensus can not be achieved when the KC graph still contains more than one sink**Issue (cont.)**• We can force processes in all sinks except one not to decide so early • They have to wait for some process outside to inform them • The system can imitate a graph with only one sink • Such an idea is viable • A connected graph with at least two sinks must own at least one non-sink SCC to link every sink together • Processes in such an SCC can know the identities of processes in all its connecting sinks by finding participants • Processes in a sink can know that they are in a sink and there is at least one process outside to inform them later**Issue (cont.)**• Our idea is to preclude processes in all sinks except one from participating in making the final decision • Deciding which sink is left in the KC graph itself is a consensus problem • Because no sinks are removed so far, the decision cannot be yet accomplished • The only way to remove sinks of the graph is to remove all current sinks • Removing a sink will not induce a non-sink SCC to become a new sink • The foregoing sink removing procedure has to be repeated again and again until there is only one sink left**Sink Removing Operation**• An operation to remove all sink SCCs of a digraph • Gkdi(Vk, Ekdi): Reducing Gdi(V, Edi) by repeatedly applying the sink-removing operation k times • G0di(V0, E0di) = Gdi(V, Edi) • Gk(Vk, Ek): The undirected version of Gkdi(Vk, Ekdi) • An edge (u, v)Ek iff (u, v)Ekdi or (v, u)Ekdi • If Gidi(Vi, Eidi) contains a certain SCC, the same one exists in Gjdi(Vj, Ejdi) for j i as well • Removing a sink will not make several SCCs merge to a new SCC**Example**B B D P1 P6 P9 P6 P2 P2 P5 P5 P7 P7 C C P3 P8 P4 P4 A G0di G1di**Sink Removing Operation**• An operation to remove all sink SCCs of a digraph • Gkdi(Vk, Ekdi): Reducing Gdi(V, Edi) by repeatedly applying the sink-removing operation k times • G0di(V0, E0di) = Gdi(V, Edi) • Gk(Vk, Ek): The undirected version of Gkdi(Vk, Ekdi) • An edge (u, v)Ek iff (u, v)Ekdi or (v, u)Ekdi • If Gidi(Vi, Eidi) contains a certain SCC, the same one exists in Gjdi(Vj, Ejdi) for j i as well • Removing a sink will not make several SCCs merge to a new SCC**Example**B B D P1 P6 P9 P6 P2 P2 P5 P5 P7 P7 C C P3 P8 P4 P4 A G0 G1**Sink Removing Operation**• An operation to remove all sink SCCs of a digraph • Gkdi(Vk, Ekdi): Reducing Gdi(V, Edi) by repeatedly applying the sink-removing operation k times • G0di(V0, E0di) = Gdi(V, Edi) • Gk(Vk, Ek): The undirected version of Gkdi(Vk, Ekdi) • An edge (u, v)Ek iff (u, v)Ekdi or (v, u)Ekdi • If Gidi(Vi, Eidi) contains a certain SCC, the same one exists in Gjdi(Vj, Ejdi) for j i as well • Removing a sink will not make several SCCs merge to a new SCC**Example**B B D P1 P6 P9 P6 P2 P2 P5 P5 P7 P7 C C P3 P8 P4 P4 A G0di G1di**d-times Removing to One Sink Reducibility (d-ROSR)**• The UKC graph G(V, E) is connected • The DAG obtained by reducing the graph Gddi(Vd, Eddi) to its SCCs has one and only one sink • The OSR class is a special case of the d-ROSR class for d = 0, without any sink-removing operation • Implementing the general d-ROSR PD has more flexibility than the special case of OSR PD • The d-ROSR PD is necessary and sufficient to solve the CUP problem**Example (1-ROSR)**B B D P1 P6 P9 P6 P2 P2 P5 P5 P7 P7 C C P3 P8 P4 P4 A G0di G1di**How to Implement**• Make a set of processes in , say , form a subgraph with the OSR property • To the extreme, the subgraph can be merely a strongly connected subgraph or even a vertex, which is trivially a sink • Add some strongly connected subgraphs or vertices with incoming edges only from the vertices contained in the subgraph constituted so far • Processes outside ’ are only known by the processes in • Repeat the previous step exactly d times to cover a few vertices, say the set **Example (3-ROSR)**SCC1 SCC5 SCC8 SCC3 SCC2 SCC6 SCC9 SCC11 SCC4 SCC7 SCC12 SCC10