Consensus and Leader Election

When several machines coordinate a shared decision, a failed response does not tell them whether a peer is dead or merely unreachable. Consensus protocols help a defined group agree on a sequence of decisions under their stated assumptions.

A leader coordinates that work, but declaring yourself leader is not sufficient. The protocol must prevent conflicting committed decisions when communication fails and leadership changes.

In a three-voter group, a majority is two; in a five-voter group it is three. Any two majorities overlap. Protocol rules about terms, votes and log history use that overlap to preserve agreement across leadership changes. Counting acknowledgements by itself is not a consensus protocol.

If a three-voter group splits into two connected voters and one isolated voter, only the majority side can make progress under an appropriate quorum protocol. The isolated node cannot promise a new committed change. This protects agreement while sacrificing progress on that side.

A replicated log records ordered decisions. Raft, for example, uses election terms and log rules to choose a leader and preserve committed entries. A minority partition cannot safely turn itself into an independent replacement simply because its own timer expired.

Leader election also does not stop a paused worker from later writing to an external system. Use fencing tokens or equivalent enforcement at the resource for protected effects. Design application idempotency separately from agreement on the log.

Use a three-voter coordination group and separate its coordination contract from the application database.

etcd uses Raft for distributed coordination and is used by Kubernetes for cluster state. ZooKeeper provides coordination through its own protocol. Database products have different leader and quorum mechanisms; consult their documentation rather than applying one generic majority rule to all reads and writes.

Place voters to match the failures you intend to tolerate. A five-voter group with three voters in one failed location loses its majority. More voters can add write coordination and network cost; they are not extra application request workers.

Read etcd’s learner and membership design before changing membership. A joining node needs to catch up; unsafe membership changes can destroy progress. Choose established software rather than implementing a new consensus protocol for an ordinary application.

Consensus preserves agreement under a defined protocol; quorum loss can stop progress. Connect leadership to resource enforcement and plan membership changes.