Service Discovery with etcd
About Apache Etcd
A distributed, reliable key-value store for the most critical data of a distributed system
etcd is a strongly consistent, distributed key-value store that provides a reliable way to store data that needs to be accessed by a distributed system or cluster of machines. It gracefully handles leader elections during network partitions and can tolerate machine failure, even in the leader node.
- Simple interface
- Key-value storage
- Watch for changes
Why etcd
strongly consistent, distributed key-value store
- Controller election - The controller is one of the most important broking entity in a Kafka ecosystem, and it also has the responsibility to maintain the leader-follower relationship across all the partitions.
- Configuration Of Topics - The configuration regarding all the topics including the list of existing topics, the number of partitions for each topic, the location of all the replicas, list of configuration overrides for all topics and which node is the preferred leader, etc.
- Membership of the cluster - Zookeeper also maintains a list of all the brokers that are functioning at any given moment and are a part of the cluster.
- Access control lists - Access control lists or ACLs for all the topics are also maintained within Zookeeper.
Book an Appointment!
Book a 30 min consulting on etcd with WeCanDoNow team
Where is etcd Useful?
- Group membership and name servicesBy having each node register an ephemeral znode for itself (and any roles it might be fulfilling), you can use ZooKeeper as a replacement for DNS within your cluster.
- Distributed mutexes and master electionWe discussed these potential uses for ZooKeeper above in connection with sequential nodes.
- Centralized configuration managementUsing ZooKeeper to store your configuration information has two main benefits.
- Asynchronous message passing and event broadcastingAlthough other tools are better suited to message passing when throughput is the main concern, I’ve found ZooKeeper to be quite useful for building a simple pub/sub system when needed.
Key Features
- Updating the node's status
- Managing the cluster
- Simplicity
- Reliability
- Ordered
- Speed
- Scalability