Service Discovery with Zookeeper
About Apache Zookeeper
What is Zookeeper? It provides a centralized infrastructure and services that enable synchronization across an Apache™ Hadoop® cluster.
Apache ZooKeeper is a software project of the Apache Software Foundation. It is essentially a centralized service for distributed systems to a hierarchical key-value store, which is used to provide a distributed configuration service, synchronization service, and naming registry for large distributed systems.
- Managing the Cluster
- Updating the Node’s Status
- Automatic Failure Recovery
Why ZooKeeper
centralized open-source server for maintaining and managing configuration information
- Controller election - The controller is one of the most important broking entity in a Kafka ecosystem, and it also has the responsibility to maintain the leader-follower relationship across all the partitions.
- Configuration Of Topics - The configuration regarding all the topics including the list of existing topics, the number of partitions for each topic, the location of all the replicas, list of configuration overrides for all topics and which node is the preferred leader, etc.
- Membership of the cluster - Zookeeper also maintains a list of all the brokers that are functioning at any given moment and are a part of the cluster.
- Access control lists - Access control lists or ACLs for all the topics are also maintained within Zookeeper.
Book an Appointment!
Book a 30 min consulting on ZooKeeper with WeCanDoNow team
Where is ZooKeeper Useful?
- Group membership and name servicesBy having each node register an ephemeral znode for itself (and any roles it might be fulfilling), you can use ZooKeeper as a replacement for DNS within your cluster.
- Distributed mutexes and master electionWe discussed these potential uses for ZooKeeper above in connection with sequential nodes.
- Centralized configuration managementUsing ZooKeeper to store your configuration information has two main benefits.
- Asynchronous message passing and event broadcastingAlthough other tools are better suited to message passing when throughput is the main concern, I’ve found ZooKeeper to be quite useful for building a simple pub/sub system when needed.
Key Features
- Updating the node's status
- Managing the cluster
- Simplicity
- Reliability
- Ordered
- Speed
- Scalability