Insidebigdata team wrote this article in which they provide 30,000 feet overview of Apache Kafka architecture. Kafka, the distributed streaming platform, features an architecture that provides application messaging that is very clear.
Kafka’s main architectural components include Producers, Topics, Consumers, Consumer Groups, Clusters, Brokers, Partitions, Replicas, Leaders, and Followers.
A fundamental explanation of Kafka’s inner workings goes as follows: Every topic is associated with one or more partitions, which are spread over one or more brokers.
It’s important to note the relationships between broker, replica, and partition components, such as:
- Kafka clusters can have one or more brokers
- Brokers can host multiple replicas
- Topics can have one or more partitions
- A broker can host zero or one replica per partition
- A partition has one leader replica and zero or more follower replicas
You will discover more relations following the article. And authors also make use of UML diagrams to explain architecture in greater detail. Great read!
[Read More]