From the course: Apache Kafka Essential Training: Getting Started
Join today to access over 25,400 courses taught by industry experts.
Consumer groups - Kafka Tutorial
From the course: Apache Kafka Essential Training: Getting Started
Consumer groups
- [Instructor] Let's discuss consumer groups in Kafka in this video. What is a consumer group? A consumer group is a group of consumers that share a topic workload. A topic may be generating hundreds of messages in a short amount of time. It may not be possible for a single consumer process to keep up with processing these messages. For scalability, multiple consumer processes can be started and the messages can be distributed among them for load balancing. A consumer group is a logical group of consumers that Kafka uses for such load distribution. Each message will be sent to only one consumer within the consumer group. That consumer is then responsible for processing the message and acknowledging back to Kafka. Consumers split workload among themselves through partitions. Kafka keeps track of the active number of consumers for each given topic. It then distributes the topics evenly between these consumers.…
Practice while you learn with exercise files
Download the files the instructor uses to teach the course. Follow along and learn by watching, listening and practicing.
Download courses and learn on the go
Watch courses on your mobile device without an internet connection. Download courses using your iOS or Android LinkedIn Learning app.