All Questions
7 questions
1
vote
0
answers
311
views
Kafka consumer : Maximum consumer rate
I am interested in learning/deducing the maximum consumption rate of a Kafka consumer in my consumer group. Maximum consumption rate is the rate at which the consumer can not keep up with the message ...
- 822
-1
votes
1
answer
818
views
Kafka: Write custom partitioner or just use Key?
So I have a Kafka topic with multiple partitions and on it I'm producing messages. I want my messages to be partitioned based on user id. I can achieve this either by using UserId as the message key ...
- 761
2
votes
2
answers
1k
views
Kafka deep healthcheck
We've recently run into an issue where a Kafka broker encountered a kernel issue which blocked IO (but was able to heartbeat back to zookeeper I guess). The result of this is that the Kafka broker ...
- 6,034
3
votes
1
answer
5k
views
Killing node with __consumer_offsets leads to no message consumption at consumers
I have 3 node(nodes0,node1,node2) Kafka cluster(broker0, broker1, broker2) with replication factor 2 and Zookeeper(using zookeeper packaged with Kafka tar) running on a different node (node 4).
I had ...
- 189
12
votes
2
answers
12k
views
Desigining Kafka Topics - Many Topics vs One Big Topic
Considering a stream of different events the recommended way would be
one big topic containing all events
multiple topics for different types of events
Which option would be better?
I understand ...
- 459
2
votes
1
answer
2k
views
Is it possible to put a binary file (say a .mp3 file) as a single message in Kafka? If yes, how?
I am relatively new to Apache Kafka. As a part of a tiny project, I was trying to put a text log file in Kafka as a single message. I encountered some encoding errors. Kafka's java API (as I ...
- 2,853
3
votes
1
answer
3k
views
Kafka cluster zookeeper failure handling
I am going to implement a kafka cluster consisting of 3 machines, one for zookeeper and other 2 as brokers. I have about 6 consumer machines and about hundred producers.
Now if one of the broker ...
- 8,370