All Questions
28 questions
-1
votes
1
answer
67
views
Sping Kafka @KafkaHandler ignores message
I am developing a project with Spring Boot + Kafka. I'm having hard times with understanding why my @KafkaHandler not catching the event I'm sedning through windows commandline. To send message I'm ...
- 355
0
votes
0
answers
2k
views
Kafka: org.apache.kafka.common.errors.DisconnectException: null & Node -1 disconnected
I used Kafka 2.5.1 before and the program ran well. Rencently, the version of Kafka has been changed to 3.4.0, and the Kafka server online is 2.8.1. Then, when I start the program, some exception info ...
- 1
1
vote
0
answers
122
views
Preserve messages during unprecedented JVM crash in a high throughput system
I am building a high volume system that will be processing up to a hundred million messages everyday. I have a microservice that is reading from a Kafka topic and doing some basic processing on them ...
- 361
0
votes
1
answer
163
views
Is it possible to combine multiple tasks results in one data entry using Kafka?
Imagine the system got an order as a JSON input. The order requires multiple tasks to be processed - some of them require local data processing, some - require responses from external API, etc., so ...
- 1,703
1
vote
1
answer
1k
views
Waiting for N Events in Kafka: How does the Group-Count-Filter Approach work?
Background
In the book Designing Event-Driven Systems, the author highlighted a common use case in business systems where there is a need to wait or N events to occur.
The example given is of an ...
- 893
1
vote
1
answer
719
views
Kafka : how consumer can know the message type before casting in receive?
I have a consumer (implements ICommonConsumer) that should call receive when receiving a message
T receive(ConsumerRecord<String, String> record, Class<T> classType, Class<?>... ...
- 2,170
-2
votes
1
answer
701
views
Is RabbitMQ or Kafka message queue a 1:1 messaging system?
As mentioned in the answer,
A message queue is a one-way pipe: one process writes to the queue, and another reads the data in the order
SysV message queue is one example
So, my understanding is,
...
0
votes
2
answers
4k
views
Kafka: Single consumer group, no partitions and multiple topics
I have 22 topics and ordering within a topic is important to me. I do not have any partitions.
Basically I have 11 tenants and I need two topics per tenant.
I am confused about whether to have a ...
- 834
1
vote
1
answer
5k
views
Kafka behaviour if consumer fail
I've looked up throught a lot of different articles about Apache Kafka transactions, recovery and exactly-once new features. Still don't understand an issue with consumer recovery. How to be sure ...
- 448
2
votes
1
answer
985
views
Removing message from Kafka intermediate topic
I have a topic named SOURCE which contains a stream of two kinds of messages, A and B. I have written a kafka-streams application that consumes that topic, finds an A and a B with the same correlation ...
- 761
0
votes
1
answer
2k
views
How to poll a remote directory for new files using kafka
I have a folder on remote location and some other process keeps putting files in it.
I am writing an application to download each new file and process it.
For now I am downloading files with rsync ...
- 5,745
7
votes
3
answers
9k
views
Does Apache Kafka provide an asynchronous subscription callback API?
My project is looking at Apache Kafka as a potential replacement for an aging JMS-based messaging approach. In order to make this transition as smooth as possible, it would be ideal if the ...
- 22k
4
votes
1
answer
2k
views
Using Kafka in a resquest/response scenario inside a REST web service
I am currently building a REST web service that needs to communicate with Kafka in a request/response pattern. (I know that Kafka is not really designed for that).
Client calls the REST web service.
...
- 91
2
votes
1
answer
779
views
Kafka rebalancing in case when one subscriber in group has been strictly assigned to specific partition
Kafka rebalancing is designed to redistribute all the partitions of topic to subscriber group alive members so that any of the topic partitions is used by only one consumer at any given moment. So in ...
- 1,104
1
vote
3
answers
2k
views
How do I get multiple clients to listen to the same consumer?
For instance, Producer A delivers a message on a queue/channel to Consumer A. Let's say it's a sports score.
How can I get multiple clients (desktops, mobile devices, etc) to listen in or connect to ...
- 161