All Questions
Tagged with message-queue apache-kafka
153 questions
1
vote
0
answers
25
views
Spring boot app cannot connect with Kafka within the container
In short
To repeat the issue, you may clone the project from:
https://github.com/ethanalef/event-driven-model
Then build both listener and publisher projects with maven, build the image with ...
- 75
-1
votes
1
answer
67
views
Sping Kafka @KafkaHandler ignores message
I am developing a project with Spring Boot + Kafka. I'm having hard times with understanding why my @KafkaHandler not catching the event I'm sedning through windows commandline. To send message I'm ...
- 355
1
vote
1
answer
81
views
Does flush.interval=1 guarantee data durablly written to disk in Kafka?
The default durability model of Kafka is to write "in memory" to multiple brokers at which point the produce request is acknowledged. Writes (and fsync) to durable storage comes later, and ...
- 65.3k
1
vote
1
answer
2k
views
How to propogate open telemetry span via kafka?
I want to propogate a trace from one service to another where I am sending data via Kafka.
I was able to create a child trace, using sort of below logic. But, opentelemetry agent doesn't send the ...
- 197
0
votes
0
answers
2k
views
Kafka: org.apache.kafka.common.errors.DisconnectException: null & Node -1 disconnected
I used Kafka 2.5.1 before and the program ran well. Rencently, the version of Kafka has been changed to 3.4.0, and the Kafka server online is 2.8.1. Then, when I start the program, some exception info ...
- 1
0
votes
1
answer
23
views
How can I make sure older messages received before newer messages?
An application receives updates from a messaging system; the messaging system uses multi-threading and message order is not respected. How can I ensure correct updating, if the app receives older ...
- 380
0
votes
0
answers
22
views
Listener Dynamic Config, listening to different types of MQ
Background
Hi, I am building an AsyncExecutor during my work. Currently, it pulls message from Kafka. (I am new in the field)
Demand
We want to refactor it as an infra or interface. Is there a way to ...
- 43
2
votes
1
answer
691
views
CDC vs Message Broker Key differences and which to use when
I have been struggling to find any key pros and cons on using one over the other. When it comes to sharing data between two microservices. Especially when it comes to scale.
What my assumption and ...
- 147
0
votes
1
answer
609
views
Message queues to trigger events for kafka and Redis from one service to other service
I have two services which runs in python2 and python3. One will be socket service other would be general calculation service based on the data from socket.
I would like to communicate between one to ...
- 5
0
votes
0
answers
261
views
How to ensure a specific consumer will process a message in RabbitMQ/Kafka?
I need to process large XMLs (2GB).
Processing the file can take longer than receiving the file.
Let's say that there is a component A that receives the message and a component B that processes the ...
- 63
0
votes
0
answers
36
views
Restrict message queue publisher based on message content
I need to organize a message queue that has multiple publishers.
Each publisher should only be allowed to publish a message with a specific header or part of message body (e.g. json field).
Having ...
- 1,136
1
vote
1
answer
2k
views
Kafka multiple consumers consuming from one partition - can the message be read twice?
I have a system with Kafka that looks like this (all consumers are within a single consumer group):
Producer ---[ 1 topic, 1 partition] ---> Consumer1
|--> ...
- 11
0
votes
1
answer
89
views
are condition variables, message queues used for inter process communication and messagebrokers(kafka) the same?
In university level course on operating systems,it is told that inter process communication can take place using message queues.
Also, in mutli-threading, condition variables are queues used to solve ...
- 17
0
votes
0
answers
232
views
Spring Apache Kafka decoupled message queue best practice
I have a problem regrading renewing an old interface of data exporting to a client of ours.
13 years ago,
A queue logic was developed in an Oracle package (Fully SQL code) - the data is inserted into ...
1
vote
1
answer
619
views
kafka - How to read and process messages in a fault tolerant way? [closed]
I am new to Kafka, and I am trying to understand how a Kafka Consumer can consume and process a message from a Kafka topic without losing messages if the consumer fails in between.
For Example:
There ...
- 3,270