All Questions
17 questions
0
votes
1
answer
54
views
Does rabbitmq have operations log?
I need to see what operations are happening inside rabbitmq similar to redis-cli monitor or redis aof. Does rabbitmq have this capability?
- 23
1
vote
1
answer
596
views
How do I achieve one by one message delivery for each user individually (in parallel)?
I need a queue/message broker that would allow me to receive by 1 message for each user.
Previously I've been using FIFO SQS and each user has its own message group id, which allows me to have 1 ...
- 1,787
3
votes
1
answer
3k
views
Using Celery with RabbitMQ as broker vs using just RabbitMQ + Pika for async tasks, advantages of using one over another
The debate I am in currently is whether we should stick with RabbitMQ implementation using Pika or move to celery, what all advantages are there if we go with Celery. From what I have understood is ...
- 872
3
votes
0
answers
665
views
Alternative for celery on Windows for long running task on remote host
I have issues while running long running task which takes around 2 and half hours to complete. The issue is either I get socket timeout error or the task result keeps on waiting forever.
So any other ...
- 715
3
votes
1
answer
895
views
Can I use Celery for publishing and subscribing to topics?
All the examples I have seen of executing/scheduling Celery tasks are like this:
add.delay()
I was wondering if I could do something like this with Celery:
celery_app.publish(topic='my-topic')
And ...
- 726
0
votes
1
answer
2k
views
Multiple subscriptions to a topic
I have been using pubsub for a bit of asynchronous work, and was wondering why someone may create multiple subscriptions for a single topic. My default values are as follows:
project_id = 'project'
...
user10332687
0
votes
1
answer
1k
views
Stop RabbitMQ consumer process current message
I'm writing a service on NodeJS that processes CPU-bound tasks. The main idea is pretty simple:
Client (browser) sends request
Webserver (NodeJS + ExpressJS) handles request: generates ID, adds a ...
0
votes
1
answer
595
views
ASP.Net Web API - scaling large number of write operations
I am working on a project using ASP.Net Web API that will be receiving a large number of POST operations where I will need to write many successive / simultaneous records to the DB. I don't have an ...
- 127
8
votes
3
answers
832
views
Distributed Task Queue Based on Sets as a Data Structure instead of Lists
I'm wondering if there's a way to set up RabbitMQ or Redis to work with Celery so that when I send a task to the queue, it doesn't go into a list of tasks, but rather into a Set of tasks keyed based ...
- 3,815
0
votes
0
answers
460
views
Using rabbitmq as both a message broker and backend
I have been working with Celery and reading about rabbitmq vs redis for the message broker and backend. I would like to use the rabbitmq backend because it sounds like it is better at guaranteeing ...
- 4,154
1
vote
2
answers
266
views
Store-and-forward failover solution for ServiceStack web services
I am developing a customer account system for a chain of recycling centers in the Northwest US. One of our key features is that our customers can set up accounts that are credited with their bottle ...
- 3,636
1
vote
1
answer
82
views
data store with a "get or block" operation?
I'm looking for a data store that has a "get or block" operation. This operation would return the value associated with a key/query if that value exists or block until that value is created.
It's ...
- 52.1k
272
votes
2
answers
59k
views
Why do we need message brokers like RabbitMQ over a database like PostgreSQL?
I am new to message brokers like RabbitMQ which we can use to create tasks / message queues for a scheduling system like Celery.
Now, here is the question:
I can create a table in PostgreSQL which ...
- 45.8k
5
votes
1
answer
2k
views
Synchronize one queue instance with multiple Redis instances
The Scenario:
We have multiple nodes distributed geographically on which we want to have queues collecting messages for that location. And then we want to send this collected data from every queue in ...
- 2,213
23
votes
1
answer
19k
views
RabbitMQ / ActiveMQ or Redis for over 250,000 msg/s
Eventhough redis and message queueing software are usually used for different purposes, I would like to ask pros and cons of using redis for the following use case:
group of event collectors write ...
- 241