Questions tagged [distributed-computing]
Distributed computing refers to architectures where software components of a same system run on several processes distributed across a network of computers and somehow coordinated.
161 questions
2
votes
2
answers
152
views
For distributed workflows - tradeoffs having stateless workers operate on central state, versus stateful workers?
I'm working on a problem right now that processes incoming data at a very high rate. Each event that flows in has an association ID, and each group of associated events will affect behaviour over time ...
- 141
0
votes
1
answer
189
views
How to rebalance data across nodes?
I am implementing a message queue where messages are distributed across nodes in a cluster. The goal is to design a system to be able to auto-scale without needing to keep a global map of each message ...
- 115
1
vote
2
answers
336
views
Generating unique number in distributed environment
I have little app that i plan to run behind load balancer, with up to 5-8 instances.
Inside the app has service, that needs to generate unique number that has up to 10 digits ( so 1-10 digits) each ...
- 227
-1
votes
3
answers
638
views
Avoid race condition using RDBMS transaction
I am working on design of a distributed system which will process some events. For simplicity lets say, multiple instance of same service will be consuming from same queue, every message will have a ...
3
votes
4
answers
1k
views
Giving multiple components access to a single database
This is in context of a distributed computing. There is a Service A which owns a database and hosts APIs for updating an entities in this database.
As time went by, the service has evolved and we are ...
- 41
3
votes
1
answer
1k
views
Long-running compute-intensive tasks in APIs: background workers?
I'm working on a design for an HTTP based API that takes in requests to perform a long-running task that requires CPU and RAM intensive processing.
To give an impression of the compute requirements, ...
- 139
0
votes
1
answer
80
views
What are the approaches for joining data in distributed processing
Let's suppose I have a processing workflow of:
Where:
The workflow can have many instances running at the same time.
A1 -> B1, C1 -> D1
A2 -> B2, C2 -> D2
and so on...
Each node is an ...
- 119
0
votes
1
answer
100
views
How to load consistent test data into multiple applications (which communicate by message streams)?
I have a system landscape consisting of multiple applications.
Each application provides REST interfaces to communicate with the outside world.
Each application has its own data storage (some kind of ...
- 207
-1
votes
1
answer
143
views
Pinning Stateful Tasks in Stateless microservice cluster
We have a microservice whose multiple instances are running for High Availablity and Scaling. But although we have multipe stateless microservice there are some tasks which needs to be singleton.
For ...
- 97
0
votes
1
answer
535
views
Large file uploading in chunks
How to upload large(mb/gb) of video files from client to app server?
The solution as of now I know is pretty simple and widely used is to break file in chunks at at client side and send http post ...
- 81
2
votes
2
answers
613
views
Is sequential consistency equivalent to performing memory accesses by a processes in program order and performing each memory access atomically?
In Fundamentals of Parallel Multicore Architecture, by Yan Solihin, p304 defines sequential consistency memory model:
Overall, we can express programmers’ implicit expectation of memory
access ...
- 5,565
4
votes
2
answers
187
views
Do coherence and transactional isolation mean the same?
About coherence vs consistency, https://en.wikipedia.org/wiki/Consistency_model says
Coherence deals with maintaining a global order in which writes to a
single location or single variable are seen ...
- 5,565
3
votes
1
answer
264
views
Database consistency for a distributed system
I'm working on a phone call monitoring project. The aim is to have one row in the calls table for each ongoing call. Each call instance may be updated by many different servers/threads as more ...
- 141
6
votes
3
answers
342
views
How can adding redunancy adversely affect performance
I have been reading about system design and was going through this popular article - A Word on Scalability
The author here says:
An always-on service is said to be scalable if adding resources to
...
- 171
1
vote
1
answer
88
views
Ordering of analytical events
My team is working on an analytics system for the web app. The web app has a service-oriented architecture, and all events are gathered in one centralized analytics service. The system is sending ...
- 77