Questions tagged [parallel-programming]
For questions about programming in such a way that a task can be divided among multiple threads/workers which work simultaneously ('in parallel') on the subtasks, leading to performance gains.
74 questions
0
votes
0
answers
93
views
How to connect to SFTP using Apache Spark 3.5 with Scala 2.12 for parallel file transfers?
I am working on a project where I need to transfer thousands of files (each sized between 50-60 MB) every hour from an SFTP server to local storage or AWS S3. I am using Apache Spark 3.5 with Scala 2....
- 11
1
vote
1
answer
477
views
Determine when distributed and parallel process is completed
In a distributred system that uses parallel processing to complete tasks and I want to determine when all sub tasks have been completed, what design method or principals can i apply ?
I am using ...
- 149
0
votes
0
answers
45
views
What are the typical properties of dependency graphs?
I want to run a complex algorithm consisting of many subtasks in parallel on a number of processors. These subtasks depend on one another, so that this algorithm is characterized by a DAG (Directed ...
- 113
-2
votes
1
answer
93
views
Efficient static race detection [closed]
I am supervising a university project on the topic of race detection for multi-threaded processes with shared memory.
We have a complex algorithm consisting of tens of thousands of subtasks, and we ...
- 113
0
votes
0
answers
106
views
Optimizing the runtime of greedy scheduling for parallel computing
I have a situation where I want to run a complex algorithm consisting of many subtasks in parallel on a number of processors. These subtasks depend on one another, so that this algorithm is ...
- 113
0
votes
1
answer
162
views
Difference between outcome of using mutex/spinlock/semaphore in this deadlock situation?
I'm not sure if my answers to the following exercise is correct:
Suppose the program is run with two threads. Further suppose that the
following sequence of events occurs:
Time Thread 0 ...
- 11
1
vote
1
answer
222
views
Distribution of processing across cores between a LINQ and PLINQ query
I am trying to create a simple demonstration of using 'parallel LINQ' (PLINQ). I have two versions of my task, in C#:
var result = Enumerable.Range(1,1000000).Where(x => IsPrime(x)).ToList();
...
- 113
3
votes
1
answer
169
views
Design for deduplicating concurrent tasks in flight simultaneously
I have 1-20 agents that issue requests over time with roughly ~50 in total in flight at any given time. Many of these requests are identical. The tasks are idempotent, but if two identical requests ...
- 141
-3
votes
2
answers
142
views
How are futures and speculations evaluated differently?
In Practical Foundation of Programming Languages
38 Futures and Speculations
A future is a computation that is performed before it is value is
needed. Like a suspension, a future represents a value ...
- 5,565
2
votes
2
answers
613
views
Is sequential consistency equivalent to performing memory accesses by a processes in program order and performing each memory access atomically?
In Fundamentals of Parallel Multicore Architecture, by Yan Solihin, p304 defines sequential consistency memory model:
Overall, we can express programmers’ implicit expectation of memory
access ...
- 5,565
1
vote
2
answers
1k
views
What are the differences between memory coherence and cache coherence?
https://en.wikipedia.org/wiki/Memory_coherence says:
Memory coherence is an issue that affects the design of computer
systems in which two or more processors or cores share a common area
of memory.[1]...
- 5,565
0
votes
2
answers
1k
views
Approaches of splitting the same type of work among multiple threads in C++
I have a reinforcement learning project.
For this I created a vectorized environment in C++, which is a handler for multiple instances of a simple game.
It is highly parallelizable. Each worker can ...
- 137
0
votes
0
answers
57
views
What is the most fitting thread model to redesign a sequential linear search algorithm?
Let's say that there is a simple sequential algorithm that performs a linear search of a unique key in an array:
public class SearchSeq {
public static int search(int[] a, int key) {
for (int i ...
-2
votes
1
answer
874
views
How to do achieve parallelism in a console application
I have a .net web application, let's say A and a .net console application, let's say B.In A, I have a queue with multiple jobs to be done by B.Once I select the jobs from A, a queue gets created. Each ...
- 23
3
votes
3
answers
2k
views
How can I improve the speed of scanning multiple directories recursively at the same?
So I am trying to speed up my program by using concurrency and/or multi-threading and/or process parallelism. The topics are pretty complex and I am sort of new to them so I am still trying to figure ...
- 263