479 questions
0
votes
1
answer
94
views
Python multiprocessing runs forever and no activity in Task manager
I'm using fuzzy matching to match a 5000 row dataset to a 2 million row dataset. I'm using extractOne() to get what I need but it's slow. I want to use multiprocess to get it done. I have multiple ...
- 18
0
votes
1
answer
109
views
Concurrent access logger for C++ [closed]
I am currently working on the modernization of a quite large application analyzing large amount of text data. The application is made of executables, launching other executables and so forth. Some of ...
- 1
9
votes
2
answers
907
views
Why does multiprocess with "fork" fail under Python 3.14 but work in 3.13 (works only with "spawn" and "forkserver")?
The following code works fine on Python 3.13, but fails on Python 3.14 with a RuntimeError related to asyncio tasks.
If I switch the multiprocessing start method from "fork" to "spawn&...
- 1,530
0
votes
0
answers
19
views
Android Execute code once per cold boot of android app
I have an Android application with a multi-process architecture where certain activities are launched in different processes. Here is the scenario:
Main Activity (Activity A) runs in Process A.
When ...
0
votes
1
answer
114
views
Why are logs not appearing in a file/console when using logging in Python with multiprocessing.Pool?
I'm trying to implement logging in a Python application where I use both threads and processes. I have a custom LoggingManager that uses a QueueHandler to push log records from multiple processes to a ...
0
votes
0
answers
128
views
multiprocess library barely works
I'm using the multiprocess library to accelerate a CPU-bound task (a method inside a user-defined class).
The function processes a page of a document, in my example a 500-page document takes around 20 ...
- 689
0
votes
0
answers
125
views
Generate a batch of Abaqus .inp files using Python Multiprocess
I am using a commercial finite element application called Abaqus/CAE with a built-in Python 2.6 interpreter and API. The inp file is for Abauqus to calculate with having geometry, material properties,...
- 1
0
votes
1
answer
60
views
Issues with synchronization with multiprocessing.Lock
I am initializing multiprocessing.Lock() in a worker process. like:
with multiprocessing.Lock(): shared_value+=1
is it any different passing a single instance from the parent process through all ...
- 19
1
vote
1
answer
65
views
How to write to the Flask Session from a child process
I have a flask web app that does image processing. The parent application calls a function that creates several temporary images, writes them to disk, and stores their paths in the flask Session. That ...
- 309
1
vote
0
answers
65
views
How to prevent multi process pool slow down overtime?
We are trying to run multiple simulation tasks using a multiprocess pool in order to reduce the overall runtime compared to running each task individually in a series. At the beginning of the run CPU ...
- 11
0
votes
0
answers
51
views
Time per process increases with number of processes when reading JSON in Python. Why?
I am trying to parallelize the processing of some JSON files using Python's multiprocess module. The amount of time required to process a fixed number of files within a subprocess seems to depend on ...
- 1,054
1
vote
0
answers
74
views
How do I launch an Android activity in the same process that is launching it?
I have some activities that I want to launch. Sometimes I want to launch the activity from my main process (e.g. another activity), and sometimes I want to launch them from my service which is running ...
- 5,471
0
votes
0
answers
173
views
Issues with Nextflow when using a parallelized Python script
I've run into a bit of an odd situation. I have a python script that counts features across chromosomes. I've parallelized the script so it sends each CPU (set by --threads) a number of chromosomes.
...
1
vote
1
answer
199
views
Writing to numpy array in shared memory failing past a certain size threshold, but only in IDEs
I'm trying to implement a shared numpy array for use in multiprocessing. It works fine on smaller arrays, but then failed when I tried to use a ~2.7GB numpy array (about 5.6Mx64 matrix). After a ...
0
votes
0
answers
26
views
NameError using multiprocess pool with function which calls another function
I would like to use multiprocessing in my jupyter notebook and AFAIK, the multiprocess package is the way to to that. My function does quite a lot of stuff, so it calls multiple subfunctions, but this ...
