All Questions
589 questions
0
votes
1
answer
77
views
Efficiently recalculating dependent values in real-time data streams using NumPy in Python
I'm currently working on a real-time data processing system for financial securities, where I need to perform calculations as soon as new data comes in. Each financial security has multiple data ...
- 305
1
vote
1
answer
82
views
Numpy: Move values according to another array
I've tried searching for it for a few hours now, but couldn't find a solution, so I hope someome can help me.
Assuming I have two numpy arrays that look like this:
data_arr = [[0 1 2]
[1 0 ...
- 23
3
votes
2
answers
178
views
NumPy on small arrays: elementary arithmetic operations performances
I am not 100% positive that this question has a solution besides "that's the overhead, live with it", but you never know.
I have a very simple set of elementary mathematical operations done ...
- 1,449
5
votes
0
answers
138
views
Why is np.sort() with order slow on multi dtype arrays? (sorting by unsigned integer ID)
Working with embedded software, I'm using numpy arrays with specified datatypes to save memory and other reasons. I'm using version '1.23.4' and Python 3.10.0, but this seems to be an issue with ...
- 372
3
votes
2
answers
143
views
Python array still needed given that numpy now exists?
I learned about numpy arrays before discovering that python has the ability to work with arrays without numpy via "import array". The book "High Performance Python" describes ...
- 53
2
votes
1
answer
168
views
Python code execution too slow due to bottleneck - seeking performance optimization
I'm currently facing a performance issue with my Python code, specifically with the execution speed. I've identified a bottleneck in my code, and I'm seeking advice on how to optimize it for better ...
- 317
3
votes
3
answers
314
views
Optimal way of counting the number of non-overlapping pairs given a list of intervals
I'm trying to count the number of non-overlapping pairs given a list of intervals.
For example:
[(1, 8), (7, 9), (3, 10), (7, 12), (11, 13), (13, 14), (9, 15)]
There are 8 pairs:
((1, 8), (11, 13))
((...
- 191
0
votes
1
answer
186
views
Performance of numpy sum and bottleneck nansum
I've got a piece of code that is quiet heavily dependent on the use of numpy sum, about 60% of my time is spent on the calculation of just a few results and an replication of the offending code is ...
- 676
0
votes
1
answer
85
views
Speed up converting huge 4D arrays to 5D arrays (converting 5D multidimensional image)
I am trying to convert huge 5D HD5 images (read using h5py) into arrays that can be read by Napari. However, I can only extract 3D stacks of it (Z,X,Y) so I have to iterate over channel and time and ...
- 1
0
votes
1
answer
250
views
How can I loop faster over an array using Cython?
I have an optics tool in which i need to calculate a set of complex amplitudes (i.e. wavefronts) for a range of time steps and wavelengths. So essentially I have two for loops and for each item I have ...
- 11
0
votes
1
answer
42
views
How to create a 2D array python
I have to build a2D array in python starting from a 1D array, where I use each element to start a new "orthogonal" 1D array. Suppose that I have an array that for each x gives me the ...
- 1
0
votes
2
answers
114
views
Python: How can I improve the efficiency of my function to check if a number is a prime
For school I am required to create a function (not stolen from a module (i.e. Sympy)) that checks if a number is a prime. The challenge for me though is that I need to check quite large numbers.
I ...
0
votes
2
answers
80
views
Fastest way to merge columns of the same type in Numpy
Say I have a numpy array A that records the distances from different 1D points. The first column records the coordinate of each destination point, the second column records the pairwise distances. Now ...
- 2,797
-1
votes
2
answers
392
views
High-speed and low-memory way to store a float number inside a Numpy array
I have this number: 19576.4125. I want to save it inside a Numpy array, and I think that while the bit count is lower, it is still better. Is this right?
I tried to save inside a half and a single, ...
- 125
0
votes
0
answers
34
views
Why appending a list is faster than changing the value of a numpy array? [duplicate]
To my surprise this code:
N = 10**8
l = []
L = np.zeros(N)
t1 = time()
for i in range(N):
l.append(1)
t2 = time()
print("Adding into list took:", t2 - t1, "s")
t1 = time()
...
- 179