Questions tagged [pandas]
Pandas is a Python data analysis library.
608 questions
4
votes
3
answers
84
views
Optimizing DataFrame iteration when generating large hierarchical text files
I have a custom object which stores dataframes in memory given a certain hierarchy, and I want to store this data in a file while maintaining the hierarchy. This hierarchy involved parents, children, ...
4
votes
4
answers
1k
views
8
votes
3
answers
491
views
Interpolating based on non-diagonal neighboring values
I have a comma-separated value (CSV) file as input, and I am supposed to interpolate all missing (nan) values based on neighboring non-diagonal values.
The CSV ...
- 361
4
votes
2
answers
353
views
rolling quarterly mean
I want to calculate the quarterly average of a time-indexed dataframe column in a rolling fashion. The mean at any timestamp should not contain information about future timestamps.
This is a code to ...
- 316
3
votes
3
answers
140
views
Increase time efficiency when writing arrays to CSV file
I have the following code to amend two rows of "test_base.csv" with the entries of the arrays "a_temp" and "b_temp," saving the result into "result.csv." .csv ...
- 33
4
votes
1
answer
214
views
Finding specific promotions from two columns [closed]
I'm trying to build a function that identifies those who are promoted into a list of jobcodes, or are promoted within that list of jobcodes.
Initially I was using ...
- 41
0
votes
1
answer
123
views
What's the fastest way to get "postcodes" for thousands of coordinates (latitudes & longitudes) in Python? [closed]
I have a dataset that contains 750,000 rows. I want to query each row and get the postcodes using the latitudes and longitudes.
Problem:
The code is executing very fast when I query like 100 rows, and ...
- 1
1
vote
1
answer
105
views
Replace iterrow loops in pandas matrices with something else to shorten the running time
This post is modified from this one: https://codereview.stackexchange.com/posts/292885/edit (Alternatives to iterrow loops in python pandas dataframes).
I have a piece of code to calculate price ...
- 81
6
votes
2
answers
748
views
Alternatives to iterrow loops in python pandas dataframes
I have a piece of code to calculate price sensitivity based on the product and its rating.
Below is the original data set with product type, reported year, customer’s rating, price per unit, and ...
- 81
2
votes
1
answer
58
views
Maintain a log containing values if certain conditions are met
I'm trying to capture profits and set a stop loss in my trading strategy. I want the stop loss to be set daily based on the past data and if the current price, i.e., price for the date falls below the ...
- 222
2
votes
1
answer
253
views
Python using generators with Excelwriter - Performance
I'm looking to understand if my code has an obvious blockage or performance pain point that will cause it to operate slower or use more memory than it should.
The current Excelfile i am processing ...
- 131
3
votes
1
answer
296
views
1
vote
1
answer
149
views
Custom neural network implementation in TensorFlow to compare normalisation vs. no normalisation on data
I am performing a sports prediction multi-class classification problem, and wanted to compare the differences in model performance between normalised and non-normalised data. You can see the 2 ...
3
votes
1
answer
245
views
Machine learning training, hyperparameter tuning and testing with 3 different models
I am trying to solve a multi-class classification involving prediction the outcome of a football match (target variable = Win, Lose or Draw). With a dataset of 2280 rows, which is 6 seasons of ...
3
votes
1
answer
89
views
Calculating premium splits for policies
Looking for a better approach to write below transformation using Python. Is it possible to avoid loop and still achieve the desired output?
It is too slow for 10 million rows.
...
- 31