Questions tagged [pandas]
Pandas is a Python data analysis library.
607 questions
4
votes
3
answers
103
views
Optimizing DataFrame iteration when generating large hierarchical text files
I have a custom object which stores dataframes in memory given a certain hierarchy, and I want to store this data in a file while maintaining the hierarchy. This hierarchy involved parents, children, ...
7
votes
5
answers
1k
views
8
votes
3
answers
500
views
Interpolating based on non-diagonal neighboring values
I have a comma-separated value (CSV) file as input, and I am supposed to interpolate all missing (nan) values based on neighboring non-diagonal values.
The CSV ...
- 361
4
votes
2
answers
365
views
rolling quarterly mean
I want to calculate the quarterly average of a time-indexed dataframe column in a rolling fashion. The mean at any timestamp should not contain information about future timestamps.
This is a code to ...
- 316
4
votes
3
answers
149
views
Increase time efficiency when writing arrays to CSV file
I have the following code to amend two rows of "test_base.csv" with the entries of the arrays "a_temp" and "b_temp," saving the result into "result.csv." .csv ...
- 43
4
votes
1
answer
217
views
Finding specific promotions from two columns [closed]
I'm trying to build a function that identifies those who are promoted into a list of jobcodes, or are promoted within that list of jobcodes.
Initially I was using ...
- 41
0
votes
1
answer
125
views
What's the fastest way to get "postcodes" for thousands of coordinates (latitudes & longitudes) in Python? [closed]
I have a dataset that contains 750,000 rows. I want to query each row and get the postcodes using the latitudes and longitudes.
Problem:
The code is executing very fast when I query like 100 rows, and ...
- 1
1
vote
1
answer
110
views
Replace iterrow loops in pandas matrices with something else to shorten the running time
This post is modified from this one: https://codereview.stackexchange.com/posts/292885/edit (Alternatives to iterrow loops in python pandas dataframes).
I have a piece of code to calculate price ...
- 81
6
votes
2
answers
755
views
Alternatives to iterrow loops in python pandas dataframes
I have a piece of code to calculate price sensitivity based on the product and its rating.
Below is the original data set with product type, reported year, customer’s rating, price per unit, and ...
- 81
2
votes
1
answer
62
views
Maintain a log containing values if certain conditions are met
I'm trying to capture profits and set a stop loss in my trading strategy. I want the stop loss to be set daily based on the past data and if the current price, i.e., price for the date falls below the ...
- 222
2
votes
1
answer
260
views
Python using generators with Excelwriter - Performance
I'm looking to understand if my code has an obvious blockage or performance pain point that will cause it to operate slower or use more memory than it should.
The current Excelfile i am processing ...
- 131
3
votes
1
answer
302
views
1
vote
1
answer
150
views
Custom neural network implementation in TensorFlow to compare normalisation vs. no normalisation on data
I am performing a sports prediction multi-class classification problem, and wanted to compare the differences in model performance between normalised and non-normalised data. You can see the 2 ...
3
votes
1
answer
247
views
Machine learning training, hyperparameter tuning and testing with 3 different models
I am trying to solve a multi-class classification involving prediction the outcome of a football match (target variable = Win, Lose or Draw). With a dataset of 2280 rows, which is 6 seasons of ...
3
votes
1
answer
94
views
Calculating premium splits for policies
Looking for a better approach to write below transformation using Python. Is it possible to avoid loop and still achieve the desired output?
It is too slow for 10 million rows.
...
- 31