Questions tagged [hyperparameter]
A parameter that is not strictly for the statistical model (or data generating process), but a parameter for the statistical method. It could be a parameter for: a family of prior distributions, smoothing, a penalty in regularization methods, or an optimization algorithm.
651 questions
2
votes
1
answer
34
views
Tune a Model on Calibrated or Uncalibrated Probabilties?
I report model performance using log loss on calibrated probabilities, where calibration is temperature scaling fitted on train-only out-of-fold (OOF) predictions.
For hyperparameter tuning, should ...
3
votes
1
answer
32
views
Why are all my tuned models (DT, GB, SVM) plateauing at ~70% F1 after rigorous data cleaning and feature engineering?
I'm working on a classification problem where the goal is to maximize the F1-score, hopefully above 80%. Despite a very thorough EDA and preprocessing workflow, I've hit a hard performance ceiling ...
- 31
0
votes
0
answers
24
views
How can I improve mAP when optimizing YOLOv8 hyperparameters with metaheuristic algorithms (e.g., GWO)?
I am working on hyperparameter optimization for YOLOv8 using a metaheuristic algorithm. Currently, I am testing the Grey Wolf Optimizer (GWO), but I plan to also experiment with other algorithms.
Here ...
- 101
1
vote
0
answers
32
views
Confused how to determine dataset splits for feature selection and hyper parameter tuning
I have a fraud detection dataset with a datetime feature. My current workflow is:
Split the dataset into train and test sets.
Feature selection on the train set by applying filter methods to drop ...
- 31
2
votes
1
answer
160
views
How many folds should a unnnested CV have compared to a nested CV
I read in the mlr3 book about nested resampling that:
Nested resampling is a method to compare models and to estimate the generalization
performance of a tuned model, however, this is the performance ...
- 43
1
vote
1
answer
87
views
Comparing two cross validation methods for hyperparameter tuning
For cross validation of hyperparameters, I have a question about which approach is generally considered better in the context of running regularized regression (specifically elastic net l1, l2 ...
- 111
0
votes
0
answers
83
views
Hyperparameter tuning for CatBoost
I'm fine-tuning hyperparameter of my CatBoost model for a multi-class problems. May I ask is this all the hyperparameter that I should tune, and if the range is the optimal range to start-tuning?
<...
- 33
0
votes
1
answer
65
views
Consistency of classical estimation of hyper-parameters from multiple samples
Let $\sigma>0$.
Suppose we observe $N$ samples of sizes $T_1,\dots,T_n$ that are each generated by the following data generating process:
$\theta_n$ is drawn from the distribution $\mathrm{N}(0,\...
- 565
2
votes
1
answer
156
views
LightGBM Hyperparameter Tuning Using Hyperopt Fmin
I'm encountering a multi-classification problem. I'm trying to use Hyperopt Fmin to perform hyperparameter tuning. However, I have no idea what's the appropriate value that I should use for the search ...
- 33
0
votes
0
answers
52
views
How to Choose Regularization Hyperparameters Based on Consistency vs. Accuracy in Model Training?
I’m training a machine learning model and optimizing the regularization hyperparameters to ensure the model generalizes well. During training, I include regularization terms in the loss function to ...
- 1
0
votes
0
answers
26
views
Does model retrain frequency in time series CV have to match production retrain frequency?
Lets assume that we retrain the model every year in production and we have accumulating 50 years of data. If using a time series CV (e.g TimeSeriesSplit in sklearn) for hyperparams recalibration at ...
- 121
3
votes
1
answer
372
views
What should the objective be when tuning hyperparameters to minimize overfitting?
I'm working on a classification problem with ~90k data rows and 12 features. I'm trying to tune the hyperparamters of an XGBoost model to minimize the overfitting. I use ROC_AUC as the metric to ...
0
votes
0
answers
90
views
How to tune hyperparameters for low calibration error under small dataset
I'm studying which variant of variational autoencoders (VAE) gives better expected calibration error (ECE) (see also this doc) under small dataset. According to google's tuning playbook, to compare ...
- 307
3
votes
2
answers
309
views
Why use nested validation when doing both hyper-parameter tuning and model selection?
The monograph Cross Validation contains a section on nested cross-validation for hyper-parameter optimisation (page 6). The author refers to this paper for a reason why it is better to decouple hp-...
- 43
1
vote
0
answers
70
views
Is my XGBoost Model Still Overfitting (Binary Classifcation)?
I am trying to build a binary classification model with XGBoost. I made sure to split my data into the training, validation and test sets. I performed feature selection, early stoppage and ...
- 11