Questions tagged [hyperparameter]
Hyperparameters of a model are the kind of parameters that cannot be directly learned during training but are set beforehand. Hyperparameters can define, for example, the complexity of the model or its capacity to learn.
153 questions
3
votes
1
answer
101
views
K Fold Cross Validation - Manual Exploration or Use the result?
I have a dataset which I split into training, testing, and out-of-time sets. Then I feed my training set into K Fold CV.
I understand that K Fold Cross Validation is used as a method to select the &...
- 131
1
vote
0
answers
52
views
Choosing the number of features via cross-validation
I have an algorithm that trains a binary predictive model for a specified number of features from the dataset (features are all of the same type, but not all important.) Thus, the number of features ...
- 117
1
vote
2
answers
195
views
xgboost - colsample_bylevel & colsample_bynode
I don't really understand the meaning of these xgboost parameters or how they differ:
If I specify exactly one of these parameters (not both at the same time),
...
- 11
1
vote
0
answers
56
views
Hyper parameter tuning LSTM network on time series data
I am trying to train LSTM model (containing four LSTM layers (500 units each) and three droupouts and a fully connected output layer to do regression) on timeseries data. To start with, I tried to ...
- 299
1
vote
0
answers
28
views
Why I am requiring tiny learning rate to overfit the model?
I am trying to train LSTM model on a timeseries data with 1.6 million records. I have taken window size of 200.
Initially I tried to overfit the model (train data = test data) on tiny dataset (few ...
- 299
2
votes
1
answer
189
views
The best algorithm(s) for finding the best hyperparameters (special case)
I would like to ask for help with the following.
Given the following dataset, which I have split into train and test sets:
...
0
votes
1
answer
239
views
Optimal combination of variables to minimise output
To be honest I'm not 100% sure how much this is purely a coding issue or a data science issue, but I'll take my chances. I've developed a matrix which is a mixture of various hyperparameters, the ...
0
votes
0
answers
60
views
Hyperparameter Widget
Could you please inform me if there exists a widget designed for the purpose of conducting hyperparameter optimization? I attempted to locate such a tool, but regrettably, I was unable to find one.
- 1
0
votes
1
answer
112
views
How does deeper or shallower trees (higher or lower max_depth) affect xgboost model?
I am doing an xgboost model for landslides assessment and I am using max_depth as one of my hyperparameters, but I don't understand how does it affect model ...
- 3
1
vote
1
answer
47
views
Hyperparameter tuning
Jane trains three different classifiers: Logistic Regression, Decision Tree, and Support Vector Machines on the
training set. Each classifier has one hyper-parameter (regularisation parameter, depth-...
- 11
1
vote
1
answer
745
views
Ordering of Train/Val/Test set use in hyperparameter tuning
The way I read almost lots of ML advice on these datasets sounds like "You train a model that's randomly chosen hyperparameters first on the training set, then you ignore this bit of the work, ...
- 121
0
votes
1
answer
234
views
Optimizing decision tree
I have a question regarding the technique/technology which could be applied for the issue:
Suppose I have a rule-based tree or decision tree which predicts a variable Y based on variables A,B,C. This ...
- 1
0
votes
1
answer
160
views
Visualizing correlation between hyper-parameters and metrics for Neural Network
I am working with a neural network and I want to investigate how different settings affect the loss and standard deviation of the network. I can change various parameters such as the loss function, ...
- 101
0
votes
1
answer
817
views
Is hyperparameter tuning done on training or validation data set?
Is hyperparameter tuning done on training or validation data set? The post here gives mixed opinion as of whether the training set should be used for hyperparameter tuning. And I would like to know ...
- 421
8
votes
1
answer
3k
views
Do transformers (e.g. BERT) have an unlimited input size?
There are various sources on the internet that claim that BERT has a fixed input size of 512 tokens (e.g. this, this, this, this ...). This magical number also appears in the BERT paper (Devlin et al. ...
- 263