1,753 questions
0
votes
1
answer
35
views
How can I decide how many epochs to train for when re-training a model on the full dataset without a validation set?
I have a BERT model that I want to fine-tune. Initially, I use a training dataset, which I split into a training and validation set. During fine-tuning, I monitor the validation loss to ensure that ...
2
votes
1
answer
81
views
How to Identify Similar Code Parts Using CodeBERT Embeddings?
I'm using CodeBERT to compare how similar two pieces of code are. For example:
# Code 1
def calculate_area(radius):
return 3.14 * radius * radius
# Code 2
def compute_circle_area(r):
return 3.14159 * ...
- 21
0
votes
0
answers
25
views
How to change last layer in finetuned model?
When I fine-tuned the model Hubert to detect phoneme, I chose a fine-tuned ASR Hubert model and I removed the last two layers and added a linear layer to the config vocab_size of phoneme. What is ...
- 1
0
votes
0
answers
49
views
How many obs per class are necessary? - transfer learning w. BERT fine-tuning
I seek advice on a classification problem in industry.
The rows in a dataset must be classified/labeled--it lacks a target/column (labels have dot-separated levels like 'x.x.x.x.x.x.x')--during every ...
- 226
0
votes
0
answers
28
views
How to detect out-of-vocabulary words in a prompt
I need to detect words an LLM has no knowledge about, to add RAG-based definition of said word to the prompt, i.e.:
What is the best way to achieve slubalisme using the new fabridocium product ?, ...
- 168
0
votes
1
answer
117
views
Why is my BERT model producing NaN loss during training for multi-label classification on imbalanced data?
I’m running into a frustrating issue while training a BERT-based multi-label text classification model on an imbalanced dataset. After a few epochs, the training loss suddenly becomes NaN, and I can’t ...
- 26
0
votes
1
answer
123
views
torch.OutOfMemoryError: CUDA out of memory. (Google Colab)
I tried to adapt the mBERT model to an existing code. However, I received the following issue even though I tried different solutions.
torch.OutOfMemoryError: CUDA out of memory. Tried to allocate 20....
- 55
0
votes
0
answers
47
views
Is it Possible to feed Embeddings generate by BERT to a LSTM based autoencoder to get the latent space?
I've just learn about how BERT produce embeddings. I might not understand it fully.
I was thinking of doing a project of leveraging those embeddings and feed it to an autoencoder to generate latent ...
0
votes
0
answers
16
views
Is it possible to evaluate Machine Translations using Sentence BERT?
I'm not referring to BERTScore. BERTScore uses token-level word embeddings, you compute pairwise cosine similarity of word embeddings and obtain scores using greedy matching.
I'm referring to Sentence ...
- 41
0
votes
0
answers
49
views
Introducing additional layers (dropout and dense layers) after BERT's output
I'm working on a BERT-based model for fake news detection. While applying additional layers(as my models encounters not getting good accuracy for only BERT model), like dropout and fully connected ...
1
vote
1
answer
231
views
ValueError: Exception encountered when calling layer 'tf_bert_model' (type TFBertModel)
I have been trying to run TFBertModel from Transformers, but it kept on throwing me this error
ValueError Traceback (most recent call last)
Cell In[9], line 1
----> 1 ...
- 13
1
vote
0
answers
140
views
Why does my PyTorch DataLoader only use one CPU core despite setting num_workers>1?
I am trying to fine-tune BERT for a multi-label classification task (Jigsaw toxic comments). I created a custom dataset and DataLoader as follows:
class CustomDataSet(Dataset):
def __init__(...
- 57
0
votes
0
answers
17
views
TypeError: Tuple indices must be integers or slices, not dict
I am trying to add a learning rate scheduler with a finBERT model. However I am getting this error " TypeError: tuple indices must be integers or slices, not dict" while training.
#...
- 1
0
votes
0
answers
24
views
PackagesNotFound error even when verified packages as installed
I am trying to follow this tutorial for BERT topic modeling:
https://jpcompartir.github.io/BertopicR/
library(reticulate)
reticulate::install_miniconda()
library(BertopicR)
BertopicR::...
- 2,109
1
vote
2
answers
65
views
dropout(): argument 'input' (position 1) must be Tensor, not str BERT Issue
I was trying to run some epochs to train my sentiment analysis model, at the very last passage, the epochs stopped with the error in the title. I attach the codes here:
Sentiment classifier:
# Build ...
