All Questions
Tagged with bert-language-model python
795 questions
0
votes
1
answer
124
views
torch.OutOfMemoryError: CUDA out of memory. (Google Colab)
I tried to adapt the mBERT model to an existing code. However, I received the following issue even though I tried different solutions.
torch.OutOfMemoryError: CUDA out of memory. Tried to allocate 20....
- 55
1
vote
0
answers
140
views
Why does my PyTorch DataLoader only use one CPU core despite setting num_workers>1?
I am trying to fine-tune BERT for a multi-label classification task (Jigsaw toxic comments). I created a custom dataset and DataLoader as follows:
class CustomDataSet(Dataset):
def __init__(...
- 57
1
vote
2
answers
65
views
dropout(): argument 'input' (position 1) must be Tensor, not str BERT Issue
I was trying to run some epochs to train my sentiment analysis model, at the very last passage, the epochs stopped with the error in the title. I attach the codes here:
Sentiment classifier:
# Build ...
0
votes
0
answers
49
views
Error: torch.dtype' object has no attribute 'base_dtype
I am trying to train the BERT model but I haven't figured out the structure of TensorFlow yet. In the line for x = self.bert_module(book) an error occurs.
Exception encountered when calling layer '...
0
votes
0
answers
78
views
Emotion Analysis with bhadresh-savani/bert-base-uncased-emotion
Hope I can get some help here please!
I am trying to run an emotion analysis model from Hugging Face rep. (bhadresh-savani/bert-base-uncased-emotion) and I am struggling with the model run as it's ...
2
votes
1
answer
39
views
How to convert character indices to BERT token indices
I am working with a question-answer dataset UCLNLP/adversarial_qa.
from datasets import load_dataset
ds = load_dataset("UCLNLP/adversarial_qa", "adversarialQA")
How do I map ...
- 632
0
votes
0
answers
17
views
Bert models show tokenizing statistics
Is there any built-in way to request some tokenizing statistics when using BertTokenizer.from_pretrained('bert-base-uncased') and BertModel.from_pretrained('bert-base-uncased') to understand how ...
0
votes
0
answers
47
views
Save Updated Weights/Size for Pretrained Bert Model (I think I'm asking this correctly)
I'm a machine learning newbie, so I'm sorry if this isn't incredibly clear but I'm going to try and be as concise as I possibly can.
I'm fine tuning a pretrained BERT model, specficially 'bert-based-...
- 23
2
votes
1
answer
58
views
Dutch sentiment analysis RobBERTje outputs just positive/negative labels, netural label is missing
When I run Dutch sentiment analysis RobBERTje, it outputs just positive/negative labels, netural label is missing in the data.
https://huggingface.co/DTAI-KULeuven/robbert-v2-dutch-sentiment
There are ...
- 473
0
votes
1
answer
65
views
Why is Keras pretrained BERT MaskedLM producing inconsistent predictions?
I am trying to use keras-nlp with a pretrained masked BERT model to predict some tokens in a sequence. However the model produces inconsistent results. What could be wrong or am i misunderstanding ...
- 67
0
votes
1
answer
48
views
OutOfMemory while training pre-trained BERT model for token classification task
I am using pre-trained BertForTokenClassification for nested Named Entities Recognition task. To define nested entities, I am using multi-labels method. In the output model returns 3 lists of logits, ...
0
votes
1
answer
326
views
How do I freeze only some embedding indices with tied embeddings?
I found in Is it possible to freeze only certain embedding weights in the embedding layer in pytorch? a nice way to freeze only some indices of an embedding layer.
However, while including it in a ...
- 11
0
votes
1
answer
303
views
How do I add a CRF layer to a BERT model for NER tasks?
I have created an NER model using BERT to detect medical entities which works great. I'm trying to add a CRF layer on top of my BERT model to enhance its performances but I'm getting an error that I ...
- 71
0
votes
1
answer
84
views
SBERT Fine-tuning always stops before finish all epochs
I'm working on a project using the SBERT pre-trained models (specifically MiniLM) for a text classification project with 995 classifications. I am following the steps laid out here for the most part ...
- 11
0
votes
0
answers
46
views
No bias Correction for Bert
im trying to take a Kfold Test on my BertModel.
First i trained my model for reveling PII and i saved the state of it in 2 file : optimazer.pt and bert_model.pt
Here's how
def save_model(self, ...
