20,416 questions
0
votes
0
answers
15
views
stuck in reducing docker image size
I'm working on a text summarization logic using Google's t5-small model. My implementation relies on the torch and transformers libraries. However, the problem is that when I build the Docker image, ...
- 15
0
votes
0
answers
15
views
import gensim binary incompatibility
import gensim
import numpy
import scipy
print("gensim version:", gensim.__version__)
print("numpy version:", numpy.__version__)
print("scipy version:", scipy.__version__)
...
-2
votes
0
answers
35
views
For training a neural network, if i have label encoded my feature, is there need to scale it or normalize it?
I am working on a project which predict customer satisfaction score. I have several categorical features. One feature has 3 unique value while some has 59 and 1600 unique values. My question is can I ...
3
votes
0
answers
30
views
Can older spaCy models be ported to future spaCy versions?
The latest spaCy versions have better performance and compatibility for GPU acceleration on Apple devices, but I have an existing project that depends on spaCy 3.1.4 and some of the specific behavior ...
- 2,105
-1
votes
1
answer
42
views
Unsupervised Topic Modeling for Short Event Descriptions
I have a dataset of approximately 750 lines containing quite short texts (less than 150 words each). These are all event descriptions related to a single broad topic (which I cannot specify for ...
1
vote
0
answers
21
views
Is there a way to reuse a heavy service across tasks in Airflow?
I'm building an Airflow DAG where some of the steps should do ML/NLP processing.
I have a service class that loads NLP model in constructor. E.g.:
class SentenceService:
def __init__(self, model: ...
- 304
3
votes
2
answers
666
views
NameError: name 'init_empty_weights' is not defined while using hugging face models
I am trying to set up hugging face locally and im running into this issue.
NameError: name 'init_empty_weights' is not defined
Here is the code I have tested my installation with
from transformers ...
- 105
0
votes
0
answers
55
views
Sentencepiece not generating models after preprocessing (SOLVED)
So this is the log that I see on the terminal:
sentencepiece_trainer.cc(78) LOG(INFO) Starts training with :
trainer_spec {
input: C:\Users\xxxx\OneDrive\Documents\Projects\py\xxxxx\data\...
0
votes
0
answers
36
views
No attention output in jinaai/jina-embeddings-v3 embedding model
When I use this model like so -
from transformers import AutoModel, AutoTokenizer
model_id = "jinaai/jina-embeddings-v3"
tokenizer = AutoTokenizer.from_pretrained(model_id, ...
0
votes
1
answer
105
views
Why does Presidio with spacy nlp engine not recognize organizations and PESEL while spaCy does?
I'm using spaCy with the pl_core_news_lg model to extract named entities from Polish text. It correctly detects both organizations (ORG) and people's names (PER):
import spacy
nlp = spacy.load("...
- 79
0
votes
1
answer
57
views
GPT-2 and other models from huggingface -100 label index for training, instead of pad token [closed]
I understand the -100 label id is used so that the predictions for these are not included when calculating the loss.
However on huggingface, they state
"complicated list comprehension here ...
- 81
0
votes
0
answers
28
views
simpler gmail Filter syntax for "word family" [verif +(y/ied/ification] + similar loanwords [term +(s/es/a)]?
Is there simpler filter that I can use for below cases?
Google has a very smart AI gemini, I hope there is a shortcut for this as I am receiving bilingual emails and loan words in Malay/Indonesia are ...
- 13
0
votes
1
answer
74
views
Creating regular expression(s) which finds capitalization errors
This is a Sentence which contains
Some capitalization errors.
So far I have this: (?<![.!?]\s)(?<!^)(?<!\sI\s)(?!I['’][a-z])(?!\b(?:Dr|Mr|Mrs)\.[\s\r\n])\b(?!I\b)[A-Z]\w*
It will find "...
- 35
0
votes
1
answer
71
views
SFTTrainer Error : prepare_model_for_kbit_training() got an unexpected keyword argument 'gradient_checkpointing_kwargs'
I'm trying to fine-tune a model using SFTTrainer from trl.
This is how my SFTConfig arguments look like,
from trl import SFTConfig
training_arguments = SFTConfig(
output_dir=output_dir,
...
0
votes
0
answers
25
views
AllenNLP all models about ccg_supertagger are unavailable. How to fix or download it?
I am trying to use AllenNLP models to parse a file to create a CCG dataset, because as a student I can't afford the CCGBank dataset, However I have to, cuz I need a dataset to help me to train a model ...
- 1