edit: cast attention_mask to long in DataCollatorCTCWithPadding

[ddobokki]

What does this PR do?

many inf values generated when training Wav2Vec2ForCTC by referring to run_speech_recognition_ctc.py using DeepSpeed library.

because Wav2Vec2ForCTC's forword has logics that sum attention_mask, so if you training model using DeepSpeed,

transformers/src/transformers/trainer.py

Line 2390 in 7e7f62b

def _prepare_input(self, data: Union[torch.Tensor, Any]) -> Union[torch.Tensor, Any]:

this method cast attention_mask's dtype int32 to float16

Wav2Vec2FeatureExtractor makes attention_mask and it's dtype int32
here is example

import torch
from transformers import Wav2Vec2FeatureExtractor

feature_extractor = Wav2Vec2FeatureExtractor(return_attention_mask=True)
data = [{'input_values':[0.1,0.1,0.1]},{'input_values':[0.2,0.2,0.2,0.2,0.2]}]
attn_mask = feature_extractor.pad(data,padding = "longest",return_tensors="pt")['attention_mask']
print(attn_mask.dtype)
-> torch.int32

so i add one line in DataCollatorCTCWithPadding that attention_mask casting long type from int32

batch['attention_mask'] = batch['attention_mask'].to(torch.long)

Fixes # 18080

Before submitting

• This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
• Did you read the contributor guideline,
  Pull Request section?
• Was this discussed/approved via a Github issue or the forum? Please add a link
  to it if that's the case.
• Did you make sure to update the documentation with your changes? Here are the
  documentation guidelines, and
  here are tips on formatting docstrings.
• Did you write any new necessary tests?

Who can review?

@patrickvonplaten

[HuggingFaceDocBuilderDev]

The documentation is not available anymore as the PR was closed or merged.

[ddobokki]

i add more line

if "attention_mask" in batch:

bcz some case, feature_extractor has config that ["return_attention_mask": false]
but is

if self.processor.feature_extractor.return_attention_mask:

more good to read? if so, i'll change that

[patrickvonplaten]

Thanks!

[sgugger]

Thanks for fixing!