#8491 语音识别阶段出错[Huggingface_ASR(内置)] The expanded size of the tensor (3000) must match the existing size (3401) at non-singleto

2408**d7be Posted at: 2 hours ago

语音识别阶段出错[Huggingface_ASR(内置)] The expanded size of the tensor (3000) must match the existing size (3401) at non-singleton dimension 1. Target sizes: [128, 3000]. Tensor sizes: [128, 3401]:Traceback (most recent call last):
File "videotrans\process\stt_pipe.py", line 61, in pipe_asr
File "D:\Video-Restoration\pyvideotrans-v4.17\_internal\transformers\pipelines\automatic_speech_recognition.py", line 247, in call

return super().__call__(inputs, **kwargs)

File "D:\Video-Restoration\pyvideotrans-v4.17\_internal\transformers\pipelines\base.py", line 1257, in call

outputs = list(final_iterator)

File "D:\Video-Restoration\pyvideotrans-v4.17\_internal\transformers\pipelines\pt_utils.py", line 126, in next

item = next(self.iterator)

File "D:\Video-Restoration\pyvideotrans-v4.17\_internal\transformers\pipelines\pt_utils.py", line 271, in next

processed = self.infer(next(self.iterator), **self.params)

File "D:\Video-Restoration\pyvideotrans-v
......
e "D:\Video-Restoration\pyvideotrans-v4.17\_internal\transformers\pipelines\pt_utils.py", line 126, in next

item = next(self.iterator)

File "D:\Video-Restoration\pyvideotrans-v4.17\_internal\transformers\pipelines\pt_utils.py", line 271, in next

processed = self.infer(next(self.iterator), **self.params)

File "D:\Video-Restoration\pyvideotrans-v4.17\_internal\torch\utils\data\dataloader.py", line 733, in next

data = self._next_data()

File "D:\Video-Restoration\pyvideotrans-v4.17\_internal\torch\utils\data\dataloader.py", line 789, in _next_data

data = self._dataset_fetcher.fetch(index)  # may raise StopIteration

File "D:\Video-Restoration\pyvideotrans-v4.17\_internal\torch\utils\data\_utils\fetch.py", line 43, in fetch

return self.collate_fn(data)

File "D:\Video-Restoration\pyvideotrans-v4.17\_internal\transformers\pipelines\base.py", line 174, in inner

padded[key] = _pad(items, key, _padding_value, padding_side)

File "D:\Video-Restoration\pyvideotrans-v4.17\_internal\transformers\pipelines\base.py", line 110, in _pad

tensor[i, : len(item[key][0])] = item[key][0]

RuntimeError: The expanded size of the tensor (3000) must match the existing size (3401) at non-singleton dimension 1. Target sizes: [128, 3000]. Tensor sizes: [128, 3401]
trk=[SpeechToText]: uuid='db73c32c97', proxy_str=None, cfg=[TaskCfgSTT]
原始输入文件名: D:/Video-Restoration/Video/Restoration/SNOS-400.restored.mp4,
输出结果保存到文件夹: D:/Video-Restoration/pyvideotrans-v4.17/output/recogn,
临时文件夹: D:/Video-Restoration/pyvideotrans-v4.17/tmp/38500/db73c32c97
已启用CUDA加速
未选 降噪
语音识别:Huggingface_ASR(内置), kotoba-tech/kotoba-whisper-v2.0
发音语言:ja
未选 降噪, precent=1, hasend=False, should_recogn=True, should_trans=False, should_dubbing=False, should_hebing=False, out_format='srt', copysrt_rawvideo=True, spk_insert=False


=
system:Windows-10-10.0.26300-SP0
version:v4.17
frozen:True
language:zh_CN
root_dir:D:/Video-Restoration/pyvideotrans-v4.17

3 Replies

#1 DeepSeek reply at 2 hours ago

这是 Huggingface_ASR(内置) 渠道在处理多个长度不一的音频切片、批量送入 pipeline 时,特征张量 padding 维度不一致导致的内部错误(kotoba-whisper-v2.0 模型与当前内置 transformers 版本存在兼容问题),并非你的操作或视频文件本身的问题。

建议处理方式:

  1. 更换语音识别渠道(最快解决)
    日语识别建议改用以下任一渠道,效果稳定:

    • openai-whisper(内置) + 模型选 large-v3
    • faster-whisper(内置) + 模型选 large-v3
    • Nemotron-3.5-asr-0.6b(内置)(英伟达多语言模型,日语支持较好)
  2. 若仍想用 Huggingface_ASR 渠道
    可尝试改选该渠道内其他日语模型(如 japanese-wav2vec2-large-rs35kh),避开 kotoba-whisper-v2.0。
  3. 重试前注意事项
    重新翻译同一视频时,请勾选主界面 清理已生成,避免复用上次出错阶段的缓存。
  4. 该错误属于特定模型在批量推理时的张量对齐缺陷,需要开发者在后续版本修复,当前阶段建议直接用上述替代渠道完成识别。

请查阅相关文档:

#2 2408**d7be reply at 2 hours ago

image
2小时日语视频 用哪种最优?

#3 DeepSeek reply at 2 hours ago

2 小时日语长视频,推荐按以下优先级选择语音识别渠道:

首选(速度与质量兼顾)

  • faster-whisper(内置) + 模型 large-v3-turbo,勾选 CUDA加速

    • 2 小时视频用 large-v3 会耗时很久,turbo 在日语上质量仍很高,速度明显更快,适合长视频。

质量优先(能接受更长耗时)

  • openai-whisper(内置) + large-v3,或 faster-whisper(内置) + large-v3

    • 准确度略高,但 large-v3 需 8GB+ 显存,2 小时耗时很长。

日语专用备选

  • Nemotron-3.5-asr-0.6b(内置):英伟达多语言模型,日语支持较好,速度较快。
  • 不建议继续用 Huggingface_ASR 的 kotoba-whisper-v2.0(即你上次报错的模型)。

提升长视频日语识别质量的关键设置

  1. 主界面 设置更多参数 中勾选 降噪(有 N 卡可在 高级选项--降噪模型 选 zipenhancer 走 GPU)。若视频有背景音乐/噪声,可勾选 分离人声背景声。
  2. 发音语言 明确选 日语,不要选“自动检测”。
  3. 建议勾选 LLM纠错,在 高级选项--语音识别参数--LLM纠错所用AI渠道 选 DeepSeek 等旗舰模型,修正错别字和补标点(2 小时视频字幕量大,纠错能明显提升后续翻译质量)。
  4. 若字幕过碎或过长,在 高级选项--语音识别参数 调整:

    • 最长语音持续:5~8 秒
    • 最短语音持续:3000 毫秒
    • 静音分割持续:600~800 毫秒

注意

  • 2 小时长视频处理时间较长,确保磁盘有充足空间(tmp/ 会生成大量缓存)。
  • 重试同一视频务必勾选 清理已生成,避免复用上次出错阶段的缓存。

请查阅相关文档:

Thinking...
Post Your Reply
If you have more than 50 messages, you will no longer receive an automatic reply. Please ask a new question!
Open source and free maintenance is not easy. If this project is helpful to you, please consider making a small donation to help the project continue to maintain and update.

Dondate via ko-fi.com

Related resource