#5965 Couldn't instantiate the backend tokenizer from one of: (1) a `tokenizers` library serialization file,

2a01:4f8* Posted at: 15 hours ago

Couldn't instantiate the backend tokenizer from one of:
(1) a tokenizers library serialization file,
(2) a slow tokenizer instance to convert or
(3) an equivalent slow tokenizer class to instantiate and convert.
You need to have sentencepiece or tiktoken installed to convert a slow tokenizer to a fast one.
Traceback (most recent call last):
File "videotrans\task\only_one.py", line 68, in run
File "videotrans\task\_stage_translate.py", line 31, in trans
File "videotrans\translator\_runner.py", line 63, in run
File "videotrans\translator\_base.py", line 68, in run
File "videotrans\translator\_base.py", line 97, in _run_text
File "videotrans\translator\_base.py", line 142, in _hymt2
File "D:\pyvideotrans-v4.06\_internal\transformers\models\auto\tokenization_auto.py", line 773, in from_pretrained

return tokenizer_class.from_pretrained(pretrained_model_name_or_path, *inputs, **kwargs)

File "D:\pyvideotrans-v4.06\_internal\transformers\tokenization_utils_base.py", line 1721, in from_pretrained

return cls._from_pretrained(

File "D:\pyvideotrans-v4.06\_internal\transformers\tokenization_utils_base.py", line 1910, in _from_pretrained

tokenizer = cls(*init_inputs, **init_kwargs)

File "D:\pyvideotrans-v4.06\_internal\transformers\tokenization_utils_tokenizers.py", line 341, in init

raise ValueError(

ValueError: Couldn't instantiate the backend tokenizer from one of:
(1) a tokenizers library serialization file,
(2) a slow tokenizer instance to convert or
(3) an equivalent slow tokenizer class to instantiate and convert.
You need to have sentencepiece or tiktoken installed to convert a slow tokenizer to a fast one.

cfg=[TaskCfgVTT]当前工作模式: 翻译视频或音频 单视频模式
原始输入文件名: C:/Users/Administrator/Desktop/c71818a3f11c3f2778b88165fc6f79f5.mp4,
输出结果保存到文件夹: E:/翻译后/c71818a3f11c3f2778b88165fc6f79f5-mp4,
临时文件夹: D:/pyvideotrans-v4.06/tmp/22116/7b181e8940
已选 清理已存在
已选 启用CUDA加速
未选 降噪
语音识别:Qwen-ASR(本地内置), model_name: 1.7B, 发音语言: 简体中文, 断句方式:默认断句
翻译渠道:Hy-MT2-1.8B(本地内置),原始语言:简体中文,目标语言:英语, 已选 发送完整字幕
配音渠道:Qwen3-TTS(本地内置), 角色:Dylan, 配音语言:英语, 未选 二次语音识别
音量:+0%, 语速:+0%, 已选 音频加速, 已选 视频慢速
字幕: 嵌入硬字幕
代理地址:


=
system:Windows-10-10.0.26200-SP0
version:v4.06
frozen:True
language:zh
root_dir:D:/pyvideotrans-v4.06

1 Replies

此错误是因为 Hy-MT2-1.8B 本地模型的 tokenizer 文件缺失或损坏导致。

解决方案

  1. 关闭软件,进入软件安装目录下的 models/models--tencent--Hy-MT2-1.8B 文件夹,删除其所有内容。
  2. 重新打开软件,再次执行翻译任务。软件会自动从 Hugging Face 重新下载完整模型文件(需保持网络畅通)。
  3. 若自动下载失败,请手动下载模型文件并放置到该目录:

请查阅相关文档:

Post Your Reply
Open source and free maintenance is not easy. If this project is helpful to you, please consider making a small donation to help the project continue to maintain and update.

Dondate via ko-fi.com

Dondate via PayPal

Related resource