翻译字幕阶段出错 [Hy-MT2-1.8B(内置)] Couldn't instantiate the backend tokenizer from one of:
(1) a tokenizers library serialization file,
(2) a slow tokenizer instance to convert or
(3) an equivalent slow tokenizer class to instantiate and convert.
You need to have sentencepiece or tiktoken installed to convert a slow tokenizer to a fast one.
Traceback (most recent call last):
File "videotrans\task\job.py", line 35, in run
File "videotrans\task\job.py", line 136, in process_task
File "videotrans\task\translate_srt.py", line 47, in trans
File "videotrans\translator\_runner.py", line 63, in run
File "videotrans\translator\_base.py", line 68, in run
File "videotrans\translator\_base.py", line 97, in _run_text
File "videotrans\translator\_base.py", line 141, in _hymt2
File "D:\FanYiTool\pyvideotrans4.08\_internal\transformers\models\auto\tokenization_auto.py", line 749, in from_pretrained
return tokenizer_class.from_pretrained(pretrained_model_name_or_path, *inputs, **kwargs)
File "D:\FanYiTool\pyvideotrans4.08\_internal\transformers\tokenization_utils_base.py", line 1721, in from_pretrained
return cls._from_pretrained(
File "D:\FanYiTool\pyvideotrans4.08\_internal\transformers\tokenization_utils_base.py", line 1910, in _from_pretrained
tokenizer = cls(*init_inputs, **init_kwargs)
File "D:\FanYiTool\pyvideotrans4.08\_internal\transformers\tokenization_utils_tokenizers.py", line 341, in init
raise ValueError(
ValueError: Couldn't instantiate the backend tokenizer from one of:
(1) a tokenizers library serialization file,
(2) a slow tokenizer instance to convert or
(3) an equivalent slow tokenizer class to instantiate and convert.
You need to have sentencepiece or tiktoken installed to convert a slow tokenizer to a fast one.
cfg=TaskCfgSTS(uuid='4d20ce1a3e', name='D:/需要翻译的/105373-720p.ja.srt', dirname='D:/需要翻译的', noextname='105373-720p.ja', basename='105373-720p.ja.srt', ext='srt', target_dir='D:/FanYiTool/pyvideotrans4.08/output/translate', cache_folder=None, is_cuda=False, source_language=None, source_language_code='-', source_sub='D:/需要翻译的/105373-720p.ja.srt', source_wav=None, source_wav_output=None, target_language=None, target_language_code='Simplified Chinese', target_sub='D:/FanYiTool/pyvideotrans4.08/output/translate/105373-720p.ja.Simplified Chinese.srt', target_wav=None, target_wav_output=None, translate_type=3)
=
system:Windows-10-10.0.19045-SP0
version:v4.08
frozen:True
language:zh
root_dir:D:/FanYiTool/pyvideotrans4.08