pyVideoTrans

Appearance

Best Results Recommendations ​

This guide recommends the best configurations for each of the four processing stages.

Stage 1: Speech Recognition ​

Goal: Convert speech in the video into subtitle files in the corresponding language.

πŸ’‘ Tip: If the original audio/video has background noise, it is recommended to click More Settings... on the main interface and enable Split Vocal/BGM. Removing noise interference before recognition will improve accuracy.

Non-Chinese Video ​

LevelProviderModelNotes
Free Recommendedfaster-whisper (Local)large-v3Best balance of speed and quality
Free Alternativeopenai-whisper (Local)large-v3Slightly higher accuracy
Paid RecommendedOpenAI Speech Recognition APIβ€”Excellent quality

Chinese Video ​

LevelProviderModelNotes
Free RecommendedQwen-ASR (Local)β€”Excellent Chinese recognition
Free AlternativeAlibaba FunASRparaformer-zhOptimized for Chinese
Paid RecommendedByteDance ASR Speed Editionβ€”Excellent Chinese recognition
Paid AlternativeAlibaba Bailian ASRβ€”Optimized for Chinese

Japanese Video ​

LevelProviderModelNotes
Free Recommendedopenai-whisper (Local)large-v3Good general performance
Free AlternativeHuggingface_ASRjapanese-wav2vec2-largeJapanese-specific model
Paid RecommendedOpenAI Speech Recognition APIβ€”Excellent quality

Minor Language Video ​

LevelProviderNotes
Free Recommendedopenai-whisper (Local) large-v3General model supporting dozens of languages
Paid RecommendedGemini LLM recognition / OpenAI APIGood for minor languages

Note: When using local models, without an NVIDIA GPU or CUDA acceleration enabled, processing will be very slow. Insufficient VRAM may cause crashes.

Stage 2: Subtitle Translation ​

Goal: Translate the subtitles generated in Stage 1 into the target language.

LevelProviderNotes
PreferredDeepSeek / OpenAI ChatGPT / Gemini (latest model)AI translation produces the best quality
FreeGoogle Translate / Microsoft TranslatorTraditional translation; fast
LocalM2M100Fully offline translation

Key settings:

  • Enable Send Full Subtitles β€” gives AI full context for more natural translation
  • When using AI providers, set AI Translation Batch Size to 100 or more, paired with a model that supports long context

Stage 3: Dubbing ​

Goal: Generate dubbed audio from the translated subtitles.

LevelProviderNotes
Free RecommendedEdge-TTSMicrosoft's free interface; natural quality; supports all languages
Local RecommendedQwen-TTS / F5-TTS / Index-TTS / GPT-SOVITS / CosyVoiceChinese, English, Japanese, Korean; good quality
Paid RecommendedByteDance TTS 2.0 / Qwen-TTS (Bailian) / 302.AI / Minimaxi / OpenAI-TTSHigh-quality commercial APIs
Voice CloningOmniVoice / Qwen-TTS / GPT-SOVITS / CosyVoice / F5-TTS / Index-TTS / ChatterBoxUse original video voice timbre

Stage 4: Subtitle, Dubbing & Video Sync ​

Goal: Synchronize subtitles, dubbing, and video.

SettingNotes
Enable STT againPerforms speech recognition on dubbed audio after dubbing for precise timecodes
Set Dubbing RateWhen translating Chinese to English, set +10 or +15 to speed up dubbing
Enable Dubbing Speedup+Automatically speeds up dubbing when it exceeds the original video length
Also enable Slow VideoCombined with audio speedup for best results
Enable Split Vocal/BGMImproves dubbing quality

Stage 5: Additional Quality Improvements ​

Basic Settings ​

  1. Enable Send Full Subtitles
  2. Enable Tools/Options(T) > More Settings... > AI Translation with Full Original Subtitles
  3. Set AI Translation Batch Size to 100 or more
  4. Must use an online AI model that supports long context

When Using clone Voice for Voice Cloning ​

  1. Do NOT use LLM Resegment β€” re-segmenting the timeline will cause reference audio alignment errors
  2. Configure Tools/Options(T) > More Settings... > ASR Parameters:
    • Min Speech Duration: 3000
    • Max Speech Duration: 10
    • Enable Whisper Pre-segment Audio
    • Enable Merge Short Subtitles
  3. If many subtitles are shorter than 3 seconds, consider using the OmniVoice-TTS provider
  4. For translation, use an AI engine (DeepSeek / OpenAI ChatGPT) and enable Send Full Subtitles
  5. For ASR: Chinese recommends ByteDance ASR Speed Edition / Qwen-ASR / FunASR; English recommends Faster-whisper + large-v3 with default segmentation
  6. To re-embed background audio, click More Settings... and enable Split Vocal/BGM; if not needed, enable Denoise instead

FAQ ​

Q: Conditions for lossless video output? ​

Original video must be H.264 MP4 + Slow Video unchecked + no embedded hard subtitles + encoder set to 264.

Q: Processing is very slow? ​

  1. Make sure GPU acceleration (CUDA) is enabled
  2. Use a smaller model
  3. Ensure your GPU drivers are up to date

Q: How to use GPU acceleration? ​

Make sure NVIDIA GPU drivers, CUDA 12.8, and cuDNN 9.11 are installed, then check Enable CUDA? on the main interface. AMD GPUs do not support CUDA acceleration.