pyVideoTrans

Appearance

Recommended Best Settings for Video Translation ​

This guide outlines recommended configuration profiles across the four main stages of the video translation pipeline.

Stage 1: Speech Recognition (STT) ​

Goal: Accurately transcribe spoken dialogue from the video into a timestamped subtitle file.

πŸ’‘ Tip: If the source media contains background music or ambient noise, click More... on the main interface and check Separate Vocal/BGM. Isolating the vocals removes interference and significantly improves recognition accuracy.

Non-Chinese Media ​

ProfileChannel / ProviderModelNotes
Free (Recommended)faster-whisper (Local)large-v3Best balance of speed and quality
Free (Alternative)openai-whisper (Local)large-v3Slightly higher accuracy
Free (Alternative)whisper.cpp (Win Built-in)large-v3Fast; pre-packaged on Windows
Free (Alternative)Nemotron-3.5-asr-0.6b (Local)β€”Fast; built-in local model
Paid (Recommended)OpenAI STT APIβ€”Excellent transcription accuracy

Chinese Media ​

ProfileChannel / ProviderModelNotes
Free (Recommended)Qwen-ASR (Local)1.7BOutstanding Chinese recognition
Free (Alternative)Alibaba FunASR (Local)paraformer-zhStrong Chinese accuracy
Free (Alternative)FireRed Chinese (Local)FireRed modelsExcellent for Chinese & dialects
Free (Alternative)Huggingface_ASR (Local)zai-org/GLM-ASR-Nano-2512Great Zhipu AI Chinese model
Paid (Recommended)ByteDance STT Turboβ€”Excellent performance
Paid (Recommended)Xiaomi mimo-v2.5-asrβ€”High Chinese accuracy
Paid (Alternative)Alibaba Bailian ASRβ€”Optimized for Chinese

Japanese Media ​

ProfileChannel / ProviderModelNotes
Free (Recommended)openai-whisper (Local)large-v3High general accuracy
Free (Recommended)Nemotron-3.5-asr-0.6b (Local)NVIDIA MultilingualGreat results
Free (Alternative)Huggingface_ASRjapanese-wav2vec2-large / Nemotron streamingTailored for Japanese
Paid (Recommended)OpenAI STT APIβ€”Excellent accuracy

Low-Resource & Minor Languages ​

ProfileChannel / ProviderNotes
Free (Recommended)openai-whisper (Local) large-v3General-purpose model supporting dozens of languages
Free (Recommended)Dolphin Asian (Local)Specialized for Asian languages (supports Auto Detect in batch STT)
Free (Recommended)Omnilingual (Local)Covers 1,600+ languages (supports Auto Detect in batch STT)
Free (Alternative)Huggingface_ASR (Local)Community fine-tuned models for specific regional languages
Paid (Recommended)Gemini STT / OpenAI APIExceptional quality on low-resource languages

Notes:

  • Explicitly setting the Spoken language yields better accuracy than relying on Auto Detect.
  • When using local models, processing will run slowly without an NVIDIA GPU and enabled CUDA acceleration. Insufficient VRAM may cause crashes.

View Usage Details for All STT Engines

Stage 2: Subtitle Translation ​

Goal: Translate the transcribed subtitles into your target language.

ProfileChannel / ProviderNotes
Top PickDeepSeek / OpenAI ChatGPT / Gemini (Latest Models)Highest translation quality
FreeGoogle Translate (Free) / Microsoft Translate (Free)Traditional MT; fast and free
LocalM2M100Completely offline local translation
LocalHy-MT2-1.8BTencent open-source translation model

Key Settings:

  • Enable Send full SRT format for AI translation (SendSRT) for natural phrasing.

  • For AI translation, set AI translation batch size (lines) to 100 or higher with a long-context model.

  • Under Advanced Options -> Subtitle Translation, check AI translation includes all original subtitles for reference? to provide full context (available from v4.12).

Stage 3: Voice Dubbing (TTS) ​

Goal: Generate synthesized audio for each translated subtitle line.

ProfileChannel / ProviderNotes
FreeEdge-TTS (Free)Free Microsoft service; natural voices across all major languages
LocalQwen-TTS, F5-TTS, OmniVoice, ConfuciusLocal models; support voice cloning
CommercialDoubao Voice 2.0 / Qwen-TTS (Bailian) / Xiaomi / MiniMax / OpenAI TTSHigh-grade commercial cloud APIs
Voice CloningOmniVoice / Confucius / Qwen-TTS / GPT-SoVITS / CosyVoice / F5-TTS / Index-TTS / ChatterBox / ZipVoice / FireRed3-TTSClones the original speaker's timbre

View All Dubbing Services and Configuration Details

Stage 4: Subtitle, Audio, and Video Sync Alignment ​

Goal: Keep dubbed speech, subtitles, and video pacing perfectly in sync.

SettingPurpose
Enable SecondarySTTRe-transcribes the generated dubbing track to create concise subtitles with frame-accurate timestamps
Adjust Speed+%When translating from Chinese to English, set +10 or +15 to match faster English syllable delivery
Enable Auto SpeedupAutomatically speeds up dubbing when audio duration exceeds the original video segment
Enable Slow VideoWorks alongside Auto Speedup to gently stretch video scenes to fit longer dubbing lines
Enable Separate Vocal/BGMAllows re-embedding original background music without vocal echo
Enable Noise ReductionCleans source audio to improve initial transcription precision

Stage 5: Advanced Quality Optimization ​

General Best Practices ​

  1. Check Send full SRT format for AI translation (SendSRT).
  2. Check Tools / Options (&T) -> Advanced Options -> AI translation includes all original subtitles for reference?.
  3. Set AI translation batch size (lines) to 100 or more.
  4. Always select an online AI LLM that supports long context windows.

Voice Cloning (clone Role) Best Practices ​

  1. Configure Tools / Options (&T) -> Advanced Options -> ASR Settings:
    • Min Speech Duration (ms): 3000
    • Max Speech Duration (sec): 10
  2. Use an AI engine (DeepSeek or OpenAI ChatGPT) for translation, with SendSRT enabled.
  3. Speech Recognition: For Chinese, use ByteDance STT Turbo, Qwen-ASR, FunASR, or FireRed Chinese. For English, choose faster-whisper + large-v3 with Default segment.
  4. If background music needs to be preserved, open More... and check Separate Vocal/BGM along with Re-embed BGM; otherwise, enable Noise Reduction.

Frequently Asked Questions (FAQ) ​

Q: What are the requirements for lossless video output? ​

The original video must be an H.264 MP4 file, Slow Video must be disabled, subtitle mode must be No Subtitles or Soft Subtitles, and codec must be set to 264.

Q: Why is processing running slowly? ​

  1. Make sure CUDA GPU acceleration is enabled.
  2. Use smaller, faster models.
  3. Verify that your NVIDIA GPU drivers are up to date.

Q: How do I enable GPU acceleration? ​

Verify that NVIDIA drivers, CUDA 12.8, and cuDNN 9.x are installed, then check Enable CUDA? on the main window. AMD graphics cards do not support CUDA.