Whisper_CPP Speech Recognition Integration

What is this?

Whisper_CPP is a high-performance C++ implementation of OpenAI's Whisper model, integrated with pyVideoTrans via pythonnet and the Whisper.net library. Compared to the native Python implementation, Whisper_CPP offers faster inference speed and lower memory usage.

Key advantages:

• Native C++ implementation for high runtime efficiency
• Supports CUDA (NVIDIA) and Vulkan (AMD) GPU acceleration
• No complex Python dependency installation required
• GGML model file format — smaller file sizes

Supported Models

Whisper_CPP uses GGML format model files (.bin extension) and supports the following models:

Model File	Parameters	VRAM	Quality	Recommended For
ggml-tiny.bin	39M	~1 GB	Basic	Quick testing
ggml-base.bin	74M	~1 GB	Good	Simple daily scenarios
ggml-small.bin	244M	~2 GB	Very good	Multilingual recognition
ggml-medium.bin	769M	~5 GB	Excellent	High-accuracy recognition
ggml-large-v1.bin	1550M	~10 GB	Very good	Large model v1
ggml-large-v2.bin	1550M	~10 GB	Excellent	Large model v2
ggml-large-v3.bin	1550M	~10 GB	Best	Highest accuracy
ggml-large-v3-turbo.bin	1550M	~10 GB	Best and faster	Recommended first choice

Recommendation: ggml-large-v3-turbo.bin maintains the highest accuracy while being faster — it's the best choice for most users.

System Requirements

Windows

• Windows 10/11 64-bit
• For GPU acceleration:
  • NVIDIA GPU: Requires the corresponding version of CUDA Toolkit (supports CUDA 11.x or 12.x)
  • AMD GPU: Requires Vulkan-capable drivers

macOS

• macOS 12.0 or higher
• Supports Apple Silicon (M1/M2/M3) and Intel processors

Installation Steps

Step 1: Download the whisper-cli executable

Based on your operating system and hardware, download the corresponding pre-compiled version:

Download: https://github.com/ggml-org/whisper.cpp/releases/tag/v1.8.2

System Configuration	Download File	Notes
Windows (no NVIDIA GPU)	whisper-bin-x64.zip	CPU version
Windows (NVIDIA GPU, CUDA 12.x)	whisper-cublas-12.4.0-bin-x64.zip	CUDA 12 GPU-accelerated
Windows (NVIDIA GPU, CUDA 11.x)	whisper-cublas-11.8.0-bin-x64.zip	CUDA 11 GPU-accelerated
macOS	whisper-v1.8.2-xcframework.zip	Apple native framework

Step 2: Extract and Organize Files

1. Extract the downloaded archive to a short path without Chinese characters or spaces

   Recommended path: D:\whispercpp or C:\whispercpp

   Note: Paths containing Chinese characters or spaces may cause the program to malfunction. Avoid this.

2. After extraction, verify that whisper-cli.exe is present

3. Ensure the dependency folder structure is correct:

   whispercpp/
   ├── whisper-cli.exe           (main program)
   ├── deps/
   │   ├── Whisper.net.dll       (.NET bindings)
   │   └── native/
   │       └── whisper.cpp native library files
   └── models/
       └── ggml-*.bin            (model files — download separately)

Step 3: Download Model Files

Download GGML format model files from:

• Official recommended source: https://huggingface.co/ggerganov/whisper.cpp/tree/main
• Mirror source (faster in some regions): https://hf-mirror.com/ggerganov/whisper.cpp/tree/main

Place the downloaded .bin files in the models folder inside the Whisper_CPP installation directory.

Step 4: Configure in pyVideoTrans

1. Open pyVideoTrans
2. Click Menu → Speech Recognition Settings → Whisper_CPP
3. Click the "Select whisper-cli" button
4. In the file browser, locate the extracted whisper-cli.exe file
5. Click OK to save the configuration

Key Parameters

Whisper_CPP supports the following adjustable parameters (configurable in source code):

Parameter	Default	Description
no_speech_threshold	-0.8	Speech detection threshold — lower values are more sensitive
logprob_threshold	-1.0	Log probability threshold for filtering low-confidence results
condition_on_previous_text	true	Whether to use previous text as context — enables better coherence

Common Issues

Issue	Possible Cause	Solution
whisper-cli.exe won't start	Missing VC++ runtime	Install Microsoft Visual C++ Redistributable
Model file not found error	Model not downloaded or wrong path	Place .bin model files in the models/ directory
GPU acceleration not working	CUDA/Vulkan environment not configured	Install the corresponding CUDA Toolkit or update GPU drivers
Very slow recognition	Using the CPU version	Download the CUDA-accelerated version and configure correctly
Inaccurate recognition	Model too small or wrong language setting	Use a larger model and confirm language settings are correct
Path error with Chinese characters	Program doesn't support Chinese paths	Move whisper-cli to a path without Chinese characters