OpenAI Whisper Large v3 Turbo: faster-whisper CTranslate2 Serving & Subtitle Automation

🎙️ Whisper Turbo’s architecture simplification and acceleration principles

The Whisper Large v3 Turbo launched by OpenAI compresses the original 32-layer Decoder to 4 layers, but retains the complete 32-layer Encoder. This asymmetric distillation architecture allows the autoregressive decoding throughput to soar 8 times while retaining the ability to represent fine speech features, and the memory usage drops to only 4GB.

⚡ Image pulling and faster-whisper (CTranslate2) deployment

Environment configuration and weight pulling

# Image pulls Turbo weight (approximately 1.6GB)
export HF_ENDPOINT="https://hf-mirror.net"
huggingface-cli download openai/whisper-large-v3-turbo \
  --local-dir /data/models/whisper-large-v3-turbo \
  --local-dir-use-symlinks False

pip install faster-whisper

📌 Highly concurrent batch audio slice transcription Python script

transcribe.py: produces transcription code

from faster_whisper import WhisperModel

# Load model (use float16)
model = WhisperModel("/data/models/whisper-large-v3-turbo", device="cuda", compute_type="float16")

segments, info = model.transcribe(
    "long_meeting.mp3",
    beam_size=5,
    language="zh",
    vad_filter=True  # Filter blank silence segments
)

print(f"Language detection: {info.language} (Accuracy: {info.language_probability:.2f})")
for seg in segments:
    print(f"[{seg.start:.2f}s -> {seg.end:.2f}s] {seg.text}")