whisper-turbo

Fast speech-to-text proxy for whisper.cpp with OpenVINO acceleration on Intel hardware.

Drop-in STT acceleration for any project using whisper.cpp -- including Willow Inference Server (WIS) for ESP32 voice assistants.

Why

whisper.cpp on CPU is slow. OpenVINO makes it 2.5x faster on Intel CPUs by optimizing the Whisper encoder -- the heaviest part of the pipeline. whisper-turbo wraps this in a clean HTTP proxy that keeps the model loaded in memory, eliminating per-request load times.

If you run whisper.cpp on an Intel machine without an NVIDIA GPU, this project is for you.

Benchmarks

Tested on a Lenovo ThinkCentre M920q (Intel Core i5-9500T, 32GB RAM, Intel UHD 630 iGPU, Ubuntu 24.04):

Setup	Encode Time	Total (11s audio)	vs. Baseline
whisper.cpp CPU (small.en)	3410 ms	4684 ms	baseline
whisper.cpp + OpenVINO CPU	1351 ms	3398 ms	2.5x faster encode
whisper.cpp + OpenVINO iGPU (UHD 630)	1961 ms	3996 ms	slower than CPU*

*The Intel UHD 630 (8th/9th gen) doesn't have enough compute units to beat OpenVINO's CPU optimizations. Newer Intel iGPUs (Iris Xe, 11th gen+) should perform better with OPENVINO_DEVICE=GPU. If you have benchmark data on newer hardware, please open an issue!

With the model kept warm in the whisper-server (no reload per request), real-world latency for a ~3 second voice command is under 2 seconds.

Architecture

Your app / ESP32 / movie upload
        |
        v
  whisper-turbo proxy (:19004)     <-- thin FastAPI layer
        |
        v
  whisper.cpp server (:19003)      <-- model stays warm in memory
        |                               OpenVINO accelerates the encoder
        v
  transcribed text (JSON) / subtitle file (SRT)

For Willow/WIS users, the proxy slots in behind WIS's nginx -- only STT is rerouted, TTS and everything else stays untouched. See Willow Integration.

Quick Start

# Clone the repo
git clone https://github.com/Blissful/whisper-turbo.git
cd whisper-turbo

# Run the install script (builds whisper.cpp, downloads model, sets up services)
sudo ./install.sh

# Edit config if needed
sudo nano /opt/whisper-turbo/.env

# Start the services
sudo systemctl start whisper-server
sudo systemctl start whisper-turbo

# Test
curl http://127.0.0.1:19004/health
curl -X POST http://127.0.0.1:19004/v1/transcribe -F "file=@test.wav"
curl -X POST http://127.0.0.1:19004/v1/subtitles -F "file=@/path/to/movie.mkv" -o movie.en.srt

Requirements

• OS: Linux (tested on Ubuntu 24.04)
• CPU: Intel (any generation -- OpenVINO optimizes for Intel instruction sets)
• RAM: 2GB free (for the whisper small.en model)
• Dependencies: cmake, gcc/g++, git, python3, python3-venv, ffmpeg
• No NVIDIA GPU required -- that's the whole point

Configuration

All settings live in a single .env file (generated by install.sh). See .env.example for all options.

Variable	Default	Description
WHISPER_MODEL	ggml-small.en.bin	Whisper model (tiny/base/small/medium, .en for English-only)
WHISPER_THREADS	4	CPU threads for inference
OPENVINO_DEVICE	CPU	CPU or GPU (GPU = Intel iGPU, best on 11th gen+)
PROXY_PORT	19004	Port for the proxy
WHISPER_SERVER_PORT	19003	Port for the whisper.cpp server
SUBTITLE_OUTPUT_DIR	/opt/whisper-turbo/subtitles	Folder where uploaded subtitle results are saved
WHISPER_LANGUAGE	en	Default language
WHISPER_SUBTITLE_TIMEOUT	1800	Timeout in seconds for full-movie subtitle inference
MEDIA_CONVERT_TIMEOUT	3600	Timeout in seconds for ffmpeg audio extraction

Choosing a Model

Model	Size	Speed	Accuracy	Best For
tiny.en	75 MB	Fastest	Basic	Single words, simple commands
base.en	142 MB	Fast	Good	Short voice commands
small.en	466 MB	Moderate	Very good	Conversational, complex requests
medium.en	1.5 GB	Slow	Excellent	Long-form, accented speech

For voice assistants, small.en is the sweet spot. For quick home automation commands, base.en is faster and accurate enough.

API

POST /v1/transcribe

Generic speech-to-text. Accepts any WAV file via multipart upload.

curl -X POST http://127.0.0.1:19004/v1/transcribe \
    -F "file=@audio.wav" \
    -F "language=en"

Response:

{
    "text": "turn off the living room lights",
    "language": "en",
    "duration_ms": 2800,
    "infer_time_ms": 920,
    "speedup": 3.0
}

POST /v1/subtitles

Upload a movie/audio file and get an SRT file back. The service also saves a copy in SUBTITLE_OUTPUT_DIR. The model is selected in .env with WHISPER_MODEL; it is not supplied per request.

curl -X POST http://127.0.0.1:19004/v1/subtitles \
    -F "file=@/home/me/movies/example.mkv" \
    -F "movie_language=en" \
    -F "srt_language=en" \
    -o example.en.srt

By default this saves the server-side copy as /opt/whisper-turbo/subtitles/example.en.srt. Change SUBTITLE_OUTPUT_DIR in .env if you want another server-side folder.

SUBTITLE_OUTPUT_DIR="/srv/subtitles"

The response body is the SRT file (application/x-subrip). The response also includes X-SRT-Path, X-Movie-Language, and X-SRT-Language headers.

Supported inputs include common video and audio containers such as mp4, mkv, mov, avi, webm, m4v, mp3, m4a, flac, and wav. ffmpeg extracts the audio before the proxy sends it to whisper.cpp.

Whisper can transcribe in the movie language, or translate supported source languages to English. For non-English movies translated to English subtitles, set srt_language=en and configure a multilingual model such as ggml-small.bin; English-only models such as ggml-small.en.bin cannot translate non-English audio.

You can also run subtitle generation from the shell while the whisper.cpp server is running. This is the SSH/sshfs path: the file path must be readable from the machine running this command.

python3 whisper_turbo.py /media/movies/example.mkv \
    --movie-language en \
    --srt-language en

POST /api/willow

Willow/WIS-compatible endpoint. Accepts raw PCM audio with ESP32 headers. See Willow Integration.

GET /health

{
    "status": "ok",
    "whisper_server": "ok",
    "language": "en",
    "device": "CPU"
}

Willow / WIS Integration

If you use Willow with ESP32 voice assistants, whisper-turbo can replace WIS's built-in STT with a single nginx config change. See the full guide: willow/WILLOW.md

Troubleshooting

whisper-server won't start:

• Check OpenVINO libs: ls /path/to/openvino/libs/libopenvino.so*
• Check logs: sudo journalctl -u whisper-server -f

Slow performance:

• Verify OpenVINO is loaded: look for OpenVINO model loaded in whisper-server logs
• Try OPENVINO_DEVICE=GPU if you have a newer Intel iGPU (11th gen+)
• Reduce model size: switch from small.en to base.en

Port conflicts:

• Change WHISPER_SERVER_PORT and PROXY_PORT in .env
• Restart both services after changing ports

Do I need both services?

• whisper-server is the underlying whisper.cpp process. It keeps the model loaded and does the actual transcription.
• whisper-turbo is the FastAPI web layer. It handles Willow compatibility, JSON responses, movie uploads, ffmpeg audio extraction, and SRT file creation.

Keep both if you want to upload movies over HTTP and avoid SSH-ing in for every movie. If you prefer SSH/sshfs and local paths, you can use the CLI mode instead and keep whisper-turbo only as the Python helper.

Model download fails:

• Manual download: cd /opt/whisper.cpp && bash models/download-ggml-model.sh small.en

Hardware Tested

Machine	CPU	iGPU	RAM	OpenVINO Device	Encode Speedup
Lenovo ThinkCentre M920q	i5-9500T	UHD 630	32 GB	CPU	2.5x

Tested on your hardware? Open an issue with your benchmarks and I'll add it to the table.

How It Differs From...

• Running whisper.cpp directly: whisper-turbo keeps the model warm (no 1-2s load per request) and adds OpenVINO acceleration. Also provides a clean HTTP API.
• faster-whisper: Requires Python + CTranslate2. whisper-turbo uses the C++ whisper.cpp binary for lower overhead.
• WIS built-in Whisper: WIS uses CTranslate2 with Python. whisper-turbo offloads the encoder to OpenVINO for faster inference on Intel hardware.

License

MIT

Acknowledgments

• whisper.cpp by Georgi Gerganov
• OpenVINO by Intel
• Willow Inference Server by Tovera