Chatterbox-TTS-Server in Docker (NVIDIA GPU)

English · 简体中文

Chatterbox-TTS-Server wraps Resemble AI's Chatterbox models in a web UI and an OpenAI-compatible API. Compared with Kokoro the voices are far more natural and expressive — the Turbo model even understands tags like [laugh] and [sigh] — and you can clone a voice from a short recording. What you give up: word timings (so Read Aloud highlights sentence by sentence), speed (a long sentence takes a few seconds), and it wants a real GPU.

The plugin uses it through the OpenAI section of its settings.

GPU only

This tutorial covers NVIDIA GPUs in Docker, nothing else, on purpose. CPU mode and the native macOS (Apple Silicon) build both work in the sense that audio comes out, but in our tests a sentence took longer to synthesize than to speak, so reading stalls after every sentence; they are not usable for Read Aloud. Plan on a card with 6–8 GB of free VRAM. For reference, on an RTX 3080 Ti a short sentence takes about 1.7 s and a long one about 8 s; the plugin prefetches a few sentences ahead, which hides that.

Prerequisites

1. Get the code

git clone https://github.com/devnen/Chatterbox-TTS-Server.git
cd Chatterbox-TTS-Server

Everything below runs in this directory. It is also where the server keeps its data: config.yaml, voices/, reference_audio/, outputs/, logs/.

2. Blank the placeholder Hugging Face token

The upstream docker-compose.yml ships HF_TOKEN=YOUR_TOKEN_HERE. Hugging Face treats that as a real, invalid token and refuses the model download. The models are public, so an empty token is all that is needed. Rather than edit the upstream file (which git pull would then fight), add an override next to it — docker-compose.override.yml:

services:
  chatterbox-tts-server:
    environment:
      HF_TOKEN: ""

Docker Compose merges it automatically.

3. Build and start

RTX 20/30/40-series:

docker compose up -d --build

RTX 50-series (Blackwell) — name both files, because an override is only picked up automatically next to the default docker-compose.yml:

docker compose -f docker-compose-cu128.yml -f docker-compose.override.yml up -d --build

The build takes around fifteen minutes (CUDA base image, PyTorch, the chatterbox package). Then the container starts, and its first start downloads the model. Follow along with

docker compose logs -f

until you see Final device selection: cuda — if it says cpu, the GPU is not reaching the container; see Troubleshooting — and, a minute or two later, TTS Model loaded successfully on cuda and Application startup complete. The container restarts with Docker from now on (restart: unless-stopped), and the model is cached in a Docker volume, so later starts take a minute.

4. Check it

The web UI is at http://localhost:8004 and the API docs at http://localhost:8004/docs. From a terminal:

curl -s http://localhost:8004/api/model-info
curl -s http://localhost:8004/v1/audio/voices
curl -s -o test.mp3 -H "Content-Type: application/json" \
  -d '{"model":"tts-1","voice":"Emily.wav","input":"Chatterbox is running on the GPU.","response_format":"mp3"}' \
  http://localhost:8004/v1/audio/speech

The first answers "device":"cuda", the second lists 28 voices, the third writes a playable test.mp3.

5. Point the plugin at it

Zotero → Settings → Zotero-TTS → OpenAI section:

Field Value
Enable OpenAI voices on
Server Chatterbox-TTS-Server — this fills in the address and grays out key, model and voices, which Chatterbox ignores
Base URL http://localhost:8004 (filled in by the preset; change it for another machine)
Extra headers empty (only needed behind a gateway, see the Cloudflare tutorial)

Test connection answers Connected. 28 voices available. Synthesis works. In Read Aloud's Local tier the voices appear as OpenAI-Emily.wav, OpenAI-Henry.wav, and so on. Highlighting is per sentence whatever the Highlight current setting says, since the server reports no word timings.

Languages

The default engine, Chatterbox Turbo, speaks English only. For other languages switch to the multilingual model (23 languages, Chinese among them): in config.yaml set

model:
  repo_id: chatterbox-multilingual
generation_defaults:
  language: zh

and restart (docker compose restart); the model downloads on the next start. The plugin sends no language with its requests, so generation_defaults. language is what every document is read in — one language per server. The web UI can also switch engines without a restart.

Your own voice

Put a clean 10–30 second WAV or MP3 of the speaker into voices/. It is listed on the next Test connection and appears in Read Aloud under its file name. (Files in reference_audio/ work too but are not listed; they would have to be typed into the Voices field.)

Day to day

docker compose stop       # free the GPU
docker compose start
docker compose logs --tail 50

Update to a newer server:

git pull
docker compose up -d --build

Troubleshooting