The quality of local speech generation has improved significantly over the years, and with models like Kokoro, it's now possible to generate realistic speech locally on a CPU [1]. This model, despite having only 82M parameters, produces high-quality speech in multiple languages, including English, Mandarin, and Hindi, and provides around 50 distinct voices [2]. To set up a server for Kokoro, you can use a pre-made container image called Kokoro-FastAPI, which includes pre-downloaded voice models [3]. The container image is around 5 GB in size, but it's easy to launch using Docker or Podman with the command podman run -p 8880:8880 ghcr.io/remsky/kokoro-fastapi-cpu [4].

Once launched, you can access a simple web UI at localhost:8880/web to generate and play audio from text. The container also serves a TTS interface compatible with the OpenAI speech API, making it easy to adapt existing programs [5]. Sample code in JavaScript and Python is available at github.com/remotebrowser/speak to facilitate testing. The generated audio will be saved as an MP3 file, and if SoX or Sound eXchange is installed, it will also play back automatically [6].

You can select a different voice by setting the TTS_VOICE environment variable, and a complete list of available voices can be found on the official Kokoro project page [7]. The synthesis speed is impressive, with measurements showing generation times of around 4.7 seconds on an Intel Core i7-4770K, 4.5 seconds on an Apple M2 Pro, and 1.5 seconds on an AMD Ryzen 7 8745HS [8]. For an alternative OpenAI-compatible containerized TTS service, consider Speaches, which offers an advantage by including Whisper, OpenAI’s renowned high-quality Speech-to-Text (STT) system [9]. When combined with a local LLM, a speech synthesis system like this allows you to enjoy listening to LLM answers instead of reading them.

Sources

  1. https://huggingface.co/hexgrad/Kokoro-82M
  2. https://huggingface.co/hexgrad/Kokoro-82M/blob/main/VOICES.md
  3. https://github.com/remsky/Kokoro-FastAPI
  4. https://github.com/remsky/Kokoro-FastAPI
  5. https://platform.openai.com/docs/guides/text-to-speech
  6. http://github.com/remotebrowser/speak
  7. https://huggingface.co/hexgrad/Kokoro-82M/blob/main/VOICES.md
  8. https://remotebrowser.substack.com/p/100-local-cpu-friendly-high-quality
  9. https://speaches.ai/