Open-Source Text-to-Speech: "Natural" Voices - AI Build & Learn
Details
Welcome to AI Build & Learn, a weekly AI engineering stream where we pick a new topic and learn by building together.
- RSVP on the Luma to get direct live links and better reminders
- You can always find the live on YouTube: https://www.youtube.com/@sagecodes/streams
This event is about turning text into speech with AI. As with the other generation events, there's no single model we're locked into — the point is to explore what's out there and actually try a few.
We'll focus on open-source models, but you're welcome to bring commercial ones (ElevenLabs and friends) if you want to compare — and it's worth seeing how close the best open models now get.
We'll look at the main axes that matter in practice: fast, lightweight narration versus the most natural, expressive voices, plus voice cloning and multilingual support. I'll research and try some of the best open-source options ahead of the stream, and we'll talk through the tradeoffs: quality, real-time latency, cloning, languages, model size / hardware, and — importantly — licensing (several strong models are non-commercial).
Some things to look up to get started:
Open-source models:
Kokoro-82M: tiny and fast — runs on CPU, Apache-2.0; 54 built-in voices but no cloning
Chatterbox (Resemble AI): natural speech with voice cloning, MIT; the Turbo variant is lightweight and beat ElevenLabs in a blind listening test
XTTS v2 (Coqui): broad multilingual voice cloning (~17 languages) — note: non-commercial license
F5-TTS / Orpheus 3B: strong research-grade voice cloning (F5-TTS weights are non-commercial; Orpheus is Apache/MIT)
Piper: ultra-light, great for CPU / Raspberry Pi and offline or edge use
Tooling:
Hugging Face TTS Arena — a crowd-ranked leaderboard: https://huggingface.co/spaces/TTS-AGI/TTS-Arena
Coqui TTS toolkit: https://github.com/coqui-ai/TTS
Resources
- GitHub: https://github.com/sagecodes/ai-build-and-learn
- Events Calendar: https://luma.com/ai-builders-and-learners
- Slack (Discuss during the week): https://slack.flyte.org/
- Hosted by Sage Elliott: https://www.linkedin.com/in/sageelliott/
In this stream
- Intro to topic
- Community Discussion
- Practical examples
Community challenge (optional)
Try spending 30–90 minutes during the week learning or building something related to the topic, then share what you’re working on in Slack.
Note on Flyte / Union
You may see Flyte used in some demos. Flyte is an open-source AI orchestration platform maintained by Union (where I work) for building scalable, durable, and observable AI workflows. You do not need to use Flyte to participate.
- Union: https://www.union.ai/
- Flyte: https://flyte.org/
Drop a comment with ideas for future topics (agents, RAG, MLOps, robotics, frameworks, and more).