Moshi is a speech-text foundation model and full-duplex spoken dialogue framework.
It’s designed for real-time spoken interaction and uses Mimi, a streaming neural audio codec, to handle low-latency audio processing. The project includes multiple inference stacks for different use cases, covering PyTorch for research, MLX for on-device inference on Apple hardware, Rust for production deployments, and a web client used for the live demo.
Mimi is a neural audio codec that processes 24 kHz audio, down to a 12.5 Hz representation with a bandwidth of 1.1 kbps, in a fully streaming manner (latency of 80ms, the frame size), yet performs better than existing, non-streaming, codecs like SpeechTokenizer (50 Hz, 4kbps), or SemantiCodec (50 Hz, 1.3kbps).
This is free and open source software.
Key Features
- Provides a speech-text foundation model for real-time dialogue.
- Supports full-duplex spoken interaction.
- Uses the Mimi streaming neural audio codec for low-latency audio processing.
- Includes PyTorch, MLX, and Rust inference stacks for research, on-device use, and production deployment.
- Offers on-device inference support for iPhone and Mac through MLX.
- Includes a web UI client for interactive browser-based use.
- Can run related Kyutai multi-stream models including simultaneous speech translation and speech generation systems.
Website: moshi-chat.kyutai.org
Support:
Developer: Kyutai Labs
License: Apache License 2.0 / MIT License
Moshi is written in Python and Rust. Learn Python with our recommended free books and free tutorials.
Related Software
| Speech Tools | |
|---|---|
| Piper | Fast, local neural text to speech system |
| Bark | Transformer-based text-to-audio model. |
| sherpa-onnx | Speech-to-text and text-to-speech software |
| Coqui TTS | Offers pretrained models in more than 1,100 different languages |
| Dia | 1.6B parameter text to speech model |
| Tortoise | Multi-voice text-to-speech system trained with an emphasis on quality |
| Festival | General multi-lingual speech synthesis system |
| PraatSpeechAnalyser | Software for speech analysis and synthesis |
| Chatterbox | Family of text-to-speech models |
| Speech Note | Speech to Text, Text to Speech and Machine Translation |
| Mimic 3 | Lightweight Text to Speech engine |
| OrcaScreenReader | Scriptable screen reader |
| MeloTTS | High-quality multi-lingual text-to-speech library |
| Parler-TTS | Lightweight text-to-speech (TTS) model |
| Flite | Small, fast run time text to speech synthesis engine |
| RHVoice | Gives the visually impaired a synthesis voice with their screen reader |
| eSpeak NG | Continuation of the eSpeak project |
| eSpeak | Speech synthesizer using a formant synthesis method |
| Orpheus-TTS-FastAPI | High-performance self-hosted text-to-speech server |
| Gespeaker | GTK-based frontend for eSpeak |
| VoiceGen | Simple text-to-speech application |
| Glate | Google Translator and Text To Speech Service |
Read our verdict in the software roundup.
 LinuxLinks helps you discover the best free and open source software for Linux. Explore our extensive collection of recommended applications, including alternatives to popular proprietary software.We also publish software reviews, Linux tutorials, hardware reviews, free programming books and articles covering the wider open source community. Browse our Linux resources to discover more. Know a useful free and open source Linux application we haven't covered? Suggest a program for inclusion. Has your software been featured on LinuxLinks? Display one of our Featured on LinuxLinks badges on your website or GitHub README. |
Please read our Comment Policy before commenting.