Speech

sherpa-onnx – offline speech processing toolkit

sherpa-onnx is a speech processing toolkit that performs automatic speech recognition, speech synthesis and related audio analysis tasks without requiring an internet connection. It uses ONNX Runtime to execute machine learning models locally, providing a common inference framework for speech applications on desktop computers, servers and mobile devices.

The project supports both streaming and non-streaming speech recognition, allowing developers to choose between processing completed recordings and handling audio as it arrives. Beyond transcription, sherpa-onnx includes voice activity detection, speaker identification, speaker diarization, keyword spotting and speech enhancement. This makes it suitable for building transcription utilities, voice-controlled interfaces, meeting analysis tools and accessibility applications.

Its architecture separates speech processing functionality from the applications that use it. Developers can access the toolkit through native libraries and bindings for several programming languages, while networking components allow speech services to be exposed to other applications. Support for different inference backends helps accommodate hardware ranging from conventional CPUs to specialised accelerators.

This is free and open source software.

Key Features

  • Perform automatic speech recognition entirely offline.
  • Support streaming recognition for live audio input.
  • Transcribe prerecorded audio using non-streaming recognition models.
  • Generate speech from text using supported synthesis models.
  • Identify individual speakers from their voice characteristics.
  • Verify whether speech belongs to a particular speaker.
  • Perform speaker diarization to identify who spoke and when.
  • Detect speech segments using voice activity detection.
  • Identify the spoken language in audio recordings.
  • Recognise predefined keywords and wake words.
  • Apply speech enhancement to improve noisy recordings.
  • Separate different sound sources from mixed audio.
  • Classify recordings using audio tagging models.
  • Restore punctuation to compatible speech recognition output.
  • Use ONNX Runtime to execute speech processing models.
  • Support multiple recognition and synthesis model architectures.
  • Expose speech processing through WebSocket servers and clients.
  • Provide native APIs and bindings for application integration.
  • Support selected hardware acceleration backends.
  • Run speech processing locally without sending recordings to cloud services.
  • Cross-platform support – runs on Linux, macOS, and Windows.

Website: github.com/k2-fsa/sherpa-onnx
Support:
Developer: sherpa-onnx contributors
License: Apache License 2.0

sherpa-onnx is written in C++ and Python . Learn C++ with our recommended free books and free tutorials. Learn Python with our recommended free books and free tutorials.


Related Software

Speech Recognition Tools
WhisperAutomatic speech recognition (system trained on 680,000 hours of data
whisper.cppRun Whisper locally with fast, efficient C/C++ speech recognition
FlashlightFast, flexible machine learning library written entirely in C++.
sherpa-onnxOffline speech recognition for many platforms and languages
KaldiC++ toolkit designed for speech recognition researchers.
FunASRVersatile speech recognition toolkit for real-world audio
faster-whisperFaster Whisper transcription with reduced memory use
SpeechBrainAll-in-one conversational AI toolkit based on PyTorch
HandyOffline speech-to-text application
ESPnetEnd-to-End speech processing toolkit
PocketSphinxLightweight speech recognition for embedded and mobile use
deepspeech.pytorchImplementation of DeepSpeech2 using Baidu Warp-CTC.
EpicenterTranscription application with global speech-to-text functionality
JuliusTwo-pass large vocabulary continuous speech recognition engine
Speech NoteWrite notes, transcribe speech, translate text and read content aloud
aTrainPrivate local transcription of recorded speech through a polished GUI
hyprwhsprNative speech-to-text designed for Arch / Omarchy
VocalinuxLocal voice dictation through a clean desktop interface
osttOpen Speech-to-Text
SimonFlexible speech recognition software

Read our verdict in the software roundup.

Speech Tools
PiperFast, local neural text to speech system
BarkTransformer-based text-to-audio model.
sherpa-onnxSpeech-to-text and text-to-speech software
Coqui TTSOffers pretrained models in more than 1,100 different languages
Dia1.6B parameter text to speech model
TortoiseMulti-voice text-to-speech system trained with an emphasis on quality
FestivalGeneral multi-lingual speech synthesis system
PraatSpeechAnalyserSoftware for speech analysis and synthesis
ChatterboxFamily of text-to-speech models
Speech NoteSpeech to Text, Text to Speech and Machine Translation
Mimic 3Lightweight Text to Speech engine
OrcaScreenReaderScriptable screen reader
MeloTTSHigh-quality multi-lingual text-to-speech library
Parler-TTSLightweight text-to-speech (TTS) model
FliteSmall, fast run time text to speech synthesis engine
RHVoiceGives the visually impaired a synthesis voice with their screen reader
eSpeak NGContinuation of the eSpeak project
eSpeakSpeech synthesizer using a formant synthesis method
Orpheus-TTS-FastAPIHigh-performance self-hosted text-to-speech server
GespeakerGTK-based frontend for eSpeak
VoiceGenSimple text-to-speech application
GlateGoogle Translator and Text To Speech Service

Read our verdict in the software roundup.


Best Free and Open Source Software LinuxLinks helps you discover the best free and open source software for Linux. Explore our extensive collection of recommended applications, including alternatives to popular proprietary software.

We also publish software reviews, Linux tutorials, hardware reviews, free programming books and articles covering the wider open source community. Browse our Linux resources to discover more.

Know a useful free and open source Linux application we haven't covered? Suggest a program for inclusion.

Has your software been featured on LinuxLinks? Display one of our Featured on LinuxLinks badges on your website or GitHub README.
Subscribe

Please read our Comment Policy before commenting.

Notify of
guest
0 Comments
Oldest
Newest Most Voted