The Eesen framework drastically simplifies the existing pipeline to build state-of-the-art ASR systems. Acoustic models in Eesen are deep bidirectional RNNs trained with the CTC objective function.
Eesen contains 4 key components to enable end-to-end ASR:
- Acoustic Model — Bi-directional RNNs with LSTM units.
- Training — Connectionist temporal classification (CTC) as the training objective.
- WFST Decoding — A principled decoding approach based on Weighted Finite-State Transducers (WFSTs), or
- RNN-LM Decoding — Decoding based on (character) RNN language models, when using Tensorflow.
Acoustic modeling in Eesen involves learning a single recurrent neural network (RNN) predicting context-independent targets (phonemes or characters). To remove the need for pre-generated frame labels, the connectionist temporal classification (CTC) objective function is adopted to infer the alignments between speech and label sequences. A distinctive feature of Eesen is a generalized decoding approach based on weighted finite-state transducers (WFSTs), which enables the efficient incorporation of lexicons and language models into CTC decoding. Experiments show that compared with the standard hybrid DNN systems, Eesen achieves comparable word error rates (WERs), while at the same time speeding up decoding significantly.
Key Features
- The WFST-based decoding approach can incorporate lexicons and language models into CTC decoding in an effective and efficient way.
- The RNN-LM decoding approach does not require a fixed lexicon.
- GPU implementation of LSTM model training and CTC learning, now also using Tensorflow.
- Multiple utterances are processed in parallel for training speed-up.
- Fully-fledged example setups to demonstrate end-to-end system building, with both phonemes and characters as labels, following Kaldi recipes and conventions.
Website: github.com/srvk/eesen
Support:
Developer: Yajie Miao and contributors
License: Apache License 2.0
Eesen is written in C++. Learn C++ with our recommended free books and free tutorials.
Related Software
| Speech Recognition Tools | |
|---|---|
| Whisper | Automatic speech recognition (system trained on 680,000 hours of data |
| whisper.cpp | Run Whisper locally with fast, efficient C/C++ speech recognition |
| Flashlight | Fast, flexible machine learning library written entirely in C++. |
| sherpa-onnx | Offline speech recognition for many platforms and languages |
| Kaldi | C++ toolkit designed for speech recognition researchers. |
| FunASR | Versatile speech recognition toolkit for real-world audio |
| faster-whisper | Faster Whisper transcription with reduced memory use |
| SpeechBrain | All-in-one conversational AI toolkit based on PyTorch |
| Handy | Offline speech-to-text application |
| ESPnet | End-to-End speech processing toolkit |
| PocketSphinx | Lightweight speech recognition for embedded and mobile use |
| deepspeech.pytorch | Implementation of DeepSpeech2 using Baidu Warp-CTC. |
| Epicenter | Transcription application with global speech-to-text functionality |
| Julius | Two-pass large vocabulary continuous speech recognition engine |
| Speech Note | Write notes, transcribe speech, translate text and read content aloud |
| aTrain | Private local transcription of recorded speech through a polished GUI |
| hyprwhspr | Native speech-to-text designed for Arch / Omarchy |
| Vocalinux | Local voice dictation through a clean desktop interface |
| ostt | Open Speech-to-Text |
| Simon | Flexible speech recognition software |
Read our verdict in the software roundup.
 Explore our carefully curated directory of recommended free and open source software, covering every major software category.The directory forms part of our extensive collection of articles for Linux enthusiasts. It includes hundreds of detailed reviews, together with free and open source alternatives to proprietary software from companies such as Google, Microsoft, Apple, Adobe, IBM, Cisco, Oracle, and Autodesk. LinuxLinks also covers interesting projects worth exploring, Linux-compatible hardware, free programming books and tutorials, and much more. Know a useful free and open source Linux application that we haven’t covered? Tell us about it using our submission form. |
Please read our Comment Policy before commenting.