OpenSeq2Seq is a toolkit for distributed and mixed precision training of sequence-to-sequence models:
- Machine translation (GNMT, Transformer, ConvS2S, …). These models have been trained with BPE vocabulary used for text tokenization.
- Speech recognition (DeepSpeech2, Wave2Letter, Jasper, …).
- Speech commands (RN-50, Jasper). Automatic speech recognition (ASR) systems can be built using a number of approaches depending on input data type, intermediate representation, model’s type and output post-processing. OpenSeq2Seq is currently focused on end-to-end CTC-based models (like original DeepSpeech model).
- Speech synthesis (Tacotron2, Tacotron2 GST, WaveNet, Centaur, …).
- Language model (LSTM with WikiText-2, LSTM with WikiText-103).
- Sentiment analysis (SST, IMDB, …).
- Image classification, a mixed precison replica of TensorFlow ResNet-50.
OpenSeq2Seq main goal is to allow researchers to most effectively explore various sequence-to-sequence models. The efficiency is achieved by fully supporting distributed and mixed-precision training.
OpenSeq2Seq is built using TensorFlow and provides all the necessary building blocks for training encoder-decoder models for neural machine translation, automatic speech recognition, speech synthesis, and language modeling.
Speech-to-text workflow uses some parts of Mozilla DeepSpeech project.
This is a research project, not an official NVIDIA product.
Key Features
- Models for:
- Neural Machine Translation.
- Automatic Speech Recognition.
- Speech Synthesis.
- Language Modeling.
- NLP tasks (sentiment analysis).
- Data-parallel distributed training:
- Multi-GPU.
- Multi-node.
- Mixed precision training for NVIDIA Volta/Turing GPUs.
- Supports two modes for parallel training: simple multi-tower approach and Horovod-based approach.
- Supports two new optimizers: Layer-wise Adaptive Rate Control (LARC) and NovoGrad. NovoGrad is a first-order SGD-based algorithm, which computes second moments per layer instead of per weight as in Adam.
- Mixed precision with existing models.
- Interactive infer – a mode that makes it easy to demo trained models.
Website: nvidia.github.io/OpenSeq2Seq
Support: GitHub Code Repository
Developer: NVIDIA
License: Apache License 2.0
OpenSeq2Seq is written in Python. Learn Python with our recommended free books and free tutorials.
Related Software
| Speech Recognition Tools | |
|---|---|
| Whisper | Automatic speech recognition (system trained on 680,000 hours of data |
| whisper.cpp | Run Whisper locally with fast, efficient C/C++ speech recognition |
| Flashlight | Fast, flexible machine learning library written entirely in C++. |
| sherpa-onnx | Offline speech recognition for many platforms and languages |
| Kaldi | C++ toolkit designed for speech recognition researchers. |
| FunASR | Versatile speech recognition toolkit for real-world audio |
| faster-whisper | Faster Whisper transcription with reduced memory use |
| SpeechBrain | All-in-one conversational AI toolkit based on PyTorch |
| Handy | Offline speech-to-text application |
| ESPnet | End-to-End speech processing toolkit |
| PocketSphinx | Lightweight speech recognition for embedded and mobile use |
| deepspeech.pytorch | Implementation of DeepSpeech2 using Baidu Warp-CTC. |
| Epicenter | Transcription application with global speech-to-text functionality |
| Julius | Two-pass large vocabulary continuous speech recognition engine |
| Speech Note | Write notes, transcribe speech, translate text and read content aloud |
| aTrain | Private local transcription of recorded speech through a polished GUI |
| hyprwhspr | Native speech-to-text designed for Arch / Omarchy |
| Vocalinux | Local voice dictation through a clean desktop interface |
| ostt | Open Speech-to-Text |
| Simon | Flexible speech recognition software |
Read our verdict in the software roundup.
 Explore our carefully curated directory of recommended free and open source software, covering every major software category.The directory forms part of our extensive collection of articles for Linux enthusiasts. It includes hundreds of detailed reviews, together with free and open source alternatives to proprietary software from companies such as Google, Microsoft, Apple, Adobe, IBM, Cisco, Oracle, and Autodesk. LinuxLinks also covers interesting projects worth exploring, Linux-compatible hardware, free programming books and tutorials, and much more. Know a useful free and open source Linux application that we haven’t covered? Tell us about it using our submission form. |
Please read our Comment Policy before commenting.