Elephas brings deep learning with Keras to Spark. Elephas intends to keep the simplicity and high usability of Keras, thereby allowing for fast prototyping of distributed models, which can be run on massive data sets.
Elephas implements a class of data-parallel algorithms on top of Keras, using Spark’s RDDs and data frames. Keras Models are initialized on the driver, then serialized and shipped to workers, alongside with data and broadcasted model parameters. Spark workers deserialize the model, train their chunk of data and send their gradients back to the driver. The “master” model on the driver is updated by an optimizer, which takes gradients either synchronously or asynchronously.
This is free and open source software.
Website: github.com/danielenricocahall/elephas
Support:
Developer: Max Pumperla
License: MIT License
Elephas is written in Python. Learn Python with our recommended free books and free tutorials.
Related Software
| Deep Learning with Python | |
|---|---|
| TensorFlow | A very popular Deep Learning framework |
| PyTorch | Tensors and Dynamic neural networks in Python |
| Keras | High-level neural networks API |
| fastai | Simplifies training fast and accurate neural networks |
| tinygrad | Compact, lightweight deep learning framework |
| PyTorch Lightning | Framework for scaling PyTorch training workflows |
| JAX | High-performance numerical computing and ML library |
| PyTensor | Library for fast numerical computation |
| MindSpore | Framework for training and deploying neural networks |
| Elephas | Distributed deep learning with Keras and Spark |
| Chainer | Powerful, flexible, and intuitive framework for neural networks |
| Equinox | Neural network library for building flexible models with JAX |
| Caffe | Convolutional Architecture for Fast Feature Embedding |
| TFlearn | Deep learning library featuring a higher-level API for TensorFlow |
| MXNet | Flexible and efficient library |
| CNTK | Distributed deep learning |
| Neupy | Python library for neural networks and deep learning |
Read our verdict in the software roundup.
 Explore our comprehensive directory of recommended free and open source software. Our carefully curated collection spans every major software category.This directory is part of our ongoing series of informative articles for Linux enthusiasts. It features hundreds of detailed reviews, along with open source alternatives to proprietary solutions from major corporations such as Google, Microsoft, Apple, Adobe, IBM, Cisco, Oracle, and Autodesk. You’ll also find interesting projects to try, hardware coverage, free programming books and tutorials, and much more. Discovered a useful open source Linux program that we haven’t covered yet? Let us know by completing this form. |
Please read our Comment Policy before commenting.