Alluxio is a distributed data access and caching platform for data-intensive computing. It sits between applications and their underlying storage systems, reducing the need for processing frameworks to communicate directly with remote data sources.
Alluxio provides a common access layer for datasets held in different storage locations. Its open-source edition focuses on analytics workloads and is designed to work with processing engines such as Apache Spark, Presto and Trino.
Rather than replacing persistent storage, Alluxio accelerates access to it. Frequently requested data can be cached closer to computing resources, helping applications avoid repeated reads from comparatively slow or distant storage services.
Alluxio is especially relevant to organisations that separate computation from storage and need several analytics applications to access shared datasets.
This is free and open source software.
Key Features
- Provides a unified namespace for accessing datasets stored in different systems.
- Supports distributed caching to reduce repeated access to remote storage.
- Offers configurable caching policies and data placement across memory and disk.
- Integrates with storage systems including HDFS and S3-compatible object stores.
- Provides interfaces for analytics frameworks such as Spark, Presto and Trino.
- Supports Hadoop-compatible filesystem access for existing data processing applications.
- Offers filesystem commands for managing paths and cached data.
- Provides metrics and monitoring facilities for tracking cache and storage activity.
- Supports configurable persistence between its caching layer and underlying storage.
- Allows applications to work with distributed data without requiring each workload to implement storage-specific access logic.
Website: github.com/Alluxio/alluxio
Support:
Developer: Alluxio community (originally UC Berkeley AMPLab)
License: Apache License 2.0
Alluxio is written in Java. Learn Java with our recommended free books and free tutorials.
Related Software
| File Systems | |
|---|---|
| HDFS | Distributed file system providing high-throughput access |
| SeaweedFS | Simple and highly scalable distributed file system |
| Lustre | File system for computer clusters |
| CephFS | Unified, distributed storage system |
| Alluxio | Virtual distributed file system |
| GlusterFS | Scale-out NAS file system |
| JuiceFS | Distributed POSIX file system |
| XtreemFS | Object-based, distributed file system for wide area networks |
| MooseFS | POSIX-compliant distributed file system |
| Quantcast File System | High-performance, fault-tolerant, distributed file system |
| OrangeFS | Multi-server scalable parallel file system |
| LeilFS | Distributed POSIX file system |
Read our verdict in the software roundup.
 LinuxLinks helps you discover the best free and open source software for Linux. Explore our extensive collection of recommended applications, including alternatives to popular proprietary software.We also publish software reviews, Linux tutorials, hardware reviews, free programming books and articles covering the wider open source community. Browse our Linux resources to discover more. Know a useful free and open source Linux application we haven't covered? Suggest a program for inclusion. Has your software been featured on LinuxLinks? Display one of our Featured on LinuxLinks badges on your website or GitHub README. |
Please read our Comment Policy before commenting.