Quantcast File System (QFS) is a distributed file system designed for large-scale data processing. It was developed by Quantcast to support workloads that read and write substantial amounts of data sequentially, particularly batch processing and MapReduce applications.
QFS divides its responsibilities among metadata servers, chunk servers and clients. The metaserver maintains the filesystem namespace and maps files to the storage servers holding their data. Chunk servers handle storage and retrieval, while clients communicate with the metaserver to locate the appropriate chunks.
This architecture allows data access to be distributed across a storage cluster. QFS is particularly relevant to workloads built around large files and sustained data throughput rather than applications requiring frequent small random updates.
This is free and open source software.
Key Features
- Stores file contents in chunks distributed across multiple storage servers.
- Supports configurable replication to protect data against storage failures.
- Provides Reed-Solomon erasure coding to reduce redundancy overhead.
- Offers metaserver replication and automatic failover when configured.
- Includes rebalancing to distribute data across available storage nodes.
- Supports file striping for parallel access to stored data.
- Uses checksums and recovery mechanisms to help maintain data integrity.
- Provides client-side metadata caching to reduce repeated metadata requests.
- Supports file leases and version information for coordinating access.
- Offers a FUSE interface for filesystem access on Linux.
- Provides tools for cluster administration, monitoring and diagnostics.
- Supports Unix-style file permissions and symbolic links.
Website: github.com/quantcast/qfs
Support:
Developer: Quantcast
License: Apache License 2.0
QFS is written in C++. Learn C++ with our recommended free books and free tutorials.
Related Software
| File Systems | |
|---|---|
| HDFS | Distributed file system providing high-throughput access |
| SeaweedFS | Simple and highly scalable distributed file system |
| Lustre | File system for computer clusters |
| CephFS | Unified, distributed storage system |
| Alluxio | Virtual distributed file system |
| GlusterFS | Scale-out NAS file system |
| JuiceFS | Distributed POSIX file system |
| 3FS | High-performance distributed file system for AI workloads |
| XtreemFS | Object-based, distributed file system for wide area networks |
| MooseFS | POSIX-compliant distributed file system |
| Quantcast File System | High-performance, fault-tolerant, distributed file system |
| OrangeFS | Multi-server scalable parallel file system |
| LeilFS | Distributed POSIX file system |
| DAOS | High-performance distributed storage for HPC and AI workloads |
| EOS | Scalable distributed storage system for scientific data |
Read our verdict in the software roundup.
LinuxLinks helps you discover the best free and open source software for Linux. Explore our extensive collection of recommended applications, including alternatives to popular proprietary software.We also publish software reviews, Linux tutorials, hardware reviews, free programming books and articles covering the wider open source community. Browse our Linux resources to discover more. Know a useful free and open source Linux application we haven't covered? Suggest a program for inclusion. Has your software been featured on LinuxLinks? Display one of our Featured on LinuxLinks badges on your website or GitHub README. |


Please read our Comment Policy before commenting.