SeaweedFS is a simple and highly scalable distributed file system.
There are two objectives: to store billions of files, and to serve the files fast.
SeaweedFS started as an Object Store to handle small files efficiently. Instead of managing all file metadata in a central master, the central master only manages volumes on volume servers, and these volume servers manage files and their metadata. This relieves concurrency pressure from the central master and spreads file metadata into volume servers, allowing faster file access (O(1), usually just one disk read operation).
There is only 40 bytes of disk storage overhead for each file’s metadata. It is so simple with O(1) disk reads that you are welcome to challenge the performance with your actual use cases.
This is free and open source software.
Key Features
- Can choose no replication or different replication levels, rack and data center aware.
- Automatic master servers failover – no single point of failure (SPOF).
- Automatic Gzip compression depending on file MIME type.
- Automatic compaction to reclaim disk space after deletion or update.
- Automatic entry TTL expiration.
- Any server with some disk space can add to the total storage space.
- Adding/Removing servers does not cause any data re-balancing unless triggered by admin commands.
- Optional picture resizing.
- Support ETag, Accept-Range, Last-Modified, etc.
- Support in-memory/leveldb/readonly mode tuning for memory/performance balance.
- Support rebalancing the writable and readonly volumes.
- Customizable Multiple Storage Tiers: Customizable storage disk types to balance performance and cost.
- Transparent cloud integration: unlimited capacity via tiered cloud storage for warm data.
- Erasure Coding for warm storage Rack-Aware 10.4 erasure coding reduces storage cost and increases availability.
Website: github.com/seaweedfs/seaweedfs
Support:
Developer: SeaweedFS developers
License: Apache License 2.0
SeaweedFS is written in Go. Learn Go with our recommended free books and free tutorials.
Related Software
| File Systems | |
|---|---|
| HDFS | Distributed file system providing high-throughput access |
| SeaweedFS | Simple and highly scalable distributed file system |
| Lustre | File system for computer clusters |
| CephFS | Unified, distributed storage system |
| Alluxio | Virtual distributed file system |
| GlusterFS | Scale-out NAS file system |
| JuiceFS | Distributed POSIX file system |
| XtreemFS | Object-based, distributed file system for wide area networks |
| MooseFS | POSIX-compliant distributed file system |
| Quantcast File System | High-performance, fault-tolerant, distributed file system |
| OrangeFS | Multi-server scalable parallel file system |
| LeilFS | Distributed POSIX file system |
Read our verdict in the software roundup.
 Explore our comprehensive directory of recommended free and open source software. Our carefully curated collection spans every major software category.This directory is part of our ongoing series of informative articles for Linux enthusiasts. It features hundreds of detailed reviews, along with open source alternatives to proprietary solutions from major corporations such as Google, Microsoft, Apple, Adobe, IBM, Cisco, Oracle, and Autodesk. You’ll also find interesting projects to try, hardware coverage, free programming books and tutorials, and much more. Discovered a useful open source Linux program that we haven’t covered yet? Let us know by completing this form. |