rapidgzip is a high-performance gzip decompressor designed to exploit multicore processors. Unlike conventional gzip tools that process compressed data sequentially, rapidgzip can divide decompression work across multiple threads.
One of its distinguishing capabilities is support for random access within compressed files. This can be particularly valuable when working with large datasets where repeatedly decompressing entire files would be inefficient.
rapidgzip provides both a command-line tool and programming interfaces. Its implementation combines indexed access, caching and parallel prefetching to improve throughput. The utility is specifically a decompressor rather than a general-purpose gzip compressor, an important distinction for users comparing it with tools such as pigz.
This is free and open source software.
Key Features
- Parallel decompression of most ordinary gzip files.
- Works with gzip data produced by conventional tools, including GNU gzip.
- Supports random access to positions within compressed files.
- Can create indexes describing compressed data block positions.
- Imports existing indexes to avoid repeating indexing operations.
- Exports indexes for reuse in later processing sessions.
- Provides compatibility with several gzip index formats.
- Uses an LRU cache and parallel prefetching to accelerate repeated access.
- Allows the number of decompression threads to be configured.
- Supports decompression through standard input and output.
- Can count decompressed bytes and newline characters.
- Includes an analysis mode for inspecting gzip and Deflate block structures.
- Provides a Python file interface supporting seek, read and close operations.
- Offers a C++ library interface for integration into other applications.
- Supports processing compressed files without creating a fully decompressed intermediate copy.
Website: github.com/mxmlnkn/rapidgzip
Support:
Developer: Maximilian Knespel
License: MIT License or Apache License 2.0
rapidgzip is written in Python. Learn Python with our recommended free books and free tutorials.
Related Software
| Compression Tools | |
|---|---|
| LZ4 | Focused on compression and decompression speed |
| Zstandard | Fast compression algorithm, providing high compression ratios |
| brotli | Generic-purpose lossless compression algorithm |
| lzop | File compressor favoring speed over compression ratio |
| pigz | Parallel implementation of gzip |
| ouch | CLI tool for compressing and decompressing for various formats |
| pixz | Parallel, indexed xz compressor |
| gzip | Provides the standard GNU file compression utilities |
| BZip3 | Spiritual successor to BZip2 |
| bzip2 | Freely available, patent free, high-quality data compressor |
| PBZIP2 | Parallel implementation of bzip2 |
| lrzip | Achieve very high compression ratios and speed when used with large files |
| lbzip2 | Multi-threaded implementation of bzip2, suited for serial and parallel processing |
| ZXC | High-performance lossless compression library and command-line tool |
| plzip | Massively parallel lossless data compressor |
| p7zip-zstd | p7zip fork with additional codecs and improvements |
| 7-Zip | File archiver with a high compression ratio |
| ncompress | Fast, simple LZW file compressor |
| PXZ | Runs LZMA compression on multiple cores and processors |
| crabz | Like pigz but written in Rust |
| Lzip | Lossless data compressor aimed at Unix-like systems |
| cmix | Optimize compression ratio at the cost of high CPU/memory usage |
| bsc | High-performance block-sorting compressor for large files |
| rapidgzip | Parallelized decompression of gzip files |
| Kanzi | Modular lossless compressor with multithreaded processing |
Read our verdict in the software roundup.
 Explore our carefully curated directory of recommended free and open source software, covering every major software category.The directory forms part of our extensive collection of articles for Linux enthusiasts. It includes hundreds of detailed reviews, together with free and open source alternatives to proprietary software from companies such as Google, Microsoft, Apple, Adobe, IBM, Cisco, Oracle, and Autodesk. LinuxLinks also covers interesting projects worth exploring, Linux-compatible hardware, free programming books and tutorials, and much more. Know a useful free and open source Linux application that we haven’t covered? Tell us about it using our submission form. |
Please read our Comment Policy before commenting.