cuniq is a command-line utility written in Rust for counting unique lines in text input.
It’s designed as a faster alternative to common shell pipelines used for line cardinality and duplicate counting, and it can work with stdin or one or more files. Alongside exact counting, the project also provides faster near-exact and estimated modes for large datasets, plus optional reporting features when you need occurrence counts for each distinct line.
This is free and open source software.
Key Features
- Counts unique lines from standard input or one or more files.
- Supports multithreaded processing.
- Can generate occurrence reports for each distinct line with the report option.
- Includes exact, near-exact, and estimated counting modes for different speed and accuracy trade-offs.
- Can sort report output alphabetically when required.
- Offers optional tuning such as hash-table sizing, thread control, and memory-mapped file reading.
Website: https://github.com/zkxs/cuniq
Support:
Developer: zkxs
License: GNU General Public License v3.0
cuniq is written in Rust. Learn Rust with our recommended free books and free tutorials.
Related Software
| Alternatives to uniq | |
|---|---|
| runiq | An efficient way to filter duplicate lines from input, à la uniq |
| huniq | Removes duplicates from stdin using a hash table |
| nauniq | Non-adjacent uniq |
| semiuniq | Removes nearby repeated lines in a file |
| uq | Removes duplicate lines from the output, regardless of the order |
| zet | Perform set operations on files considered as sets of lines |
| anew | Adds new lines to files, skipping duplicates |
| anewer | Appends lines from stdin to a file if they don't already exist in the file |
 Explore our comprehensive directory of recommended free and open source software. Our carefully curated collection spans every major software category.This directory is part of our ongoing series of informative articles for Linux enthusiasts. It features hundreds of detailed reviews, along with open source alternatives to proprietary solutions from major corporations such as Google, Microsoft, Apple, Adobe, IBM, Cisco, Oracle, and Autodesk. You’ll also find interesting projects to try, hardware coverage, free programming books and tutorials, and much more. Discovered a useful open source Linux program that we haven’t covered yet? Let us know by completing this form. |