xuniq is a Rust command-line utility for text-processing workflows. It’s designed for fast line deduplication and fits neatly into shell pipelines when you need a lightweight tool focused on speed and streaming input rather than sorting-based approaches.
This is free and open source software.
Key Features
- Removes duplicate lines from standard input and writes unique lines to standard output.
- Doesn’t require input to be sorted before processing.
- Preserves the first occurrence of each line as it appears in the input stream.
- Offers a
--capacityoption to set the expected number of lines. - Provides a
--safemode that uses exact string matching to avoid collisions, at the cost of higher memory use.
Website: github.com/lalitshankarch/xuniq
Support:
Developer: Lalit Shankar Chowdhury
License: MIT License
xuniq is written in Rust. Learn Rust with our recommended free books and free tutorials.
Related Software
| Alternatives to uniq | |
|---|---|
| runiq | An efficient way to filter duplicate lines from input, à la uniq |
| huniq | Removes duplicates from stdin using a hash table |
| nauniq | Non-adjacent uniq |
| semiuniq | Removes nearby repeated lines in a file |
| uq | Removes duplicate lines from the output, regardless of the order |
| zet | Perform set operations on files considered as sets of lines |
| anew | Adds new lines to files, skipping duplicates |
| anewer | Appends lines from stdin to a file if they don't already exist in the file |
 Explore our comprehensive directory of recommended free and open source software. Our carefully curated collection spans every major software category.This directory is part of our ongoing series of informative articles for Linux enthusiasts. It features hundreds of detailed reviews, along with open source alternatives to proprietary solutions from major corporations such as Google, Microsoft, Apple, Adobe, IBM, Cisco, Oracle, and Autodesk. You’ll also find interesting projects to try, hardware coverage, free programming books and tutorials, and much more. Discovered a useful open source Linux program that we haven’t covered yet? Let us know by completing this form. |