uq – remove duplicate lines without sorting

uq is a command-line utility that removes repeated lines from textual input without sorting the data beforehand.

Traditional uniq only removes adjacent duplicates, which means it is commonly combined with sort when processing unsorted files. uq provides a simpler alternative by remembering lines already encountered and discarding subsequent occurrences.

Because the input does not need to be sorted, uq preserves the order in which distinct records first appear. This is useful when processing logs, lists and command output where the original sequence matters.

The application also works with continuous input streams, allowing it to filter repeated lines as data arrives through a shell pipeline.

uq deliberately focuses on a small, straightforward set of operations rather than providing the extensive options found in some other text-processing utilities.

This is free and open source software.

Key Features

• Removes repeated lines even when duplicates are not adjacent.
• Preserves the first occurrence of each distinct line.
• Processes unsorted input without requiring a separate sorting operation.
• Accepts continuous streams of text through standard input.
• Works as a filter in shell pipelines.

Website: github.com/lostutils/uq
Support:
Developer: Tamir Bahar
License: MIT License

uq is written in Rust. Learn Rust with our recommended free books and free tutorials.

Alternatives to uniq

Explore Linux
	Best Free and Open Source Software - A huge collection of recommended free and open source software covering every major category.
	In-depth Linux Software Reviews - Detailed assessments of Linux software covering features, usability and overall quality.
	The Big List of Active Linux Distros - An extensive collection of actively developed Linux distributions.
	Open Source Alternatives - Alternatives to software and services from Google, Microsoft, Apple, Adobe, IBM, Autodesk, Oracle and others.
	Essential Linux System Tools - Indispensable utilities for monitoring, maintaining and managing Linux systems.
	Linux Productivity Tools - Useful utilities for improving productivity and streamlining everyday tasks.
	Alternatives to Popular CLI Tools - Modern alternatives to familiar command-line utilities supplied with Linux systems.
	Awesome Free Linux Game Tools - Tools, utilities and companion software that improve gaming on Linux.
	Machine Learning on Linux - Machine learning and deep learning software, self-hosted applications and practical tools.
	Saving Money with Linux - Practical ways to reduce electricity consumption and lower computing costs with Linux.
	Linux Candy - Fun, entertaining and sometimes delightfully quirky software from the lighter side of Linux.