TwinHunter is a command-line utility for finding and cleaning up duplicate files.
It scans a directory, groups files with identical content and reports the number of duplicate copies, their combined size and the amount of storage that can be recovered.
The software uses a two-stage hashing process, combining the speed of BLAKE3 with SHA-256 verification. Searches can be narrowed by file size, extension, directory, glob or regular-expression filters. After reviewing the results, users can delete redundant copies, replace them with links or move them to a backup directory. Dry-run and interactive confirmation modes help guard against unintended changes.
This is free and open source software.
Key Features
- Finds duplicate files by comparing their content.
- Uses BLAKE3 for initial hashing and SHA-256 for verification.
- Concurrent hashing with a configurable number of worker processes.
- Automatically uses all available CPU cores when the worker count is set to zero.
- Scan a single directory or recurse through its subdirectories.
- Set minimum and maximum file sizes.
- Exclude files using glob or regular-expression patterns.
- Exclude specified directory names and file extensions.
- Ignores common directories such as .git, node_modules, .svn and __pycache__ by default.
- Reports duplicate groups, duplicate-file counts, wasted space and recoverable storage.
- Detects files that are already hard linked.
- Sort duplicate groups by size, number of copies or file path.
- Keep the oldest file, newest file or file with the shortest path.
- Delete redundant copies.
- Replace duplicates with hard or symbolic links.
- Move duplicate files to a backup directory.
- Interactive menu for selecting a cleanup action and keep strategy.
- Confirmation prompts before modifying files.
- Dry-run mode previews cleanup operations without changing the filesystem.
- Pretty, JSON, CSV and silent output modes.
- Export reports in JSON, CSV or HTML format.
- Perform cleanup later using a previously saved report.
- Verbose mode displays hashes, sizes and modification times.
- Live progress indicator during scanning.
- Shell completion for Bash, Zsh, Fish and PowerShell.
Website: github.com/mrinjamul/TwinHunter
Support:
Developer: Injamul Mohammad Mollah
License: MIT License
TwinHunter is written in Go. Learn Go with our recommended free books and free tutorials.
Related Software
| Find and Delete Duplicate Files with these CLI Tools | |
|---|---|
| Czkawka | Find duplicate files, big files, empty files, similar images, and much more |
| fdupes | Great CLI tool that's written in C |
| fclones | Efficient duplicate file finder and remover |
| rmlint | Fast tool to remove duplicates and other lint |
| jdupes | Powerful CLI duplicate file finder and 'enhanced' fork of fdupes |
| smash | Find duplicate files super fast |
| rdfind | CLI redundant data find tool written in C++ |
| duff | Command-line utility for finding duplicate files |
| rmdupes | Option to use a reference directory |
| Periscope | Organize storage and safely remove redundant files |
| Go Find Duplicates | Scans directories for duplicate files and directories |
| dskDitto | Parallel duplicate-file detector with interactive menus |
| samanlainen | Delete duplicate files with SHA512 hashing |
| FSlint | Python based CLI and GUI tool |
| sdupes | Fast duplicate file detection utility. |
| dupefi | Duplicate file finder designed with Linux philosophy |
| Dupster | Duplicate file finder |
| duple | Find and remove duplicate files |
| ddh | Directory Differential hTool |
| backdown | Safely and ergonomically remove duplicate files |
Read our verdict in the software roundup.
 Explore our carefully curated directory of recommended free and open source software, covering every major software category.The directory forms part of our extensive collection of articles for Linux enthusiasts. It includes hundreds of detailed reviews, together with free and open source alternatives to proprietary software from companies such as Google, Microsoft, Apple, Adobe, IBM, Cisco, Oracle, and Autodesk. LinuxLinks also covers interesting projects worth exploring, Linux-compatible hardware, free programming books and tutorials, and much more. Know a useful free and open source Linux application that we haven’t covered? Tell us about it using our submission form. |
Please read our Comment Policy before commenting.