Windows
Linux
Text Processing
Windows sort /unique equivalent in Linux
Removes or counts duplicate lines. Must be used after sorting — it only removes adjacent duplicates.
Windows
→
sort /unique
Linux equivalents
sort | uniq
sort | uniq
sort | uniq
Overview
What this command does
Removes or counts duplicate lines. Must be used after sorting — it only removes adjacent duplicates.
Migration tip:
uniq only removes adjacent duplicates — always run sort first. uniq -c adds a count prefix, making it great for frequency analysis.Practical tasks
Common use cases
Filter duplicate adjacent lines
Removes or counts duplicate lines. Must be used after sorting — it only removes adjacent duplicates.
sort file | uniq
Remove all duplicate lines
uniq -c file
Count occurrences of each line
uniq -d file
Show only lines that are duplicated
Ready to run
Copyable Linux commands
Review paths, device names, package names and permissions before running any command.
sort names.txt | uniq
awk '{print $1}' access.log | sort | uniq -c | sort -rn | head -20
sort file.txt | uniq -d
Walkthrough
Examples with explanations
# No direct Windows CMD equivalent
# Remove duplicates from a sorted list:
sort names.txt | uniq
# Top 20 IPs by hit count:
awk '{print $1}' access.log | sort | uniq -c | sort -rn | head -20
# Find duplicate lines only:
sort file.txt | uniq -dReference
Common options and variations
| Command or option | Use |
|---|---|
sort file | uniq | Remove all duplicate lines |
uniq -c file | Count occurrences of each line |
uniq -d file | Show only lines that are duplicated |
uniq -u file | Show only unique (non-duplicate) lines |
Distro differences
Debian/Ubuntu vs Fedora vs Arch
This command is the same on Debian/Ubuntu, Fedora and Arch Linux when the relevant tool is installed.
| Distribution | Package manager / base | Equivalent command | Difference to notice |
|---|---|---|---|
| Debian/Ubuntu | same command | sort | uniq |
Same command on this distribution. |
| Fedora | same command | sort | uniq |
Same command on this distribution. |
| Arch | same command | sort | uniq |
Same command on this distribution. |
Keep learning