How can I remove duplicate lines from my text file?
The easiest way to remove duplicate files from a text file as by use sort and uniq commands. Consider a following text file:
$ cat file.txt 
How can I remove duplicate lines from my text file?
use sort and uniq commands
use sort and uniq commands
How can I remove duplicate lines from my text file?

To remove all duplicate lines we first need to pipe the content of the file to sort. Once the content is sorted we use uniq command to remove duplicates:
$ sort file.txt | uniq 
How can I remove duplicate lines from my text file?
use sort and uniq commands


Go to top