Command-line string manipulation tips

I just wanted to share some command line one-liners I used today to make my life easier.

Getting unique entries from a list

I had a long list of names and I had to find all the unique entries. I used common Linux commands,
sort and uniq. Input file is a text file with all the names on their own lines.

The file is first sorted by sort command and then uniq returns all the unique lines. The results are saved to output file.

# cat input-all.txt | sort | uniq > output-unique-entries.txt

Appending string to a list of entries

I had to convert a long list of dates in YYYY format to YYYY-MM-DD format with default dates of 1st Jan. I decided to do that with sed regex. Here is the command:

# cat input.txt | sed -e 's/^.*$/&-01-01/g' > output.txt

The command takes an input file (each date on its own line) and just appends ‘-01-01’ string after the year string. The & character is the key in the regex as is keeps the year in the result. The results are saved to output file.

Example: 1980 -> 1980-01-01