Convert input sets of numbers to numerical sequences


I wrote a function for shell (basically bash) that makes it possible to convert a series of numbers such as “1,5-8,15” into a completely enumerated sequence, so 1 5 6 7 8 15.

I needed this to facilitate passing parameters to another function, but with the ability to give arbitrarily-grouped sets of numbers.

You can see my gist on github.

convert_to_seq() {
  printf "${@}" | xargs -n1 -d',' | tr '-' ' ' | awk 'NF == 2 { system("/bin/seq "$1" "$2); } NF != 2 { print $1; }' | xargs

convert_to_seq "$1"

Try it out for yourself! If you are looking for such a function, here you go.


Input: 1,5,8-10
Output: 1 5 8 9 10

Input: 500-510,37
Output: 500 501 502 503 504 505 506 507 508 509 510 37

Remove only certain duplicate lines with awk

Basic solution demonstrates and explains how to use awk to remove duplicate lines in a stream without having to sort them. This statement is really useful.
awk '!x[$0]++'

The fancy solution

But if you need certain duplicated lines preserved, such as the COMMIT statements in the output of iptables-save, you can use this one-liner:
iptables-save | awk '!asdf[$0]++; /COMMIT|Completed on|Generated by/;' | uniq
The second awk rule prints again any line that matches “COMMIT” or “Completed on” or “Generated by,” which appear multiple times in the iptables-save output. I was programmatically adding rules and one host in particular was just adding new ones despite the identical rule already existing. So I had to remove the duplicates and save the output, but keep all the duplicate “COMMIT” statements. I also wanted to keep all the comments as well.