AWK tricks and tips
remove duplicate lines without sorting (will keep first instance)
awk '!a[$0]++' filefiltering - if the 5th column is 0 then skip it and print the rest
awk '{ if ($5 == 0) next; print }' file
remove duplicate lines without sorting (will keep first instance)
awk '!a[$0]++' file
filtering - if the 5th column is 0 then skip it and print the rest
awk '{ if ($5 == 0) next; print }' file