Thursday, 11 April 2013

Unix: Count the number of delimiters for each line in a huge file

Found this here:

*THIS* is the one to use:

awk 'BEGIN { FS = ";" } ; { print NF-1 }' <million_lines.csv>

note that this outputs to console, so pipe to file for easier use
(add this at the end: > output_file)


input file is "addresses.txt" and i want to

awk 'BEGIN { FS = "\t" } ; { print NF-1 }' addresses.txt > tab_count.txt

The other approach using echo |tr|wc is really really slow.

Note to self: learn awk and sed!

No comments: