DEV Community

Cover image for Append text in each line in a large CSV (unix style)

Append text in each line in a large CSV (unix style)

leyluj profile image Leylow Lujuo ・1 min read

Say you have a 10GB csv file and you want to add a text in the end of each line.

Obviously, opening it in vim work cut it

oh will it? ... let me confirm ... I tried and vim crushed :( ...

"So, what do we do?" I asked my conscious self while googling and stack-overflow-ing with 10+ tabs opened.

Viola! one of my stack-overflow tab came through and the answer was sed - a stream editor.

sed -r 's/text_to_find/text_to_replace/' input_file > output_file
# example
sed -r 's/\r/|the_text\r/' big_file.csv > output.csv

Basically, what it does here is;

-r this option represent extended regular expression

's/**\r**/|**the_text\r**/' the \r represent EOL, so replace EOL with the_text plus EOL(so that the output won't be a single line file)

The rest i.e... input and output file is self explanatory.

Writing is exhausting, I'm done. I have being told to be positive, so i'm scratching this and the line below.

Now i have to search for a cover image, smh.

Thanks for your one minute, hopefully I will write again.

Discussion (0)

Editor guide