DEV Community

Nam Hoang
Nam Hoang

Posted on

Handy Sed One-Liners for Text Manipulation and Processing

FILE SPACING:

Double space a file:

sed G
Enter fullscreen mode Exit fullscreen mode

Double space a file which already has blank lines, output file should contain no more than one blank line between lines of text:

sed '/^$/d;G'
Enter fullscreen mode Exit fullscreen mode

Triple space a file:

sed 'G;G'
Enter fullscreen mode Exit fullscreen mode

Undo double-spacing (assumes even-numbered lines are always blank):

sed 'n;d'
Enter fullscreen mode Exit fullscreen mode

Insert a blank line above every line which matches "regex":

sed '/regex/{x;p;x;}'
Enter fullscreen mode Exit fullscreen mode

Insert a blank line below every line which matches "regex":

sed '/regex/G'
Enter fullscreen mode Exit fullscreen mode

Insert a blank line above and below every line which matches "regex":

sed '/regex/{x;p;x;G;}'
Enter fullscreen mode Exit fullscreen mode

NUMBERING:

Number each line of a file (simple left alignment):

sed = filename | sed 'N;s/\n/\t/'
Enter fullscreen mode Exit fullscreen mode

Number each line of a file (number on left, right-aligned):

sed = filename | sed 'N; s/^/     /; s/ *\(.\{6,\}\)\n/\1  /'
Enter fullscreen mode Exit fullscreen mode

Number each line of file, but only print numbers if line is not blank:

sed '/./=' filename | sed '/./N; s/\n/ /'
Enter fullscreen mode Exit fullscreen mode

Count lines (emulates "wc -l"):

sed -n '$='
Enter fullscreen mode Exit fullscreen mode

TEXT CONVERSION AND SUBSTITUTION:

Convert DOS newlines (CR/LF) to Unix format:

sed 's/.$//'
Enter fullscreen mode Exit fullscreen mode

Convert Unix newlines (LF) to DOS format:

sed "s/$/`echo -e \\\r`/"
Enter fullscreen mode Exit fullscreen mode

Delete leading whitespace (spaces, tabs) from front of each line:

sed 's/^[ \t]*//'
Enter fullscreen mode Exit fullscreen mode

Delete trailing whitespace (spaces, tabs) from end of each line:

sed 's/[ \t]*$//'
Enter fullscreen mode Exit fullscreen mode

Delete both leading and trailing whitespace from each line:

sed 's/^[ \t]*//;s/[ \t]*$//'
Enter fullscreen mode Exit fullscreen mode

Insert 5 blank spaces at beginning of each line (make page offset):

sed 's/^/     /'
Enter fullscreen mode Exit fullscreen mode

Align all text flush right on a 79-column width:

sed -e :a -e 's/^.\{1,78\}$/ &/;ta'
Enter fullscreen mode Exit fullscreen mode

Center all text in the middle of 79-column width:

sed  -e :a -e 's/^.\{1,77\}$/ & /;ta'
Enter fullscreen mode Exit fullscreen mode

Substitute (find and replace) "foo" with "bar" on each line:

sed 's/foo/bar/'
Enter fullscreen mode Exit fullscreen mode

Substitute "foo" with "bar" EXCEPT for lines which contain "baz":

sed '/baz/!s/foo/bar/g'
Enter fullscreen mode Exit fullscreen mode

Reverse order of lines (emulates "tac"):

sed '$!G;h;$!d'
Enter fullscreen mode Exit fullscreen mode

Join pairs of lines side-by-side (like "paste"):

sed '$!N;s/\n/ /'
Enter fullscreen mode Exit fullscreen mode

If a line ends with a backslash, append the next line to it:

sed -e :a -e '/\\$/N; s/\\\n//; ta'
Enter fullscreen mode Exit fullscreen mode

Add a blank line every 5 lines:

gsed '0~5G'
Enter fullscreen mode Exit fullscreen mode

SELECTIVE PRINTING OF CERTAIN LINES:

Print first 10 lines of file (emulates behavior of "head"):

sed 10q
Enter fullscreen mode Exit fullscreen mode

Print first line of file (emulates "head -1"):

sed q
Enter fullscreen mode Exit fullscreen mode

Print the last 10 lines of a file (emulates "tail"):

sed -e :a -e '$q;N;11,$D;ba'
Enter fullscreen mode Exit fullscreen mode

Print only lines which match regular expression (emulates "grep"):

sed -n '/regexp/p'
Enter fullscreen mode Exit fullscreen mode

Print only lines which do NOT match regexp (emulates "grep -v"):

sed '/regexp/!d'
Enter fullscreen mode Exit fullscreen mode

Print paragraph if it contains AAA:

sed -e '/./{H;$!d;}' -e 'x;/AAA/!d;'
Enter fullscreen mode Exit fullscreen mode

Print section of file from regular expression to end of file:

sed -n '/regexp/,$p'
Enter fullscreen mode Exit fullscreen mode

SELECTIVE DELETION OF CERTAIN LINES:

Print all of file EXCEPT section between 2 regular expressions:

sed '/Iowa/,/Montana/d'
Enter fullscreen mode Exit fullscreen mode

Delete duplicate, consecutive lines from a file (emulates "uniq"):

sed '$!N; /^\(.*\)\n\1$/!P; D'
Enter fullscreen mode Exit fullscreen mode

Delete all blank lines from a file:

sed '/^$/d'
Enter fullscreen mode Exit fullscreen mode

Delete all consecutive blank lines from file except the first:

sed '/./,/^$/!d'
Enter fullscreen mode Exit fullscreen mode

Delete all trailing blank lines at end of file:

sed -e :a -e '/^\n*$/{$d;N;ba' -e '}' 
Enter fullscreen mode Exit fullscreen mode

SPECIAL APPLICATIONS:

Remove nroff overstrikes (char, backspace) from man pages:

sed "s/.`echo \\\b`//g"
Enter fullscreen mode Exit fullscreen mode

Get Usenet/e-mail message header:

sed '/^$/q'
Enter fullscreen mode Exit fullscreen mode

Get Usenet/e-mail message body:

sed '1,/^$/d'
Enter fullscreen mode Exit fullscreen mode

Add a leading angle bracket and space to each line (quote a message):

sed 's/^/> /'
Enter fullscreen mode Exit fullscreen mode

Delete leading angle bracket & space from each line (unquote a message):

sed 's/^> //'
Enter fullscreen mode Exit fullscreen mode

Remove most HTML tags:

sed -e :a -e 's/<[^>]*>//g;/</N;//ba'
Enter fullscreen mode Exit fullscreen mode

Extract multi-part uuencoded binaries:

sed '/^end/,/^begin/d' file1 file2 ... fileX | uudecode
Enter fullscreen mode Exit fullscreen mode

SORTING PARAGRAPHS OF FILE ALPHABETICALLY:

sed '/./{H;d;};x;s/\n/={NL}=/g' file | sort | sed '1s/={NL}=//;s/={NL}=/\n/g'
Enter fullscreen mode Exit fullscreen mode

OPTIMIZING FOR SPEED:
Substitution executes more quickly if the "find" expression is specified before the "s/.../.../" instruction:

sed '/foo/ s/foo/bar/g' filename
Enter fullscreen mode Exit fullscreen mode

Top comments (0)