Introduction
sed
is a powerful tool often used in the Linux terminal to edit text documents without opening them. The word "sed" is a contraction of the phrase "Stream Editor". It is essentially a tool for editing streams of text. sed
takes in text from a file or text received from standard input (stdin) and lets you perform some operations on it. In this tutorial, I will demonstrate how sed
can be used to make string substitutions in text files in the Linux command-line environment.
Creating a File
First, open your Linux terminal and navigate to your home directory if you aren't there already. Create a new file with the cat
command by following the succeeding code block:
cat << EOF > myfile.txt
This is my box.
It is mine.
My box is big, it is a big box.
My box is black, my box is strong, my box is the best.
My mother bought it for me.
Let's go to the sandbox to play with my box.
EOF
This is the sample file which we will perform most of our sed
operations on. In the next section, you will begin to see how the sed
command is fundamentally used to alter and substitute text.
The Basic Syntax of sed
The most common way in which you will use sed
is this:
sed 's/OLD-WORD/NEW-WORD/' FILENAME
You start with typing the sed
keyword which lets the computer know you want to access the stream editor tool.
s
is the search operator that looks through the file.
Next, you enter the word you want to substitute out or replace which is the OLD-WORD
, and the word you want to substitute in which is the NEW-WORD
.
Then you enter the name of the target file FILENAME
you wish to perform this sed
operation on.
The forward slash /
which separates each keyword is called a delimiter, more on delimiters will be explained later.
Now, using the sed
command in the file you created earlier, myfile.txt, you will attempt to replace instances of the old word box with a new word bag.
Enter the following command and press ENTER
:
sed 's/box/bag/' myfile.txt
Immediately, the following output is displayed:
This is my bag. It is mine. My bag is big, it is a big box. My bag is black, my box is strong, my box is the best. My mother bought it for me. Let's go to the sandbag to play with my box.
You may notice that the word sandbox has been altered to sandbag. This occurs because the sed
command acts on text patterns it encounters, not simply the standalone word.
You may also observe that on each line, only the first instance of box is changed to bag. The rest are left unchanged. You can look at the highlighted words in the generated output to observe this.
Note: The file myfile.txt itself remains unchanged. By default the
sed
command prints the result of its operation to standard output (stdout). More on how to permanently save your edits will be covered later.
Substituting Specific Instances
By default, only the first instance of the specified text is altered. If you wish to change the order of the affected instance, you type in the following:
sed 's/box/bag/2' myfile.txt
The following output is displayed in your terminal upon execution:
This is my box. It is mine. My box is big, it is a big bag. My box is black, my bag is strong, my box is the best. My mother bough it for me. Let's go to the sandbox to play with my bag.
In the command entered, you'll notice that we've added a number 2. What this does is, it makes it so that on each line where box appears, it isn't the first instance that is altered, but rather, the second instance. Looking at the generated output, you will be able to observe this.
You may replace 2 with any number to denote which instance of the old word you wish to substitute out for your new word.
Substituting All instances
If you wish for every instance of the old word box in the file to be replaced with the new word bag, enter the following command:
sed 's/box/bag/g' myfile.txt
You should see this output:
This is my bag. It is mine. My bag is big, it is a big bag. My bag is black, my bag is strong, my bag is the best. My mother bought it for me. Let's go to the sandbag to play with my bag.
The g
flag added indictates you want to make the substitution globally, i.e everywhere the old word occurs in the file.
Navigating Case-Insensitive Substitutions
The sed
command is by default case-sensitive. Using the i
flag in your sed
operation helps you make case-insensitive substitutions. This is for situations where you wish to find and replace instances of the old word regardless of the letter case they may be in. Below is an example:
sed 's/BOX/bag/i' myfile.txt
The output:
This is my bag. It is mine. My bag is big, it is a big box. My bag is black, my box is strong, my box is the best. My mother bought it for me. Let's go to the sandbag to play with my box.
You'll observe that even though BOX is used in the command, instances of its lower-case, box, in the file are altered. The substitution wouldn't work without the i
flag, you can try this out in your terminal environment to see.
In the next section, we will see how to make changes on designated lines.
Substituting Text on Specific Lines
To find and replace a text pattern on a specific line, instead of all the lines in the file, use the following command:
sed '3 s/box/bag' myfile.txt
Generated output:
This is my box. It is mine. My bag is big, it is a big box. My box is black, my box is strong, my box is the best. My mother bought it for me. Let's go to the sandbox to play with my box.
In the preceding generated output, you will observe that only the first instance of box on the third line has been substituted out. Placing the number 3
before s
tells the computer that in myfile.txt
you want to find and replace the word box for bag in the third line only.
Also, you can change that number to whatever line number you want your substitution made instead.
Note: If you have a large file, to display the line numbers, enter this command:
sed = myfile.txt
Substituting Text in a Range of Lines
To find and replace text in a range of lines:
sed '1,3 s/box/bag/' myfile.txt
The displayed output
This is my bag. It is mine. My bag is big, it is a big box. My box is black, my box is strong, my box is the best. My mother bought it for me. Let's go to the sandbox to play with my box
In the sed
command entered, the first number is followed by the second number and a comma separates them. This command lets the computer know that you want your sed
substitutions to be made within the range of line 1 to line 3.
In the next section, we will experiment with different ways of how to execute multiple sed
substitutions at a go.
Making Multiple sed
Substitutions
At times, you may wish to use the sed
command multiple times on the same text file, there are various ways for you to execute this in one command entry in the Linux terminal.
To demonstrate this, you will substitute the words box with bag and black with red in the text file we created, myfile.txt. This can be done using any of the following methods:
1. Using Pipe |
Enter in the following command:
sed 's/box/bag/' myfile.txt | sed 's/black/red/'
The output:
This is my bag. It is mine. My bag is big, it is a big box. My bag is red, my box is strong, my box is the best. My mother bought it for me. Let's go to the sandbag to play with my box
Remember, sed
is a Stream Editor. It prints out to standard output(stdout) and can also take in text piped in from standard input(stdin). In the above scenario, the output from the first sed
substitution is piped in to the next sed
command to make its own substitutions too. This lets us make multiple sed
substitutions at once.
2. Using the -e
Option
You may use the -e
option provided by sed
to conveniently execute multiple sed
substitutions at a go. See below:
sed -e 's/box/bag/' -e 's/black/red/' myfile.txt
Output:
This is my bag. It is mine. My bag is big, it is a big box. My bag is red, my box is strong, my box is the best. My mother bought it for me. Let's go to the sandbag to play with my box
3. Using the Semicolon ;
The semicolon ;
allows you to quickly conjoin your multiple sed
commands and execute them at once on the same file. See below:
sed 's/box/bag/;s/black/red/' myfile.txt
Printed output:
This is my bag. It is mine. My bag is big, it is a big box. My bag is red, my box is strong, my box is the best. My mother bought it for me. Let's go to the sandbag to play with my box
We've seen the different ways available that help facilitate executing multiple sed
substitutions at a go. In the next section we will go over what happens when you only want the lines you've altered returned to output.
Printing Edited Lines Only
If you are working on a huge file, and you don't want the output of your sed
commands on the file filling up your terminal environment each time you make an execution. There is a method you can use to isolate and output only the lines of the file your command has altered. See below:
sed -n 's/box/bag/p' myfile.txt
This is the generated output:
This is my bag. My bag is big, it is a big box. My bag is black, my box is strong, my box is the best. Let's go to the sandbag to play with my box
In the command executed we use the -n
option and the p
flag. The -n
option suppresses automatic printing of the output. Meanwhile, the p
flag causes double printing of the edited lines. The combination of these two will cause the command to print out the edited lines only.
Note:
sed
provides a whole lot of options and flags you can use to make all kinds of tweaks to your substitutions. To access them, you simply entersed --help
into your terminal orman sed
for the full user manual.
Delimiters
So far, we've been using the forward-slash /
as a delimiter to separate the relevant categories of our substitution commands in the terminal. Although, there will be situations where this will not suffice.
For example if you're trying to use sed
on a URL, a URL already has forward slashes /
so an error will occur if you try to use a forward-slash(/) as a delimiter. Fortunately, other special characters can be used as delimiters too.
Let us try to make a sed
substitution on the URL https://www.wikipedia.org/homepage.html
. We will take the text pattern org/homepage and replace it with inc/search. See below:
echo "https://www.wikipedia.org/homepage.html" | sed 's$org/homepage$inc/search$'
The resulting output:
https://www.wikipedia.inc/search.html
In the command entered, notice that we now use a dollar-sign $
as a delimiter to separate the text we want to substitute out org/homepage and the text to be substituted in inc/search. If we had tried using the forward-slash /
as a delimiter, the operation wouldn't have been successful.
Saving Changes
It is not enough to only make our edits in stream, you may wish to have the files altered permanently. To make the changes permanent, you must pass the -i
option. See below for an example acting on the earlier created myfile.txt file:
sed -i 's/black/red/' myfile.txt
To confirm that the file has been altered, you will open it with the cat
command like this:
cat myfile.txt
The result:
This is my box. It is mine. My box is big, it is a big box. My box is red, my box is strong, my box is the best. My mother bought it for me. Let's go to the sandbox to play with my box.
Owing to the -i
option, we have now confirmed that the old word black
has been replaced with a new word red
permanently.
Note: It is not advised to use the
-i
option casually because doing so could cause you to permanently lose your data. Use it only when you are certain you want to make irreversible changes.
Saving Changes to a New File
Suppose instead you wanted to save your changes to a new file leaving the first one unaltered. Enter the following command:
sed 's/box/bag/g' myfile.txt > secondfile.txt
If the file secondfile.txt did not exist, it is created automatically in your present working directory with the output generated from the sed
command executed.
If you open the file with the command cat secondfile.txt
, you should see the following:
This is my bag. It is mine. My bag is big, it is a big bag. My bag is red, my bag is strong, my bag is the best. My mother bought it for me. Let's go to the sandbag to play with my bag.
Making a Backup
You can also pass an option that tells the computer to permanently edit the file but create a backup of the original document. To do this, you type out a variation of the -i
option, which is the -i.bak
option. It is used like this:
sed -i.bak 's/box/bag/g' myfile.txt
The above command will save your changes to the myfile.txt file, but also creates a backup of the original file with a .bak extension in the same directory. So the backup created from the command above will be myfile.txt.bak.
Running cat myfile.txt
gets you:
This is my bag. It is mine. My bag is big, it is a big bag. My bag is red, my bag is strong, my bag is the best. My mother bought it for me. Let's go to the sandbag to play with my bag.
Running cat myfile.txt.bak
gets you:
This is my box. It is mine. My box is big, it is a big box. My box is red, my box is strong, my box is the best. My mother bought it for me. Let's go to the sandbox to play with my box.
You can see that myfile.txt is permanently altered while myfile.txt.bak contains its original content.
Conclusion
Now you know the different ways of using sed
to make basic substitutions in text files in the linux command line, how to make multiple sed changes in one entry, and how to save your changes permanently.
Top comments (0)