sed

Introduction

This is an article about sed.

There are also articles about awk and awk

Usually examples are shown in WSL2 or in Bash in Linux .

Sed basics

In order to apply SED, just enter the command line

echo ice | sed s/ice/fire/

Result:

fire

Note that using / is not required.

You can use s followed by any other character, for example : or , or |

The result will be the same, the main thing is that all three delimiters are the same and the character itself has no additional meaning.

echo mice | sed s/m/r/
echo mice | sed s,m,r,
echo mice | sed s:m:r:

rice
rice
rice

If you chose |, the command must be put in quotes - | has a special role in bash - pipeline

echo mice | sed 's|m|r|'

rice

If you are editing file paths (and they contain /) then this is exactly the case when it is convenient to choose another delimiter

For example, you need to replace /bin/bash with /bin/sh

It is much more convenient to use @ as a delimiter than to escape every slash.

Compare the two identical commands

sed 's@/bin/bash@/bin/sh@' /etc/passwd

sed 's/\/bin\/bash/\/bin\/sh/' /etc/passwd

Delete something from a file

Deletion is handled by option d you can read a separate article about it sed d

You can also delete by replacing with an empty string

s/something//

And delete using other options, for example, q

Make a replacement

Replacement is handled by option s you can read a separate article about it sed s - substitute

Escaping characters in sed

Special characters are escaped with \

What to include in special characters depends on which sed you use, but I advise always escaping $.*[\^ as well as spaces and quotes.

A space can also be replaced with \s

. in regular expressions means any single character except the beginning of a new line \n so if you want to write a url use \

heihei\.ru

An example of escaping dots and quotes to change the locale in CentOS you can study here

Suppose there is a file input.txt with the following content

Here is a String / it has a Name Here is an Integer / it has a Name Here is a Float it / has a Name

We want to discard everything to the left of /a, including /a, and write it to a file.

sed 's/^.*/a//' > output.txt

As a result we get an error

-e expression #1, char 15: unknown option to `s'

To make the command work you need to add \ before /

sed 's/^.*\/a//' > output.txt

Result:

Here is a String Here is an Integer Here is a Float

Escaping spaces can come in handy when replacing one phrase with another

In the script sites.sh from the directory /opt/andrei/scripts/ replace the phrase Bike website topbicycle.ru with Travel website heihei.ru you need to run

sed -i s/Bike\ website\ topbicycle.ru/Travel\ website\ heihei.ru/ /opt/andrei/scripts/sites.sh

Two conditions at once in Sed

Suppose we have a file input.txt with the following content

Here is a String /b it has a Name Here is an Integer /b it has a Name Here is a Float /b it has a Name

We want to discard everything to the left of /b, including /b, and everything to the right has.

Thus, only the word it.

You need to take into account the need to escape the special character / and we also want to direct the output to a file.

sed 's/^.*\/b// ; s/has.*//' input.txt > output.txt

Result:

it it it

Get a range of lines

When you work with large files, for example with logs, you often need to get only certain lines, for example, at the moment a bug appears.

Copying from the command line UI is not always convenient, but if you roughly know the range of the lines you need - you can copy only them and write them to a separate file.

For example, you need lines 9570 through 9721

sed -n '9570,9721p;9722q' project-2019-10-03.log > bugFound.txt

Replace everything between certain characters

Delete everything between square brackets including the brackets

sed 's/\[.*\]//' input.txt > output.txt

Create a function

So as not to recall sed commands every time, you can create a function

Let's take the command that deletes comments and empty lines from the previous example and write it as a function clean_file.

First, in the console you need to type in the terminal function clean_file { and press Enter

Then enter the expression sed -i '/^#/d ; /^$/d' $1

$1 means that the function will take one argument. This, of course, will be the file name.

Then you need to press again Enter and on a new line write } and press Enter once more

$ function clean_file { > sed -i '/^#/d;/^$/d' $1 > }

Make sure the file contains comments and empty lines. If not - create them to keep the experiment clean.

cat websites

# Travel https://www.heihei.ru # Bicycles https://www.topbicycle.ru # IT https://www.eth1.ru

clean_file websites
cat websites

https://www.heihei.ru https://www.topbicycle.ru https://www.eth1.ru

Discard everything to the left of a certain word

Suppose we have a file input.txt with the following content

Here is a String it has a Name Here is an Integer it has a Name Here is a Float it has a Name

We want to discard everything to the left of the word it, including the word it, and write it to a file.

sed 's/^.*it//' input.txt > output.txt

^ means that we start from the beginning of the line Result:

 has a Name

 has a Name

 has a Name

For clarity I will explain the syntax by comparing two commands. Look carefully when we replace the word Here with There.

There is located between two slashes. I will color them green and red for clarity.

sed 's/Here/There/'

And when we want to delete something, we first describe what we want to delete. For example, everything from the beginning of the line to the word it.

Now on the right side of the condition, where the replacement value used to be, we write nothing, i.e. we replace with an empty string. I hope the logic is clear.

sed 's/^.*it//' > output.txt

Discard everything to the right of a certain word

Suppose we have a file input.txt with the following content

Here is a String / it has a Name Here is an Integer / it has a Name Here is a Float / it has a Name

We want to discard everything to the right of the word is, including the word is, and write it to a file.

sed 's/is.*//' > output.txt

Result:

Here Here Here

Removing newline breaks

sed ':a;N;$!ba;s/\n//g' file ;

Delete everything after a certain line

Suppose you want to delete all lines after the third

sed 3q input.txt > output.txt

Article author: Andrei Olegovich