Editing Files Using SED: Unwrapped

Editing Files Using SED The Linux Stream Editor

The Linux text editor sed is the stream editor which allows you to edit a file non-interactively. This means that you can edit a file from within a script or via SSH over the network without opening a terminal. Linux administrators and devops will benefit in this automation and will want to learn how easily file changes can be made with sed.

Simple File Cleaning

One of the best uses of editing files using sed that I find and celebrate is the cleaning of configuration files. Although I am not against comments in configuration files, the place for documentation is in the manual page and comments in the file should explain why the setting has been implemented.

In Ubuntu 20.04 the time server configuration /etc/chrony/chrony.conf has 45 lines. Now, I am not suggesting that 45 lines is excessive it is more than is required. A cluttered configuration file can make it difficult to focus on what has been configured. Using sed_we can easily filter or permanently delete commented and empty lines. To avoid any issues for the demonstration we will copy the configuration file to the home directory and work with the copy. We also need to make sure that chrony is installed.

$ sudo apt update
$ sudo apt install -y chrony
$ cp /etc/chrony/chrony.conf $HOME/chrony.conf
$ sed -E '/^(#|$)/d' $HOME/chrony.conf
pool ntp.ubuntu.com iburst maxsources 4
pool 0.ubuntu.pool.ntp.org iburst maxsources 1
pool 1.ubuntu.pool.ntp.org iburst maxsources 1
pool 2.ubuntu.pool.ntp.org iburst maxsources 2
keyfile /etc/chrony/chrony.keys
driftfile /var/lib/chrony/chrony.drift
logdir /var/log/chrony
maxupdateskew 100.0
rtcsync
makestep 1 3

With this simple command we have reduced the file from 45 lines to 10 lines and it easily fits on a single screen. There are further edits we could make but this has been a simple start.

Explaining the Steps

The options are listed below:

-E : Set the enhanced regular expression engine. We need this to process the group we specify in the regular expression

” : We use single quote to encapsulate the complete regular expression and preventing the shell from reading any item as a meta character

// : The regular expression is delimited using the forward slash to open and close”

^ : The first part of the regular expression is the caret that specifies that the lines starts with
() : The parenthesis specifies a group and why we need the option -E to sed. This is an enhanced regular expression.

^(#|$) : The grouping in this case along with the rest of the regular expression allows the commented or empty lines to be deleted.

The command only filters lines and does not touch the underlying file. At least not yet and it is good to test the command before editing the file.

Deleting Other Lines When Editing Files Using Sed

As we are concentrating on editing files with sed we are likely to need to change the configured time sources. In an enterprise you will likely source time from a local infrastructure device on your own network. We can easily delete the configured pool entries too:

$ sed -Ee '/^(#|$)/d' -e '/^pool/d' $HOME/chrony.conf
keyfile /etc/chrony/chrony.keys
driftfile /var/lib/chrony/chrony.drift
logdir /var/log/chrony
maxupdateskew 100.0
rtcsync
makestep 1 3

Although just extending the expressions on the CLI by adding the -e option for each new expression we want is possible it will also get messy and may not be repeatedly correct. Wehn editing files using sed we can also use sed script files. We will use this to first insert two of our own entries and the delete the lines we don’t want.

SED Script Files Help Automate Editing

To create a script file we simply use our prefered text editor:

$ vim chrony.sed
1i server 192.168.1.1 iburst prefer
2i pool uk.pool.ntp.org 
/^(#|$)/d
/^pool/d

We have added an entry to add our infrastructure time source at line one and at line 2 a failover to a public time source. It is easier to add the lines before any deletions and the line addresses change when lines are added or deleted. With the script in place:

$ sed -Ef chrony.sed '/^(#|$)/d' $HOME/chrony.conf
server 192.168.1.1 iburst prefer
pool uk.pool.ntp.org
keyfile /etc/chrony/chrony.keys
driftfile /var/lib/chrony/chrony.drift
logdir /var/log/chrony
maxupdateskew 100.0
rtcsync
makestep 1 3

In-Place Edit

The file script now works and we can implement the final changes now by editing files using sed and the option -i for an in-place edit.

$ sed -i -Ef chrony.sed '/^(#|$)/d' $HOME/chrony.conf