Sed Inserts Deletes and Appends

Having the power to be able edit files non-interactively via a script or directly across many servers from the CLI is so important to speed your administration of Linux systems. It can also be very simple and this is what we would to show you in this blog where we look at sed inserts deletes and appends. We are working on a Alma Linux 8 system but vaguely this would be the same on any Linux distribution. The demonstration file that we work with in the chrony.conf, the time server configuration file.

• In Ubuntu this is /etc/chrony/chrony.conf
• In RedHat based systems this is /etc/chrony.conf

Although you could work with any file you may prefer working with the same file so your results will be similar. If you need to install the chrony time server:

• On Ubuntu: sudo apt update && sudo apt install -y chrony
• On RedHat based systems: sudo yum install -y chrony

Now that we have the file we will make a copy of it within our own home directories. Especially when learning sed, it is good to work on non-live files to avoid potential problems.

Replace the path to the file with that used by your distribution

cp /etc/chrony.conf ~

We can count the number of lines in the file:

wc -l ~/chrony.conf

In general, we will have close to 40 lines in this file, most of which are comments or empty lines. One of the most effective and impressive tasks that we can use with sed is to delete the extraneous information from the file. Deleting commented lines and empty lines:

sed -Ei.bak '/^($|#)/d' ~/chrony.conf

This command first creates a backup named chrony.conf.bak and edits the original removing the comments and blank lines.

wc -l ~/chrony.conf
wc -l ~/chrony.conf.bak

Observing the output, we can see the new file has 7 or so lines depending on the distribution and the backup file 38 or so lines. As I say, this is very impressive and a quick and easy win.

Explaining the options that we have used we annotate the command for you, don’t forget that we have a video to help you at the end of the blog:

• -E use the enhanced regular expression engine. We need this as we use groups in the regular expression
• -i the inplace edit, were we edit the file rather than just displaying to the screen
• .bak this is the extension that we want to apply to the backup. If we do not include it, no backup will be made. The extension we use must be pushed right next to the -i option without any space
• ” the single quotes enclose the expression  that is passed to sed. The quotes protect all characters from being expanded by bash. We need them passed through as they are to the command sed
• // the forward slashes denote that we are using a regular expression rather than a line number. The forward slashed enclose the complete regular expression
• ^ the first regular expression metacharacter that we use is the caret or ‘hat’. This means lines begin with 
• () the parentheses enclose groups, we use the option -E to allow groups. Ther groups allow us to use OR expressions here we use # or $. Lines begin with a comment or the dollar being the end of line. So the line starts with a comment ot the lines start with the end of line marker
• $ indicates the end of line in Linux file. If the line begins with a $ it is and empty line
• | the logical OR operator
• # literally just the # symbol. If the line starts with a # it is a commented line
• d the final d is the sed command to delete the matching lines
• The complete expression within the single quotes allow sed to delete the comments and empty lines from the file

Having now created are new clean file it is much easier to read, we always have help maintained in the documentation should we need it. But, or course, we are just getting practice here, this is not the live file. We will continue with more practice with sed inserts deletes and appends!

man 5 chrony.conf

Having deleted a few lines then using the regular expression, we can also look at inserting lines. To insert a new first line, maybe even a new comment explaining the file:

sed -i '1i #My clean chrony.conf' ~/chrony.conf

We insert before the current line 1 the new commented line. Using the line number we do not need the forward slashes surrounding a regular expression.

If we decide we do not need this new line, it is very easy the delete any numbered line:

sed -i '1d' ~/chrony.conf

A very simple and sparse command is all we need.

To delete the last line by line number without counting the lines we can use the $ symbol, it mis not read as a regular expression as we won’t add the forward slashes.

sed -i '$d' ~/chrony.conf

Should we want to add the last line back we can append the line after the current last line

sed -i '$a logfile /var/log/chrony.log'

Now hopefully we are becoming a little more confident using sed to edit our files, managing sed inserts deletes and appends. The video follows

More sed tutorials here: https://www.theurbanpenguin.com/linux-sed-match-first-occurrence/