VDO Data Optimizer

RHCSA 8 Study Guide

The VDO Data Optimizer was one of those features that was first shown in RHEL 7.5 as was the Web Interface Cockpit. This does mean that you can test VDO for free using CentOS 7.5 or higher including CentOS 8. The VDO Data Optimizer is a kernel driver that ships with CentOS 8 to reduce the data footprint used by filesystems. The kernel modules are kvdo for compression and uds for data-duplication.

To Install VDO and Start the VDO Service, (the service is used to start VDO volumes):

# yum install -y vdo kmod-kvdo
# systemctl enable --now vdo

We can create a new VDO device, this acts as a block device created by the DevMapper. These are thinly provisioned allowing us to set a maximum size larger than available at the start. If we miss the size option it will be sized to the physical device size. It is important to let the filesystem know that more space is available as it is unaware of any black magic with compression or de-dup.

# modprobe kvdo
# vdo create --name=vdo1 --device=/dev/sdb --vdoLogicalSize=20G
# vdo status --name=vdo1 | grep -E '(Deduplication|Compression)'
    Compression: enabled
    Deduplication: enabled

We can see that compression and de-dup are enabled. We can also check the loaded modules and the link back to the DevMapper:

# lsmod | grep -E '(uds|kvdo)'
kvdo                  581632  1
uds                   315392  1 kvdo
dm_mod                151552  12 kvdo,dm_log,dm_mirror

This is just a block device, to be able to use we must add a filesystem. This then needs to be mounted and we need to ensure it is mounted on each boot. The normal deal here. So, we format the filesystem and create a mount point, we call it /data. Using -K with mkfs.xfs allows the command to run more quickly on large volumes.  We append a new line to the /etc/fstab file. This then mounts the filesystem on boot. DevMapper devices will be uniquely named so we don’t particularly need the UUID of the filesystem. We do though, want to make sure that we wait for the vdo service before mounting the device. We can test the entry in the /etc/fstab file using mount -a.

# mkfs.xfs -K /dev/mapper/vdo1
# mkdir /data
# echo "/dev/mapper/vdo1 /data xfs x-systemd.requires=vdo.service 0 0" >> /etc/fstab
# mount -a
# mount -t xfs
/dev/mapper/cl-root on / type xfs (rw,relatime,seclabel,attr2,inode64,noquota)
/dev/mapper/vdo1 on /data type xfs (rw,relatime,seclabel,attr2,inode64,noquota,x-systemd.requires=vdo.service)

Yes, all look good. It may be worthwhile rebooting the system as a double check. In the exam, Red Hat will want all settings to persist a reboot.

Listing the space available from the filesystem we see the 20G of space:

# df -h /data
Filesystem        Size  Used Avail Use% Mounted on
/dev/mapper/vdo1   20G  176M   20G   1% /data

The real space available and used can be seen using vdostats:

# vdostats --human-readable
Device                    Size      Used Available Use% Space saving%
/dev/mapper/vdo1          8.0G      4.0G      4.0G  50%           98%

We can see the need to start with the very minimum of a 4G disk as vdo uses space from the start.

The largest file in the /boot directory is the rescue system RAM disk, 57M. We can copy this across multiple times to show the disk optimization:

# for i in {1..100}; \
   do cp /boot/initramfs-0-rescue-06558bd39a6542229483bde088da049b.img \
  /data/$i.img; done

We make 100 copies of the file in the /data directory. That should take 5.7G of disk space, (57M * 100), but ends up taking only about 100M.

First of all, let’s look at what df says after the copy, previously, it did have 176M:

# df -h /data
Filesystem        Size  Used Avail Use% Mounted on
/dev/mapper/vdo1   20G  5.8G   15G  29% /data

So, it shows about 5.7G extra, about right for the real data. As the data can be compressed and de-duplicated the actual storage is much less, about 100M.

# vdostats --human-readable
Device                    Size      Used Available Use% Space saving%
/dev/mapper/vdo1          8.0G      4.1G      3.9G  50%           99%

ENABLING / DISABLING FEATURES

As we have previously seen, both de-duplication  and compression is enabled by default. Compression will work on blocks identified as unique and de-deduplication on blocks already stored on the device. To disable de-duplication:

# vdo disableDeduplication --name=vdo1
Disabling deduplication on vdo vdo1

This can be re-enabled:

# vdo enableDeduplication --name=vdo1
Enabling deduplication on vdo vdo1

Similarly, working with the compression feature of VDO:

# vdo disableCompression --name=vdo1
Disabling compression on vdo vdo1
Stopping compression on VDO vdo1
# vdo enableCompression --name=vdo1
Enabling compression on vdo vdo1
Starting compression on VDO vdo1

INCREASING THE LOGICALSIZE

If you have found that the compression or de-duplication of data was better than planned for you can increase the vdoLogicalSize of a device allowing VDO to show more space to the filesystem.

# vdo growLogical --name=vdo1 --vdoLogicalSize=40G

The filesystem will need to be increased, using XFS, this would be:

# xfs_growfs /data
meta-data=/dev/mapper/vdo1       isize=512    agcount=4, agsize=1310720 blks
         =                       sectsz=4096  attr=2, projid32bit=1
         =                       crc=1        finobt=1, sparse=1, rmapbt=0
         =                       reflink=1
data     =                       bsize=4096   blocks=5242880, imaxpct=25
         =                       sunit=0      swidth=0 blks
naming   =version 2              bsize=4096   ascii-ci=0, ftype=1
log      =internal log           bsize=4096   blocks=2560, version=2
         =                       sectsz=4096  sunit=1 blks, lazy-count=1
realtime =none                   extsz=4096   blocks=0, rtextents=0
data blocks changed from 5242880 to 10485760

Online Instructor-Led Training