Solaris 11 Creating and maintaining ZFS Pools

Even though ZFS, the Zetabyte File System made its appearance during the life of Solaris 10; in 11 the open Source marvel becomes the default file-system and the root file-system is ZFS. At the heart of the file-system are ZFS pools which group disks together use able data sets. In this tutorial we look at the zpool and zfs commands but the focus and concentration will be on zpool.

During the installation of Solaris 11 the root file system will be created on the pool called rpool; the root pool. This can be seen using the command

zpool list

We can run the command as is and we will see all pools, in this case we have just the one, rpool. However if we have more and we just want to see the single pool we could issue the command

zpool list rpool

In this case we would only see the single pool’s details. The pool represents the disk space used and creates the initial file-system. So in addition to the ZFS pool we can list the default ZFS file system witht eh command

zfs list

Here we see the ZFS file system rpool and its data sets, note that there is no need to format the file system it is created with the pool, data sets allow for different file system attributes such as compression, quotas etc. to be in place for different directories within the same file system:

This tutorial will look at ZFS pools, file systems and data sets will be covered in the next Solaris installment.

We can create ZFS pools with the zpool create sub-command. We will need additional disks in our system or disk files. For the purpose of the example I will use disk files as it is easy to create as many as I need. In your own environment you can use files or physical disks. A physical disk can be referred to by its full name or just the device name, ie

c8t1d0

or

/dev/dsk/c8t1d0

Disk files have to referred to by their name and full path, ie

/root/disk1

To create the disk files I use the command mkfile (/usrsbin/mkfile)

mkfile 128M /root/disk{1,2,3,4}

This command creates 4 files of 128M each. We can use these files as alternatives to physical disks when creating pools.

The simplest ZFS pool can consist of just one disk. If we add more disk to the pool the data will be striped across the disks but no fault tolerance is provided.

zpool create pool1 /root/disk1

Here we have created a pool, pool1, consisting of the single 128M disk. Not only will the pool have been created but the file system too and it will be mounted to a directory /pool1.

We can view the pool with

zpool list pool1

We can view the file system with

zfs list pool1

The remarkable feature here is that the formatting of the file system, creation of the mount point and mounting were all automated. If we need the pool to be mounted to a certain directory, then we use the -m option in the create sub-command. The directory /data, in the following command will be used as the mount point and will be created if it does not exist. If it does exist then it will need to be empty.

zpool create -m /data pool1 /root/disk1

We can see from the screen capture that the pool is created and mounted now to the /data directory.

If we need to add additional storage to this pool we can so and the file system will be increased online accordingly.

zpool add pool1 /root/disk2

Viewing the information from zpool list poool1 and zfs list pool1 confirm the increase to the pool and file system respectively.

We can now start to see the power and speed of management of this file system.

We can add fault-tolerance to the pool if we create it with a specified RAID level. The first that we will look at is a Mirrored Pool using the keyword mirror . We will use mirror in this example but possibilities include:

• mirror
• raidz
• raidz1
• raidz2
• raids3

A mirror being RAID level1 and the other raidz being implementations of RAID5.

A mirror consists of two or more disks. Whatever is written to one disk in the array is replicated, mirrored, to all other disks in the array. Usually a mirror will consist of two disks and allows for a failure of one disk. If we had three disks in the mirror we would add durability allowing for two disk failures. The cost is the disk space. If I use two 128M disks, I have 128M space, if I create a mirror on three 128M disks I still have just the 128M of space, and so on with 4. We add durability with extra disk not space.

First of all we need to destroy the existing pool and re-use the disk files. This does exactly what it says on the tin, and of course, all data on the pool will be lost.

zpool destroy pool1

With the pool gone we can re-sue the original disk files to create the new mirror.

zpool create pool1 mirror /root/disk1 /root/disk2

Using the list command we can see that only the single 128M of space becomes available even though we have used two disk. The same would apply no matter how many disks we included into the mirror, we have the single storage which is replicated to all other disks in the array.

zpool list pool1

We can see the two disks participating within the mirror using the status sub-command

zpool status pool1

The status shows that both disks are online as is the mirror and pool.

Well we need to, in order to prepare for disaster we have to know that the mirror will continue with one failed disk in our case. We also need to know how to replace the failed disk and rebuild the mirror.

We will start by adding some data to the mirror whilst it is in operation and then we will simulate a disk failure by destroying the MBR (master boot record) on one of the disks.

• cp /etc/hosts /pool1 #adds the hosts file to the m1rror to show that data will not be lost
• dd if=/dev/zero of=/root/disk1 count=1 bs=512 #destroys the MBR on disk1
• cat /pool1/hosts #proves we can read the file still

We can scrub the pool to detect errors and then display the status. We know the pool is still active as we can read and write to the pool; but we have a failed disk.

zpool scrub pool1
zpool status pool1

To repair the mirror, we will remove disk1 and add in a new disk to the mirror.

zpool detach pool1 /root/disk1
zpool attach pool1 /root/disk2 /root/disk3

Note: We attach disk3 to the existing disk in the mirror. Data will be copied from disk2 to disk3 in this case.

Now, when we run the status sub-command we will see that the mirror has been re-silvered or the data copied to the new drive.

zpool status pool1

As we mentioned earlier, as well as mirroring we can implement RAID5 style arrays, where a disk is used to hold pariy information. In the event of a failed drive any data can be evaluated based on the parity data. If we use just two drives then this is similar to mirror, we have the data drive and parity information.

Using 2 x 128M drives

1 drive data = 128M
1 drive parity
Storage = 128M

Using 3 X 128M drives

2 drives data = 256M
1 drive parity
Storage = 256M

So we two drives we have no advantage over mirroring but for each additional drive added we can make full use of the space. This would use the keyword raidz  or raidz1.

If we wanted to use two parity drives we can allow for up to to failed drives. We need a minimum of 3 drives for this, but should have 4 or more to have advantages over mirroring; we would use raidz2 to set this up.

If we wanted to allow for 3 failed drives, the maximum, then we could use raidz3. For this we would need a minimum of 4 drives but practically would use a minimum of 5 to have any advantage of mirroring.

To set this up we can first destroy the current pool so we can reuse the disks.

zpool destroy pool1

and define the new pool

zpool create pool1 /root/disk2 /root/disk3 /root/disk4

Using the three disks, each of 128M will gives us 240M or so free space and one disk used for parity. We have to look at the file system, not the pool space as the pool space will show all the space available including the parity information.

zfs list pool1

We have seen how we can create simple pools without fault tolerance with ZFS. This still simplifies our filesystem management as so much of the leg work is done for us. Building on this we have seen how we can allow for single and multiple disk failures with mirroring and raidz in ZFS pools.

The video can be seen on YouTube: