Alternate ZFS Layout for FreeBSD

In previous post, I introduced how to install FreeBSD to ZFS. Now I discover another layout for FreeBSD’s ZFS root. Most of the parts are the same as previous ones. There are only a few differences.

Harware environment
Asus N73SV Notebook
CPU:Intel Core i7 2630QM
12GB DDR3 RAM
750GB HDD

Installation media
FreeBSD-9.1-RELEASE-amd64-memstick.img

Installation media preparation
Download FreeBSD-9.1-RELEASE-amd64-memstick.img, and burn it into your USB flash disk.

Boot FreeBSD 9.1 from USB flash disk.

Enter LiveCD mode
Choose “LiveCD” instead of install
Login name is root without password, then you could enter LiveCD environment.

Create a ramdisk, and mount it at /tmp to store zpool.cache (optional)

1
2
# umount /dev/md1
# mdmfs -s 512M md1 /tmp

Create GPT+ZFS partition
Find out your HDD device name, usually, the first AHCI HDD should be /dev/ada0 (FreeBSD-9.x has AHCI support builtin).

1
# ls /dev/ad*

Check if there are partitions already in the HDD.

1
# gpart show ada0

If there are, use gpart to clean them.

Since your HDD partition layout is different from mine, please man gpart to learn about gpart delete and gpart erase by yourself.

After the partitions are clean, we can now create GPT.

1
# gpart create -s gpt ada0

Create a small parition for FreeBSD to boot from GPT. Attention, if your HDD is AF (Advanced Format) with 4KB sector size, please append “-a 4K” parameter to make sure your partition is correctly aligned.

1
# gpart add -s 64k -t freebsd-boot ada0

Leave other free space for ZFS. Attention, if your HDD is AF (Advanced Format) with 4KB sector size, please append “-a 4K” parameter to make sure your partition is correctly aligned.

1
# gpart add -t freebsd-zfs ada0

Write FreeBSD GPT+ZFS bootcode into first partition.

1
# gpart bootcode -b /boot/pmbr -p /boot/gptzfsboot -i 1 ada0

Create ZFS as single pool mode in second parition, and assign the “/“.

1
2
3
4
# zpool create -R /mnt -m none rpool /dev/ada0p2
# zfs create -o mountpoint=/ rpool/ROOT
# zpool set bootfs=rpool/ROOT rpool
# zfs set checksum=fletcher4 rpool/ROOT

Use zfs create command to create swap volume in ZFS volume group.

1
2
3
# zfs create -o sync=always -o primarycache=metadata -o secondarycache=none -V 8G rpool/swap
# zfs set checksum=off rpool/swap
# zfs set org.freebsd:swap=on rpool/swap

Use zfs create command to create system layout.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
# zfs create -o mountpoint=/usr rpool/USR
# zfs create -o mountpoint=/var rpool/VAR
# zfs create -o mountpoint=/home rpool/HOME
# zfs create -o compression=on -o exec=on -o setuid=off -o mountpoint=/tmp rpool/TMP
# zfs create -o mountpoint=none rpool/FREEBSD
# zfs create -o compression=lzjb -o setuid=off -o mountpoint=/usr/ports rpool/FREEBSD/ports
# zfs create -o compression=off -o exec=off -o setuid=off -o mountpoint=/usr/ports/distfiles rpool/FREEBSD/distfiles
# zfs create -o compression=off -o exec=off -o setuid=off -o mountpoint=/usr/ports/packages rpool/FREEBSD/packages
# zfs create -o compression=lzjb -o exec=off -o setuid=off -o mountpoint=/usr/src rpool/FREEBSD/src
# zfs create -o compression=lzjb -o exec=off -o setuid=off -o mountpoint=/var/crash rpool/FREEBSD/crash
# zfs create -o exec=off -o setuid=off -o mountpoint=/var/db rpool/FREEBSD/db
# zfs create -o compression=lzjb -o exec=on -o setuid=off -o mountpoint=/var/db/pkg rpool/FREEBSD/pkg
# zfs create -o exec=off -o setuid=off -o mountpoint=/var/empty rpool/FREEBSD/empty
# zfs create -o compression=lzjb -o exec=off -o setuid=off -o mountpoint=/var/log rpool/FREEBSD/log
# zfs create -o compression=gzip -o exec=off -o setuid=off -o mountpoint=/var/mail rpool/FREEBSD/mail
# zfs create -o exec=off -o setuid=off -o mountpoint=/var/run rpool/FREEBSD/run
# zfs create -o compression=lzjb -o exec=on -o setuid=off -o mountpoint=/var/tmp rpool/FREEBSD/tmp

Mount ZFS under /mnt through zpool export and zpool import for installation.

1
2
# zpool export rpool
# zpool import -o cachefile=/tmp/zpool.cache -R /mnt rpool

Now, your ZFS is mounted under /mnt, you can see there would be /mnt/usr, /mnt/var and /mnt/tmp directories. Please type below commands.

1
2
# chmod 1777 /mnt/tmp
# chmod 1777 /mnt/var/tmp

Let’s install FreeBSD 9.x into ZFS under /mnt. It is better to change your shell to sh. Please note the line start with “for”, since it is too long, so it’s better to use \ to split it.

1
2
3
4
# sh
# cd /usr/freebsd-dist
# export DESTDIR=/mnt
# for file in base.txz lib32.txz kernel.txz doc.txz games.txz ports.txz src.txz; \ do (cat $file | tar --unlink -xpJf - -C ${DESTDIR:-/}); done

Congratulations! You have installed FreeBSD base system into /mnt. Remember to do the following steps, or your system will fail to boot. Copy zpool.cache to your new system.

1
# cp /tmp/zpool.cache /mnt/boot/zfs/zpool.cache

At last, set /etc/rc.conf, /etc/fstab and /boot/loader.conf for your new system.

Set rc.conf

1
# echo 'zfs_enable="YES"' >> /mnt/etc/rc.conf

Set loader.conf

1
2
# echo 'zfs_load="YES"' >> /mnt/boot/loader.conf
# echo 'vfs.root.mountfrom="zfs:rpool/ROOT"' >> /mnt/boot/loader.conf

Set fstab (usually, this file should be empty, because all mountpoints and swap will be automatically mounted after you boot from ZFS)

1
# touch /mnt/etc/fstab

One last step! Set your ZFS mountpoints.

1
# zfs set readonly=on rpool/FREEBSD/empty

Then you can reboot, and you should be able to boot into your new FreeBSD-9.x based on ZFS. Now you can link /usr/home to /home.

1
# ln -sf /usr/home /home

Set timezone

1
# tzsetup

Set root password

1
# passwd

OK! Everything is done, enjoy your new FreeBSD system!