I have put my eyes on ZoL (ZFS on Linux) project for almost one year. At the beginning, I feel excited because now we can use ZFS as Linux rootfs, but now, I do not think it is reliable.
Compared with other operating systems which can also use ZFS as rootfs (such as FreeBSD, OpenIndiana, Nexenta and so on), ZoL project seems to fall behind on both stability and porting works. Typically, when you use ZFS as your Linux rootfs, a simple kernel upgrade would make your whole system not bootable. Moreover, you cannot import your zpool because ZoL does not support your new kernel yet. Linux kernel is under fast developing and releasing, ZoL cannot always catch up the Linux kernel. Sometimes, there are even API changes in new kernel versions, which make things worse. Currently, there are only a few LiveCDs which contains ZFS kernel module. So, if your Linux failed to boot, you cannot recover it soon.
Such cases will not happen with FreeBSD or OpenIndiana, not only because they do not often release new versions, but also because they have ZFS module built in kernel. Let’s take FreeBSD as an example, after you upgrade your zpool (new zpool release usually comes with new FreeBSD release versions), what you need to do is only re-write the bootcode. Then you can just reboot your system and boot into upgraded zpool.
So, if you want to use ZFS on your Linux, do NOT put your rootfs on ZFS! It is better if you put your non-system data on ZFS.
Personally, I like ZFS very much, and I’m glad ZoL project could port ZFS to Linux. However, as a user, I feel that I am taking a large risk. Although many people in #zfsonlinux have put their production environment on ZFS, there is always someone say ZoL would cause data loss, or their HDD cannot be recognized.
In a word, if you are running Linux only, and you do not have extra backup, take care when you use ZFS as Linux system root. However, having a separate /boot is never a bad idea, and if you already use ZFS as your rootfs, then do NOT upgrade your kernel often.