Riaan's SysAdmin Blog

My tips, howtos, gotchas, snippets and stuff. Use at your own risk!

UbuntuZFS

Ubuntu server 20.04 zfs root and OCI

admin

My experiment to:

  • create an Ubuntu 20.04 (not final release as of Feb 20) server in Virtualbox
  • setup server with a ZFS root disk
  • enable serial console
  • Virtualbox export to OCI

As you probably know newer desktop versions of Ubuntu will offer ZFS for root volume during installation. I am not sure if that is true in Ubuntu 20.04 server installs and when I looked at the 19.10 ZFS installation I did not necessarily want to use the ZFS layout they did. My experiment is my custom case and also tested on Ubuntu 16.04 and 18.04.

Note this is an experiment and ZFS layout, boot partition type, LUKS, EFI, multiple boot disks(mirrored) and netplan are all debatable configurations. Mine may not be ideal but it works for my use case.

Also my goal here was to export a bootable/usable OCI (Oracle Cloud Infrastructure) compute instance.

Start by booting a recent desktop live CD. Since I am testing 20.04 (focal) I used that. In the live cd environment open a terminal, sudo and apt install ssh. Start the ssh service and set ubuntu user password.

$ ssh ubuntu@192.168.1.142
$ sudo -i

##
apt-add-repository universe
apt update
apt install --yes debootstrap gdisk zfs-initramfs

## find correct device name for below
DISK=/dev/disk/by-id/ata-VBOX_HARDDISK_VB26c080f2-2bd16227
USER=ubuntu
HOST=server
POOL=ubuntu

##
sgdisk --zap-all $DISK
sgdisk --zap-all $DISK
sgdisk -a1 -n1:24K:+1000K -t1:EF02 $DISK
sgdisk     -n2:1M:+512M   -t2:EF00 $DISK
sgdisk     -n3:0:+1G      -t3:BF01 $DISK
sgdisk     -n4:0:0        -t4:BF01 $DISK
sgdisk --print $DISK

##
zpool create -o ashift=12 -d \
    -o feature@async_destroy=enabled \
    -o feature@bookmarks=enabled \
    -o feature@embedded_data=enabled \
    -o feature@empty_bpobj=enabled \
    -o feature@enabled_txg=enabled \
    -o feature@extensible_dataset=enabled \
    -o feature@filesystem_limits=enabled \
    -o feature@hole_birth=enabled \
    -o feature@large_blocks=enabled \
    -o feature@lz4_compress=enabled \
    -o feature@spacemap_histogram=enabled \
    -o feature@userobj_accounting=enabled \
    -O acltype=posixacl -O canmount=off -O compression=lz4 -O devices=off \
    -O normalization=formD -O relatime=on -O xattr=sa \
    -O mountpoint=/ -R /mnt bpool ${DISK}-part3

zpool create -o ashift=12 \
    -O acltype=posixacl -O canmount=off -O compression=lz4 \
    -O dnodesize=auto -O normalization=formD -O relatime=on -O xattr=sa \
    -O mountpoint=/ -R /mnt rpool ${DISK}-part4

zfs create -o canmount=off -o mountpoint=none rpool/ROOT
zfs create -o canmount=off -o mountpoint=none bpool/BOOT

zfs create -o canmount=noauto -o mountpoint=/ rpool/ROOT/ubuntu
zfs mount rpool/ROOT/ubuntu

zfs create -o canmount=noauto -o mountpoint=/boot bpool/BOOT/ubuntu
zfs mount bpool/BOOT/ubuntu

## Note: I skipped creating datasets for home, root, var/lib/ /var/log etc etc

##
debootstrap focal /mnt
zfs set devices=off rpool

## 
cat > /mnt/etc/netplan/01-netcfg.yaml<< EOF
network:
  version: 2
  ethernets:
    enp0s3:
      dhcp4: true
EOF

##
cat > /mnt/etc/apt/sources.list<< EOF
deb http://archive.ubuntu.com/ubuntu focal main universe
EOF

##
mount --rbind /dev  /mnt/dev
mount --rbind /proc /mnt/proc
mount --rbind /sys  /mnt/sys
chroot /mnt /usr/bin/env DISK=$DISK bash --login

##
locale-gen --purge en_US.UTF-8
update-locale LANG=en_US.UTF-8 LANGUAGE=en_US
dpkg-reconfigure --frontend noninteractive locales
echo US/Central > /etc/timezone    
dpkg-reconfigure -f noninteractive tzdata

##
passwd

##
apt install --yes --no-install-recommends linux-image-generic
apt install --yes zfs-initramfs
apt install --yes grub-pc
grub-probe /boot

##update-initramfs -u -k all  <- this does not work. try below 
KERNEL=`ls /usr/lib/modules/ | cut -d/ -f1 | sed 's/linux-image-//'`
update-initramfs -u -k $KERNEL

# edit /etc/default/grub
GRUB_DEFAULT=0
#GRUB_TIMEOUT_STYLE=hidden
GRUB_TIMEOUT=5
GRUB_CMDLINE_LINUX_DEFAULT=
GRUB_CMDLINE_LINUX=root=ZFS=rpool/ROOT/ubuntu console=tty1 console=ttyS0,115200
GRUB_TERMINAL=serial console
GRUB_SERIAL_COMMAND=serial --unit=0 --speed=115200

##
update-grub
grub-install $DISK

##
cat > /etc/systemd/system/zfs-import-bpool.service<< EOF
[Unit]
  DefaultDependencies=no
  Before=zfs-import-scan.service
  Before=zfs-import-cache.service

[Service]
  Type=oneshot
  RemainAfterExit=yes
  ExecStart=/sbin/zpool import -N -o cachefile=none bpool

[Install]
  WantedBy=zfs-import.target
EOF

systemctl enable zfs-import-bpool.service

##
zfs set mountpoint=legacy bpool/BOOT/ubuntu
echo bpool/BOOT/ubuntu /boot zfs \
    nodev,relatime,x-systemd.requires=zfs-import-bpool.service 0 0 >> /etc/fstab
zfs snapshot bpool/BOOT/ubuntu@install
zfs snapshot rpool/ROOT/ubuntu@install

##
systemctl enable serial-getty@ttyS0
apt install ssh
systemctl enable ssh

##
exit
##
mount | grep -v zfs | tac | awk '/\/mnt/ {print $3}' | xargs -i{} umount -lf {}
zpool export -a

** reboot

** detach live cd

NOTE: grub had a prompt on reboot but no options. try below

KERNEL=ls /usr/lib/modules/ | cut -d/ -f1 | sed 's/linux-image-//'
update-initramfs -u -k $KERNEL
update-grub

REF: https://github.com/zfsonlinux/zfs/wiki/Ubuntu-18.04-Root-on-ZFS

admin

Bio Info for Riaan