My test to recover an Ubuntu server OS from a backup.
Note the following:
- I used Ubuntu 20.04 (focal) which is still beta at the time of this POC. In theory Ubuntu 18.04 should work the same or better.
- For an OS recovery I documented the backup elsewhere. It was something like this for me and yours will vary of course:
restic –exclude={/dev/*,/media,/mnt/*,/proc/*,/run/*,/sys/*,/tmp/*,/var/tmp/*,/swapfile} backup / /dev/{console,null} - For the partition recovery I saved it on the source server to a file for easy copy/paste during the recovery: sfdisk -d > /tmp/partition-table
- I tested restic repo’s with both sftp and AWS s3.
- Used a Virtualbox VM named u20.04-restic-os-restored. Made the recover server disk 15G (5G larger than the original 10G where backup was done)
- My POC consist of a very simple disk layout ie one ext4 partition only. It was just the default install from this Ubuntu 20.04 desktop install. Complicated boot disk layouts may be very different. I am not interested in recovering servers with complicated OS disk layouts. To me that does not fit with modern infrastructure and concepts like auto scaling. Boot disks should be lean and easily recovered/provisioned through scripting; and configuration applied with configuration management tools.
- boot liveCD, set ubuntu user password and install and start ssh so we can ssh into and easier to copy/paste etc.
- Abbreviated commands (removed most output to shorten)
$ ssh ubuntu@192.168.1.160 $ sudo -i # export AWS_ACCESS_KEY_ID=<secret..> # export AWS_SECRET_ACCESS_KEY=<secret..> # export RESTIC_PASSWORD=<secret..> # export RESTIC_REPOSITORY="sftp:rr@192.168.1.111:/ARCHIVE/restic-os-restore-poc" # cd /usr/local/bin/ # wget https://github.com/restic/restic/releases/download/v0.9.6/restic_0.9.6_linux_amd64.bz2 # bzip2 -d restic_0.9.6_linux_amd64.bz2 # mv restic_0.9.6_linux_amd64 restic # chmod +x restic # mkdir /mnt/restore # sfdisk -d /dev/sda < partition-table # mkfs.ext4 /dev/sda1 # mkdir /mnt/restore/ # mount /dev/sda1 /mnt/restore/ # /usr/local/bin/restic snapshots # time /usr/local/bin/restic restore latest -t /mnt/restore --exclude '/etc/fstab' --exclude '/etc/crypttab' --exclude '/boot/grub/grub.cfg' --exclude '/etc/default/grub' # mount --bind /dev /mnt/restore/dev # mount -t sysfs sys /mnt/restore/sys # mount -t proc proc /mnt/restore/proc # mount -t devpts devpts /mnt/restore/dev/pts # mount -t tmpfs tmp /mnt/restore/tmp # mount --rbind /run /mnt/restore/run # mount -t tmpfs tmp /mnt/restore/tmp # chroot /mnt/restore /bin/bash # lsblk | grep sda # grub-install /dev/sda # update-grub # blkid | grep sda # UUID=$(blkid | grep sda | cut -d' ' -f2 | cut -d\" -f2) # echo "$UUID / ext4 errors=remount-ro 0 1" > /etc/fstab # sync # exit # init 0
Note:
New server booted and worked but graphics (GNOME login) login for ubuntu account stalled on login. This fixed it: dconf reset -f /org/gnome/
My restic backup command works but just for reference since restic has no include flag rsync seem to have a better exclude/include functionality syntax like this: rsync –include=/dev/{console,null} –exclude={/dev/,/proc/,/sys/,/tmp/,/run/,/mnt/,/media/*,/lost+found}