I have been playing around with some options to utilize Object Storage for backups. Since I am working on Oracle Cloud Infrastructure (OCI) I am doing my POC using the OCI Object Storage. OCI object storage does have Swift and S3 Compatibility API's to interface with. Of course if you want commercial backups many of them can use object storage as back-ends now so that would be the correct answer. If your needs does not warrant commercial backups solutions you can try several things. A few options I played with.<\/p>\n
1. Bareos server\/client with the object storage droplet. Not working reliably. Too experimental with droplet?
\n2. Rclone and using tar to pipe with rclone's rcat feature. This works well but is not a backup solution as in incrementals etc.
\n3. Duplicati. In my case using rclone as connection since S3 interface on OCI did not work.
\n4. Dupliciti. Could not get this one to work to S3 interface on OCI.
\n5. Restic. In my case using rclone as connection since S3 interface on OCI did not work.<\/p>\n
So far duplicati was not bad but had some bugs. It is beta software so probably should expect problems. Restic is doing a good job so far and I show a recipe of my POC below:<\/p>\n
Out of scope is setting up rclone, rclone.conf. Make sure you test that rclone is accessing your bucket first.<\/p>\n
Restic binary<\/p>\n
\r\n# wget https:\/\/github.com\/restic\/restic\/releases\/download\/v0.9.1\/restic_0.9.1_linux_amd64.bz2\r\n2018-08-03 10:25:10 (3.22 MB\/s) - \u2018restic_0.9.1_linux_amd64.bz2\u2019 saved [3786622\/3786622]\r\n# bunzip2 restic_0.9.1_linux_amd64.bz2 \r\n# mv restic_0.9.1_linux_amd64 \/usr\/local\/bin\/\r\n# chmod +x \/usr\/local\/bin\/restic_0.9.1_linux_amd64 \r\n# mv \/usr\/local\/bin\/restic_0.9.1_linux_amd64 \/usr\/local\/bin\/restic\r\n# \/usr\/local\/bin\/restic version\r\nrestic 0.9.1 compiled with go1.10.3 on linux\/amd64\r\n<\/pre>\nInitialize repo<\/p>\n
\r\n# rclone ls s3_servers_phoenix:oci02a\r\n# export RESTIC_PASSWORD="WRHYEjblahblah0VWq5qM"\r\n# \/usr\/local\/bin\/restic -r rclone:s3_servers_phoenix:oci02a init\r\ncreated restic repository 2bcf4f5864 at rclone:s3_servers_phoenix:oci02a\r\n\r\nPlease note that knowledge of your password is required to access\r\nthe repository. Losing your password means that your data is\r\nirrecoverably lost.\r\n\r\n# rclone ls s3_servers_phoenix:oci02a\r\n 155 config\r\n 458 keys\/530a67c4674b9abf6dcc9e7b75c6b319187cb8c3ed91e6db992a3e2cb862af63\r\n<\/pre>\nRun a backup<\/p>\n
\r\n# time \/usr\/local\/bin\/restic -r rclone:s3_servers_phoenix:oci02a backup \/opt\/applmgr\/12.2\r\nrepository 2bcf4f58 opened successfully, password is correct\r\n\r\nFiles: 1200934 new, 0 changed, 0 unmodified\r\nDirs: 2 new, 0 changed, 0 unmodified\r\nAdded: 37.334 GiB\r\n\r\nprocessed 1200934 files, 86.311 GiB in 1:31:40\r\nsnapshot af4d5598 saved\r\n\r\nreal\t91m40.824s\r\nuser\t23m4.072s\r\nsys\t7m23.715s\r\n\r\n# \/usr\/local\/bin\/restic -r rclone:s3_servers_phoenix:oci02a snapshots\r\nrepository 2bcf4f58 opened successfully, password is correct\r\nID Date Host Tags Directory\r\n----------------------------------------------------------------------\r\naf4d5598 2018-08-03 10:35:45 oci02a \/opt\/applmgr\/12.2\r\n----------------------------------------------------------------------\r\n1 snapshots\r\n<\/pre>\nRun second backup<\/p>\n
\r\n# \/usr\/local\/bin\/restic -r rclone:s3_servers_phoenix:oci02a backup \/opt\/applmgr\/12.2\r\nrepository 2bcf4f58 opened successfully, password is correct\r\n\r\nFiles: 0 new, 0 changed, 1200934 unmodified\r\nDirs: 0 new, 0 changed, 2 unmodified\r\nAdded: 0 B \r\n\r\nprocessed 1200934 files, 86.311 GiB in 47:46\r\nsnapshot a158688a saved\r\n<\/pre>\nExample cron entry<\/p>\n
\r\n# crontab -l\r\n05 * * * * \/usr\/local\/bin\/restic -r servers_phoenix:oci02a backup -q \/usr; \/usr\/local\/bin\/restic -r servers_phoenix:oci02a forget -q --prune --keep-hourly 2 --keep-daily 7\r\n<\/pre>\n","protected":false},"excerpt":{"rendered":"I have been playing around with some options to utilize Object Storage for backups. Since I am working on Oracle<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[3,87,96,99],"tags":[],"class_list":["post-1235","post","type-post","status-publish","format-standard","hentry","category-backups","category-oci","category-rclone","category-restic"],"_links":{"self":[{"href":"https:\/\/blog.ls-al.com\/wp-json\/wp\/v2\/posts\/1235","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/blog.ls-al.com\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/blog.ls-al.com\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/blog.ls-al.com\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/blog.ls-al.com\/wp-json\/wp\/v2\/comments?post=1235"}],"version-history":[{"count":0,"href":"https:\/\/blog.ls-al.com\/wp-json\/wp\/v2\/posts\/1235\/revisions"}],"wp:attachment":[{"href":"https:\/\/blog.ls-al.com\/wp-json\/wp\/v2\/media?parent=1235"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/blog.ls-al.com\/wp-json\/wp\/v2\/categories?post=1235"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/blog.ls-al.com\/wp-json\/wp\/v2\/tags?post=1235"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}