{"id":1191,"date":"2018-03-29T09:35:03","date_gmt":"2018-03-29T14:35:03","guid":{"rendered":"http:\/\/blog.ls-al.com\/?p=1191"},"modified":"2018-03-29T09:35:03","modified_gmt":"2018-03-29T14:35:03","slug":"rclone-and-oci-s3-interface","status":"publish","type":"post","link":"https:\/\/blog.ls-al.com\/rclone-and-oci-s3-interface\/","title":{"rendered":"Rclone and OCI S3 Interface"},"content":{"rendered":"<p>I am testing rclone to the Oracle Cloud Interface object storage and recording what worked for me.<\/p>\n<p>Note I could not get the swift interface to work with rclone, duplicity or swiftclient yet.  Although straightforward curl does work to the swift interface.<\/p>\n<p>rclone configuration generated with rclone config<\/p>\n<pre class=\"brush: bash; title: ; notranslate\" title=\"\">\r\n# cat \/root\/.config\/rclone\/rclone.conf\r\n&#x5B;s3_backups]\r\ntype = s3\r\nenv_auth = false\r\naccess_key_id = ocid1.credential.oc1..a&lt;redacted&gt;ta\r\nsecret_access_key = K&lt;redacted&gt;6s=\r\nregion = us-ashburn-1\r\nendpoint = &lt;tenancy&gt;.compat.objectstorage.us-ashburn-1.oraclecloud.com\r\nlocation_constraint = \r\nacl = private\r\nserver_side_encryption = \r\nstorage_class = \r\n<\/pre>\n<p>Issue with max-keys.  This problem although very difficult to find was also preventing copy\/sync of folders although a single file was working. rclone v1.36 was installed form Ubuntu repos and issue resolved with newer version.<\/p>\n<pre class=\"brush: bash; title: ; notranslate\" title=\"\">\r\n# rclone ls s3_backups:repo1\r\n2018\/03\/29 08:55:44 Failed to ls: InvalidArgument: The 'max-keys' parameter must be between 1 and 1000 (it was 1024) status code: 400, request id: fa704a55-44a8-1146-1b62-688df0366f63\r\n<\/pre>\n<p>Update and try again.<\/p>\n<pre class=\"brush: bash; title: ; notranslate\" title=\"\">\r\n# curl https:\/\/rclone.org\/install.sh | sudo bash\r\n&#x5B;..]\r\nrclone v1.40 has successfully installed.\r\n\r\n# rclone -V\r\nrclone v1.40\r\n- os\/arch: linux\/amd64\r\n- go version: go1.10\r\n\r\n# rclone ls s3_backups:repo1\r\n      655 config\r\n       38 hints.3\r\n\r\n# rclone copy \/root\/backup\/repo1 s3_backups:repo1\r\n\r\n# rclone sync \/root\/backup\/repo1 s3_backups:repo1\r\n\r\n# rclone ls s3_backups:repo1\r\n       26 README\r\n      655 config\r\n       38 hints.3\r\n    82138 index.3\r\n  5245384 data\/0\/1\r\n  3067202 data\/0\/3\r\n\r\n# rclone lsd s3_backups:\r\n          -1 2018-03-27 21:07:11        -1 backups\r\n          -1 2018-03-29 13:39:42        -1 repo1\r\n          -1 2018-03-26 22:23:35        -1 terraform\r\n          -1 2018-03-27 14:34:55        -1 terraform-src\r\n<\/pre>\n<p>References:<br \/>\nhttps:\/\/rclone.org\/docs\/<br \/>\nhttps:\/\/docs.us-phoenix-1.oraclecloud.com\/api\/#\/en\/s3objectstorage\/20160918\/<\/p>\n<blockquote class=\"wp-embedded-content\" data-secret=\"BkXMhU8lRZ\"><p><a href=\"https:\/\/blog.selectel.com\/rclone-rsync-cloud-storage\/\">Rclone: Rsync for Cloud Storage<\/a><\/p><\/blockquote>\n<p><iframe loading=\"lazy\" class=\"wp-embedded-content\" sandbox=\"allow-scripts\" security=\"restricted\" style=\"position: absolute; clip: rect(1px, 1px, 1px, 1px);\" title=\"&#8220;Rclone: Rsync for Cloud Storage&#8221; &#8212; Selectel Blog\" src=\"https:\/\/blog.selectel.com\/rclone-rsync-cloud-storage\/embed\/#?secret=BkXMhU8lRZ\" data-secret=\"BkXMhU8lRZ\" width=\"600\" height=\"338\" frameborder=\"0\" marginwidth=\"0\" marginheight=\"0\" scrolling=\"no\"><\/iframe><\/p>\n<p>In a future article I will add my testing around BorgBackup + rclone + OCI objectstorage from this interesting idea: https:\/\/opensource.com\/article\/17\/10\/backing-your-machines-borg  <\/p>\n","protected":false},"excerpt":{"rendered":"<p>I am testing rclone to the Oracle Cloud Interface object storage and recording what worked for me. Note I could<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[87,96],"tags":[],"class_list":["post-1191","post","type-post","status-publish","format-standard","hentry","category-oci","category-rclone"],"_links":{"self":[{"href":"https:\/\/blog.ls-al.com\/wp-json\/wp\/v2\/posts\/1191","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/blog.ls-al.com\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/blog.ls-al.com\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/blog.ls-al.com\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/blog.ls-al.com\/wp-json\/wp\/v2\/comments?post=1191"}],"version-history":[{"count":0,"href":"https:\/\/blog.ls-al.com\/wp-json\/wp\/v2\/posts\/1191\/revisions"}],"wp:attachment":[{"href":"https:\/\/blog.ls-al.com\/wp-json\/wp\/v2\/media?parent=1191"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/blog.ls-al.com\/wp-json\/wp\/v2\/categories?post=1191"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/blog.ls-al.com\/wp-json\/wp\/v2\/tags?post=1191"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}