Terraform can utilize a http backend for maintaining state. This is a test of a Terraform http backend using a server implemented with python.<\/p>\n
NOTE: checked source into https:\/\/github.com\/rrossouw01\/terraform-stateserver-py\/<\/a><\/p>\n Using Virtualbox Ubuntu 20.10 and followed links:<\/p>\n Terraform can utilize a http backend for maintaining state. This is a test of a Terraform http backend using a<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[13,88],"tags":[],"class_list":["post-1722","post","type-post","status-publish","format-standard","hentry","category-python","category-terraform"],"_links":{"self":[{"href":"https:\/\/blog.ls-al.com\/wp-json\/wp\/v2\/posts\/1722","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/blog.ls-al.com\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/blog.ls-al.com\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/blog.ls-al.com\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/blog.ls-al.com\/wp-json\/wp\/v2\/comments?post=1722"}],"version-history":[{"count":0,"href":"https:\/\/blog.ls-al.com\/wp-json\/wp\/v2\/posts\/1722\/revisions"}],"wp:attachment":[{"href":"https:\/\/blog.ls-al.com\/wp-json\/wp\/v2\/media?parent=1722"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/blog.ls-al.com\/wp-json\/wp\/v2\/categories?post=1722"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/blog.ls-al.com\/wp-json\/wp\/v2\/tags?post=1722"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}recipe and components<\/h2>\n
\n
setup<\/h2>\n
$ mkdir tf-state-server\n$ cd tf-state-server\n\n$ virtualenv -p python3 venv\ncreated virtual environment CPython3.8.6.final.0-64 in 174ms\n creator CPython3Posix(dest=\/home\/rrosso\/tf-state-server\/venv, clear=False, global=False)\n seeder FromAppData(download=False, pip=bundle, setuptools=bundle, wheel=bundle, via=copy, app_data_dir=\/home\/rrosso\/.local\/share\/virtualenv)\n added seed packages: pip==20.1.1, pkg_resources==0.0.0, setuptools==44.0.0, wheel==0.34.2\n activators BashActivator,CShellActivator,FishActivator,PowerShellActivator,PythonActivator,XonshActivator\n\n$ source venv\/bin\/activate\n\n(venv) $ pip install -U -r requirements.txt\nCollecting flask\n Using cached Flask-1.1.2-py2.py3-none-any.whl (94 kB)\nCollecting flask_restful\n Downloading Flask_RESTful-0.3.8-py2.py3-none-any.whl (25 kB)\nCollecting itsdangerous>=0.24\n Using cached itsdangerous-1.1.0-py2.py3-none-any.whl (16 kB)\nCollecting Werkzeug>=0.15\n Using cached Werkzeug-1.0.1-py2.py3-none-any.whl (298 kB)\nCollecting Jinja2>=2.10.1\n Using cached Jinja2-2.11.3-py2.py3-none-any.whl (125 kB)\nCollecting click>=5.1\n Using cached click-7.1.2-py2.py3-none-any.whl (82 kB)\nCollecting pytz\n Downloading pytz-2021.1-py2.py3-none-any.whl (510 kB)\n |\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588| 510 kB 3.0 MB\/s \nCollecting six>=1.3.0\n Using cached six-1.15.0-py2.py3-none-any.whl (10 kB)\nCollecting aniso8601>=0.82\n Downloading aniso8601-9.0.1-py2.py3-none-any.whl (52 kB)\n |\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588| 52 kB 524 kB\/s \nCollecting MarkupSafe>=0.23\n Using cached MarkupSafe-1.1.1-cp38-cp38-manylinux2010_x86_64.whl (32 kB)\nInstalling collected packages: itsdangerous, Werkzeug, MarkupSafe, Jinja2, click, flask, pytz, six, aniso8601, flask-restful\nSuccessfully installed Jinja2-2.11.3 MarkupSafe-1.1.1 Werkzeug-1.0.1 aniso8601-9.0.1 click-7.1.2 flask-1.1.2 flask-restful-0.3.8 itsdangerous-1.1.0 pytz-2021.1 six-1.15.0\n\n(venv) $ python3 stateserver.py\n * Serving Flask app \"stateserver\" (lazy loading)\n * Environment: production\n WARNING: This is a development server. Do not use it in a production deployment.\n Use a production WSGI server instead.\n * Debug mode: off\n * Running on http:\/\/192.168.1.235:5000\/ (Press CTRL+C to quit)\n...<\/code><\/pre>\nterraform point to remote http<\/h2>\n
\u279c cat main.tf \nterraform {\n backend \"http\" {\n address = \"http:\/\/192.168.1.235:5000\/terraform_state\/4cdd0c76-d78b-11e9-9bea-db9cd8374f3a\"\n lock_address = \"http:\/\/192.168.1.235:5000\/terraform_lock\/4cdd0c76-d78b-11e9-9bea-db9cd8374f3a\"\n lock_method = \"PUT\"\n unlock_address = \"http:\/\/192.168.1.235:5000\/terraform_lock\/4cdd0c76-d78b-11e9-9bea-db9cd8374f3a\"\n unlock_method = \"DELETE\"\n }\n}\n\n\u279c source ..\/env-vars \n\n\u279c terraform init \n\nInitializing the backend...\nDo you want to copy existing state to the new backend?\n Pre-existing state was found while migrating the previous \"local\" backend to the\n newly configured \"http\" backend. No existing state was found in the newly\n configured \"http\" backend. Do you want to copy this state to the new \"http\"\n backend? Enter \"yes\" to copy and \"no\" to start with an empty state.\n\n Enter a value: yes\n\nSuccessfully configured the backend \"http\"! Terraform will automatically\nuse this backend unless the backend configuration changes.\n\nInitializing provider plugins...\n\nThe following providers do not have any version constraints in configuration,\nso the latest version was installed.\n\nTo prevent automatic upgrades to new major versions that may contain breaking\nchanges, it is recommended to add version = \"...\" constraints to the\ncorresponding provider blocks in configuration, with the constraint strings\nsuggested below.\n\n* provider.oci: version = \"~> 4.17\"\n\nTerraform has been successfully initialized!\n\nYou may now begin working with Terraform. Try running \"terraform plan\" to see\nany changes that are required for your infrastructure. All Terraform commands\nshould now work.\n\nIf you ever set or change modules or backend configuration for Terraform,\nrerun this command to reinitialize your working directory. If you forget, other\ncommands will detect it and remind you to do so if necessary.\n\nserver shows\n---\n````bash\n...\n192.168.1.111 - - [16\/Mar\/2021 10:50:00] \"POST \/terraform_state\/4cdd0c76-d78b-11e9-9bea-db9cd8374f3a?ID=84916e49-1b44-1b32-2058-62f28e1e8ee7 HTTP\/1.1\" 200 -\n192.168.1.111 - - [16\/Mar\/2021 10:50:00] \"DELETE \/terraform_lock\/4cdd0c76-d78b-11e9-9bea-db9cd8374f3a HTTP\/1.1\" 200 -\n192.168.1.111 - - [16\/Mar\/2021 10:50:00] \"GET \/terraform_state\/4cdd0c76-d78b-11e9-9bea-db9cd8374f3a HTTP\/1.1\" 200 -\n...\n\n$ ls -la .stateserver\/\ntotal 24\ndrwxrwxr-x 2 rrosso rrosso 4096 Mar 16 10:50 .\ndrwxrwxr-x 4 rrosso rrosso 4096 Mar 16 10:43 ..\n-rw-rw-r-- 1 rrosso rrosso 4407 Mar 16 10:50 4cdd0c76-d78b-11e9-9bea-db9cd8374f3a\n-rw-rw-r-- 1 rrosso rrosso 5420 Mar 16 10:50 4cdd0c76-d78b-11e9-9bea-db9cd8374f3a.log\n\n$ more .stateserver\/4cdd0c76-d78b-11e9-9bea-db9cd8374f3a*\n::::::::::::::\n.stateserver\/4cdd0c76-d78b-11e9-9bea-db9cd8374f3a\n::::::::::::::\n{\n \"lineage\": \"9b756fb7-e41a-7cd6-d195-d794f377e7be\",\n \"outputs\": {},\n \"resources\": [\n{\n\"instances\": [\n{\n\"attributes\": {\n\"compartment_id\": null,\n\"id\": \"ObjectStorageNamespaceDataSource-0\",\n\"namespace\": \"axwscg6apasa\"\n},\n\"schema_version\": 0\n}\n],\n...\n \"serial\": 0,\n \"terraform_version\": \"0.12.28\",\n \"version\": 4\n}\n::::::::::::::\n.stateserver\/4cdd0c76-d78b-11e9-9bea-db9cd8374f3a.log\n::::::::::::::\nlock: {\n \"Created\": \"2021-03-16T15:49:34.567178267Z\",\n \"ID\": \"7eea0d70-5f53-e475-041b-bcc393f4a92d\",\n \"Info\": \"\",\n \"Operation\": \"migration destination state\",\n \"Path\": \"\",\n \"Version\": \"0.12.28\",\n \"Who\": \"rrosso@desktop01\"\n}\nunlock: {\n \"Created\": \"2021-03-16T15:49:34.567178267Z\",\n \"ID\": \"7eea0d70-5f53-e475-041b-bcc393f4a92d\",\n \"Info\": \"\",\n \"Operation\": \"migration destination state\",\n \"Path\": \"\",\n \"Version\": \"0.12.28\",\n \"Who\": \"rrosso@desktop01\"\n}\nlock: {\n \"Created\": \"2021-03-16T15:49:45.760917508Z\",\n \"ID\": \"84916e49-1b44-1b32-2058-62f28e1e8ee7\",\n \"Info\": \"\",\n \"Operation\": \"migration destination state\",\n \"Path\": \"\",\n \"Version\": \"0.12.28\",\n \"Who\": \"rrosso@desktop01\"\n}\nstate_write: {\n \"lineage\": \"9b756fb7-e41a-7cd6-d195-d794f377e7be\",\n \"outputs\": {},\n \"resources\": [\n{\n\"instances\": [\n{\n\"attributes\": {\n\"compartment_id\": null,\n\"id\": \"ObjectStorageNamespaceDataSource-0\",\n\"namespace\": \"axwscg6apasa\"\n},\n\"schema_version\": 0\n}\n...<\/code><\/pre>\nstateserver shows during plan<\/h2>\n
...\n192.168.1.111 - - [16\/Mar\/2021 10:54:12] \"PUT \/terraform_lock\/4cdd0c76-d78b-11e9-9bea-db9cd8374f3a HTTP\/1.1\" 200 -\n192.168.1.111 - - [16\/Mar\/2021 10:54:12] \"GET \/terraform_state\/4cdd0c76-d78b-11e9-9bea-db9cd8374f3a HTTP\/1.1\" 200 -\n192.168.1.111 - - [16\/Mar\/2021 10:54:15] \"DELETE \/terraform_lock\/4cdd0c76-d78b-11e9-9bea-db9cd8374f3a HTTP\/1.1\" 200 -\n...<\/code><\/pre>\nsource<\/h2>\n
$ more requirements.txt stateserver.py \n::::::::::::::\nrequirements.txt\n::::::::::::::\nflask\nflask_restful\n::::::::::::::\nstateserver.py\n::::::::::::::\n#!\/usr\/bin\/python3\n\nimport flask\nimport flask_restful\nimport json\nimport logging\nimport os\n\napp = flask.Flask(__name__)\napi = flask_restful.Api(app)\n\n@app.before_request\ndef log_request_info():\n headers = []\n for header in flask.request.headers:\n headers.append('%s = %s' % (header[0], header[1]))\n\n body = flask.request.get_data().decode('utf-8').split('\\n')\n\n app.logger.debug(('%(method)s for %(url)s...\\n'\n ' Header -- %(headers)s\\n'\n ' Body -- %(body)s\\n')\n % {\n 'method': flask.request.method,\n 'url': flask.request.url,\n 'headers': '\\n Header -- '.join(headers),\n 'body': '\\n '.join(body)\n })\n\nclass Root(flask_restful.Resource):\n def get(self):\n resp = flask.Response(\n 'Oh, hello<\/body><\/html>',\n mimetype='text\/html')\n resp.status_code = 200\n return resp\n\nclass StateStore(object):\n def __init__(self, path):\n self.path = path\n os.makedirs(self.path, exist_ok=True)\n\n def _log(self, id, op, data):\n log_file = os.path.join(self.path, id) + '.log'\n with open(log_file, 'a') as f:\n f.write('%s: %s\\n' %(op, data))\n\n def get(self, id):\n file = os.path.join(self.path, id)\n if os.path.exists(file):\n with open(file) as f:\n d = f.read()\n self._log(id, 'state_read', {})\n return json.loads(d)\n return None\n\n def put(self, id, info):\n file = os.path.join(self.path, id)\n data = json.dumps(info, indent=4, sort_keys=True)\n with open(file, 'w') as f:\n f.write(data)\n self._log(id, 'state_write', data)\n\n def lock(self, id, info):\n # NOTE(mikal): this is racy, but just a demo\n lock_file = os.path.join(self.path, id) + '.lock'\n if os.path.exists(lock_file):\n # If the lock exists, it should be a JSON dump of information about\n # the lock holder\n with open(lock_file) as f:\n l = json.loads(f.read())\n return False, l\n\n data = json.dumps(info, indent=4, sort_keys=True)\n with open(lock_file, 'w') as f:\n f.write(data)\n self._log(id, 'lock', data)\n return True, {}\n\n def unlock(self, id, info):\n lock_file = os.path.join(self.path, id) + '.lock'\n if os.path.exists(lock_file):\n os.unlink(lock_file)\n self._log(id, 'unlock', json.dumps(info, indent=4, sort_keys=True))\n return True\n return False\n\nstate = StateStore('.stateserver')\n\nclass TerraformState(flask_restful.Resource):\n def get(self, tf_id):\n s = state.get(tf_id)\n if not s:\n flask.abort(404)\n return s\n\n def post(self, tf_id):\n print(flask.request.form)\n s = state.put(tf_id, flask.request.json)\n return {}\n\nclass TerraformLock(flask_restful.Resource):\n def put(self, tf_id):\n success, info = state.lock(tf_id, flask.request.json)\n if not success:\n flask.abort(423, info)\n return info\n\n def delete(self, tf_id):\n if not state.unlock(tf_id, flask.request.json):\n flask.abort(404)\n return {}\n\napi.add_resource(Root, '\/')\napi.add_resource(TerraformState, '\/terraform_state\/