Terraform Stateserver Using Go
Terraform can utilize a http backend for maintaining state. This is a test of a Terraform http backend using a server implemented with go.
recipe and components
• Using Virtualbox Ubuntu 20.10 and links listed below
• Run as http server
• Change to https
• For a future test terraform http backend locking feature and/or include locking in serverlinks
• https://www.terraform.io/docs/language/settings/backends/http.html
• https://linuxhint.com/install-golang-ubuntu/
• https://github.com/MerlinDMC/go-terraform-stateserver
• https://github.com/denji/golang-tlssetup go and test
# go version
go version go1.14.7 linux/amd64
$ pwd
/home/rrosso/tf-go-server$ cat main.go
package main
import "fmt"
func main() {
fmt.Println("Hello World");
}$ go run main.go
Hello World
$ go build main.go
$ ./main
Hello Worldrun server and test response
$ go run tf-stateserver.go -data_path=/home/rrosso/tf-go-server -listen_address=192.168.1.235:8080
$ curl -sL http://192.168.1.235:8080 | xxd
00000000: 3430 3420 7061 6765 206e 6f74 2066 6f75 404 page not fou
00000010: 6e64 0and.terraform init with http backend
➜ terraform-poc tail -8 main.tf
#}
## POC to test a go server
terraform {
backend "http" {
address = "http://192.168.1.235:8080/states/terraform.tfstate"
}
}
➜ terraform-poc terraform init
Initializing the backend...
Do you want to copy existing state to the new backend?
Pre-existing state was found while migrating the previous "local" backend to the
newly configured "http" backend. No existing state was found in the newly
configured "http" backend. Do you want to copy this state to the new "http"
backend? Enter "yes" to copy and "no" to start with an empty state.
Enter a value: yes
Successfully configured the backend "http"! Terraform will automatically
use this backend unless the backend configuration changes.
Initializing provider plugins...
The following providers do not have any version constraints in configuration,
so the latest version was installed.
To prevent automatic upgrades to new major versions that may contain breaking
changes, it is recommended to add version = "..." constraints to the
corresponding provider blocks in configuration, with the constraint strings
suggested below.
* provider.oci: version = "~> 4.17"
Terraform has been successfully initialized!
You may now begin working with Terraform. Try running "terraform plan" to see
any changes that are required for your infrastructure. All Terraform commands
should now work.
If you ever set or change modules or backend configuration for Terraform,
rerun this command to reinitialize your working directory. If you forget, other
commands will detect it and remind you to do so if necessary.observe server log messages
• file not found if no file exist during init is expected
• if file same and already there GET fine during init
• plan does a GET
• apply does a POST
• did not observe a DELETE$ go run tf-stateserver.go -data_path=/home/rrosso/tf-go-server -listen_address=192.168.1.235:8080
2021/03/18 09:57:41 received GET for state file: /home/rrosso/tf-go-server/states/terraform.tfstate
...
2021/03/18 09:58:33 received POST for state file: /home/rrosso/tf-go-server/states/terraform.tfstate
...
2021/03/18 10:00:50 received GET for state file: /home/rrosso/tf-go-server/states/terraform.tfstate
2021/03/18 10:00:50 during GET cannot open file: open /home/rrosso/tf-go-server/states/terraform.tfstate: no such file or directory
2021/03/18 10:00:50 received GET for state file: /home/rrosso/tf-go-server/states/terraform.tfstatechange to self signed cert for https test
NOTE:
• use sudo since linux instance is not running low ports like 443 for normal users
• need terraform skip_cert_verification for this self signed certificate
$ openssl genrsa -out server.key 2048
Generating RSA private key, 2048 bit long modulus (2 primes)
..........................+++++
.......................................................+++++
e is 65537 (0x010001)
$ openssl ecparam -genkey -name secp384r1 -out server.key
$ openssl req -new -x509 -sha256 -key server.key -out server.crt -days 3650
...
$ sudo go run tf-stateserver.go -certfile="server.crt" -keyfile="server.key" -data_path=./ -listen_address=192.168.1.235:443
$ curl -sL http://192.168.1.235:443 | xxd
00000000: 436c 6965 6e74 2073 656e 7420 616e 2048 Client sent an H
00000010: 5454 5020 7265 7175 6573 7420 746f 2061 TTP request to a
00000020: 6e20 4854 5450 5320 7365 7276 6572 2e0a n HTTPS server..
➜ terraform-poc tail -8 main.tf
#}
## POC to test a go server
terraform {
backend "http" {
address = "https://192.168.1.235/states/terraform.tfstate"
skip_cert_verification = true
}
}NOTE: terraform init, plan, apply works
additional links
• https://appdividend.com/2019/11/29/golang-json-example-how-to-use-json-with-go/
• https://www.golangprograms.com/web-application-to-read-and-write-json-data-in-json-file.html
• https://tutorialedge.net/golang/parsing-json-with-golang/
• https://dzone.com/articles/how-to-write-a-http-rest-api-server-in-go-in-minutsource
NOTE: subsequently checked source into github here: https://github.com/rrossouw01/terraform-stateserver-go/blob/main/server.go
Can use the file directly with simple wget to the raw link for example:
$ wget https://raw.githubusercontent.com/rrossouw01/terraform-stateserver-go/main/server.go$ cat tf-stateserver.go
package main
//https://github.com/MerlinDMC/go-terraform-stateserver/blob/master/main.go
import (
"flag"
"io"
"log"
"net/http"
"os"
"path/filepath"
)
var (
flagDataPath string
flagCertFile string
flagKeyFile string
flagListenAddress string
)
func init() {
flag.StringVar(&flagCertFile, "certfile", "", "path to the certs file for https")
flag.StringVar(&flagKeyFile, "keyfile", "", "path to the keyfile for https")
flag.StringVar(&flagDataPath, "data_path", os.TempDir(), "path to the data storage directory")
flag.StringVar(&flagListenAddress, "listen_address", "0.0.0.0:8080", "address:port to bind listener on")
//flag.StringVar(&flagListenAddress, "listen_address", "192.168.1.235:8080", "address:port to bind listener on")
}
func main() {
if !flag.Parsed() {
flag.Parse()
}
router := http.NewServeMux()
router.HandleFunc("/", requestHandler)
if flagCertFile != "" && flagKeyFile != "" {
log.Fatal(http.ListenAndServeTLS(flagListenAddress, flagCertFile, flagKeyFile, router))
} else {
log.Fatal(http.ListenAndServe(flagListenAddress, router))
}
}
func requestHandler(res http.ResponseWriter, req *http.Request) {
if req.URL.Path == "/" {
http.NotFound(res, req)
return
}
stateStorageFile := filepath.Join(flagDataPath, req.URL.Path)
stateStorageDir := filepath.Dir(stateStorageFile)
switch req.Method {
case "GET":
log.Printf("received GET for state file: %s\n", stateStorageFile)
fh, err := os.Open(stateStorageFile)
if err != nil {
log.Printf("during GET cannot open file: %s\n", err)
goto not_found
}
defer fh.Close()
res.WriteHeader(200)
io.Copy(res, fh)
return
case "POST":
log.Printf("received POST for state file: %s\n", stateStorageFile)
if err := os.MkdirAll(stateStorageDir, 0750); err != nil && !os.IsExist(err) {
log.Printf("during POST cannot create parent directories: %s\n", err)
goto not_found
}
fh, err := os.OpenFile(stateStorageFile, os.O_RDWR|os.O_CREATE|os.O_TRUNC, 0666)
if err != nil {
log.Printf("cannot open file: %s\n", err)
goto not_found
}
defer fh.Close()
if _, err := io.Copy(fh, req.Body); err != nil {
log.Printf("failed to stream data into statefile: %s\n", err)
goto not_found
}
res.WriteHeader(200)
return
case "DELETE":
log.Printf("received DELETE for state file: %s\n", stateStorageFile)
//if os.RemoveAll(stateStorageFile) != nil {
if err := os.RemoveAll(stateStorageFile); err != nil {
log.Printf("cannot remove file: %s\n", err)
goto not_found
}
res.WriteHeader(200)
return
}
not_found:
http.NotFound(res, req)
}