I want to talk about writing a Terraform provider, but instead of focusing on creating a resource, as in many articles on the Internet, I want to cover a data source.
There usually are two reasons to write a terraform provider:
- To wrap a remote API to manage remote infrastructure as code
- To create additional utility functions in Terraform
I will show you how to extend Terraform functionality and add an additional utility function using a custom provider.
This article will also give your a bare-bone setup to start with plugin development that you can use for your project. Sounds good? Let's dive into it.
Configure your dev environment
To start, you need to set up the Go environment. Depending on the OS you are using, instructions could be different. Take a look here to find instructions for your system. Examples in this article are for OSX because that's what I use.
brew install go
You, of course, will need to install Terraform. If you haven't tried tfenv, it is a straightforward way to manage different versions of Terraform on your local machine.
brew install tfenv
tfenv install 1.1.9
There are currently two standards for building Terraform plugins: the old one SDKv2 and the new one Plugin Framework. Though Plugin Framework is the future, it was still under technical review when I wrote this, so we will look into SDKv2 as it is a more stable way of creating a plugin currently.
Hashicorp has a scaffolding repo to help you start developing a plugin, and it makes sense to review it. But I would like us to start with an empty repo and add files as we go along. I think it will be easier to learn how to write a plugin this way.
First, we should choose the name for our provider and follow the pattern when the plugin name is always terraform-provider-<name>
. Even though we don't connect to any external provider's API, we still call it a Terraform provider.
It's essential to pick a name in the beginning because later on, this name will be used all over the code, and it will be a hassle to change it. The name will be required as a prefix to any resource or data source you will define, so it's better if the name is concise and has no special symbols. I will call our provider test
, and the project name is terramform-provider-test
.
Let's create a new folder and fill it out with the primary files we will need for this project.
mkdir terraform-provider-test
cd terraform-provider-test
touch main.go provider.go main.tf
That's the list of files we will start with. Typically you would follow folder structure as in the scaffolding repo,, placing Go files inside the internal/provider
folder and Terraform files inside the examples
folder. But let us first understand how it all works together before imposing this structure.
We will look into the content of the files in the next section. But before we do that, there is one last thing to set up. Terraform has a particular configuration in your user's Home folder called .terraformrc
. You probably don't have it if you never developed for Terraform. This file will help us create so-called developer overrides, which are needed to specify the path to the plugin object file which Terraform can load in.
touch ~/.terraformrc
Fill it out with the path to the project folder you have created.
provider_installation {
dev_overrides {
"test" = "/Users/~your~path~/terraform-provider-test/"
}
direct {}
}
Plugin entry point
First thing, let's initialise Go dependency tracking.
go mod init terraform-provider-test
And now, we are ready to create an entry point for our plugin, starting with the main.go
file.
package main
import (
"github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema"
"github.com/hashicorp/terraform-plugin-sdk/v2/plugin"
)
func main() {
plugin.Serve(&plugin.ServeOpts{
ProviderFunc: func() *schema.Provider {
return Provider()
},
})
}
In this file, we start the plugin and initialise a Provider object. Let's create that object in provider.go
.
package main
import (
"github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema"
)
func Provider() *schema.Provider {
return &schema.Provider{
Schema: nil,
DataSourcesMap: nil,
ProviderMetaSchema: map[string]*schema.Schema{},
TerraformVersion: "",
}
}
An empty Provider that doesn't do anything just to make the project build successfully. After that, install SDKv2 dependencies.
go get github.com/hashicorp/terraform-plugin-sdk/v2/plugin
go get github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema
And build the plugin.
go build -o terraform-provider-test
If you see a terraform-provider-test
executable in your folder, congratulations, you built your first Terraform plugin. Now we can try using it with some Terraform code.
Our Terraform code
We will now look at creating a simple data source that takes some information in and allows printing something out. It will be easier to write code when you know what kind of result you expect, so let's map out our idea in Terraform code before implementing it in Go. The following goes into main.tf
.
terraform {
required_providers {
test = {
version = "0"
source = "test"
}
}
}
data "test_msg" "my_msg" {
message = "My output message"
}
I thought of doing something very simple for starters. As you can see from the code, the data source takes in some text information in the message
field. It doesn't have to do anything with this message yet. We will look at how to load that plugin in and make terraform plan
pass on that example.
When you develop a plugin, you don't have to initialise your repository code with terraform init
, just use plan
.
tfenv use 1.1.9
terraform plan
It should fail with something like this.
╷
│ Error: Invalid data source
│
│ on main.tf line 10, in data "test_msg" "my_msg":
│ 10: data "test_msg" "my_msg" {
│
│ The provider hashicorp/test does not support the data source "test_msg".
This error indicates that Terraform could load our provider in memory and communicate with it, meaning all previous steps were correct. And we are finally ready to add the data source code.
Define the input values schema
Terraform plugin SDK helps you define and validate schema for the input values for resources you create. When you make a resource, you start by defining its schema. Let's open a new file called data_source_msg.go
and create our schema.
package main
import (
"context"
"github.com/hashicorp/terraform-plugin-log/tflog"
"github.com/hashicorp/terraform-plugin-sdk/v2/diag"
"github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema"
)
func dataSourceMsg() *schema.Resource {
return &schema.Resource{
Description: "Output a custom message",
ReadContext: dataSourceMsgRead,
Schema: map[string]*schema.Schema{
"message": {
Type: schema.TypeString,
Required: true,
},
},
}
}
func dataSourceMsgRead(ctx context.Context, d *schema.ResourceData, meta any) diag.Diagnostics {
tflog.Info(ctx, "test-msg data source has been loaded")
return nil
}
In this file, we define the data source schema, and the read function, which will be called by Terraform when it finds your data source in the infra configuration.
Now return to the provider.go
file and add DataSourcesMap
to point to the resource we have just defined.
DataSourcesMap: map[string]*schema.Resource{
"test_msg": dataSourceMsg(),
}
So our plugin exposes a new data source at this point, and we can test that everything works by building it and running terraform plan
.
go build -o terraform-provider-test
TF_LOG=INFO terraform plan
tflog is the standard way to debug your provider code, and it comes in handy here to print a little message once the provider is loaded. We should be able to see the message in the output log.
2022-10-02T20:06:55.212+0200 [INFO] provider.terraform-provider-test: configuring server automatic mTLS: timestamp=2022-10-02T20:06:55.211+0200
2022-10-02T20:06:55.250+0200 [INFO] provider.terraform-provider-test: test-msg data source has been loaded: tf_provider_addr=provider tf_req_id=dd639fe4-d927-04fc-0d17-cf1c89fe8f9e @caller=/terraform-provider-test/data_source_msg.go:27 @module=provider tf_rpc=ReadDataSource tf_data_source_type=test_msg timestamp=2022-10-02T20:06:55.250+0200
2022-10-02T20:06:55.252+0200 [INFO] backend/local: plan operation completed
No changes. Your infrastructure matches the configuration.
Conclusion
If everything worked out well for you in this article, you should have set up the basis for Terraform plugin development. In the next part, I will show you how to add more functionality to your plugin.
Top comments (0)