DEV Community

loading...
Cover image for How to allow dynamic Terraform Provider Configuration
Camptocamp Infrastructure Solutions

How to allow dynamic Terraform Provider Configuration

Raphaël Pinson
Infrastructure Developer & Trainer ⚙ | DevOps ∞ | Family Historian 🌳 | Puppet Champion 🦊 | Terraform 🌍 | Docker 🐳 | Kubernertes ☸ | 🇫🇷 living in 🇨🇭
・4 min read

Terraform relies heavily on the concept of providers, a base brick which consists of Go plugins enabling the communication with an API.

Each provider gives access to one or more resource types, and these resources then manage objects on the target API.

Most of the time, a provider's configuration is static, e.g.

provider "aws" {
  region = "us-east-1"
}
Enter fullscreen mode Exit fullscreen mode

However, in some cases, it is useful to configure a provider dynamically, using the attribute values from other resources as input for the provider's configuration.

I'll use the example of the Argo CD provider. In a single Terraform run, we would like to:

  • install a Kubernetes cluster (using a DevOps Stack K3s Terraform module)
  • install Argo CD on the the cluster using the Helm provider
  • instantiate Argo CD resources (projects, applications, etc.) on this new Argo CD server.

Our code will look like this:

# Install Kubernetes & Argo CD using a local module
# (from https://devops-stack.io)
module "cluster" {
  source = "git::https://github.com/camptocamp/devops-stack.git//modules/k3s/docker?ref=master"

  cluster_name = "default"
  node_count   = 1
}

# /!\ Setup the Argo CD provider dynamically
# based on the cluster module's output
provider "argocd" {
  server_addr = module.cluster.argocd_server
  auth_token  = module.cluster.argocd_auth_token
  insecure    = true
  grpc_web    = true
}

# Deploy an Argo CD resource using the provider
resource "argocd_project" "demo_app" {
  metadata {
    name      = "demo-app"
    namespace = "argocd"
  }

  spec {
    description  = "Demo application project"
    source_repos = ["*"]

    destination {
      server    = "https://kubernetes.default.svc"
      namespace = "default"
    }

    orphaned_resources {
      warn = true
    }
  }

  depends_on = [ module.cluster ]
}
Enter fullscreen mode Exit fullscreen mode

This requires to configure Argo CD dynamically, using the output of the Kubernetes cluster's resources.

Provider Initialization

Providers are initialized early in a Terraform run, as their initialization is required to compute the graph which defines in which order the resources are applied.

This means it is actually not possible to make a provider initialize after a secondary resource is created.

Officially, the story stops here, and Terraform has a bug report to track the feature allowing to dynamically configure providers.

So… it's game over then? 🎮 👾
Not really!

Leveraging Pointers

When a provider is configured in Terraform, it triggers a configuration function:

func Provider() *schema.Provider {
    return &schema.Provider{
      ConfigureFunc: func(d *schema.ResourceData) (interface{}, error) {
        // Create someObject
        return someObject, nil
      }
    }
}
Enter fullscreen mode Exit fullscreen mode

This ConfigureFunc method is usually used to create a static client for the target API. In the Argo CD provider for example, it returns a ServerInterface structure, with pointers to several clients, instantiated from the provider parameters:

type ServerInterface struct {                                                   
    ApiClient            *apiclient.Client                                      
    ApplicationClient    *application.ApplicationServiceClient                  
    ClusterClient        *cluster.ClusterServiceClient                          
    ProjectClient        *project.ProjectServiceClient                          
    RepositoryClient     *repository.RepositoryServiceClient                    
    RepoCredsClient      *repocreds.RepoCredsServiceClient                      
    ServerVersion        *semver.Version                                        
    ServerVersionMessage *version.VersionMessage                                                                                                              
}
Enter fullscreen mode Exit fullscreen mode

The return statement from the ConfigureFunc eventually looks like this:

return ServerInterface{                                                             
    &apiClient,                                                                     
    &applicationClient,                                                             
    &clusterClient,                                                                 
    &projectClient,                                                                 
    &repositoryClient,                                                              
    &repoCredsClient,                                                               
    serverVersion,                                                                  
    serverVersionMessage}, err
Enter fullscreen mode Exit fullscreen mode

Let's add a new field to the ServerInterface to store the pointer to the provider's ResourceData object, which gives access to the provider's parameters:

type ServerInterface struct {                                                       
    ApiClient            *apiclient.Client                                          
    ApplicationClient    *application.ApplicationServiceClient                      
    ClusterClient        *cluster.ClusterServiceClient                              
    ProjectClient        *project.ProjectServiceClient                              
    RepositoryClient     *repository.RepositoryServiceClient                        
    RepoCredsClient      *repocreds.RepoCredsServiceClient                          
    ServerVersion        *semver.Version                                            
    ServerVersionMessage *version.VersionMessage                                    
    ProviderData         *schema.ResourceData                                       
}
Enter fullscreen mode Exit fullscreen mode

Now in the ConfigureFunc, we'll instantiate the ServerInterface, providing only the ProviderData pointer. The first resource that needs to use the provider will then instantiate the clients, when the provider parameters are available. We'll need the ConfigureFunc method to return a pointer to a ServerInterface, so we can later cache the clients and avoid recreating them for every resource:

ConfigureFunc: func(d *schema.ResourceData) (interface{}, error) {                  
    server := ServerInterface{ProviderData: d}                                      
    return &server, nil                                                             
},
Enter fullscreen mode Exit fullscreen mode

Initialize the Clients

Now we need to actually initialize the clients in each resource.

Each resource method gets the interface returned by the ConfigureFunc function as an empty interface parameter, usually called meta:

func resourceArgoCDProjectCreate(ctx context.Context, d *schema.ResourceData, meta interface{}) diag.Diagnostics {
Enter fullscreen mode Exit fullscreen mode

These methods currently simply cast the meta parameter as a ServerInterface structure and use the pre-initialized clients:

server := meta.(ServerInterface)
Enter fullscreen mode Exit fullscreen mode

We now need to cast meta as a pointer to a ServerInterface structure instead (since we'll need to modify the clients from within the resources), and initialize the clients:

server := meta.(*ServerInterface)                                                   
if err := server.initClients(); err != nil {                                        
    return []diag.Diagnostic{                                                       
        diag.Diagnostic{                                                            
            Severity: diag.Error,                                                   
            Summary:  fmt.Sprintf("Failed to init clients"),                        
            Detail:   err.Error(),                                                  
        },                                                                          
    }                                                                           
}
Enter fullscreen mode Exit fullscreen mode

The initClients() method of the ServerInterface structure will be called, allowing to set up the clients using the current provider parameters.

Client Pool Caching

In the ServerInterface#initClients() method, we want to make sure we reuse existing clients. This is rather simple, since each client is stored as a pointer in the structure, so it defaults to nil:

func (p *ServerInterface) initClients() error {                                 
    d := p.ProviderData                                                         

    if p.ApiClient == nil {                                                     
        apiClient, err := initApiClient(d)                                      
        if err != nil {                                                         
            return err                                                          
        }                                                                       
        p.ApiClient = &apiClient                                                
    }

    // etc for all clients

    return nil
}
Enter fullscreen mode Exit fullscreen mode

Conclusion

That's it, we're done. With these modifications, terraform plan now works. The resources get applied in the proper order, and the outputs from the cluster module get properly passed as configuration to the Argo CD clients.

Discussion (0)