The Case for Terraform Modules: Why Not Just Use Providers Directly?

When it comes to large-scale Infrastructure as Code (IaC) for complex cloud environments, managing the sheer number of resources is not the only challenge. The real differentiator lies in provisioning infrastructure according to strict governance, security, and availability requirements without introducing unnecessary complexity or increasing the risk of human error during the design and implementation process.

In large-scale environments, we encounter a set of pre-established definitions that encompass everything from corporate standards to security practices and cost management. These guidelines, though essential, often make the process of writing infrastructure code more challenging, demanding speed, efficiency, and compliance.

Developing modules in Terraform is a key strategy to manage this complexity. Modules not only help create consistent patterns but also enable engineers to describe and provision the required infrastructure without worrying about all the compliance and security details. Rather than limiting creativity and agility, modules provide a solid foundation that ensures best practices are followed.

Security awareness is crucial in cloud development, but it should not be an obstacle. On the contrary, it should be a core part of the organizational culture, naturally integrated into the design and implementation process of cloud solutions.

To address the main question, let's start with the basics: What is Terraform?

"Terraform is an infrastructure as code tool developed by HashiCorp that allows you to define both cloud and on-premises resources in human-readable configuration files that can be versioned, reused, and shared. You can then use a consistent workflow to provision and manage your entire infrastructure throughout its lifecycle. Terraform can manage low-level components like compute, storage, and networking resources, as well as high-level components like DNS entries and SaaS features." — HashiCorp.

What is a provider in Terraform?

A provider in Terraform is a plugin that allows Terraform to interact with various platforms, services, or APIs. Providers are a crucial component in Terraform's architecture, enabling the tool to manage and configure resources across different systems.

Key Points about Terraform Providers:

• Platform Interaction: Providers are responsible for translating Terraform's declarative code into API calls that integrate with an infrastructure platform. This platform can be a cloud provider like Azure or AWS, a Software as a Service (SaaS) like GitHub or Datadog, or even on-premises systems.
• Resource Management: Each provider defines a set of resources it can manage. These resources are declared in Terraform configuration files.
• State Management: Providers work with Terraform to manage the state of the infrastructure, allowing Terraform to understand what changes are necessary during a terraform apply.
• Installation and Versioning: Providers are distributed as plugins, and Terraform automatically downloads the required providers when you initialize your configuration (terraform init). Each provider has its own versioning, and you can specify the version you need in your configuration.
• Extensibility: If a platform or service doesn't have an official provider, you can develop a custom provider using the Terraform SDK.

Example:

If you want to manage resources in AWS using Terraform, you need to use the AWS provider. In your configuration file, you would declare your provider like this:

provider "aws" {
  region = "us-west-2"
}

resource "aws_instance" "example" {
  ami           = "ami-123456"
  instance_type = "t2.micro"
}

In this example:

• The aws provider allows Terraform to integrate with AWS.
• The aws_instance resource is defined by the AWS provider and represents an EC2 instance in AWS.

Providers are essential to Terraform's operation because they abstract the complexity of interacting with different platforms, making infrastructure as code easily manageable across multiple environments.

What is a module in Terraform?

A module in Terraform is a container for one or more resources that are used together. It is a fundamental unit of code organization in Terraform, serving to group related resources and make configurations more modular, reusable, and easier to manage.

Essentially, a Terraform module is a set of .tf files in a directory. Every configuration file (.tf) in a directory is implicitly considered part of the module. When you start a project and create the main.tf, output.tf, and variables.tf files, this set of files in the root directory is called the "root module."

You can explicitly use modules by placing related configurations in a directory and calling the module from your configuration file (.tf) in the root module or any other module.

Reusability:

Modules allow you to encapsulate patterns in your resource declarations and reuse them in multiple projects or environments. This reduces code duplication and makes your configuration more DRY (Don't Repeat Yourself).

For example, you can create a module to handle common tasks like provisioning an EC2 instance, configuring a network, or provisioning a Kubernetes cluster. Once the module is created, it can be used in different configurations by simply referencing it.

Modules typically accept inputs (variables) and produce outputs. This allows you to customize the module's behavior based on your needs and pass information from one module to other parts of your Terraform configuration.

How to use a module?

module "ec2_instance" { 
    source = "./modules/ec2-instance" 
    instance_type = "t2.micro" 
    ami_id = "ami-123456" 
    vpc_id = "vpc-123456" 
}

In this example, the ec2_instance module is referenced from a directory called ./modules/ec2-instance. Variables like instance_type, ami_id, and vpc_id are passed to the module.

Modules can be referenced from various sources:

• Locally: source = "./modules/my-module"
• Terraform Registry: source = "terraform-aws-modules/vpc/aws"
• Git Repositories: source = "git::https://github.com/user/repo.git//path/to/module"

Terraform Module Registry

The Terraform Module Registry is a public repository where you can find and share modules created by the community. It offers various commonly used modules that can save significant development time when provisioning infrastructure.

What are the advantages of using Terraform modules?

There are several advantages to using Terraform modules. The ability to reuse code and standardize it are key factors driving teams to adopt Terraform modules.

Example without a module:

Here’s what the code to create a virtual machine in Azure would look like:

# main.tf

terraform {
    required_providers {
        azurerm = {
            source  = "hashicorp/azurerm"
            version = "3.116.0"
        }
    }
}

provider "azurerm" {
    features {}
}

resource "azurerm_resource_group" "rg" {
    name     = "myresourcegroup"
    location = "West Europe"
}

resource "azurerm_virtual_network" "vnet" {
    name                = "myvnet"
    resource_group_name = azurerm_resource_group.rg.name
    location            = azurerm_resource_group.rg.location
    address_space       = ["10.0.0.0/16"]
}

resource "azurerm_subnet" "subnet" {
    name                 = "mysubnet"
    resource_group_name  = azurerm_resource_group.rg.name
    virtual_network_name = azurerm_virtual_network.vnet.name
    address_prefixes     = ["10.1.0.0/24"]
}

resource "azurerm_network_interface" "nic" {
    name                = "myvmnic"
    location            = azurerm_resource_group.rg.location
    resource_group_name = azurerm_resource_group.rg.name
    ip_configuration {
        name                          = "myvmnicconfig"
        private_ip_address_allocation = "Dynamic"
    }
}

resource "azurerm_virtual_machine" "vm" {
    name                  = "myvm"
    location              = azurerm_resource_group.rg.location
    resource_group_name   = azurerm_resource_group.rg.name
    network_interface_ids = [azurerm_network_interface.nic.id]
    vm_size               = "Standard_DS1_v2"

    os_profile_linux_config {
        disable_password_authentication = false
    }

    os_profile {
        computer_name  = "hostname"
        admin_username = "testadmin"
        admin_password = "Password1234!"
    }

    storage_image_reference {
        publisher = "Canonical"
        offer     = "0001-com-ubuntu-server-jammy"
        sku       = "22_04-lts"
        version   = "latest"
    }

    storage_os_disk {
        name              = "myosdisk"
        caching           = "ReadWrite"
        create_option     = "FromImage"
        managed_disk_type = "Standard_LRS"
    }
}

In this example, to create a virtual machine, you also need to create a virtual network, a subnet, a network interface, and a resource group. This adds some complexity due to the need to manage resources related to the virtual machine, which could be managed separately.

Example with the use of a module:

Now see how using a module can simplify the code:

# main.tf

provider "azurerm" {
  features {}
}

resource "azurerm_resource_group" "example" {
  name     = "example-resources"
  location = "East US"
}

module "network" {
  source              = "./network/vnet"
  location            = azurerm_resource_group.example.location
  resource_group_name = azurerm_resource_group.example.name
  vnets = {
    vnet1 = {
      name          = "vnet1"
      address_space = ["10.0.0.0/16"]
      subnets       = {


        subnet1 = {
          name            = "subnet1"
          address_prefixes = ["10.0.1.0/24"]
        }
      }
    }
  }
}

module "nic" {
  source              = "./network/nic"
  location            = azurerm_resource_group.example.location
  resource_group_name = azurerm_resource_group.example.name
  nics = {
    nicvm1 = {
      name      = "nicvm1"
      subnet_id = module.network.vnet_ids["vnet1"].subnets["subnet1"]
    }
  }
}

module "compute" {
  source              = "./compute/vm"
  location            = azurerm_resource_group.example.location
  resource_group_name = azurerm_resource_group.example.name
  vms = {
    vm1 = {
      name            = "vm1"
      size            = "Standard_B1s"
      admin_username  = "adminuser"
      admin_password  = "P@ssw0rd1234!"
      nic_name        = "nicvm1"
      image_publisher = "Canonical"
      image_offer     = "UbuntuServer"
      image_sku       = "18.04-LTS"
      image_version   = "latest"
    }
  }
}

In this example, three modules were used, reducing the complexity of creating one or more virtual machines.

Another important factor is abstraction and encapsulation. By developing a module, you can simplify resource management and design infrastructure that is more user-friendly for those who will consume these modules.

Imagine that your company decides that only Ubuntu version 22.04 will be available for creating virtual machines. You can set this as a default value in the module and omit it from the configuration. Another important aspect is security: for example, you could enforce that Linux virtual machines are only created with SSH key authentication, disabling password authentication.

From a governance perspective, you can establish naming standards and tags that are transparent to the user but ensure your infrastructure remains compliant.

Modules are a great strategy, but using providers directly can be advantageous when the complexity is low and the number of resources doesn't justify the use of modules.

Modules make your infrastructure as code more secure, reusable, and reliable. They shouldn't be seen as a complicating factor; whenever you choose to use modules, ensure they are up to date with provider versions, test your modules, and ensure they are an aid to those who use them, not a burden.

See more on: https://github.com/rafaelherik/terraform-samples