Testing IaC Scripts 🧪

Lack of testing can introduce defects in IaC scripts, which in turn can have serious conse-quences. A synthesis of IaC testing practices can be helpful for practitioners to mitigate defects in IaC scripts. (Hasan et al., 2020)

Although IaC is more and more adopted by various companies in their operation process and integrated in many cloud providers using APIs, the quality assurance and maintenance of the scripts created still has to undergo further research and development to achieve a certain level of quality in form of system security and resilience. Because of the undeniable importance of this topic, IaC providers like Terraform and Pulumi, external companies like Snyk and Gruntwork as well as cloud providers like Azure and AWS are currently working on improving the development conditions with IaC.

Testing Guidelines Provided by IaC Providers

Starting with the most well-known IaC provider HashiCorp, Terraform provides documentations and guidelines describing how to implemented unit and snapshot testing capabilities in their developer documentation. Other than that, there is an experimental feature related to module testing and community content for integration tests such as shown by the Github User Ansgarm. As for Pulumi, documentations for unit, property and integration tests are available.

Testing in Terraform 🌱

Testing in Terraform heavily depends on the language that was used to write the Terraform code itself. When using HCL for example the testing framework to favour is Jest by Facebook, which provides testing capabilities to JavaScript and TypeScript, yet is still compatible with Terraform. It was included in version 0.6 for snapshot tests and assertions.

An important factor is the existence of the so-called cdktf (Cloud Development Kit for Terraform) package for the testing framework at use. This package provides an interface for interacting with the Terraform framework itself. When writing common unit test assertion statements in TypeScript a so-called synthScope is used to test part of the application. It returns a JSON string representing the synthesized HCL-JSON on which assertions can then by applied. This way important assertions such as toHa-veResource (whether a resource of a certain type exists) and toHaveResourceWithProperties can applied.

Testing in Pulumi 🧊

To get a broader view on common or experimental testing techniques for IaC tools in general, Pulumi will also be considered in this series. It provides the same base functionality as Terraform, however in addition to unit and integration tests, Pulumi provides the concept of so-called Property Test, which are used to assert properties on a resource-level while infrastructure is being deployed. The key difference between the three testing techniques documented are illustrated below.

Key differences of testing styles documented by Pulumi

With each of the techniques a specific point in time of the infrastructure can be tested. The unit tests are mostly mock-based implemented, meaning the engine normally used for the deployment of Pulumi is replaced by a mock, thus removing the need for out-of-process calls and increasing isolation. Property testing can be seen as a form of policy-as-code to ensure that the deployed infrastructure complies with desired policies. Finally, the integration tests are labelled as deploy-check-destroy tests by Pulumi as they deploy cloud resources and validate their actual behavior.

Next Steps 🪜

After discussing the testing approaches suggested by the two IaC providers Terraform and Pulumi, in the next post we will take a look at the dedicated IaC testing providers takes on this topic. Here we will have a look at Gruntwork and Snyk. So, stay tuned if you are interessted!

Cover Image by Martin Lopez! 🥳