DEV Community 👩‍💻👨‍💻

Maxime Guilbert
Maxime Guilbert

Posted on

How to improve terragrunt pipeline ?

Working with Terragrunt is really cool because we can apply a lot of terraform stuff at once! But when we have a lot of stuff to init, validate, plan, apply... it can take a lot of time and disk space.


Cache

If you are working with 50 terragrunt declaration in a project, by default terraform will download the provider for each declaration! Because, by default, providers are downloaded in the folder of the declaration.

So if you are using always the same providers, use a cache!

From the terraform documentation, we can see that we only need to :

  • declare an environment variable

export TF_PLUGIN_CACHE_DIR="$HOME/.terraform.d/plugin-cache"

  • or declare in the CLI configuration

plugin_cache_dir = "$HOME/.terraform.d/plugin-cache"
``

And then your stuff will go faster and you won't need to download each and every time all the providers.


Parallel execution

With the previous solution, your pipeline will accelerate a lot, but maybe to much thread will start at the same time and will try to do some update on cache files.

So you can have some "text file busy" error.

In this case (or if you want to optimize your terragrunt configurations), you can use the following option :

`hcl
--terragrunt-parallelism 4
`

With this, you will limit the number of parallel execution. (Terragrunt documentation).

For sure it will slow the execution, but you will be sure to avoid file conflicts (for example).


I hope it will help you! 🍺

Top comments (0)

Build Anything...


Use any Linode offering to create something for the DEV x Linode Hackathon 2022. A variety of prizes are up for grabs, inculding $1,000 USD. 👀

Join the Hackathon <-