Delete a bucket that is not empty

Remove a Cloud Object Storage (COS) bucket that is not empty. Use a Terraform script to recursively delete all the objects of a Cloud Object Storage (COS) bucket using MinIO client.

I am working on a use-case where I use Cloud Object Storage(COS) bucket to store a lot of files. The cloud service provisioning for the use-case happens using Terraform scripts. The tough part was to delete the COS bucket using terraform destroy. The destroy fails if the bucket is not empty.

Use this workaround if and only if your terraform resource doesn't support force-delete=true attribute or if don’t want to use the attribute.

So, I started looking for options to use with Terraform and got reminded of the Opensource MinIO client.

MinIO is a High-Performance Object Storage released under Apache License v2.0. It is API compatible with Amazon S3 cloud storage service. Use MinIO to build high-performance infrastructure for machine learning, analytics and application data workloads.

But, the challenge here is I want to use MinIO client only with terraform destroy . Here comes the Terraform Destroy provisioner to the rescue

If when = destroy is specified, the provisioner will run when the resource it is defined within is destroyed.

resource "cos_instance" "cos" {
  # ...

  provisioner "local-exec" {
    when      = destroy
    command   = "echo 'Destroy-time provisioner'"
  }
}

Destroy provisioners are run before the resource is destroyed. If they fail, Terraform will error and rerun the provisioners again on the next terraform apply. Due to this behaviour, care should be taken to destroy provisioners to be safe to run multiple times.

This is awesome. I thought I am all set and ready to rock-n-roll. But, then I hit a roadblock — You can’t set environment variables in a Destroy provisioner and MinIO expects HMAC credentials and bucket name.

To see how I cracked this, check the main.tf file in the GitHub repository

1. Clone the repository to deploy COS

 git clone https://github.com/VidyasagarMSC/cos-object-cleanup.git

1. Create terraform.tfvars file from the template and update the file with your details

cp terraform.tfvars.template terraform.tfvars

1. Run both terraform commands

terraform init 

terraform apply

Destroy

terraform destroy

By now, you should see that I have used a null_resource with triggers to set the required environment variables for the shell script. Alternatively, you can also use local_file to generate a shell script on-the-fly in Terraform.

If the workaround worked for you, don’t forget to drop an 🌟on the GitHub repo and 👏 for this article.

If you see an error, feel free to open an issue in the GitHub repository.