DEV Community

Cover image for Handling an ACME challenge response with a temporary Azure Container Instance
Kai Walter
Kai Walter

Posted on • Edited on

Handling an ACME challenge response with a temporary Azure Container Instance

#azure #aci #acme #certbot

ACME brings a high level of convenience for issuing certificates for cloud services (global & local gateways, web services).

That works pretty fine in many variations, but what if

  • I don't have an actual server where I can host some certbot / nginx magic to handle a HTTP-01 challenge - see challenge types
  • I do not want to spoil an infrastructure and pass down a request to the bowels of my business compute see sample architecture
  • I'm not able to maintain DNS entries via automation for a DNS-01 challenge

and I want to automate anyway?

This post describes how to

What is required

  • an existing Application Gateway which currently routes traffic and which is bound (custom DNS) to the domain, for which the certificate is to be requested - having no handling for port 80 / HTTP so the route to the container instance can be injected
  • or (as in my case) an existing Front Door from which the HTTP / port 80 traffic is routed to a temporary / dedicated Application Gateway

What I use

  • Azure CLI 2.15.1 to handle Azure resources
  • PowerShell Core 7.x to drive the process - because I like the primitives for handling JSON emitted by Azure CLI - but this can be easily converted to bash / jq
  • a Storage Account mounted to ACI to capture certbot output
  • an ACR to build and provide the Docker image
  • a Service Principal to allow ACI to pull image from ACR - I would have preferred Managed Identity but there is currently a limitation
  • nginx to handle incoming web traffic for HTTP-01 challenge
  • certbot to drive ACME process

Structure

Script snippets provided in this post follow this structure

  1. build up permanent components (ACR, Service Principal, create Storage, create Application Gateway HTTP route)
  2. build up temporary components (renew Service Principal credentials, build image, create ACI, establish Application Gateway to ACI link)
  3. execute certificate request
  4. destroy temporary components (see 2. - but reverse)

assuming between steps 1 and 2, after a certificate has been initially created, some time passes until certificate is to be renewed.

Generally all components could be created and wired up temporarily - which would make it a even more secure and kind of stateless approach. However while building and trying I collected these topics to consider:

a) deploying a temporary dedicated Application Gateway with all the settings can easily take 20-30 minutes
b) in my case - connecting this temporary Application Gateway to Front Door also can take another 20-30 minutes, until configuration really is effective
c) in one version I created Service Principal temporarily; just because I ended up in a lot of iterations I splitted this process into a creation and renewal of credentials
d) Storage also can be handled temporarily - just a matter of a few minutes of deployment time - but then with a GUID like name to guarantee uniqueness

0. common variables and initialization

I did not spend any energy in abstration of commonly used configuration information - hence I have these variables copied at the start of each of the scripts:

$location = "{your-location}"
$resourceGroupName = "{your-resourcegroup}"

$appGwName = "myacmepoc" # name of the Application Gateway to create / manage
$containerGroupName = "myacmepoc" # name of the Azure Container Instance to be created
$publicIpName = "myacmepoc" # name of Public IP used for Application Gateway
$registryName = "myacmepoc" # name of Azure Container registry
$servicePrincipalName = "myacmepoc" # name of Service Principal to be used to pull from ACR to ACI
$storageAccountName = "myacmepoc" # name of Storage Account where certificates are captured (from ACI)
$storageAccountShareName = "certificates" # name of share on Storage Account to be mounted to ACI
$subscriptionId = "{your-subscription-id}"

az account set -s $subscriptionId

az configure -d location=$location group=$resourceGroupName
Enter fullscreen mode Exit fullscreen mode

1. build up permanent components

Container Registry with a Service Principal 

az acr create -n $registryName --sku Basic

$registryResourceId = $(az acr show --name $registryName --query id --output tsv)
az ad sp create-for-rbac --name $servicePrincipalName --scopes $registryResourceId --role acrpull
Enter fullscreen mode Exit fullscreen mode

dedicated Application Gateway

SKIP this section when you're working with an existing Application Gateway, which is already used for your backend exposure

Create a basic setup with HTTPS which allows modification of HTTP configuration without breaking the gateway.

PFX path/password - in my case I already had an existing certificate which I could bind to the HTTPS configuration

az network application-gateway create `
    --name $appGwName `
    --capacity 1 `
    --sku Standard_v2 `
    --http-settings-cookie-based-affinity Enabled `
    --public-ip-address $publicIpName

## - HTTPS configuration

az network application-gateway http-settings create `
    -n appGatewayBackendHttpsSettings `
    --gateway-name $appGwName `
    --port 443 --protocol Https --cookie-based-affinity Enabled

az network application-gateway ssl-cert create `
    -n httpsCert `
    --gateway-name $appGwName `
    --cert-file $(Read-Host "PFX path") `
    --cert-password $(Read-Host "PFX password")

az network application-gateway frontend-port create `
    -n appGatewayHttpsPort `
    --gateway-name $appGwName `
    --port 443

az network application-gateway http-listener create `
    -n appGatewayHttpsListener `
    --gateway-name $appGwName `
    --frontend-ip appGatewayFrontendIP `
    --frontend-port appGatewayHttpsPort `
    --ssl-cert httpsCert

az network application-gateway rule create `
    -n ruleHttps `
    --gateway-name $appGwName `
    --address-pool appGatewayBackendPool `
    --http-listener appGatewayHttpsListener `
    --http-settings appGatewayBackendHttpsSettings

## - HTTP configuration - clean-up

az network application-gateway rule delete `
    -n rule1 `
    --gateway-name $appGwName

az network application-gateway http-settings delete `
    -n appGatewayBackendHttpSettings `
    --gateway-name $appGwName
Enter fullscreen mode Exit fullscreen mode

add HTTP configuration to gateway

Create the HTTP configuration with a dummy backend - which will be set to the address of running ACI container later. Add a probe so that health can be checked.

## - HTTP configuration - build-up

az network application-gateway probe create `
    -n probeContainer `
    --gateway-name $appGwName `
    --path "/index.html"  --protocol Http `
    --host-name-from-http-settings true

az network application-gateway address-pool create `
    -n appGatewayBackendContainerPool `
    --gateway-name $appGwName `
    --servers 10.0.0.1

az network application-gateway http-settings create `
    -n appGatewayBackendHttpSettings `
    --gateway-name $appGwName `
    --probe probeContainer `
    --port 80 `
    --host-name-from-backend-pool true

az network application-gateway rule create `
    -n ruleContainer `
    --gateway-name $appGwName `
    --address-pool appGatewayBackendContainerPool `
    --http-listener appGatewayHttpListener `
    --http-settings appGatewayBackendHttpSettings
Enter fullscreen mode Exit fullscreen mode

(optional) link to Front Door

At this step I would link the exposed public IP on port 80 to Front Door.

2. build up temporary components

initialization

Initialize some values for the certificate generation process. My provider works with External Account Binding - hence these additional credential information.

$downloadFolder = "./certificates"
$domain = "webservice.mydomain.com"
$emailAddress = "my.email@mydomain.com"
$serverUrl = "https://{providers-ACME-directory-url}"
$eabkid = "{external-account-binding-key-identifier}"
$eabhmac = "{HMAC-key-for-external-account-binding}"
Enter fullscreen mode Exit fullscreen mode

[Console]::ResetColor() comes in handy in case console colors are messed up from a previous failed run

I like to specifically tag those kind of images just to be absolutely sure that the correct version is pulled and used.

[Console]::ResetColor()

$registry = az acr show -n $registryName -o json | ConvertFrom-Json
$tag = Get-Date -AsUTC -Format yyMMdd_HHmmss
$image = "$($registry.loginServer)/certbot:$tag"

$storageAccountKey = $(az storage account keys list --account-name $storageAccountName --query "[0].value" --output tsv)
az storage share create --name $storageAccountShareName --account-name $storageAccountName
Enter fullscreen mode Exit fullscreen mode

Renew credentials on Service Principal so that I do not need to store those somewhere.

$servicePrincipal = az ad sp credential reset --name $servicePrincipalName -o json | ConvertFrom-Json

$delay = 30
Write-Host "wait $delay seconds for service principal credential change"
Start-Sleep -Seconds $delay
Enter fullscreen mode Exit fullscreen mode

build and spin up container

Build container on ACR (no local Docker required) and spin up in ACI. With that mount Storage Account share to the container.

az acr build -t $image -r $registryName .

az container create --name $containerGroupName `
    --image $image `
    --registry-login-server $registry.loginServer --registry-username $credentials.username --registry-password $credentials.passwords[0].value `
    --ip-address public `
    --azure-file-volume-account-name $storageAccountName `
    --azure-file-volume-account-key $storageAccountKey `
    --azure-file-volume-share-name $storageAccountShareName `
    --azure-file-volume-mount-path "/tmp/certificates" `
    --environment-variables DOMAIN=esb-dev.$dnsSuffix `
    --secure-environment-variables EMAIL=$EmailAddress SERVERURL=$ServerUrl EABKID=$EABKID EABHMACKEY=$EABHMACKEY
$containerGroup = az container show -n $containerGroupName -o json | ConvertFrom-Json
Enter fullscreen mode Exit fullscreen mode

Sample configuration for Docker and nginx is based on this post.

Dockerfile 

FROM nginx:alpine

RUN apk add python3 python3-dev py3-pip build-base libressl-dev musl-dev libffi-dev
RUN apk add gcc musl-dev openssl-dev cargo
RUN pip3 install pip --upgrade
RUN pip3 install certbot-nginx
RUN mkdir /etc/letsencrypt

COPY default.conf /etc/nginx/conf.d/default.conf
COPY challenge.sh /root/challenge.sh
RUN chmod u+x /root/challenge.sh
Enter fullscreen mode Exit fullscreen mode

challenge.sh

in an earlier version of this post, the sequence of commands was executed directly over az container exec. However when revisiting the scenario in summer 2021, this direct execution was not possible anymore - see my StackOverflow post.

##!/bin/sh
rm -rf /tmp/certificates/*
certbot register --email $EMAIL --server $SERVERURL --eab-kid $EABKID --eab-hmac-key $EABHMACKEY --agree-tos -n
certbot certonly --nginx --email $EMAIL --server $SERVERURL -d $DOMAIN
cp -avr /etc/letsencrypt /tmp/certificates
cp /var/log/letsencrypt/letsencrypt.log  /tmp/certificates/
Enter fullscreen mode Exit fullscreen mode

default.conf 

server {
    listen       80;
    server_name  _;

    location / {
        root   /usr/share/nginx/html;
        index  index.html index.htm;
    }

    error_page   500 502 503 504  /50x.html;
    location = /50x.html {
        root   /usr/share/nginx/html;
    }
}
Enter fullscreen mode Exit fullscreen mode

modify Application Gateway and retrieve health 

az network application-gateway address-pool update `
    -n appGatewayBackendContainerPool `
    --gateway-name $appGwName `
    --set "backendAddresses[0].ipAddress=$($containerGroup.ipAddress.ip)"

$health = az network application-gateway show-backend-health -n $appGwName -o json | ConvertFrom-Json
$serverHealth = $health.backendAddressPools | ?{$_.backendAddressPool.id -match "appGatewayBackendContainerPool"} | %{$_.backendHttpSettingsCollection[0].servers}
$serverHealth
Enter fullscreen mode Exit fullscreen mode

3. execute certificate request

When Application Gateway seems healthy, execute ACME server registration and certificate creation. Backup whole folder used during the process onto the Storage Account share. These folder is then downloaded locally.

In this sample, container group on ACI is started with a public IP address. This is also possible with linking the container group to a virtual network with a private IP address. However the machine driving the process and submitting az container exec also needs to be linked to this virtual network then.

if($serverHealth[0].health -eq "Healthy") {

    "=" * 80

    Read-Host "Hit enter to start certificate challenge"

    az container exec -n $containerGroupName --exec-command "/root/challenge.sh"

    "=" * 80

    Read-Host "Hit enter to download"

    if (Test-Path $downloadFolder -PathType Container) {
        Remove-Item $downloadFolder -Recurse -Force
    }
    New-Item $downloadFolder -ItemType Directory

    az storage file download-batch --account-name $storageAccountName --account-key $storageAccountKey -s $storageAccountShareName -d $downloadFolder
}
Enter fullscreen mode Exit fullscreen mode

4. destroy temporary components 

Read-Host "Hit enter to clean up"

az container stop -n $containerGroupName

az container delete -n $containerGroupName --yes

az storage share delete --name $storageAccountShareName --account-name $storageAccountName
Enter fullscreen mode Exit fullscreen mode

variations

When working with a dedicated Application Gateway it is possible to start & stop it at the beginning and the end of the process.

az network application-gateway start --name $appGwName
Enter fullscreen mode Exit fullscreen mode 
az network application-gateway stop --name $appGwName
Enter fullscreen mode Exit fullscreen mode

conclusion

I know, the depicted configuration might look very specific and some may wonder "Why all this hassle when there are services like Let's Encrypt?". However I hope that there is worthwhile information or some insights for you people out there. Please let me know what you think or where I could improve.

Top comments (0)

Read next

shivamchamoli18 profile image

How to Choose the Right Cloud Certification?

Shivam Chamoli -

dotnetfullstackdev profile image

CI/CD in Azure DevOps for .NET Microservices Product Order Services

DotNet Full Stack Dev -

dazevedo profile image

AI-900 Series: Understanding AI Workloads and Considerations

Daniel Azevedo -

omiossec profile image

Azure Virtual Network, new undocumented feature, subnet peering

Olivier Miossec -