Motivation
I've been setting up Azure Application Gateway now for some years in various scenarios. As it is a quite versatile resource and I really always have to get my ducks in a row before I start applying it to a certain scenario. So this post here is merely a reference for myself - my future dummy me. By taking you along, maybe you also gain some information out of this post.
What is Application Gateway?
Azure Application Gateway is a layer 7 - application layer - load balancer and reverse proxy including an optional WAF - Web Application Firewall - to inspect and even block traffic towards a web application. Web application already suggests that it is only designed with HTTP/HTTPS traffic. On the other hand Azure Loadbalancer works on layer 4 - the transport layer - and hence is independent of application layer protocols - but of course can also handle HTTP/HTTPS.
More material:
- What is Azure Application Gateway? - MS docs
- Adam Gordon proving a really good introduction to what Azure Application Gateway is in this video
Fun fact: although Application Gateway seems to be intended to front a bunch of VMs producing ones web application content and functionality, I never used it for such a scenario; I don't do web applications and APIs like that - I always try to follow a PaaS-first approach
What are the primitives in Application Gateway?
I do not want to repeat all the material that is already out there and rather focus on the various components and their role in the orchestration of the gateway.
For that I assembled this schematic overview with the most important components - or the components for the most common scenarios I had so far:
I will try to explain these components by following a hypothetical traffic flow - so that it should be easier at which point of a flow the components come into play. This textual explanation is complemented with a corresponding Bicep template in API version 2021-05-01 - using Bicep here because it offers the least noise (compared to ARM, Pulumi, Terraform, ...) to show the essence of what configuration options are available.
To understand configuration options in detail I usually check out Azure REST API for Application Gateways - Create Or Update (or any other resource) and then transfer back to the Infrastructure-as-Code stack I use.
Abbreviations used
short | long |
---|---|
AG | Azure Application Gateway |
APIM | Azure API Management |
CNAME | DNS CNAME Resource Record |
FQDN | Fully qualified domain name |
VNET | virtual network |
Ingress
First let's pick up incoming traffic. This is done with an entry in frontendIPConfigurations
. Here I can specify whether to pick up from a public IP address - another Azure resource external to AG, a private address in the gateway subnet or a private link configuration - specified in another section of the AG.
For the most common scenario, public IP, while DNS related referencing (e.g. putting a CNAME on the FQDN provided by Azure) is targetted towards this public IP resource, actual SSL certificate valid for the FQDN has to be linked or uploaded to AG.
frontendIPConfigurations: [
{
id: 'string'
name: 'string'
properties: {
privateIPAddress: 'string'
privateIPAllocationMethod: 'string'
privateLinkConfiguration: {
id: 'string'
}
publicIPAddress: {
id: 'string'
}
subnet: {
id: 'string'
}
}
}
]
privateLinkConfigurations: [
{
id: 'string'
name: 'string'
properties: {
ipConfigurations: [
{
id: 'string'
name: 'string'
properties: {
primary: bool
privateIPAddress: 'string'
privateIPAllocationMethod: 'string'
subnet: {
id: 'string'
}
}
}
]
}
}
]
a sample for a private link configuration can be found in my post, where I use it to make an APIM instance available for another VNET without peering
Next element is an entry in httpListeners
, frontendPorts
and sslCertificates
(in case HTTPS traffic is to be processed). httpListener
picks up from a frontendIPConfiguration
(to be referenced) and specifies which port (443, 80, 8080, ...) and protocol (Https, Http) is to be used. A configuration for Https requires that a certificate is made available to AG, either by specifying with data
or keyVaultSecretId
in sslCertificates
. When using a HTTP-01 certificate challenge I often add a HTTP configuration like in this post which points to a temporary backend handling the challenge. Also I then make the Http / Https configuration switchable like in this sample - so as long as no certificate is yet bound on the AG I configure e.g. with protocol/port Http/8080
and then switch immediately to protocol/port Https/443
when certificate is uploaded.
Best is not to point to actual backends as long as Https is not activated!
httpListeners: [
{
id: 'string'
name: 'string'
properties: {
customErrorConfigurations: [
{
customErrorPageUrl: 'string'
statusCode: 'string'
}
]
firewallPolicy: {
id: 'string'
}
frontendIPConfiguration: {
id: 'string'
}
frontendPort: {
id: 'string'
}
hostName: 'string'
hostNames: [
'string'
]
protocol: 'string'
requireServerNameIndication: bool
sslCertificate: {
id: 'string'
}
sslProfile: {
id: 'string'
}
}
}
]
frontendPorts: [
{
id: 'string'
name: 'string'
properties: {
port: int
}
}
]
sslCertificates: [
{
id: 'string'
name: 'string'
properties: {
data: 'string'
keyVaultSecretId: 'string'
password: 'string'
}
}
]
SSL processing can additionally be adapted to your requirements with these elements - but would not necessarily be required:
sslPolicy: {
cipherSuites: [
'string'
]
disabledSslProtocols: [
'string'
]
minProtocolVersion: 'string'
policyName: 'string'
policyType: 'string'
}
sslProfiles: [
{
id: 'string'
name: 'string'
properties: {
clientAuthConfiguration: {
verifyClientCertIssuerDN: bool
}
sslPolicy: {
cipherSuites: [
'string'
]
disabledSslProtocols: [
'string'
]
minProtocolVersion: 'string'
policyName: 'string'
policyType: 'string'
}
trustedClientCertificates: [
{
id: 'string'
}
]
}
}
]
trustedClientCertificates: [
{
id: 'string'
name: 'string'
properties: {
data: 'string'
}
}
]
trustedRootCertificates: [
{
id: 'string'
name: 'string'
properties: {
data: 'string'
keyVaultSecretId: 'string'
}
}
]
Specifying backend(s)
No we assume traffic is inside the gateway and we have to determine where to send it to. An entry in backendAddressPools
with the backendAddresses
array specifies traffic targets. Either FQDN or IP addresses can be specified. When using FQDNs AG needs to be able to DNS resolve the domain - so DNS configuration needs to facilitate what is required here.
A common mistake in scenarios where AG is used to integrate APIM in an internal VNET is, that {apim-service-name}.azure-api.net
would not point to the internal IP address and hence FQDN would not resolve correctly. In this scenario ipAddress
has to be used but when integrated APIM services is operated on HTTPS, the service expects to be addressed with the correct Host
HTTP header {apim-service-name}.azure-api.net
, hence either pickHostNameFromBackendAddress
or a specific hostName
has to be specified in the corresponding backendHttpSettingsCollection
entry. Same goes for the entry in probes
.
IMPORTANT: the probe really has to be able to find and get a valid response from the backend. When the probe is not healthy - this can by checked in Azure Portal or az network application-gateway show-backend-health -n {appGwName}
- AG will not forward traffic and reward you with a 502 bad gateway error.
backendAddressPools: [
{
id: 'string'
name: 'string'
properties: {
backendAddresses: [
{
fqdn: 'string'
ipAddress: 'string'
}
]
}
}
]
backendHttpSettingsCollection: [
{
id: 'string'
name: 'string'
properties: {
affinityCookieName: 'string'
authenticationCertificates: [
{
id: 'string'
}
]
connectionDraining: {
drainTimeoutInSec: int
enabled: bool
}
cookieBasedAffinity: 'string'
hostName: 'string'
path: 'string'
pickHostNameFromBackendAddress: bool
port: int
probe: {
id: 'string'
}
probeEnabled: bool
protocol: 'string'
requestTimeout: int
trustedRootCertificates: [
{
id: 'string'
}
]
}
}
]
probes: [
{
id: 'string'
name: 'string'
properties: {
host: 'string'
interval: int
match: {
body: 'string'
statusCodes: [
'string'
]
}
minServers: int
path: 'string'
pickHostNameFromBackendHttpSettings: bool
port: int
protocol: 'string'
timeout: int
unhealthyThreshold: int
}
}
]
For APIM the status endpoint can be used to be referenced in a probe:
probes: [
{
name: 'api-gateway-probe'
properties: {
protocol: 'Https'
port: 443
path: '/status-0123456789abcdef'
interval: 15
timeout: 15
host: apiGatewayHostname
unhealthyThreshold: 3
match: {
statusCodes: [
'200'
]
}
}
}
]
Request processing
Having specified ingress and backend, those components can we wired together with an entry in requestRoutingRules
. ruleType
Basic
forwards all traffic 1:1. Limiting fowarding only for certain paths can be achieved with ruleType
PathBasedRouting
and entries in urlPathMaps
. With rewriteRuleSets
some basic rewriting of the request is possible.
requestRoutingRules: [
{
id: 'string'
name: 'string'
properties: {
backendAddressPool: {
id: 'string'
}
backendHttpSettings: {
id: 'string'
}
httpListener: {
id: 'string'
}
loadDistributionPolicy: {
id: 'string'
}
priority: int
redirectConfiguration: {
id: 'string'
}
rewriteRuleSet: {
id: 'string'
}
ruleType: 'string'
urlPathMap: {
id: 'string'
}
}
}
]
rewriteRuleSets: [
{
id: 'string'
name: 'string'
properties: {
rewriteRules: [
{
actionSet: {
requestHeaderConfigurations: [
{
headerName: 'string'
headerValue: 'string'
}
]
responseHeaderConfigurations: [
{
headerName: 'string'
headerValue: 'string'
}
]
urlConfiguration: {
modifiedPath: 'string'
modifiedQueryString: 'string'
reroute: bool
}
}
conditions: [
{
ignoreCase: bool
negate: bool
pattern: 'string'
variable: 'string'
}
]
name: 'string'
ruleSequence: int
}
]
}
}
]
urlPathMaps: [
{
id: 'string'
name: 'string'
properties: {
defaultBackendAddressPool: {
id: 'string'
}
defaultBackendHttpSettings: {
id: 'string'
}
defaultLoadDistributionPolicy: {
id: 'string'
}
defaultRedirectConfiguration: {
id: 'string'
}
defaultRewriteRuleSet: {
id: 'string'
}
pathRules: [
{
id: 'string'
name: 'string'
properties: {
backendAddressPool: {
id: 'string'
}
backendHttpSettings: {
id: 'string'
}
firewallPolicy: {
id: 'string'
}
loadDistributionPolicy: {
id: 'string'
}
paths: [
'string'
]
redirectConfiguration: {
id: 'string'
}
rewriteRuleSet: {
id: 'string'
}
}
}
]
}
}
]
sample ruleset for client certificate extraction
One common scenario is to extract the client certificate with AG and then pass it to a downstream services:
rewriteRuleSets: [
{
name: 'extract-client-cert'
properties: {
rewriteRules: [
{
ruleSequence: 100
name: 'extract-cllient-cert'
actionSet: {
requestHeaderConfigurations: [
{
headerName: 'X-ARR-ClientCert'
headerValue: '{var_client_certificate}'
}
]
}
}
]
}
}
]
Egress
The entry in gatewayIPConfigurations
tells AG to which subnet the traffic is sent to.
Some limitations I experienced:
- the gateway subnet can only contain AG resources, no other Azure resources; also when migrating from AG v1 to v2 those could not be mixed in the same subnet; this also means when doing the AG to APIM internal integrations, that APIM has to reside in a different subnet
- a private link on the AG has to be in the same VNET, but a different subnet
gatewayIPConfigurations: [
{
id: 'string'
name: 'string'
properties: {
subnet: {
id: 'string'
}
}
}
]
Conclusion
I know I did not touch all components and configuration options in Application Gateway with this post. I only showed the essential ones which are required to get traffic passing through AG.
Anyway I hope this information is useful to anybody out there. As mentioned and as you can see from the samples I provided, Application Gateway is an elementary component in many of the VNET isolated workloads I build. Hence my future me will certainly come back here from time to time ... fix typos and probably also add more findings and specifics.
Top comments (7)
added sample ruleset for client certificate extraction
Hey Kai,
One of the possibilities in SSL profile is to add multiple trustedclientcertificates. In my bicep code I loop through multiple SSL profiles, but I'm struggling adding a child loop to add multiple trustedclientcertificates associated with a SSL profile. Is this something you are familiar with?
Hi @odenor , so basically a nested loop? Not yet. I just checked the 2 main huge repositories with Bicep templates I have at hand but did not see anything that could help.
Does this maybe help: ochzhen.com/blog/nested-loops-in-a...
That page you link, solution 1 is a bit weird. He says you can't nest loops but then still does it. Must be me that doesn't understand i guess. Anyway...
With application gateway bicep implementation the real problem starts when you have multiple rewriteRuleSets, that have multiple rewriteRules, that have multiple conditions.
I have fixed this in the past by using modules. For instance to loop the creation of subnets within a bicep that creates multiple vnets.
Since rewriteRuleSets are not a subresource you can't make modules of it as far as I understand.
I'm now looking at that page's solution 2 where you make your module return an array object of the nested parameters.
If you ever figure this out, do share :)
@jayded Is this what you want to achieve?
I am feeding in 2 arrays:
Build variables to hold the trusted client certs, their resource Ids and the SSL profiles:
and then later use the variables in the resource:
Hey, thanks for posting your code.
It's very similar.
I also solved it by making a module that outputs arrays. Then looping those arrays into my rewrireRuleSets.
I realy hope at a certain point they'll allow us to nest for's. Would make our lives a lot easier.
ps: agw was by far the hardest part to put into code to be fair. Even stuff like our apim was a lot easier to implement.
I know - that is why I made this post