With the removal of the mixer component in istio 1.5, the configuration of rate limiting changed. Since istio > 1.5, rate limiting is done with an EnvoyFilter that applies to your proxies. Therefore it makes sense to have a closer look at the istio data plane to gain a better understanding about envoy.
Image from https://istio.io/latest/docs/concepts/what-is-istio/
As you can see in the above image, the service mesh dataplane consists of envoy proxies sitting in front of each of your services. An Ingressgateway or Egressgateway is also just an envoy proxy at the edge. Because all traffic in the mesh goes through these proxies, you gain observability of your network traffic as well as control by configuring the proxies to your needs. To better understand the possibilites of a service mesh, we should take a deeper look into the envoy proxy itself.
What is envoy?
In the envoy docs we read:
Envoy is an L7 proxy and communication bus designed for large modern service oriented architectures. The project was born out of the belief that:
"The network should be transparent to applications. When network and application problems do occur it should be easy to determine the source of the problem."
Since envoy runs as a sidecar in your pods, you can update the proxy configuration on the fly without touching the different services itself.
Every Proxy has a built in filter chain. In this chain there are pluggable filters which provide the traffic management capabilites. The filters perform different tasks like buffering, rate limiting and, last but not least routing. You can think of the filter chain like a middleware in your webserver.
To avoid confusion you should have a look at the envoy terminology.
We cited the most important terms for this article from the envoy docs.
Host: An entity capable of network communication (application on a mobile phone, server, etc.). In this documentation a host is a logical network application. A physical piece of hardware could possibly have multiple hosts running on it as long as each of them can be independently addressed.
Downstream: A downstream host connects to Envoy, sends requests, and receives responses.
Upstream: An upstream host receives connections and requests from Envoy and returns responses.
Listener: A listener is a named network location (e.g., port, unix domain socket, etc.) that can be connected to by downstream clients. Envoy exposes one or more listeners that downstream hosts connect to.
Cluster: A cluster is a group of logically similar upstream hosts that Envoy connects to. Envoy discovers the members of a cluster via service discovery. It optionally determines the health of cluster members via active health checking. The cluster member that Envoy routes a request to is determined by the load balancing policy.
Now, that we have a glimpse of the istio data plane. Let's solve an actual problem with it.
Rate limit a service
Envoy has local (non-distributed) and global rate limiting capabilities. This article will focus on the global rate limiting architecture.
To use global rate limiting you need an external rate limiter service that keeps track of the domains that have to be rate limited. Luckily envoy provides a redis based ratelimit service. In the following sections you will learn how to configure the redis based ratelimit service and how to rate limit specific routes.
Image created with draw.io by Natalia Sattler and Moritz Rieger
Deploy the ratelimit service
For the deployment of the ratelimit service you can use the ratelimitservice.yaml as a starting point. If you already have a redis instance in your kubernetes cluster, then feel free to use it for the rate limiter service as well and remove redis related parts.
Adjust the ConfigMap ratelimit-config with your rate limiting rules and specify the REDIS_URL in the ratelimit Deployment. Now you can deploy the ratelimit service to your kubernetes cluster.
Rate limiting rules
You should check the ratelimit service documentation for details on how to configure the rate limiting rules. For now you can just use the example configuration from below and adjust it to your needs.
Example configuration:
apiVersion: v1
kind: ConfigMap
metadata:
name: ratelimit-config
data:
config.yaml: |
domain: foo-domain
descriptors:
- key: BAR
value: "/foo"
rate_limit:
unit: minute
requests_per_unit: 1
- key: BAR
rate_limit:
unit: minute
requests_per_unit: 100
From the rate limit docs
Domain: A domain is a container for a set of rate limits. All domains known to the Ratelimit service must be globally unique.
Descriptor: A descriptor is a list of key/value pairs owned by a domain that the Ratelimit service uses to select the correct rate limit to use when limiting.
In the example we use foo-domain to group our rate limiting rules:
- all descriptors with key
BARand value/foowill have a rate limit of 1 request per minute - all other values of the descriptor with key
BARwill have a rate limit of 100 requests per minute. - all other keys are not rate limited
The rate limit service determines whether a request should be limited based on the descriptors provided in the check call. If the call is made with ("BAR","/foo"), rule number one of the rate limiter will be used, if it is called with ("BAR","/foobar"), the second rule kicks in. Because there is no special handling for the /foobar value. If another request with ("BAR","/test") comes in, the rule number two will be used as well. Note, that the rate limit is not shared with ("BAR","/foobar"). Each of the requests can be queried 100 times per minute. The implicit rule number three gets applied if the rate limit service is invoked with something like ("FOO","/foo"). For an unspecified key, no ratelimiting is applied at all.
One important thing to mention is that the value of the descriptor, though it looks like a part of an URI, does not have to denote the requested URI. It is merely a value of the descriptor and could be anything. In the example it is specified this way because of the envoy filter we configure later on.
When the configuration of the rate limiter is done and the service is beamed into your kubernetes cluster, it is time to ensure that all incoming requests will be guarded by this service.
Introduce the ratelimit service to Envoy
At first, register the ratelimit service as an envoy cluster. This is done via an envoy filter patch which is applied at the cluster level. The config patch specifies the new envoy cluster with a name of your choice and the according endpoints of the rate limiter service (line 11-36).
This cluster can then be referenced from the ratelimit http filter so that the filter knows where to send the rate limit queries.
This is exactly what the second path does. We insert the rate limit http filter into to the filter chain of the ingress gateway proxy. In line 55-62 we reference to the previously created cluster and rate limiter domain in the filter definition.
Example
apiVersion: networking.istio.io/v1alpha3
kind: EnvoyFilter
metadata:
name: filter-ratelimit
namespace: istio-system
spec:
workloadSelector:
labels:
istio: ingressgateway
configPatches:
- applyTo: CLUSTER
match:
proxy:
proxyVersion: ^1\.15.*
cluster:
# kubernetes dns of your ratelimit service
service: ratelimit.default.svc.cluster.local
patch:
operation: ADD
value:
name: rate_limit_cluster
type: STRICT_DNS
connect_timeout: 10s
lb_policy: ROUND_ROBIN
http2_protocol_options: {}
load_assignment:
# arbitrary name
cluster_name: rate_limit_cluster
endpoints:
- lb_endpoints:
- endpoint:
address:
socket_address:
# kubernetes dns of your ratelimit service
address: ratelimit.default.svc.cluster.local
port_value: 8081
- applyTo: HTTP_FILTER
match:
context: GATEWAY
proxy:
proxyVersion: ^1\.15.*
listener:
filterChain:
filter:
name: 'envoy.http_connection_manager'
subFilter:
name: 'envoy.router'
patch:
operation: INSERT_BEFORE
value:
name: envoy.filters.http.ratelimit
typed_config:
'@type': type.googleapis.com/envoy.extensions.filters.http.ratelimit.v3.RateLimit
# arbirary domain, ensure it matches with the domain used in the ratelimit service config
domain: foo-domain
failure_mode_deny: true
rate_limit_service:
grpc_service:
envoy_grpc:
# must match load_assignment.cluster_name from the patch to the CLUSTER above
cluster_name: rate_limit_cluster
timeout: 10s
transport_api_version: V3
Use rate limit filter in an ingress gateway
So far we have configured the rate limit filter and the cluster. Now it is time to use this filter in our ingress gateway to rate limit incoming requests.
In the folowing yaml we attach the rate limit filter to the ingress gateway virtual host filter chain and specify the descriptor to be used with rate limiter service queries.
Example
apiVersion: networking.istio.io/v1alpha3
kind: EnvoyFilter
metadata:
name: filter-ratelimit-svc
namespace: istio-system
spec:
workloadSelector:
labels:
istio: ingressgateway
configPatches:
- applyTo: VIRTUAL_HOST
match:
proxy:
proxyVersion: ^1\.15.*
context: GATEWAY
routeConfiguration:
# Should be in the namespace/name format. Use this field in conjunction with the portNumber and portName to accurately select the Envoy route configuration for a specific HTTPS server within a gateway config object.
gateway: istio-system/istio-gateway
portNumber: 443
portName: https
patch:
operation: MERGE
value:
rate_limits:
- actions:
# This action results in the following descriptor ("BAR","/foo") where "/foo" is the requested path.
# :path is resolved to the actual requested path at runtime and used as the descriptor value
- request_headers:
header_name: ':path'
descriptor_key: 'BAR'
Test envoy configuration
Test yor configuration by just issuing the following request to your ingress gateway two times in a row curl -i https://<your host>/foo. The second request should result in an 429 HTTP error.
If not, troubleshoot your configuration.
1. Check if the rate limiter cluster is registered
Verify that your rate limiter cluster is properly registered
istioctl proxy-config cluster <your-istio-ingressgateway-pod>.istio-system -o json
This command should produce something like
[
{
"name": "rate_limit_cluster",
"type": "STRICT_DNS",
"connectTimeout": "10s",
"loadAssignment": {
"clusterName": "rate_limit_cluster",
"endpoints": [
{
"lbEndpoints": [
{
"endpoint": {
"address": {
"socketAddress": {
"address": "ratelimit.default.svc.cluster.local",
"portValue": 8081
}
}
}
}
]
}
]
},
"http2ProtocolOptions": {}
},
...
]
If not, use the istioctl proxy-status command to check the status of the cluster.
istioctl proxy-status <your-istio-ingressgateway-pod>.istio-system
2. Check if the rate limiter filter is attached to the http filter chain
Verify that the rate limiter filter is attached to the http filter chain
istioctl proxy-config listener <your-istio-ingressgateway-pod>.istio-system -o json
This command should produce something like
[
{
"name": "0.0.0.0_8443",
"address": {
"socketAddress": {
"address": "0.0.0.0",
"portValue": 8443
}
},
"filterChains": [
{
"filterChainMatch": {
"serverNames": [
"www.example.com"
]
},
"filters": [
{
"name": "envoy.filters.network.http_connection_manager",
"typedConfig": {
"@type": "type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager",
"statPrefix": "outbound_0.0.0.0_8443",
"rds": {
"configSource": {
"ads": {},
"resourceApiVersion": "V3"
},
"routeConfigName": "https.443.https.istio-gateway.istio-system"
},
"httpFilters": [
...
{
"name": "envoy.filters.http.ratelimit",
"typedConfig": {
"@type": "type.googleapis.com/envoy.extensions.filters.http.ratelimit.v3.RateLimit",
"domain": "foo-domain",
"failureModeDeny": true,
"rateLimitService": {
"grpcService": {
"envoyGrpc": {
"clusterName": "rate_limit_cluster"
},
"timeout": "10s"
},
"transportApiVersion": "V3"
}
}
},
...
If not, check the problems with the same command as above.
3. Check if the rate limiter action configuration is applied to your ingress gateway virtual host configuration
Verify the route configuration with the following command
istioctl proxy-config route <your-istio-ingressgateway-pod>.istio-system -o json
This command should produce something like
[
{
"name": "https.443.https.istio-gateway.istio-system",
"virtualHosts": [
{
"name": "www.example.com:443",
"domains": [
"www.example.com",
"www.example.com:*"
],
"routes": [...],
"rateLimits": [
{
"actions": [
{
"requestHeaders": {
"headerName": ":path",
"descriptorKey": "PATH"
}
}
]
}
],
...
If not, check the problems with the same command as above.
Be aware of pitfalls!
Query parameters make a path unique
After you have applied all the configurations from above and verified that ratelimiting works. You may expect to get same rate limiting rules applied to the request https://<your host>/foo?param=value as to https://<your host>/foo. But this is not the case. The applied rule will be the second one (with 100 req/min).
Image from giphy.com
This is because the pseudo-header field :path, which we used for the descriptor value includes the path and all query parts of the target URI. The ratelimiter service on its part does not detect any special configuration for the descriptor ("BAR", "/foo?param=value") and therfore use the default of key "BAR".
To make the search params work as expected with the rate limiter we have to cut the off the value. This can be done with just another filter envoy.filters.http.header_to_metadata.
To use this filter you have to change your filter definitions:
filter-ratelimit
apiVersion: networking.istio.io/v1alpha3
kind: EnvoyFilter
metadata:
name: filter-ratelimit
namespace: istio-system
spec:
workloadSelector:
labels:
istio: ingressgateway
configPatches:
- applyTo: CLUSTER
match:
proxy:
proxyVersion: ^1\.15.*
cluster:
# kubernetes dns of your ratelimit service
service: ratelimit.default.svc.cluster.local
patch:
operation: ADD
value:
name: rate_limit_cluster
type: STRICT_DNS
connect_timeout: 10s
lb_policy: ROUND_ROBIN
http2_protocol_options: {}
load_assignment:
# arbitrary name
cluster_name: rate_limit_cluster
endpoints:
- lb_endpoints:
- endpoint:
address:
socket_address:
# kubernetes dns of your ratelimit service
address: ratelimit.default.svc.cluster.local
port_value: 8081
- applyTo: HTTP_FILTER
match:
proxy:
proxyVersion: ^1\.15.*
context: GATEWAY
listener:
filterChain:
filter:
name: 'envoy.http_connection_manager'
subFilter:
name: 'envoy.router'
patch:
operation: INSERT_BEFORE
value:
name: envoy.filters.http.header_to_metadata
typed_config:
"@type": type.googleapis.com/envoy.extensions.filters.http.header_to_metadata.v3.Config
request_rules:
- header: ':path'
on_header_present:
# use an arbitary name for the namespace
# will be used later to extract descriptor value
metadata_namespace: example
# use an arbitary key for the metadata
# will be used later to extract descriptor value
key: uri
regex_value_rewrite:
pattern:
# regex matcher
google_re2: {}
# truncates parameters from path
regex: '^(\/[\/\d\w-]+)\??.*$'
substitution: '\1'
- applyTo: HTTP_FILTER
match:
proxy:
proxyVersion: ^1\.15.*
context: GATEWAY
listener:
filterChain:
filter:
name: 'envoy.http_connection_manager'
subFilter:
name: 'envoy.filters.http.header_to_metadata'
patch:
operation: INSERT_BEFORE
value:
name: envoy.filters.http.ratelimit
typed_config:
'@type': type.googleapis.com/envoy.extensions.filters.http.ratelimit.v3.RateLimit
# ensure the domain matches with the domain used in the ratelimit service config
domain: foo-domain
failure_mode_deny: true
rate_limit_service:
grpc_service:
envoy_grpc:
# must match load_assignment.cluster_name from the patch to the CLUSTER above
cluster_name: rate_limit_cluster
timeout: 10s
transport_api_version: V3
This adds a new filter, which is registered just before the rate limiter filter and which rewrites the :path header and adds it to the metadata. To use the metadata for the descriptor value update the following envoy filter definition.
filter-ratelimit-svc
apiVersion: networking.istio.io/v1alpha3
kind: EnvoyFilter
metadata:
name: filter-ratelimit-svc
namespace: istio-system
spec:
workloadSelector:
labels:
istio: ingressgateway
configPatches:
- applyTo: VIRTUAL_HOST
match:
context: GATEWAY
routeConfiguration:
# Should be in the namespace/name format. Use this field in conjunction with the portNumber and portName to accurately select the Envoy route configuration for a specific HTTPS server within a gateway config object.
gateway: istio-system/istio-gateway
portNumber: 443
portName: https
patch:
operation: MERGE
value:
rate_limits:
- actions:
- dynamic_metadata:
descriptor_key: BAR
metadata_key:
key: example
path:
- key: uri
When you now call your service with curl -i https://<your host>/foo?param=value followed by curl -i https://<your host>/foo. You should receive 429 on the second call which verifies that both URIs are limited by the same rule.
Envoy filter syntax change
When you introduce EnvoyFilter into your istio service mesh configuration, you should pay special attention to Envoy Filters, even on minor upgrades.
As EnvoyFilter is a break glass API without backwards compatibility guarantees, we recommend users explicitly bind EnvoyFilters to specific versions and appropriately test them prior to upgrading.
Envoy version ≠ istio version
Be aware which version of envoy is deployed in your istio service mesh. The Version of istio and envoy is not in sync.
You can determine your version of envoy with the following command:
kubectl exec -it <PODNAME-WITH-ENVOY-SIDECAR> -c istio-proxy -n istio-system -- pilot-agent request GET server_info
{
"version": "dc78069b10cc94fa07bb974b7101dd1b42e2e7bf/1.15.1-dev/Clean/RELEASE/BoringSSL",`
...
}
```
<span>Title image by <a href="https://unsplash.com/@punttim?utm_source=unsplash&utm_medium=referral&utm_content=creditCopyText">Tim Gouw</a> on <a href="https://unsplash.com/?utm_source=unsplash&utm_medium=referral&utm_content=creditCopyText">Unsplash</a></span>
{% user nsattler %}
{% user moritzrieger %}
Top comments (6)
@moritzrieger I tried regex match but it is not working in istio 1.13. Any ideas to debug this would help.
The link to
ratelimitservice.yamlis not working, is this file still available?Hi @jeroendk
Thanks for the hint that the link is broken. Luckily Istio also provides us with an example of a ratelimiter service deployment config.
Path variable also make a path unique
Example:
/abc.com/v1/test/account1/generate
/abc.com/v1/test/account2/generate
How to apply regex for path variable?
@agk2 Were you able to achieve this ?
The key, 'config.yaml' to be used in configMap is fixed or can be anything. Also for providing multiple domain, what needs to be done?