DEV Community

Cover image for Sticky sessions and canary releases in Kubernetes
Daniele Polencic
Daniele Polencic

Posted on

Sticky sessions and canary releases in Kubernetes

Sticky sessions or session affinity is a convenient strategy to keep subsequent requests always reaching the same pod.

Let's look at how it works by deploying a sample application with three replicas and one service.

In this scenario, requests directed to the service are load-balanced amongst the available replicas.

A deployment and a service in Kubernetes

Let's deploy the ingress-nginx controller and create an Ingress manifest for the deployment.

In this case, the ingress controller skips the services and load balances the traffic directly to the pods.

A deployment, a service and ingress controller in Kubernetes

While the two scenarios end up with the same outcome (i.e. requests are distributed to all replicas), there's a subtle (but essential) distinction: the Service operates on L4 (TCP/UDP), whereas the Ingress is L7 (HTTP).

Difference between Kubernetes service and ingress controller

Unlike the service, the Ingress controller can route traffic based on paths, headers, etc.

You can also use it to define weights (e.g. 20-80 traffic split) or sticky sessions (all requests from the same origin always land on the same pod).

Sticky sessions, weighted traffic and canary release in an Ingress controller

The following Ingress implements sticky sessions for the nginx-ingress controller.

The ingress writes a cookie on your browser to keep track of what instance you visited.

Session affinity in ingress-nginx

There are two convenient settings for affinity:

  1. balanced — requests are redistributed if the deployment scales up.
  2. persistent — no matter what, the requests always stick to the same pod.

nginx-ingress can also be used for canary releases.

If you have two deployments and you wish to test a subset of the traffic for a newer version of that deployment, you can do so with a canary release (and impact a minimal amount of users).

A canary release with ingress-nginx

In a canary release, each deployment has its own Ingress manifest.

However, one of those is labelled as a canary.

You can decide how the traffic is forwarded: for example, you could inspect a header or cookie.

Canary release beased on a header value

In this example, all traffic labelled east-us is routed to the canary deployment.

Example of routing traffic with a canary release

You can also decide which fraction of the total traffic is routed to the canary with weights.

Setting weights to a canary release

But if the header is omitted in a subsequent request, the user will return to see the previous deployment.

How can you fix that?

Traffic in canary releases is not sticky

With sticky sessions!

You can combine canary releases and sticky sessions with ingress-nginx to progressively (and safely) roll out new deployments to your users.

Combining canary releases and sticky sessions

It's important to remember that those types of canary releases are only possible for front-facing apps.

To roll out a canary release for internal microservices, you should look at alternatives (e.g. service mesh).

Traffic to internal services is not forwarded by the ingress controller

Is nginx-ingress the only option for sticky sessions and canary releases?

Not really, but the annotations might be different to other ingress controllers.

At Learnk8s, we've put together a spreadsheet to compare them.

And finally, if you've enjoyed this thread, you might also like:

While authoring this post, I also found the following resources valuable:

Top comments (2)

Collapse
 
symeta profile image
symeta

Hi, Daniel, I followed the steps the blog dev.to/danielepolencic/binding-aws...
and found out an error when executing

aws sts assume-role-with-web-identity   --role-arn $AWS_ROLE_ARN   --role-session-name test   --web-identity-token $AWS_WEB_IDENTITY_TOKEN_FILE   --duration-seconds 1000
Enter fullscreen mode Exit fullscreen mode

the error msg is as below:

An error occurred (InvalidIdentityToken) when calling the AssumeRoleWithWebIdentity operation: Couldn't retrieve verification key from your identity provider,  please reference AssumeRoleWithWebIdentity documentation for requirements
Enter fullscreen mode Exit fullscreen mode

both .well-known/openid-configuration and openid/v1/jwks could be curl ed.
could you give some guidance on how to solve the issue?
Many thanks.

Collapse
 
datrader profile image
DaTrader • Edited

Hi Daniele,

We have this necessity to have all the requests (via two different endpoints) relating to the same user to be directed to the same pod. Ideally, it'd be something like Sticky Sessions but with the identifier supplied in a header, instead of using cookies. Is there a way to achieve this?

If not and if we use Sticky sessions instead (making it thus all about the session and not the related user), how do you suggest I go about with the API endpoint requests then? Good thing is the API endpoint requests are only considered once there's at least one browser connection for the related user, so the cookie can be communicated to the client API before it makes its first request for the related user/session.

Thanks