User Geo Data in a Kubernetes cluster

At some point, we've faced a rather classic task - to determine the geo-data of our users (IP address, country, and city).

But since all our services/applications are running in a Kubernetes cluster on Amazon EKS at Cooper Pet Care, the IP addresses in the end applications are local addresses within the cluster. It is impossible to determine the real location of customers from them.

We conducted a research and tried to patch the main Nginx Ingress Controller but it was impossible. Maybe we didn’t pay enough attention to details, but another solution came to the rescue - to use the geo-data that AWS CloudFront can provide in the request headers.

Here is the approach we had:

Client → (request) → Route53 → Nginx Ingress Controller → (intra-cluster routing) → Application

And it became the following:

Client → (request) → Route53 → CloudFront Distribution → Network Load Balancer → Application

It is quite easy to reconfigure Amazon Route53 → CloudFront Distribution routing in the AWS interface. As for the application itself, it was necessary to update the Service (change its type and add a few annotations so that EKS understands how the service works).

Here is what we had:

kind: Service
metadata:
 name: my-service-name
 namespace: my-namespace
spec:
 type: ClusterIP
 ports:
  - port: 3000
   targetPort: 3000

Here is what it has become:

kind: Service
metadata:
 name: my-service-name
 annotations:
  service.beta.kubernetes.io/aws-load-balancer-type: "nlb"
  service.beta.kubernetes.io/aws-load-balancer-backend-protocol: "http"
  service.beta.kubernetes.io/aws-load-balancer-ssl-ports: "https"
  service.beta.kubernetes.io/aws-load-balancer-ssl-cert: "arn:aws:acm:..."
  service.beta.kubernetes.io/aws-load-balancer-proxy-protocol: "*"
spec:
 type: LoadBalancer
 ports:
  - name: http
   port: 80
   targetPort: 3000
  - name: https
   port: 443
   targetPort: 3000

Apart from it, for the new Load Balancer, we started using an annual SSL certificate issued by Amazon (its ARN is indicated in the aws-load-balancer-ssl-cert annotation) instead of LetsEncrypt certificates valid for 3 months.

You can easily add any additional headers you want to receive with each request to the application at the Distribution settings (Behaviors tab).

That's about it. No need to use any external services for obtaining geo-data outside of the Amazon AWS infrastructure.