Using ArgoCD on EKS + Fargate

In this post, I will describe how to use ArgoCD on existing EKS cluster.

I will not go through all the steps required to configure ArgoCD, but some troubles I encountered.

This documentation from ArgoCD provides all the information about how to install ArgoCD into Kubernetes cluster, and sync(deploy) applications.

But I encountered some troubles when trying to access the ArgoCD API Server on EKS which is using Fargate.

Since Fargate pods are only allowed to be launched within private subnet, we have to use Kubernetes Ingress to route requests from outside world into actual pods. And with AWS Load-Balancer-Controller add-on, when we create a Kubernetes Ingress, it will provision an AWS Application Load Balancer on behalf, accepting all the requests in the frontier.

You can install AWS Load-Balancer-Controller add-on following this documentation.

When we install both ArgoCD and AWS Load-Balancer-Controller add-on into EKS Cluster, we will fail to connect to ArgoCD API Server. This is because the pods running ArgoCD API Server are exposed using port 8080, and security group attached to pods do not accept inbound requests from 8080 by default. This results in ALB's health check failure on its target groups.

So we have to define new security group for ArgoCD API Server pods, and apply it.

To do this, you can simply follow this documentation.

The new security group to attach to pods needs to meet requirements below.

• Allow inbound request on port 53(TCP) from security group of the EKS cluster.
• Allow inbound request on port 53(UDP) from security group of the EKS cluster.
• Allow inbound request on port 8080(TCP).

Let's create new security group, and apply the yaml file below.

# argocd-pod-sg-policy.yaml
apiVersion: vpcresources.k8s.aws/v1beta1
kind: SecurityGroupPolicy
metadata:
  name: argo-sgp
  namespace: argocd
spec:
  podSelector:
    matchLabels:
      app.kubernetes.io/name: argocd-server
  securityGroups:
    groupIds:
      - ${NEW_CREATED_SECURITY_GROUP_ID}
      - ${EKS_CLUSTER_SECURITY_GROUP_ID}

Let's apply the yaml file above and create new SecurityGroupPolicy. Also, this new SecurityGroupPolicy doesn't apply to already-running pods, so we have to restart all ArgoCD API Server pods.

kubectl delete --all pod -n argocd

Lastly, let's define a Kubernetes Ingress, which will eventually provision a new AWS Application Load Balancer.

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: argocd-ingress
  namespace: argocd
  annotations:
    alb.ingress.kubernetes.io/load-balancer-name: argocd-alb
    alb.ingress.kubernetes.io/scheme: internet-facing
    alb.ingress.kubernetes.io/target-type: ip
    alb.ingress.kubernetes.io/subnets: ${PUBLIC_SUBNET_IDS}
    alb.ingress.kubernetes.io/certificate-arn: ${ACM_CERT_ARN}
    alb.ingress.kubernetes.io/security-groups: ${ALB_SECURITY_GROUP}
    alb.ingress.kubernetes.io/backend-protocol: HTTPS
    alb.ingress.kubernetes.io/conditions.argogrpc: |
      [{"field":"http-header","httpHeaderConfig":{"httpHeaderName": "Content-Type", "values":["application/grpc"]}}]
    alb.ingress.kubernetes.io/listen-ports: '[{"HTTPS":443}]'
spec:
  ingressClassName: alb
  rules:
  - host: dev-argocd.planit-study.com
    http:
      paths:
      - path: /
        backend:
          service:
            name: argogrpc
            port:
              number: 443
        pathType: Prefix
      - path: /
        backend:
          service:
            name: argocd-server
            port:
              number: 443
        pathType: Prefix

On the yaml file above, we used service named argogrpc.
ArgoCD API Server runs on both gRPC and HTTP/HTTPS.
gRPC is used to serve requests from ArgoCD CLI, and HTTP/HTTPS is used for UI(web) requests.

It is ideal to create new service only serving gRPC requests, so let's create one, which is also in the ArgoCD documentation.

apiVersion: v1
kind: Service
metadata:
  annotations:
    alb.ingress.kubernetes.io/backend-protocol-version: HTTP2
  labels:
    app.kubernetes.io/name: argocd-server
  name: argogrpc
  namespace: argocd
spec:
  ports:
  - name: "443"
    port: 443
    protocol: TCP
    targetPort: 8080
  selector:
    app.kubernetes.io/name: argocd-server
  sessionAffinity: None
  type: NodePort

Now we can successfully access ArgoCD API Server via both CLI and Web(UI)! And that's it! Hope this post helps you :)