Amazon EKS Auto Mode now supports security groups for Pods

Assign Security Groups to Individual Pods in EKS Auto Mode

EKS Auto Mode simplifies cluster management, but assigning security groups to individual Pods requires leveraging the AWS VPC CNI plugin capabilities. Below is a detailed guide to achieve this securely and effectively.

Prerequisites

1. EKS Cluster: Running in EKS Auto Mode (Kubernetes v1.27+ recommended).
2. VPC CNI Plugin: Must support security groups for Pods (enabled by default in recent versions).
3. IAM Permissions:
   • The node IAM role or Fargate profile execution role must have permissions for ec2:AssignPrivateIpAddresses, ec2:AttachNetworkInterface, and ec2:CreateTags.
   • Ensure the VPC CNI Daemonset has the required policies (e.g., AmazonEKSVPCResourceController).
4. Security Groups:
   • Pre-create security groups in the same VPC as your EKS cluster.
   • Define inbound/outbound rules (e.g., allow specific ports, IPs, or other security groups).

Step 1: Enable Security Groups for Pods (if not already enabled)

EKS Auto Mode uses the VPC CNI plugin. Ensure it’s configured to support security groups.

1.1 Check Current CNI Configuration

kubectl describe daemonset aws-node -n kube-system | grep -A 10 Environment

Look for these environment variables:

- name: ENI_ENABLE_SECURITY_GROUPS
  value: "true"
- name: ENI_SEC_GROUP_FOR_PODS
  value: "true"

1.2 Enable Security Groups (if disabled)

If the variables are missing or set to false, edit the Daemonset:

kubectl edit daemonset aws-node -n kube-system

Add/UPDATE the environment variables:

env:
  - name: ENI_ENABLE_SECURITY_GROUPS
    value: "true"
  - name: ENI_SEC_GROUP_FOR_PODS
    value: "true"

Restart the Daemonset pods to apply changes:

kubectl delete pod -n kube-system -l k8s-app=kube-proxy

Note: In EKS Auto Mode, some configurations may be managed automatically. If you cannot edit the Daemonset, raise a support ticket or use AWS Managed Policies to ensure the cluster supports security groups.

Step 2: Assign Security Groups to Individual Pods

Use Pod annotations to attach security groups. You can apply this at the Deployment, StatefulSet, or Pod level.

2.1 Example: Assign to a Deployment

apiVersion: apps/v1
kind: Deployment
metadata:
  name: my-app
spec:
  replicas: 2
  selector:
    matchLabels:
      app: my-app
  template:
    metadata:
      labels:
        app: my-app
      annotations:
        # Replace sg-12345678 with your security group ID(s)
        vcp.amazonaws.com/security-group: "sg-12345678,sg-87654321"
    spec:
      containers:
      - name: app-container
        image: nginx:latest
        ports:
        - containerPort: 80

2.2 Example: Assign to a Single Pod

apiVersion: v1
kind: Pod
metadata:
  name: single-pod
  annotations:
    vcp.amazonaws.com/security-group: "sg-12345678"
spec:
  containers:
  - name: container
    image: nginx

Multiple Security Groups: Comma-separate IDs (max 5 per Pod).

Namespace-Level: Apply annotations to the Namespace to default all Pods in that namespace.

Step 3: Verify Security Group Assignment

3.1 Check Pod ENI in AWS Console

1. Go to VPC Dashboard → Network Interfaces.
2. Find the ENI associated with your Pod (look for kubernetes.io Created For tag).
3. Under Security Groups, confirm the attached groups match your annotation.

3.2 Test Connectivity

• Use kubectl exec to test access from the Pod to external resources.
• Validate that traffic aligns with your security group rules.

Step 4: Best Practices & Compliance

Security & Compliance Considerations

1. Least Privilege:

   • Assign only required rules to security groups (e.g., restrict ingress to specific CIDRs).
   • Avoid using 0.0.0.0/0 unless absolutely necessary.

2. Audit & Monitoring:

   • Enable AWS CloudTrail to log ENI and security group changes.
   • Use Amazon GuardDuty to detect anomalous traffic.

3. Tagging:

   • Tag security groups with metadata (e.g., Project: MyApp, Environment: Prod) for cost allocation and governance.

4. Infrastructure as Code (IaC):

   • Define security groups and Pod annotations in Terraform or AWS CloudFormation for reproducibility.

Example Terraform Snippet:

   resource "aws_security_group" "pod_sg" {
     name        = "my-pod-sg"
     description = "Allow HTTP traffic"
     vpc_id      = aws_vpc.main.id

     ingress {
       from_port   = 80
       to_port     = 80
       protocol    = "tcp"
       cidr_blocks = ["10.0.0.0/16"]
     }
   }

1. Fargate Support:
   • If using Fargate profiles, ensure the Fargate pod execution role has permissions for security group assignment.

Troubleshooting

Summary

By enabling the VPC CNI plugin’s security group support and using Pod annotations, you can assign granular security groups to individual Pods in EKS Auto Mode. Always follow AWS best practices for least privilege, monitoring, and IaC to maintain security and compliance.

Reference: Assign Security Groups to Individual Pods in EKS Auto Mode