Secure builds on Kubernetes with Kaniko (AWS ECR)

Kaniko is a suitable choice for scenarios where security, isolation, compatibility with container orchestration platforms, and efficient resource utilization are important factors. It provides an alternative to traditional Docker builds, particularly in environments where running Docker daemons might not be optimal or secure.

All the files are included in gitlab

Prerequisites:

• Kubernetes Cluster
• Helm Installed
• gitlab-registration token
• IAM User from AWS with full ECR ACCESS

Lets start by adding the AWS credentials as variables into our repository

The helm repository will enable us to use the gitlab-runner chart

helm repo add gitlab https://charts.gitlab.io

Gitlab runner config yaml

A deeper overview of the gitlab configuration is located in the official repo ["https://gitlab.com/gitlab-org/charts/gitlab-runner/-/blob/main/values.yaml"]

gitlabUrl: https://gitlab.com
runnerRegistrationToken: $REGISTRATION_TOKEN
concurrent: 2
rbac:
  create: true
  name: gitlab-runner
runners:
  tags: kube,aws
  runUntagged: true
  imagePullPolicy: if-not-present

• gitalb-runner/values.yml contains the values to overwrite in the helm repostiry
• tags: kube,aws can be anything that makes sense to you
• runnerRegistrationToken should be obtained from the CI/CD section.

Create a namespace for gitlab: kubectl create ns gitlab

Using helm to instal gitlab-runner

helm install --namespace gitlab gitlab-runner -f values.yml gitlab/gitlab-runner

kubectl get pods -n gitlab should have active gitlab pods

Now if you navigate to the runners section in your gitlab, there should be a new runner registered:

For the project build you can use the Dockerfile & index.html from my gitlab-repoisitory

.gitlab-ci.yml

To further understand kaniko please reference to the documentation kaniko Gitlab

stages:
  - build

variables:
  AWS_REGION: eu-north-1
  REGISTRY: "${AWS_ACCOUNT_ID}.dkr.ecr.eu-north-1.amazonaws.com"

build:
  stage: build
  image:
    name: gcr.io/kaniko-project/executor:debug
    entrypoint: [""]
  script:
    - mkdir -p /kaniko/.docker
    - echo "{\"credsStore\":\"ecr-login\",\"credHelpers\":{\"$REGISTRY/contoso\":\"ecr-login\"}}" > /kaniko/.docker/config.json
    - >-
      /kaniko/executor
      --context "${CI_PROJECT_DIR}" 
      --dockerfile "${CI_PROJECT_DIR}/Dockerfile" 
      --build-arg AWS_REGION=$AWS_REGION 
      --build-arg AWS_ACCESS_KEY_ID=$AWS_ACCESS_KEY_ID 
      --build-arg AWS_SECRET_ACCESS_KEY=$AWS_SECRET_ACCESS_KEY 
      --destination "${REGISTRY}/contoso:${CI_COMMIT_TAG:-$CI_COMMIT_REF_SLUG}"
  tags:
    - aws,kube

"{\"credsStore\":\"ecr-login\",\"credHelpers\":{\"$REGISTRY/contoso\":\"ecr-login\"}}":

• credsStore: This specifies the Docker credentials store to be used for authentication. In this case, it's set to "ecr-login", in which the Amazon Elastic Container Registry (ECR) authentication mechanism is being used.

• credHelpers: Is an object that defines custom credential helpers for specific Docker registries. In this case, a custom helper is being set up for the $REGISTRY/contoso registry.

Once we push gitlab-ci.yml and trigger the pipeline we see the build is running, it Could take up to 30 secs for the runner to pick up the job

You can see the full pipelinepipeline log:

Navigate to the ECR repository to confirm the image build

Now that you have the pipeline all working start experimenting and see what you can do :)