Daniel J. Saldaña

Posted on Oct 28, 2023 • Originally published at danieljsaldana.dev on Jun 11, 2023

Desplegar app en Okteto con Gitlab CI III

#devops #gitlab #spanish #kubernetes

Continuamos con la tercera parte de esta serie de tutoriales de Gitlab CI. En este tutorial nos enfocaremos en realizar un refactor para cambiar el flujo de los pipelines y añadir unos nuevos jobs y steps.

Aquí os dejo los objetivos que abordaremos:

Añadimos un job para ejecutar test de integración, “en este tutorial solo usaremos unos test de Cypress, en un futuro añadiremos unos test unitarios”.
Integrar Cypress para validar test e2e.
Analizaremos con Trivy el contenido del repositorio para detectar posibles usuarios y contraseñas
Analizaremos la imagen con Trivy para detectar el tipo de licencia que dispone nuestro software.
Redefinir flujo de CI, se añade el entorno de Staging.

Como siempre aquí os dejo el repositorio que utilizaremos.

Daniel J. Saldaña / app-okteto · GitLab

Recordamos que el tutorial lo podrá encontrar en la rama main, ya que verá que actualmente existen más ramas, las cuales están enfocadas en los siguientes labs.

stages:
  - test
  - scanning
  - review
  - build
  - staging
  - production

variables:
  IMAGE_NODE: node:latest
  IMAGE_GITGUARDIAN: gitguardian/ggshield:latest
  IMAGE_SONAR: sonarsource/sonar-scanner-cli:latest
  IMAGE_OKTETO: okteto/okteto:1.13.4
  IMAGE_CYPRESS: cypress/browsers:node18.12.0-chrome107
  IMAGE_DOCKER: docker:stable
  IMAGE_KANIKO: gcr.io/kaniko-project/executor:v1.9.0-debug
  IMAGE_TRIVY: docker.io/aquasec/trivy:latest
  IMAGE_HELM: alpine/helm
  RELEASE:

cache:
  key: ${CI_COMMIT_REF_SLUG}
  paths:
    - node_modules/
    - .npm/

.charts_rules:
  rules:
    - if: $CI_COMMIT_BRANCH && $CI_OPEN_MERGE_REQUESTS
      when: never
    - if: $CI_COMMIT_BRANCH != 'main'
      when: never
    - if: $CI_MERGE_REQUEST_TARGET_BRANCH_NAME == 'devops'
      when: always
    - if: $CI_COMMIT_BRANCH != 'devops'
      changes:
        - charts/*
      when: always
.code_rules:
  rules:
    - if: $CI_COMMIT_BRANCH && $CI_OPEN_MERGE_REQUESTS
      when: never
    - if: $CI_COMMIT_BRANCH != 'main'
      when: never
    - if: $CI_MERGE_REQUEST_TARGET_BRANCH_NAME == 'devops'
      when: always
    - if: $CI_COMMIT_BRANCH != 'devops'
      changes:
        - src/*
        - index.html
        - package*
        - tsconfig*
        - vite.config.ts
        - Dockerfile
      when: always
.code_rules_stop:
  rules:
    - if: $CI_COMMIT_BRANCH && $CI_OPEN_MERGE_REQUESTS
      when: never
    - if: $CI_COMMIT_BRANCH != 'main'
      when: never
    - if: $CI_COMMIT_BRANCH != 'devops'
      changes:
        - src/*
        - index.html
        - package*
        - tsconfig*
        - vite.config.ts
        - Dockerfile
      when: delayed
      start_in: 30 minutes
    - if: $CI_MERGE_REQUEST_TARGET_BRANCH_NAME == 'devops'
      when: always
.tests_rules:
  rules:
    - if: $CI_COMMIT_BRANCH && $CI_OPEN_MERGE_REQUESTS
      when: never
    - if: $CI_COMMIT_BRANCH != 'main'
      when: never
    - if: $CI_MERGE_REQUEST_TARGET_BRANCH_NAME == 'devops'
      when: always
    - if: $CI_COMMIT_BRANCH != 'devops'
      changes:
        - cypress/*
      when: always
.tests_rules_stop:
  rules:
    - if: $CI_COMMIT_BRANCH && $CI_OPEN_MERGE_REQUESTS
      when: never
    - if: $CI_COMMIT_BRANCH != 'main'
      when: never
    - if: $CI_COMMIT_BRANCH != 'devops'
      changes:
        - cypress/*
      when: delayed
      start_in: 30 minutes
    - if: $CI_MERGE_REQUEST_TARGET_BRANCH_NAME == 'devops'
      when: always
.devops_rules:
  rules:
    - if: $CI_COMMIT_BRANCH == 'devops' && $RELEASE == 'yes'
      when: always
    - if: $CI_COMMIT_BRANCH != 'main'
      when: never
    - if: $CI_MERGE_REQUEST_TARGET_BRANCH_NAME == 'main'
      when: always
.devops_rules_stop:
  rules:
    - if: $CI_COMMIT_BRANCH == 'devops' && $RELEASE == 'yes'
      when: delayed
      start_in: 30 minutes
    - if: $CI_COMMIT_BRANCH != 'main'
      when: never
    - if: $CI_MERGE_REQUEST_TARGET_BRANCH_NAME == 'main'
      when: always
.go_to_production_rules:
  rules:
    - if: $CI_MERGE_REQUEST_TARGET_BRANCH_NAME == 'main'
.production_rules:
  rules:
    - if: $CI_COMMIT_BRANCH == 'main'

.deploy:
  image:
    name: $IMAGE_HELM
    entrypoint: [""]
  script:
    - export KUBECONFIG=${ENV_KUBECONFIG}:${KUBECONFIG:-$HOME/.kube/config}
    - sed -i 's/_CI_PROJECT_TITLE/'"${CI_PROJECT_TITLE}"'/' "charts/templates/NOTES.txt"
    - sed -i 's/_VA_ENV/'""'/' "charts/templates/NOTES.txt"
    - sed -i 's/_CI_COMMIT_REF_SLUG/'""'/' "charts/templates/NOTES.txt"
    - sed -i 's/_OKTETO_USERNAME/'"-${OKTETO_USERNAME}"'/' "charts/templates/NOTES.txt"
    - helm upgrade $CI_PROJECT_NAME ./charts
      --values=./charts/values.yaml
      --namespace ${VA_ENV}-${OKTETO_USERNAME}
      --install --atomic

kubelinter:
  stage: test
  image: $IMAGE_DOCKER
  services:
    - docker:dind
  script:
    - docker run -v $PWD/charts:/dir -v $PWD/lint/config.yaml:/etc/config.yaml stackrox/kube-linter lint /dir --config /etc/config.yaml
  rules:
    - !reference [.charts_rules, rules]
    - !reference [.devops_rules, rules]

sonarcloud-check:
  stage: test
  image:
    name: $IMAGE_SONAR
  variables:
    SONAR_USER_HOME: "${CI_PROJECT_DIR}/.sonar"
    GIT_DEPTH: "0"
  cache:
    key: "${CI_JOB_NAME}"
    paths:
      - .sonar/cache
  script:
    - sonar-scanner
  rules:
    - !reference [.code_rules, rules]
    - !reference [.devops_rules, rules]

integration-testing:
  stage: test
  image: $IMAGE_CYPRESS
  script:
    - npm ci
    - npm run integration-testing &
    - npx cypress run --env baseUrl=http://localhost:3000 --config-file cypress/integration-testing.config.ts --browser chrome --reporter junit --reporter-options "mochaFile=results/test-integration-[hash].xml"
  artifacts:
    when: always
    paths:
      - results/test-integration-*.xml
      - cypress/videos/**/*.mp4
      - cypress/screenshots/**/*.png
    reports:
      junit: results/test-integration-*.xml
    expire_in: 1 day
  rules:
    - !reference [.tests_rules, rules]
    - !reference [.devops_rules, rules]

gitguardian:
  stage: scanning
  image:
    name: $IMAGE_GITGUARDIAN
  script:
    - ggshield secret scan ci
  rules:
    - !reference [.code_rules, rules]
    - !reference [.tests_rules, rules]
    - !reference [.devops_rules, rules]

type-detection:
  stage: scanning
  image:
    name: $IMAGE_TRIVY
    entrypoint: [""]
  script:
    - trivy conf --severity HIGH,CRITICAL .
  cache:
    paths:
      - .trivycache/
  artifacts:
    expire_in: 1 week
    name: "gl-type-detection-report-${CI_BUILD_ID}-${CI_COMMIT_SHA}.json"
    paths:
      - ${CI_PROJECT_DIR}
  rules:
    - !reference [.code_rules, rules]
    - !reference [.devops_rules, rules]

deploy-review:
  stage: review
  image: $IMAGE_OKTETO
  variables:
    APP: $CI_COMMIT_REF_SLUG
    VA_ENV: review
  script:
    - okteto preview deploy $VA_ENV-$CI_COMMIT_REF_SLUG-$OKTETO_USERNAME --scope personal --branch $CI_COMMIT_REF_NAME --repository $CI_REPOSITORY_URL --wait
  environment:
    name: review/$CI_COMMIT_REF_SLUG
    url: https://${CI_PROJECT_TITLE}-${VA_ENV}-${CI_COMMIT_REF_SLUG}-${OKTETO_USERNAME}.cloud.okteto.net
    on_stop: stop-review
  rules:
    - !reference [.code_rules, rules]
    - !reference [.tests_rules, rules]

e2e:
  stage: review
  image: $IMAGE_CYPRESS
  needs: ["deploy-review"]
  variables:
    APP: $CI_COMMIT_REF_SLUG
    VA_ENV: review
  script:
    - npm ci
    - npx cypress run --env baseUrl=https://${CI_PROJECT_TITLE}-${VA_ENV}-${CI_COMMIT_REF_SLUG}-${OKTETO_USERNAME}.cloud.okteto.net --config-file cypress/e2e.config.ts --browser chrome --reporter junit --reporter-options "mochaFile=results/test-e2e-[hash].xml"
  artifacts:
    when: always
    paths:
      - results/test-e2e-*.xml
      - cypress/videos/**/*.mp4
      - cypress/screenshots/**/*.png
    reports:
      junit: results/test-e2e-*.xml
    expire_in: 1 day
  rules:
    - !reference [.code_rules, rules]
    - !reference [.tests_rules, rules]

stop-review:
  stage: review
  image: $IMAGE_OKTETO
  needs: ["e2e"]
  variables:
    APP: $CI_COMMIT_REF_SLUG
    VA_ENV: review
  environment:
    name: review/$CI_COMMIT_REF_SLUG
    action: stop
  script:
    - okteto preview destroy $VA_ENV-$CI_COMMIT_REF_SLUG-$OKTETO_USERNAME
  rules:
    - !reference [.code_rules_stop, rules]
    - !reference [.tests_rules_stop, rules]

build-docker:
  stage: build
  image: $IMAGE_DOCKER
  services:
    - docker:dind
  variables:
    REGISTRY_HOST: docker.io
    DOCKER_HUB_IMAGE: app-okteto
    REGISTRY_IMAGE: index.docker.io/danieljesussp/app-okteto
  script:
    - docker login -u "$DOCKER_HUB_USER" -p "$DOCKER_HUB_PASSWORD" $REGISTRY_HOST
    - >
      docker build
      --no-cache
      --tag $REGISTRY_IMAGE:$CI_COMMIT_SHA
      .
    - docker push $REGISTRY_IMAGE:$CI_COMMIT_SHA
  rules:
    - !reference [.devops_rules, rules]

build-kaniko:
  stage: build
  image:
    name: $IMAGE_KANIKO
    entrypoint: [""]
  variables:
    DOCKER_HUB_REGISTRY: registry.gitlab.com
  script:
    - /kaniko/executor
      --context ${CI_PROJECT_DIR}
      --dockerfile ${CI_PROJECT_DIR}/Dockerfile
      --destination ${DOCKER_HUB_REGISTRY}/${CI_PROJECT_PATH}:${CI_COMMIT_SHA}
  rules:
    - !reference [.devops_rules, rules]

container_scanning:
  stage: build
  image:
    name: $IMAGE_TRIVY
    entrypoint: [""]
  needs:
    - job: build-kaniko
  variables:
    DOCKER_HUB_REGISTRY: registry.gitlab.com
    FULL_IMAGE_NAME: ${DOCKER_HUB_REGISTRY}/${CI_PROJECT_PATH}:${CI_COMMIT_SHA}
  script:
    - trivy --version
    - trivy image --clear-cache
    - trivy image --exit-code 0 --cache-dir .trivycache/ --no-progress --security-checks vuln "$FULL_IMAGE_NAME"
    - trivy image --exit-code 0 --cache-dir .trivycache/ --severity CRITICAL --no-progress --security-checks vuln "$FULL_IMAGE_NAME"
  cache:
    paths:
      - .trivycache/
  artifacts:
    expire_in: 1 week
    name: "gl-container-scanning-report-${CI_BUILD_ID}-${CI_COMMIT_SHA}.json"
    paths:
      - ${CI_PROJECT_DIR}
  rules:
    - !reference [.devops_rules, rules]

checks-license:
  stage: build
  image:
    name: $IMAGE_TRIVY
    entrypoint: [""]
  needs:
    - job: build-kaniko
  variables:
    DOCKER_HUB_REGISTRY: registry.gitlab.com
    FULL_IMAGE_NAME: ${DOCKER_HUB_REGISTRY}/${CI_PROJECT_PATH}:latest
  script:
    - trivy --version
    - trivy image --clear-cache
    - trivy image --security-checks license --severity UNKNOWN,HIGH,CRITICAL --license-full "$FULL_IMAGE_NAME"
  cache:
    paths:
      - .trivycache/
  artifacts:
    expire_in: 1 week
    name: "gl-license-scanning-report-${CI_BUILD_ID}-${CI_COMMIT_SHA}.json"
    paths:
      - ${CI_PROJECT_DIR}
  rules:
    - !reference [.devops_rules, rules]

deploy-staging:
  stage: staging
  extends: .deploy
  variables:
    VA_ENV: staging
  environment:
    name: staging
    url: https://${CI_PROJECT_TITLE}-${VA_ENV}-${OKTETO_USERNAME}.cloud.okteto.net
  rules:
    - !reference [.devops_rules, rules]

uninstall-staging:
  stage: staging
  needs: ["deploy-staging"]
  image:
    name: $IMAGE_HELM
    entrypoint: [""]
  variables:
    VA_ENV: staging
  environment:
    name: staging
    url: https://${CI_PROJECT_TITLE}-${VA_ENV}-${OKTETO_USERNAME}.cloud.okteto.net
  script:
    - export KUBECONFIG=${ENV_KUBECONFIG}:${KUBECONFIG:-$HOME/.kube/config}
    - helm uninstall $CI_PROJECT_NAME
      --namespace ${VA_ENV}-${OKTETO_USERNAME}
      --wait
  rules:
    - !reference [.devops_rules_stop, rules]

deploy-production:
  stage: production
  extends: .deploy
  variables:
    VA_ENV: production
  environment:
    name: production
    url: https://${CI_PROJECT_TITLE}-${VA_ENV}-${OKTETO_USERNAME}.cloud.okteto.net
  rules:
    - !reference [.production_rules, rules]

Ahora vamos a explicar los cambios que hemos introducido en este nuevo labs.

integration-testing:
  stage: test
  image: $IMAGE_CYPRESS
  script:
    - npm ci
    - npm run integration-testing &
    - npx cypress run --env baseUrl=http://localhost:3000 --config-file cypress/integration-testing.config.ts --browser chrome --reporter junit --reporter-options "mochaFile=results/test-integration-[hash].xml"
  artifacts:
    when: always
    paths:
      - results/test-integration-*.xml
      - cypress/videos/**/*.mp4
      - cypress/screenshots/**/*.png
    reports:
      junit: results/test-integration-*.xml
    expire_in: 1 day
  rules:
    - !reference [.tests_rules, rules]
    - !reference [.devops_rules, rules]

Lo primero es que este job se desarrollara en el próximo y seguramente último laboratorio, donde integraremos test unitarios. Pero nos vale para explicar la esencia de este job, el cual el objetivo mismo será realizar test unitarios.

type-detection:
  stage: scanning
  image:
    name: $IMAGE_TRIVY
    entrypoint: [""]
  script:
    - trivy conf --severity HIGH,CRITICAL .
  cache:
    paths:
      - .trivycache/
  artifacts:
    expire_in: 1 week
    name: "gl-type-detection-report-${CI_BUILD_ID}-${CI_COMMIT_SHA}.json"
    paths:
      - ${CI_PROJECT_DIR}
  rules:
    - !reference [.code_rules, rules]
    - !reference [.devops_rules, rules]

Ahora hemos agregado otro job para detectar contraseñas o token, esta solución es parecida a GitGuardian, con la diferencia de que esta solución está basada en Trivy y es totalmente gratuita.

e2e:
  stage: review
  image: $IMAGE_CYPRESS
  needs: ["deploy-review"]
  variables:
    APP: $CI_COMMIT_REF_SLUG
    VA_ENV: review
  script:
    - npm ci
    - npx cypress run --env baseUrl=https://${CI_PROJECT_TITLE}-${VA_ENV}-${CI_COMMIT_REF_SLUG}-${OKTETO_USERNAME}.cloud.okteto.net --config-file cypress/e2e.config.ts --browser chrome --reporter junit --reporter-options "mochaFile=results/test-e2e-[hash].xml"
  artifacts:
    when: always
    paths:
      - results/test-e2e-*.xml
      - cypress/videos/**/*.mp4
      - cypress/screenshots/**/*.png
    reports:
      junit: results/test-e2e-*.xml
    expire_in: 1 day
  rules:
    - !reference [.code_rules, rules]
    - !reference [.tests_rules, rules]

Este job se ejecuta justo después de desplegar nuestra aplicación y nos permite probar nuestra aplicación sin necesidades de ejecutar pruebas manuales. Tal como funciona el pipeline actualmente, vinculamos el reporte en Gitlab CI y guardamos evidencias y videos en el pipeline.

checks-license:
  stage: build
  image:
    name: $IMAGE_TRIVY
    entrypoint: [""]
  needs:
    - job: build-kaniko
  variables:
    DOCKER_HUB_REGISTRY: registry.gitlab.com
    FULL_IMAGE_NAME: ${DOCKER_HUB_REGISTRY}/${CI_PROJECT_PATH}:latest
  script:
    - trivy --version
    - trivy image --clear-cache
    - trivy image --security-checks license --severity UNKNOWN,HIGH,CRITICAL --license-full "$FULL_IMAGE_NAME"
  cache:
    paths:
      - .trivycache/
  artifacts:
    expire_in: 1 week
    name: "gl-license-scanning-report-${CI_BUILD_ID}-${CI_COMMIT_SHA}.json"
    paths:
      - ${CI_PROJECT_DIR}
  rules:
    - !reference [.devops_rules, rules]

Este job lo considero bastante interesante y creo que cada vez existe una mayor necesidades por parte de las empresas de saber qué tipo de licenciamiento tiene sus productos de software. En este paso, analizamos y generamos un reporte de tipos de licenciamiento que tiene nuestra imagen.

deploy-staging:
  stage: staging
  extends: .deploy
  variables:
    VA_ENV: staging
  environment:
    name: staging
    url: https://${CI_PROJECT_TITLE}-${VA_ENV}-${OKTETO_USERNAME}.cloud.okteto.net
  rules:
    - !reference [.devops_rules, rules]

Ahora hablaremos de unos de los cambios en el flujo del CI. En este lab, hemos creado el entorno de Staging, para ello hemos establecido un flujo de trabajo. Cada programamos creará una rama de trabajo con la tarea que vaya a desarrollar. Una vez finalizada la tarea, se abrirá una pull request de la rama de su tarea a la rama devops. Una vez que tenemos todo preparado para ir a producción, abriremos otra pull request de la rama devops a main. Cuando se mergea a main, es cuando se realizara el despliegue a producción.

DEV Community

Desplegar app en Okteto con Gitlab CI III

Top comments (0)

Read next

AWS DevOps — Establishing IAM for Finance Controller Role with Billing Access

How I Conquered the AWS Certified DevOps Engineer Professional Exam in 60 Days

Setup NGINX

Docker networking, how to connect different containers