Continuamos con la tercera parte de esta serie de tutoriales de Gitlab CI. En este tutorial nos enfocaremos en realizar un refactor para cambiar el flujo de los pipelines y añadir unos nuevos jobs y steps.
Aquí os dejo los objetivos que abordaremos:
- Añadimos un job para ejecutar test de integración, “en este tutorial solo usaremos unos test de Cypress, en un futuro añadiremos unos test unitarios”.
- Integrar Cypress para validar test e2e.
- Analizaremos con Trivy el contenido del repositorio para detectar posibles usuarios y contraseñas
- Analizaremos la imagen con Trivy para detectar el tipo de licencia que dispone nuestro software.
- Redefinir flujo de CI, se añade el entorno de Staging.
Como siempre aquí os dejo el repositorio que utilizaremos.
Daniel J. Saldaña / app-okteto · GitLab
Recordamos que el tutorial lo podrá encontrar en la rama main, ya que verá que actualmente existen más ramas, las cuales están enfocadas en los siguientes labs.
stages:
- test
- scanning
- review
- build
- staging
- production
variables:
IMAGE_NODE: node:latest
IMAGE_GITGUARDIAN: gitguardian/ggshield:latest
IMAGE_SONAR: sonarsource/sonar-scanner-cli:latest
IMAGE_OKTETO: okteto/okteto:1.13.4
IMAGE_CYPRESS: cypress/browsers:node18.12.0-chrome107
IMAGE_DOCKER: docker:stable
IMAGE_KANIKO: gcr.io/kaniko-project/executor:v1.9.0-debug
IMAGE_TRIVY: docker.io/aquasec/trivy:latest
IMAGE_HELM: alpine/helm
RELEASE:
cache:
key: ${CI_COMMIT_REF_SLUG}
paths:
- node_modules/
- .npm/
.charts_rules:
rules:
- if: $CI_COMMIT_BRANCH && $CI_OPEN_MERGE_REQUESTS
when: never
- if: $CI_COMMIT_BRANCH != 'main'
when: never
- if: $CI_MERGE_REQUEST_TARGET_BRANCH_NAME == 'devops'
when: always
- if: $CI_COMMIT_BRANCH != 'devops'
changes:
- charts/*
when: always
.code_rules:
rules:
- if: $CI_COMMIT_BRANCH && $CI_OPEN_MERGE_REQUESTS
when: never
- if: $CI_COMMIT_BRANCH != 'main'
when: never
- if: $CI_MERGE_REQUEST_TARGET_BRANCH_NAME == 'devops'
when: always
- if: $CI_COMMIT_BRANCH != 'devops'
changes:
- src/*
- index.html
- package*
- tsconfig*
- vite.config.ts
- Dockerfile
when: always
.code_rules_stop:
rules:
- if: $CI_COMMIT_BRANCH && $CI_OPEN_MERGE_REQUESTS
when: never
- if: $CI_COMMIT_BRANCH != 'main'
when: never
- if: $CI_COMMIT_BRANCH != 'devops'
changes:
- src/*
- index.html
- package*
- tsconfig*
- vite.config.ts
- Dockerfile
when: delayed
start_in: 30 minutes
- if: $CI_MERGE_REQUEST_TARGET_BRANCH_NAME == 'devops'
when: always
.tests_rules:
rules:
- if: $CI_COMMIT_BRANCH && $CI_OPEN_MERGE_REQUESTS
when: never
- if: $CI_COMMIT_BRANCH != 'main'
when: never
- if: $CI_MERGE_REQUEST_TARGET_BRANCH_NAME == 'devops'
when: always
- if: $CI_COMMIT_BRANCH != 'devops'
changes:
- cypress/*
when: always
.tests_rules_stop:
rules:
- if: $CI_COMMIT_BRANCH && $CI_OPEN_MERGE_REQUESTS
when: never
- if: $CI_COMMIT_BRANCH != 'main'
when: never
- if: $CI_COMMIT_BRANCH != 'devops'
changes:
- cypress/*
when: delayed
start_in: 30 minutes
- if: $CI_MERGE_REQUEST_TARGET_BRANCH_NAME == 'devops'
when: always
.devops_rules:
rules:
- if: $CI_COMMIT_BRANCH == 'devops' && $RELEASE == 'yes'
when: always
- if: $CI_COMMIT_BRANCH != 'main'
when: never
- if: $CI_MERGE_REQUEST_TARGET_BRANCH_NAME == 'main'
when: always
.devops_rules_stop:
rules:
- if: $CI_COMMIT_BRANCH == 'devops' && $RELEASE == 'yes'
when: delayed
start_in: 30 minutes
- if: $CI_COMMIT_BRANCH != 'main'
when: never
- if: $CI_MERGE_REQUEST_TARGET_BRANCH_NAME == 'main'
when: always
.go_to_production_rules:
rules:
- if: $CI_MERGE_REQUEST_TARGET_BRANCH_NAME == 'main'
.production_rules:
rules:
- if: $CI_COMMIT_BRANCH == 'main'
.deploy:
image:
name: $IMAGE_HELM
entrypoint: [""]
script:
- export KUBECONFIG=${ENV_KUBECONFIG}:${KUBECONFIG:-$HOME/.kube/config}
- sed -i 's/_CI_PROJECT_TITLE/'"${CI_PROJECT_TITLE}"'/' "charts/templates/NOTES.txt"
- sed -i 's/_VA_ENV/'""'/' "charts/templates/NOTES.txt"
- sed -i 's/_CI_COMMIT_REF_SLUG/'""'/' "charts/templates/NOTES.txt"
- sed -i 's/_OKTETO_USERNAME/'"-${OKTETO_USERNAME}"'/' "charts/templates/NOTES.txt"
- helm upgrade $CI_PROJECT_NAME ./charts
--values=./charts/values.yaml
--namespace ${VA_ENV}-${OKTETO_USERNAME}
--install --atomic
kubelinter:
stage: test
image: $IMAGE_DOCKER
services:
- docker:dind
script:
- docker run -v $PWD/charts:/dir -v $PWD/lint/config.yaml:/etc/config.yaml stackrox/kube-linter lint /dir --config /etc/config.yaml
rules:
- !reference [.charts_rules, rules]
- !reference [.devops_rules, rules]
sonarcloud-check:
stage: test
image:
name: $IMAGE_SONAR
variables:
SONAR_USER_HOME: "${CI_PROJECT_DIR}/.sonar"
GIT_DEPTH: "0"
cache:
key: "${CI_JOB_NAME}"
paths:
- .sonar/cache
script:
- sonar-scanner
rules:
- !reference [.code_rules, rules]
- !reference [.devops_rules, rules]
integration-testing:
stage: test
image: $IMAGE_CYPRESS
script:
- npm ci
- npm run integration-testing &
- npx cypress run --env baseUrl=http://localhost:3000 --config-file cypress/integration-testing.config.ts --browser chrome --reporter junit --reporter-options "mochaFile=results/test-integration-[hash].xml"
artifacts:
when: always
paths:
- results/test-integration-*.xml
- cypress/videos/**/*.mp4
- cypress/screenshots/**/*.png
reports:
junit: results/test-integration-*.xml
expire_in: 1 day
rules:
- !reference [.tests_rules, rules]
- !reference [.devops_rules, rules]
gitguardian:
stage: scanning
image:
name: $IMAGE_GITGUARDIAN
script:
- ggshield secret scan ci
rules:
- !reference [.code_rules, rules]
- !reference [.tests_rules, rules]
- !reference [.devops_rules, rules]
type-detection:
stage: scanning
image:
name: $IMAGE_TRIVY
entrypoint: [""]
script:
- trivy conf --severity HIGH,CRITICAL .
cache:
paths:
- .trivycache/
artifacts:
expire_in: 1 week
name: "gl-type-detection-report-${CI_BUILD_ID}-${CI_COMMIT_SHA}.json"
paths:
- ${CI_PROJECT_DIR}
rules:
- !reference [.code_rules, rules]
- !reference [.devops_rules, rules]
deploy-review:
stage: review
image: $IMAGE_OKTETO
variables:
APP: $CI_COMMIT_REF_SLUG
VA_ENV: review
script:
- okteto preview deploy $VA_ENV-$CI_COMMIT_REF_SLUG-$OKTETO_USERNAME --scope personal --branch $CI_COMMIT_REF_NAME --repository $CI_REPOSITORY_URL --wait
environment:
name: review/$CI_COMMIT_REF_SLUG
url: https://${CI_PROJECT_TITLE}-${VA_ENV}-${CI_COMMIT_REF_SLUG}-${OKTETO_USERNAME}.cloud.okteto.net
on_stop: stop-review
rules:
- !reference [.code_rules, rules]
- !reference [.tests_rules, rules]
e2e:
stage: review
image: $IMAGE_CYPRESS
needs: ["deploy-review"]
variables:
APP: $CI_COMMIT_REF_SLUG
VA_ENV: review
script:
- npm ci
- npx cypress run --env baseUrl=https://${CI_PROJECT_TITLE}-${VA_ENV}-${CI_COMMIT_REF_SLUG}-${OKTETO_USERNAME}.cloud.okteto.net --config-file cypress/e2e.config.ts --browser chrome --reporter junit --reporter-options "mochaFile=results/test-e2e-[hash].xml"
artifacts:
when: always
paths:
- results/test-e2e-*.xml
- cypress/videos/**/*.mp4
- cypress/screenshots/**/*.png
reports:
junit: results/test-e2e-*.xml
expire_in: 1 day
rules:
- !reference [.code_rules, rules]
- !reference [.tests_rules, rules]
stop-review:
stage: review
image: $IMAGE_OKTETO
needs: ["e2e"]
variables:
APP: $CI_COMMIT_REF_SLUG
VA_ENV: review
environment:
name: review/$CI_COMMIT_REF_SLUG
action: stop
script:
- okteto preview destroy $VA_ENV-$CI_COMMIT_REF_SLUG-$OKTETO_USERNAME
rules:
- !reference [.code_rules_stop, rules]
- !reference [.tests_rules_stop, rules]
build-docker:
stage: build
image: $IMAGE_DOCKER
services:
- docker:dind
variables:
REGISTRY_HOST: docker.io
DOCKER_HUB_IMAGE: app-okteto
REGISTRY_IMAGE: index.docker.io/danieljesussp/app-okteto
script:
- docker login -u "$DOCKER_HUB_USER" -p "$DOCKER_HUB_PASSWORD" $REGISTRY_HOST
- >
docker build
--no-cache
--tag $REGISTRY_IMAGE:$CI_COMMIT_SHA
.
- docker push $REGISTRY_IMAGE:$CI_COMMIT_SHA
rules:
- !reference [.devops_rules, rules]
build-kaniko:
stage: build
image:
name: $IMAGE_KANIKO
entrypoint: [""]
variables:
DOCKER_HUB_REGISTRY: registry.gitlab.com
script:
- /kaniko/executor
--context ${CI_PROJECT_DIR}
--dockerfile ${CI_PROJECT_DIR}/Dockerfile
--destination ${DOCKER_HUB_REGISTRY}/${CI_PROJECT_PATH}:${CI_COMMIT_SHA}
rules:
- !reference [.devops_rules, rules]
container_scanning:
stage: build
image:
name: $IMAGE_TRIVY
entrypoint: [""]
needs:
- job: build-kaniko
variables:
DOCKER_HUB_REGISTRY: registry.gitlab.com
FULL_IMAGE_NAME: ${DOCKER_HUB_REGISTRY}/${CI_PROJECT_PATH}:${CI_COMMIT_SHA}
script:
- trivy --version
- trivy image --clear-cache
- trivy image --exit-code 0 --cache-dir .trivycache/ --no-progress --security-checks vuln "$FULL_IMAGE_NAME"
- trivy image --exit-code 0 --cache-dir .trivycache/ --severity CRITICAL --no-progress --security-checks vuln "$FULL_IMAGE_NAME"
cache:
paths:
- .trivycache/
artifacts:
expire_in: 1 week
name: "gl-container-scanning-report-${CI_BUILD_ID}-${CI_COMMIT_SHA}.json"
paths:
- ${CI_PROJECT_DIR}
rules:
- !reference [.devops_rules, rules]
checks-license:
stage: build
image:
name: $IMAGE_TRIVY
entrypoint: [""]
needs:
- job: build-kaniko
variables:
DOCKER_HUB_REGISTRY: registry.gitlab.com
FULL_IMAGE_NAME: ${DOCKER_HUB_REGISTRY}/${CI_PROJECT_PATH}:latest
script:
- trivy --version
- trivy image --clear-cache
- trivy image --security-checks license --severity UNKNOWN,HIGH,CRITICAL --license-full "$FULL_IMAGE_NAME"
cache:
paths:
- .trivycache/
artifacts:
expire_in: 1 week
name: "gl-license-scanning-report-${CI_BUILD_ID}-${CI_COMMIT_SHA}.json"
paths:
- ${CI_PROJECT_DIR}
rules:
- !reference [.devops_rules, rules]
deploy-staging:
stage: staging
extends: .deploy
variables:
VA_ENV: staging
environment:
name: staging
url: https://${CI_PROJECT_TITLE}-${VA_ENV}-${OKTETO_USERNAME}.cloud.okteto.net
rules:
- !reference [.devops_rules, rules]
uninstall-staging:
stage: staging
needs: ["deploy-staging"]
image:
name: $IMAGE_HELM
entrypoint: [""]
variables:
VA_ENV: staging
environment:
name: staging
url: https://${CI_PROJECT_TITLE}-${VA_ENV}-${OKTETO_USERNAME}.cloud.okteto.net
script:
- export KUBECONFIG=${ENV_KUBECONFIG}:${KUBECONFIG:-$HOME/.kube/config}
- helm uninstall $CI_PROJECT_NAME
--namespace ${VA_ENV}-${OKTETO_USERNAME}
--wait
rules:
- !reference [.devops_rules_stop, rules]
deploy-production:
stage: production
extends: .deploy
variables:
VA_ENV: production
environment:
name: production
url: https://${CI_PROJECT_TITLE}-${VA_ENV}-${OKTETO_USERNAME}.cloud.okteto.net
rules:
- !reference [.production_rules, rules]
Ahora vamos a explicar los cambios que hemos introducido en este nuevo labs.
integration-testing:
stage: test
image: $IMAGE_CYPRESS
script:
- npm ci
- npm run integration-testing &
- npx cypress run --env baseUrl=http://localhost:3000 --config-file cypress/integration-testing.config.ts --browser chrome --reporter junit --reporter-options "mochaFile=results/test-integration-[hash].xml"
artifacts:
when: always
paths:
- results/test-integration-*.xml
- cypress/videos/**/*.mp4
- cypress/screenshots/**/*.png
reports:
junit: results/test-integration-*.xml
expire_in: 1 day
rules:
- !reference [.tests_rules, rules]
- !reference [.devops_rules, rules]
Lo primero es que este job se desarrollara en el próximo y seguramente último laboratorio, donde integraremos test unitarios. Pero nos vale para explicar la esencia de este job, el cual el objetivo mismo será realizar test unitarios.
type-detection:
stage: scanning
image:
name: $IMAGE_TRIVY
entrypoint: [""]
script:
- trivy conf --severity HIGH,CRITICAL .
cache:
paths:
- .trivycache/
artifacts:
expire_in: 1 week
name: "gl-type-detection-report-${CI_BUILD_ID}-${CI_COMMIT_SHA}.json"
paths:
- ${CI_PROJECT_DIR}
rules:
- !reference [.code_rules, rules]
- !reference [.devops_rules, rules]
Ahora hemos agregado otro job para detectar contraseñas o token, esta solución es parecida a GitGuardian, con la diferencia de que esta solución está basada en Trivy y es totalmente gratuita.
e2e:
stage: review
image: $IMAGE_CYPRESS
needs: ["deploy-review"]
variables:
APP: $CI_COMMIT_REF_SLUG
VA_ENV: review
script:
- npm ci
- npx cypress run --env baseUrl=https://${CI_PROJECT_TITLE}-${VA_ENV}-${CI_COMMIT_REF_SLUG}-${OKTETO_USERNAME}.cloud.okteto.net --config-file cypress/e2e.config.ts --browser chrome --reporter junit --reporter-options "mochaFile=results/test-e2e-[hash].xml"
artifacts:
when: always
paths:
- results/test-e2e-*.xml
- cypress/videos/**/*.mp4
- cypress/screenshots/**/*.png
reports:
junit: results/test-e2e-*.xml
expire_in: 1 day
rules:
- !reference [.code_rules, rules]
- !reference [.tests_rules, rules]
Este job se ejecuta justo después de desplegar nuestra aplicación y nos permite probar nuestra aplicación sin necesidades de ejecutar pruebas manuales. Tal como funciona el pipeline actualmente, vinculamos el reporte en Gitlab CI y guardamos evidencias y videos en el pipeline.
checks-license:
stage: build
image:
name: $IMAGE_TRIVY
entrypoint: [""]
needs:
- job: build-kaniko
variables:
DOCKER_HUB_REGISTRY: registry.gitlab.com
FULL_IMAGE_NAME: ${DOCKER_HUB_REGISTRY}/${CI_PROJECT_PATH}:latest
script:
- trivy --version
- trivy image --clear-cache
- trivy image --security-checks license --severity UNKNOWN,HIGH,CRITICAL --license-full "$FULL_IMAGE_NAME"
cache:
paths:
- .trivycache/
artifacts:
expire_in: 1 week
name: "gl-license-scanning-report-${CI_BUILD_ID}-${CI_COMMIT_SHA}.json"
paths:
- ${CI_PROJECT_DIR}
rules:
- !reference [.devops_rules, rules]
Este job lo considero bastante interesante y creo que cada vez existe una mayor necesidades por parte de las empresas de saber qué tipo de licenciamiento tiene sus productos de software. En este paso, analizamos y generamos un reporte de tipos de licenciamiento que tiene nuestra imagen.
deploy-staging:
stage: staging
extends: .deploy
variables:
VA_ENV: staging
environment:
name: staging
url: https://${CI_PROJECT_TITLE}-${VA_ENV}-${OKTETO_USERNAME}.cloud.okteto.net
rules:
- !reference [.devops_rules, rules]
Ahora hablaremos de unos de los cambios en el flujo del CI. En este lab, hemos creado el entorno de Staging, para ello hemos establecido un flujo de trabajo. Cada programamos creará una rama de trabajo con la tarea que vaya a desarrollar. Una vez finalizada la tarea, se abrirá una pull request de la rama de su tarea a la rama devops. Una vez que tenemos todo preparado para ir a producción, abriremos otra pull request de la rama devops a main. Cuando se mergea a main, es cuando se realizara el despliegue a producción.
Top comments (0)