Why Bitbucket Pipelines is Slow and How to Optimize

Bitbucket Pipelines can be slow due to poor CI infrastructure performance and the time-consuming process of installing dependencies for each build. Here's how to address these issues:

Performance Issues:

• Infrastructure Delays: Delays in provisioning and build setup can occur, with performance inconsistencies noted in monitoring projects.
• Dependency Installation: Installing libraries, tools, packages, and third-party dependencies each time consumes significant build time.

Optimization Approach:

Pre-configure Docker Image:

1. Dockerfile Creation: Move dependency installation commands to a Dockerfile.
2. Build and Push Image: Use Docker Buildx to create an image and push it to Docker Hub.
3. Update Pipeline: Modify bitbucket-pipelines.yml to use the new Docker image, eliminating repetitive setup steps.

Example Dockerfile:

FROM python:3.11.7-slim-bullseye
RUN apt-get update
RUN apt-get install -y build-essential postgresql postgresql-client curl
RUN curl https://dl.min.io/client/mc/release/linux-amd64/mc --create-dirs -o ~/minio-binaries/mc
RUN chmod +x ~/minio-binaries/mc
COPY requirements.txt .
RUN pip install --ignore-installed -r requirements.txt

Example Updated bitbucket-pipelines.yml:

image: dntl/1l2c-backend-for-tests:latest
pipelines:
  default:
    - step:
        script:
          - export PATH=$PATH:~/minio-binaries/
          - mc alias set testing-minio http://localhost:9000 root root-pass
          - mc admin user svcacct add --access-key "test_access_key" --secret-key "test_secret_key" testing-minio root
          - mc mb testing-minio/dean-testing
          - PGPASSWORD=root psql -h localhost -p 5432 -U root postgres -c 'create database dean_unit_tests_db;'
          - python -m pytest . --test_application_name=dean --junitxml=./test-reports/dean_out_report.xml
        services:
          - postgres
          - elasticsearch
          - minio

Results:

Setup Time Reduction: Dependencies are pre-configured, reducing setup time from several minutes to less than 10 seconds.

Overall Efficiency: Test execution time decreased significantly, saving valuable build minutes.

Conclusion:

By pre-configuring a Docker image, you can optimize Bitbucket Pipelines, ensuring faster and more reliable builds. This approach minimizes dependency on CI infrastructure performance and reduces the total build time.

For more detailed information, visit the full article here.