Level Up Your Dev Workflow: Branch-Specific Data Volumes with Docker Compose

Imagine this: you're working on a new feature and you're deep into it, only to be suddenly interrupted by a critical bug in production. You want to switch to a new branch to work on the bug fix, but your local database is still a work-in-progress with changes from your feature branch, hindering you from being able to test any changes for the bug fix.

Yikes! If only there was a way to do this without pulling your hair out 🤔

One approach to solve this problem is using branch-specific volumes in Docker. This blog post will guide you through a script that leverages Docker Compose to spin up services with branch-specific data volumes, streamlining your development process.

With branch-specific data volumes, each branch gets its own isolated data, preventing such conflicts. No more fear of breaking your main development environment or other branches!

Benefits of Using Docker Compose for Development:

• Consistent Environments: Docker Compose ensures your development environment is consistent across different machines and setups.
• Simplified Setup: Define and manage multiple services in a single docker-compose.yml file.
• Faster Development: Quickly spin up and tear down development environments.
• Improved Collaboration: Share your development environment with ease.

The Challenge with Default Docker Compose Volume Behavior

A common limitation of Docker Compose is that a single instance of a volume, defined within a Docker Compose file, exists for the entire project. This can be problematic when working with multiple branches, as each branch might require its own isolated data.

The Solution: Dynamic Volume Creation

To overcome these limitations, we'll use a script to dynamically create and manage branch-specific data volumes. Here's how it works:

1. Detect the Current Branch and Calculate a Hash:

   CURRENT_BRANCH=$(git symbolic-ref --short HEAD)
   CURRENT_BRANCH_HASH=$(echo -n "$CURRENT_BRANCH" | gzip -1 -c | tail -c8 | hexdump -n4 -e '"%08x"')

1. Define Volume Names Based on the Branch or Hash:

   if [[ "$CURRENT_BRANCH" == "master" ]]; then
       export MYSQL_VOLUME="$APP_NAME-mysql-master-data"
       export REDIS_VOLUME="$APP_NAME-redis-master-data"
       # ... other volumes
   else
       export MYSQL_VOLUME="$APP_NAME-mysql-$CURRENT_BRANCH_HASH-data"
       export REDIS_VOLUME="$APP_NAME-redis-$CURRENT_BRANCH_HASH-data"
       # ... other volumes
   fi

   # Note that the '$APP_NAME' variable has been defined earlier
   # which is basically the project name for the docker compose manifest.

1. Create Volumes and Copy Data from master if Necessary:

   VOLUMES=(MYSQL_VOLUME REDIS_VOLUME ...)

   for VOLUME in "${VOLUMES[@]}"; do
       if [[ -z "$(docker volume ls -q -f name="${!VOLUME}")" ]]; then
           echo "Creating volume: ${!VOLUME}..."
           docker volume create "${!VOLUME}" > /dev/null

           if [[ "$CURRENT_BRANCH" != "master" ]]; then
               MASTER_VOLUME=$(echo -n "${!VOLUME}" | sed -E "s/(.*?)-$CURRENT_BRANCH_HASH-(.*?)/\1-master-\2/g")

               if [[ -z "$(docker volume ls -q -f name="${MASTER_VOLUME}")" ]]; then
                   echo "Master volume not found: $MASTER_VOLUME"
                   docker volume rm "${!VOLUME}" > /dev/null
                   exit 1
               fi

               echo "Copying data from master volume..."
               docker run --rm -v "$MASTER_VOLUME:/from" -v "${!VOLUME}:/to" busybox sh -c "cd /from; cp -a . /to" > /dev/null
           fi
       fi
   done

1. Start Docker Compose: You can use the defined volume names as environment variables in your docker-compose.yml file:

   services:
       mysql:
           image: mysql:latest
           volumes:
               - mysql-data:/var/lib/mysql
           environment:
               MYSQL_ROOT_PASSWORD: root
               MYSQL_DATABASE: my_database
       redis:
           image: redis:latest
           volumes:
               - redis-data:/data
       # ... other services
   volumes:
       mysql-data:
           name: ${MYSQL_VOLUME}
           external: true
       redis-data:
           name: ${REDIS_VOLUME}
           external: true
       # ... other volumes

By dynamically creating volumes based on the branch hash, we ensure that each branch has its own isolated data, even if the branch name contains invalid characters for Docker volume names.

Level Up Your Workflow

By incorporating this script and Docker Compose into your development process, you'll enjoy a smoother, more efficient, and less stressful development experience.

For a full working example, check out this GitHub repository: https://github.com/sikhlana/laravel-template/blob/master/bin/run