I've used CircleCI a fair bit over the past couple of years - I love how you can run the tests locally before you push - it's a real safety net (although it still doesn't guarantee I didn't do something silly).
Because my builds tend to be Docker Compose stacks, there's usually a bit of service ordering to do - or more precisely, service waiting. You can't install into a database that isn't ready yet.
Jason Wilder's Dockerize has always been the go-to for ensuring that correct order is restored.
My first exposure to this involved including dockerize into a docker image used specifically to build out the tests - it can then be easily called via docker-compose
- run: name: check if mariadb is responding command: docker-compose exec test dockerize -wait tcp://mariadb:3306 -timeout 1
This is a great first step, but surely we shouldn't need to add a binary to an image so we can check if a different container is up and running?
In CircleCI, the most common pattern is actually to install it into the CircleCI environment every time
- run: name: Install dockerize command: wget https://github.com/jwilder/dockerize/releases/download/$DOCKERIZE_VERSION/dockerize-linux-amd64-$DOCKERIZE_VERSION.tar.gz && tar -C /usr/local/bin -xzvf dockerize-linux-amd64-$DOCKERIZE_VERSION.tar.gz environment: DOCKERIZE_VERSION: v0.6.1 - run: name: Wait for db command: dockerize -wait tcp://HOST:PORT -timeout 1m
This works (why else would hundreds of sites use it?), and indeed, now CircleCI is onto it, and now includes dockerize in their provided base images alongside docker and docker-compose - nifty huh!
- run: name: Wait for db command: dockerize -wait tcp://HOST:PORT -timeout 1m
However, both these CircleCI based solutions have one catch when used with docker-compose - trying to get to the database endpoint to check it - because we use
setup_remote_docker, it's abstracted away from the container building the images.
In my case, the mariadb exposes its 3306 port to a 32xxx one, but only on the bridge network created by docker-compose. We can still use this to get to the port, but it's complex to achieve, even locally (I've installed dockerize here from homebrew to demo).
toby@pop-os:~/sites/example$ docker-compose port mariadb 3306 0.0.0.0:32781 toby@pop-os:~/sites/example$ docker network inspect example_default | grep Gateway "Gateway": "192.168.144.1" toby@pop-os:~/sites/example$ dockerize -wait tcp://192.168.144.1:32781 -timeout 1m 2020/05/31 02:43:15 Waiting for: tcp://192.168.144.1:32781 2020/05/31 02:43:15 Connected to tcp://192.168.144.1:32781 2020/05/31 02:43:15 Command finished successfully.
Victory - now all we need to do is artisan code some bash script to retrieve the port, find the network gateway and craft a command to dockerize, make a cup of tea, and sit back in satisfaction (which will probably last until you realise that CircleCI generates a random name for the network it creates...)
But wait - surely there's a better, more Docker-ish way?
There's three key pieces of information to this puzzle:
- Docker containers can be used to run single purpose applications (d'oh)
- Those standalone containers can be joined to a named docker-compose network
- You can control the name of CircleCI's (and docker-compose) default network - using
COMPOSE_PROJECT_NAMEas an environment variable
So, if we know what the network is called, surely we can join a container to that network, run dockerize (or something else...) inside that network, then dispose of it?
We make sure to add the COMPOSE_PROJECT_NAME env var to the CircleCI config (this has the benefit of naming your stack logically - which also makes it easier to control the stack independently)
docker-compose: docker: - image: circleci/php:cli user: root environment: COMPOSE_PROJECT_NAME: circleci ... - run: name: Check mariadb is responsive with dockerize command: docker run --rm --net circleci_default jwilder/dockerize dockerize -wait tcp://mariadb:3306 -timeout 1m
Dockerize is available on dockerhub as an image - so we can run it (and remove afterwards), attach it to the predictably named network, and use the docker-compose service name and internal port.
toby@pop-os:~/sites/example$ docker run --rm --net example_default jwilder/dockerize dockerize -wait tcp://mariadb:3306 -timeout 1m 2020/05/31 03:54:58 Waiting for: tcp://mariadb:3306 2020/05/31 03:54:58 Problem with dial: dial tcp: lookup mariadb on 127.0.0.11:53: no such host. Sleeping 1s 2020/05/31 03:54:59 Problem with dial: dial tcp: lookup mariadb on 127.0.0.11:53: no such host. Sleeping 1s 2020/05/31 03:55:00 Problem with dial: dial tcp: lookup mariadb on 127.0.0.11:53: no such host. Sleeping 1s 2020/05/31 03:55:01 Connected to tcp://mariadb:3306 2020/05/31 03:55:01 Command finished successfully.
There is another image out there that does a similar thing - dokku/wait - their documentation recommends the use of
--link to join it to running containers, but given --link is possibly being deprecated, we can use the same
--net strategy as before:
toby@pop-os:~/sites/example-$ docker run --net example_default --rm dokku/wait -c mariadb:3306 Waiting for mariadb:3306 .nc: getaddrinfo: Name does not resolve .nc: getaddrinfo: Name does not resolve .nc: getaddrinfo: Name does not resolve .nc: getaddrinfo: Name does not resolve .. up! Everything is up
Both these packages allow for multiple endpoints to be checked - so for more complex stacks, you can run multiple stages if needed.
I personally really like using the COMPOSE_PROJECT_NAME in my CircleCI projects - given that I like to run them up locally, I also want to be able to control and view them once running - you can use the
-p switch to docker-compose to specify a different project to the folder you're currently in - e.g. to clear it all away once you're done!
toby@pop-os:~/sites/example$ docker-compose -p circleci down -v --remove-orphans Stopping circleci_php_1 ... done Stopping circleci_mariadb_1 ... done Removing circleci_php_1 ... done Removing circleci_mariadb_1 ... done Removing network circleci_default Removing volume circleci_files
Hopefully you've found this useful - please let me know if I've missed any obvious options (I've deliberately not veered into log tailing here!)