Use Case
I want to run multiple/long commands for my service(s) in my docker-compose file, like below:
---
# docker-compose.yaml
services:
server: &server
build:
context: .
dockerfile: ./compose/staging/server/Dockerfile
image: someproject_slug_staging_server:staging_shortsha_unixtime
depends_on:
- postgres
env_file:
- ./.envs/.staging/.server
- ./.envs/.staging/.postgres
command: python /app/manage.py collectstatic --no-input
command: python /app/manage.py migrate
command: gunicorn config.wsgi --bind 0.0.0.0:5000 --chdir=/app
postgres:
build:
context: .
dockerfile: ./compose/staging/postgres/Dockerfile
image: someproject_slug_staging_postgres:staging_shortsha_unixtime
volumes:
- staging_postgres_data:/var/lib/postgresql/data:Z
- staging_postgres_data_backups:/backups:z
env_file:
- ./.envs/.staging/.postgres
However it won't work since it's not possible.
Solutions
Script File
One solution is that putting all your commands in a script file -let's say start.sh, put it into your image, build with it and call it directly.
As an example here is a docker compose file:
---
# staging.yaml
version: '3.7'
volumes:
staging_postgres_data: {}
staging_postgres_data_backups: {}
staging_traefik: {}
staging_redis: {}
services:
server: &server
build:
context: .
dockerfile: ./compose/staging/server/Dockerfile
image: someproject_slug_server:staging_shortsha_unixtime
depends_on:
- postgres
- redis
env_file:
- ./.envs/.staging/.server
- ./.envs/.staging/.postgres
- ./.envs/.staging/.redis
command: /start
postgres:
build:
context: .
dockerfile: ./compose/staging/postgres/Dockerfile
image: someproject_slug_postgres:staging_shortsha_unixtime
volumes:
- staging_postgres_data:/var/lib/postgresql/data:Z
- staging_postgres_data_backups:/backups:z
env_file:
- ./.envs/.staging/.postgres
traefik:
build:
context: .
dockerfile: ./compose/staging/traefik/Dockerfile
image: someproject_slug_traefik:staging_shortsha_unixtime
depends_on:
- server
volumes:
- staging_traefik:/etc/traefik/acme:z
ports:
- "0.0.0.0:80:80"
- "0.0.0.0:443:443"
- "0.0.0.0:5555:5555"
redis:
image: docker.io/bitnami/redis:6.2
env_file:
- ./.envs/.staging/.redis
ports:
- "6379:6379" # WARNING: In production never bind 0.0.0.0 wildcard adress of the host
volumes:
- staging_redis:/bitnami/redis/data:z
celeryworker:
<<: *server
command: /start-celeryworker
celerybeat:
<<: *server
command: /start-celerybeat
celeryflower:
<<: *server
command: /start-celeryflower
aws-backup:
build:
context: .
dockerfile: ./compose/staging/aws/Dockerfile
env_file:
- ./.envs/.staging/.server
volumes:
- staging_postgres_data_backups:/backups:z
WARNING
Don't bind your host's
0.0.0.0wildcard adress in production environment. (Actually instead of deploying viadocker-composein production you should consider to use more reliable, and robust distributed systems likeKubernetesbut this is a topic for another post.)
6379:6379means0.0.0.0:6379:6379, so you open your redis instance to whole open world. And if you don't useAUTHor don't take required security precaution -like TLS etc. you may get some headache. So use it only if you
know the related consequences.For more redis security topic look at this: Redis Security
And as an example here is content of start from server service:
#!/usr/env/bin bash
# start
set -o errexit
set -o pipefail
set -o nounset
python /app/manage.py collectstatic --noinput
python /app/manage.py migrate # WARNING: Not a good idea for prod
gunicorn config.wsgi --bind 0.0.0.0:5000 --chdir=/app
WARNING
Migration is not a command you want to run every container start but since it's staging environment let's assume it's OK.
Multiline Scalar
Running in a script file is OK however maybe it's local or staging environment which means you need to debug and tweak these commands one by one; maybe some flags are not working etc. and you don't want to re-build the image every time.
How are we gonna do this?
The answer lines in the multiline capability of yaml syntax:
command: >
bash -c "python /app/manage.py collectstatic --noinput
&& python /app/manage.py migrate
&& gunicorn config.wsgi --bind 0.0.0.0:5000 --chdir=/app"
This is called Folded Style. The indention in each line will be ignored. A line break (\n) will be inserted at the end.
Or you can use Literal Style:
command:
- /bin/bash
- -c
- |
python /app/manage.py collectstatic --noinput
python /app/manage.py migrate
gunicorn config.wsgi --bind 0.0.0.0:5000 --chdir=/app
This turns every newline within the string into a literal newline, and adds one at the end.
More on these styles:
Folded Style
Key: >
this is my very very very
long command
Output:
this is my very very very long command\n
Mind new line-\n (end of line-EOL)
Literal Style
Key: |
this is my very very very
long command
Output:
this is my very very very\nlong command\n
Mind new lines-\n, and compare to it above output.
Fore more detailed style and yaml syntax you can look at this amazing explanation in this link.
These were the solutions for running multiple or long commands in your docker-compose file.
All done!
Top comments (0)