Notes
Requires Docker and at least 4 GB RAM (see comment section bellow for further details)
Not applicable for single app
Why I use Docker images
Using version managers like pyenv and nvm are obsolete.
For example, virtualenv/pyenv only encapsulates Python dependencies.
Docker containers encapsulate an entire OS with several dependencies.
Docker already allows to pull several one-click images like python, nodejs, golang, etc ...
docker pull python:3.9.0
# Dockerfile
FROM python:3.9.0
Docker images and containers
A Docker image includes the elements needed to run an application as a container -- such as code, config files, environment variables, libraries and run time.
A Docker container can be seen as a replica or as a printout of that image.
For example, you can have one python image and multiple containers related to this image.
Isolation with the OS (Clean Install)
You may not have python installed on your OS and you can pull the python 3.9 docker image.
So you can only use python inside a docker container.
docker pull python:3.9.0
You may have the python 2.7 installed on your OS and pull the python 3.9 docker image.
They will not conflict together, but you can only use python 3.9 inside a docker container.
If you don't want the python 3.9 anymore , you just have to delete the docker container related to the python 3.9 image and the python 3.9 image itself.
Multiple Versions
You can pull the python 2.7 docker image and the python 3.9 docker image.
They will not conflict together.
Multiple Layers
You can pull the python 2.9 docker image and install additional dependencies.
# Dockerfile
FROM python:3.9.0
...
COPY requirements.txt /app/requirements.txt
RUN pip install -r requirements.txt
Upgrade - Switch versions
You can pull the python 3.8 docker image and then switch to the 3.9 image.
# Dockerfile
FROM python:3.8.0 -> FROM python:3.9.0
Example : python
# Dockerfile
FROM python:3.9.0
# Dockerfile
FROM python:3.9.0-alpine
Example : node
# Dockerfile
FROM node:14.16.0
# Dockerfile
FROM node:14.16.0-alpine
Top comments (41)
I'd disagree that version managers are obsolete. I have great success with nvm and brew, with no complications. When I use Docker, a lot of RAM is used. I have a 32GB Mac and yet it still slows things down.
I do use Docker for some things successfully, but I can't imagine running all these different containers to do my day-to-day work. Wherever possible, I run things locally on my machine as close to the metal as possible.
Thanks for the post, I'm curious what your day-to-day looks like with all of these different images running?
I have also a 32 GB RAM Mac and I assigned 12-14 GB RAM for Docker (+2 GB for Swap).
I agree that the Docker Virtual Machine (Mac/Windows) could take a lot RAM. In my case 12 RAM is used whether there is any container or not.
For Linux, It takes all of the available Memory. So if for some reason there is a High Memory or Out of Memory, the System crash. For Mac/Windows, only Docker crash and then restart.
Everyday, I run a fullstack containers (api, app, db, broker, etc ) with at least 10 containers.
I know that there is a limitation with the number of containers on the machine (depend on the memory and cpu) but If you want, you can control the memory and the cpu usage of your containers.
If these containers launch dev-servers (like API or APP) so it is the same to launch manually servers on the OS because it is not costly.
I noticed that containers take a lot of RAM when building the image. In my case, I build the image only once and then bind it with a volume.
If you have a single app with few dependencies, so docker could be useless.
My servers are mostly 1GB or 2GB, so I can't fit many containers there.
Laptop is 8GB and no Docker. I have everything on Windows or in WSL2.
I wish I can afford 32GB.
Are you sure about that? I have used docker in Ubuntu. It almost used around 500mb memory. Though I don't always use docker. But docker is very close to linux. Correct me if I am wrong. And linux has a way to kill the process if it's using too much ram.
You're indeed correct.
Sincerely,
Another Ubuntu Docker user
32 GB RAM!!! I have begun questioning my mere existence... I got a 3GB RAM junkbox ๐๐
Haha yeah. This was provided by work, I've never had so much RAM before!
Same for me
This! Docker is resource hungry and I constantly need to go back and cleanup storage.
I use Docker, but not for these cases.
agree
Docker is great for production deployment but painful for development.
You don't have to necessary need another container for your database.
You could also have it directly installed on your OS.
In my case, I prefer to containerised all of my stack (db, api, app, broker, etc,)
To quote:
You don't have to necessary need another container for your database..You could also have it directly installed on your OS.
In my opinion that's the WORST WAY to use Docker. I rather run the 10 containers Confluent provides for Kafka and have everything working and setup than waste time configuring Kafka for my OS (which most probably won't work if it's Windows).
For development, you can do theses things :
1) Keep the container awake with a non-stop command ( ex: CMD tail -f /dev/null)
2) Bind a volume on your container (with docker-compose.yml)
3) Use a development server with hot-reloader (runserver for Django, run for flask, react-scripts start for Reactjs, etc)
About running a database on another container I see that as an advantage as it helps isolate parts of your application. But if you want to run your db and app server on the same container, that can also be achieved.
I'd suggest checking out FreeBSD with Jails then. They are the containerization system that Linux modeled after (but IMO didn't do it anywhere near as well). Jails are functionally containers, but from a UX perspective are closer to full blown virtual machines. You can SSH into them, local console access into them, they have the full FreeBSD package manager, can install and run virtually anything that could run on the host. They also have their own independent network stack via VNET, which fully supports DHCP, IPv6, the whole array of networking protocols, or optionally share the host's networking stack instead. Additionally, you get native ZFS support accessible on the host, and optionally accessible inside the jail for each snapshotting/backup/replication/deployment.
Our entire development environment runs in Docker. To address your points.
I like the idea of Docker, but it does cause pain for local development. Others in the comments have already expressed the main issues, but for me it's mainly just the slowness (e.g. having to rebuild) and extra complexity for simple tasks (e.g. setting up git pre-commit hooks).
For simple Node.js-only projects (no dependency on Postgres or Redis), Node.js version managers like NVM work great and keeps setups very simple. It's not a complete replication of the production environment, but you should have that in a deployed development environment.
Conceptually, we should use containers for everything. In practice (at least in my own experience), it causes unnecessary problems for local development.
Ok I will mention it that version managers could be used for simple project
Or...perhaps just use what works best for your workflow, no matter how many articles say you should "Stop using that one thing that literally does the job for you".
Docker is a regular part of my workflow, but no, I'm not going to spin up a Docker container every single time I want to run a Python file with particular packages. I'm quite proficient with virtualenv, and it works very well.
Meanwhile,
nvmsaved my hide when testing a PR in an Electron app in an actual user environment, which Docker is not (without a lot of wasted time building it, at least.)To say "only use Docker images for building" is like saying "always rent a food truck when you need to cook supper, rather than wasting your time cleaning and setting up your home kitchen." It's overkill. There's a place for both, and one cannot just replace the other.
But then, "don't use X, use Y" is virtually always fad-chasing, rather than fact-based.
I use nvm and pyenv with huge success most of the times, but I also know that it can be a huge struggle when these tools fail what they promise.
That's why I don't use Docker for these tasks but I often use it for starting a local test database with one command and throwing it away when I don't need it anymore.
But I also use Docker when I don't want to install a certain tool in development. Then I quickly write a
docker-compose.ymland mount the development folder into the container. Then there is no need to rebuild the container for every change and I can run the tool with shell inside the container.I respect all developer point of views and theirs laptop configuration (memory, cpu)
It is a pleasure to have this kind of debate in dev.to (I couldn't do the same at the office).
I will notice your feedbacks at the top of my post.
Thanks for all replies.
You should add a disclaimer:
When you have tons of RAM and disk space.
Docker images consume a lot of disk space and running containers use a lot of RAM due to the need of a virtualization tool in the OS (in case of macos) or because of the containers themselves.
I really need to get comfortable with Docker and by extension Kubernetes.
I recommend Mumshad's course on Udemy. I used it when I prepared for the CKA exam and it was the best online course I have taken in a long time. I'm not affiliated with the man in any way so this is coming from my heart.. (:
Thanks for the recommandation.
I would definitely add security to the benefits of using Docker vs. virtualenvs.
PyPI repositories are a wild west and you could end up running all sorts of crap on your host OS if you don't contain it.
Of course you have to audit your dependencies nevertheless because eventually the code will be deployed somewhere. But still, it at least makes me sleep my nights better.
I totally agree with you.
There is the security layer.
And I would also add the famous quote of : "it doesn't work on my machine".
Now sharing images will help to build app on several similar environment (local, development, staging, production) and look like the same environement (almost)
I've used Docker on GKE for federated (multiregion) delivery, but I found LXD containers to be easy to manage, secure, and far less resource intensive for our dev env. It's really simple to spin up a cloned container using your favorite distro in seconds and work in a familiar linux server env. Each solution has it's upside and critters to deal with, but that's what works for us. Thanks for the good write up.