I'm looking for those with docker experience to help me wrap my head around moving from development and into production. I would normally throw this type of stuff onto stackoverflow but I find the feedback and insights from our dev.to community being really constructive and useful for others to learn too.
Current situation:
I'm a first-time docker user and I've been working on a side project which I've got running fine in containers and lifted with docker-compose in development as a full stack. Theres a producer app, rabbitmq worker queue, consumer app, caching database and a sql database with a website front end amounting to 6 images. I have registered these within my private gitlab instance and I have a semi-decent setup for keeping my images up to date after changes.
Whenever I dip into the docker documentation it throws out things such as deploying with swarm, compose and machine. I've spent a couple of pain staking days getting docker-machine installed on a private vps to allow remote management since docker docs seem to suggest this is the way to go, however this is where the problems have been endless.
So far docker-machine is completely horrific to work with using the generic driver. I've hit problem after problem and spent a great deal of my time digging through various fixes in their github issues list. I'm not overly confident in whether it's going to run smoothly when my application is eventually deployed so I wanted to double check with the great dev.to community as to the route you typically follow.
At the moment I'm fine running on a single VPS which should hopefully simplify things. In the future, there are workers in my architecture which pull from a worker queue and if I need to scale it would most likely be spinning up more workers to get through the tasks.
As soon as I look outside of the docker documentation I start to read things like.. rancher, kubernetes, portainer etc. I'm not completely sure whether these are relevant or even needed in my case so it would be great to hear some feedback on those too.
Actually just found this .. what a mine field https://landscape.cncf.io/
Top comments (3)
Know your options
Well, you're on the right track but no one actually puts things clear when it comes to containers cuz everyone wants you to use their products 😆
Google Cloud, Azure, and AWS all have different solutions for your problem, but you need to choose the one that gives you, as a solution architect, what you need at minimum costs (time+money).
The options
Those are the available options to deploy your stuff into production:
1. docker-compose
First, you should know the difference between docker-compose and the normal docker run... we have docker-compose to orchestrate/manage a multi-services solution.
I know many people would come and shout "docker-compose isnt for production"... well, lemme tell you a story.
We built our programming academy:
coretabs.net
And at the time of building it, we were using a small linux box (we didnt have the wealth to run 4 VPSs or so), so we SSHed into it, installed git, then cloned the repo, filled the secrets and env vars, run docker-compose... and voila it worked perfectly fine to serve hundreds of user!
So I really dont care if people say it's not production ready, cuz it works perfectly fine on small scale.
2. AWS Elastic Beanstalk
This does exactly the same as docker-compose, but provides you more of control over the instances (in Amazon EC2) running those containers.
Ah... there is also a nice web GUI instead of controlling things using commands in docker-compose.
Here is the story behind it, time passed and we needed to scale, so we found that elastic beanstalk is easy to use like docker-compose and it offers scaling multi-image environment so easy... so it was time to move.
We have written the dockerrun aws file (similar to docker-compose file):
github.com/coretabs-academy/websit...
Then created an environment on elastic beanstalk, wrote a small script on gitlab to automate the deployment:
github.com/coretabs-academy/websit...
And now we can deploy in 4 mins and scale into any number of instances.
3. Azure containers
If you're using Azure in your company, then you might need to use their services... there is no advantage that I know from any cloud provider to another... cuz the big three MS, Google, and Amazon are giving you superior power.
4. Kubernetes:
Same goes here, you can use it to manage/orchestrate your containers from Google.
5. Docker swarm
If you dont wanna do things using any cloud provider, go ahead and use docker swarm mode... it's like docker-compose
But the extra thing is the master-slave pattern, where you setup a master to manage containers on other machines (slaves).
There is a catch here, you wont have the spoil of GUI and you will have to setup every new instance (linux box or so) as a slave.
My recommendation is avoid it and use instead any cloud provider that makes your life easier.
6. Dozens of other container management solutions
Surely, there are many others... but seriously I would personally suggest to stick to the big three cuz they've got you covered in most cases.
Running a Docker project locally for development and production could be quite a different scenario.
For local development, it is very likely that all containers and volumes are run in a single Docker server which is ur local machine. And you already get it done with tool like docker-compose.
But when it comes to production environment, running everything in a single docker server doesn't sound good as it couldn't utilize the benefit of container-base application which is the ease of scaling because you could only vertically scale it by giving more computational power and RAM to that single server.
That's why there are clustering tools like docker swarm and kubernetes for container orchestration. In that case, you could scale your application horizontally by creating more containers across multiple docker servers to handle the workload.
To run a container-base application on a docker cluster has many new concepts. Ex.
On the other hand, you also need to consider things like
I suggest either docker swarm and kubernetes is good to go. Docker swarm should be easier to setup and should have a more gentle learning curve compared with k8s.
I haven't played with docker machine before and as far as i know it is quite similar to vagrant for provisioning a server instance. If u find it problematic, i think u could skip it and just provision the docker servers manually and start playing with docker swarm directly.
P.S. I also hv some freelance projects with production running on a single docker server because they are small projects and probably don't need scaling up. =P
You should invest in this udemy.com/course/docker-mastery/
Also depending on the cloud provider you will deploy to it will be different. Also focus on docker multistage builds and pushing your production images to a private registry