This article was originally posted on Feb 20th 2018 at: https://nickjanetakis.com/blog/setting-up-a-python-development-environment-with-and-without-docker
You might be wondering what "setting up your computer" means exactly. To me, that means coming up with a plan to solve the following problems:
- What code editor should I use?
- How do I install Python?
- How do I install my app's dependencies?
- How do I install required external services?
- How do I run my application?
Technically these problems are the same for any language / framework, but let's stick with Python for this article. Specifically web development with Python.
What Code Editor Should I Use?
This one is easy because whether or not you use Docker, the answer is going to be the same. Just install whatever code editor you prefer.
Personally I'm a fan of using VSCode, but I've used Sublime Text for many years in the past.
Getting Your Python App Running without Docker
The first thing you need to do is install Python on your operating system of choice. Are you using Windows, MacOS or Linux? If Linux, what distro are you using?
I'm not going to show you how to do this for each OS choice because I have a low tolerance for self induced pain, but please Google for something like "how to install Python on Windows 10" and you'll find all sorts of answers.
By the way, you should also consider on whether or not you want to use Python 2.7 or something more modern such as Python 3.6. Both have their pros and cons. You may end up wanting to install both versions, since they can live along side of each other.
Next up, you need to deal with your app's dependencies. There's a few different types of dependencies here. There's installing packages that your application needs to run (such as Flask if you're building a web application, or Django if you prefer that), but there's also external services too, such as your database (which we'll go over shortly).
Let's focus on your app's packages first. Technically you could just install pip (which is Python's package manager) and start installing everything directly on your main OS, but that's going down a dangerous path.
If you install your packages to your system's version of Python directly then you're forfeiting your right to be able to work on more than 1 project that requires 2 different versions of the same package.
This comes up all the time in web development. You might be working on a project today but then 6 months from now you start a new project and you want to use the shiny new version of a package you started using 6 months ago.
Well, you can't just use the new version in your new app without caring because if you upgrade it, it will likely break your old project from 6 months ago due to backwards compatibility issues.
This is a really big problem but fortunately smart people have solved this problem.
You can install a tool called Virtualenv. Virtualenv is a Python tool that lets you create these little virtual environments (not to be confused with virtual machines btw).
Now when you start a new project, you also create a Virtualenv for it too. When you install all of your app's required packages, they all get stored in their own separate Virtualenv.
You can even tell it to use a specific version of Python. That's pretty sweet.
The problem is now totally solved. All of your projects have their own package versions, and your system's version of Python is not polluted by anything. It can simply act as a runtime for your Virtualenv wrapped apps.
Onwards we go, by installing external services. Real world web applications often consist of at least a database, and could easily require many more services such as Redis, etc..
It's a really bad idea to use SQLite in development because when you eventually run your application "for real" on a server, you would end up using Postgres or MySQL most likely -- not SQLite. Also some of your code may depend on using features that simply don't exist with SQLite. You have no choice.
So the best thing you could do is install Postgres on your computer. That's going to be wildly different depending on what OS you use. You'll also want to make sure you get the same version you plan to run "for real" on your server later.
Technically it will never be the same because the MacOS or Windows version of Postgres is not the same binaries that run on a Linux server, but it's likely "close enough".
You'll likely have to do the same thing for Redis too because most real world web apps will have a background worker (such as Celery) and Redis is great to use as a broker and cache, but now we're getting pretty deep in the woods on a side topic, so let's get back to services.
By the way, Redis doesn't even run on Windows officially. You'll have to settle for running a custom port from Microsoft, or alternatively, you'll need to delve into the wonderful world of virtual machines and potentially use tools like Vagrant.
WARNING: You're about to enter a world of serious pain and confusion with VMs if you're new to software development. It's also going to delay writing your app by quite a while while you soak in how to set up all of this stuff. Expect many sleepless nights.
You've made it, now it's time to run your app. If you made it this far, you're serious about web development for sure. Too bad the bar of entry was so high before you could even start writing your application.
I mean, we're still not there yet, because now we have to figure out how to run our app.
We have a few choices, but there is 1 thing to be aware of. In order for your application to work, you'll need to make sure Postgres and maybe Redis is running first.
You could potentially run them all the time, but that means they will be eating up your system's memory and CPU cycles even when you're not developing. Sure, you can start and stop them manually between development sessions, but that's not fun.
Oh yeah, there's also the issue of installing multiple versions of Postgres and Redis at the same time, because you'll definitely want to use newer versions as they come out, but your older apps may require older versions due to compatibility related changes.
Once you figure out how you want to run Postgres and Redis, you need to worry about starting up your Python app too. I'm a big fan of Flask for web development, which is a micro web framework for Python. I like it so much that I created a whole course on it.
Anyways, you'll need to start your web app's server along with your background worker, which means you'll be juggling multiple terminals. It's going to get worse too as you introduce more services to your app over time.
However, like Virtualenv, this is a solved problem. There's a really nice tool called Honcho. It lets you create this thing called a Procfile and then define what services you want when you run it. It runs everything through one terminal window.
You could add your web server, background worker and even Postres / Redis to this file and be good to go. Now if you want to start everything up, you just run honcho start and you're golden. It even color codes the terminal output for all of your services.
To recap, before you could even start writing your app, you had to:
Hunt down how to install Python, get pip installed, install virtualenv and now deal with using virtualenv on a very regular basis, otherwise you're going to enter dependency hell.
Then you had to learn how to install and configure Postgres, Redis and any other services that your application may use. Finally after that, you had to figure out how to run everything in a pleasant way with Honcho.
That's a long road, and it may have taken weeks or months to get there if you were uncovering things as you went.
Unfortunately if you ever plan to run your application on a real server in production you'll have to start from scratch and learn how to do all of the above on Linux, but hey, let's only focus on development problems for now!
Getting Your Python App Running with Docker
If you're expecting me to say "but with Docker, it's so easy, all you have to do is run 1 command and you're done", then you're mistaken!
While, it's true, once you have things set up and understand Docker, it really will be so easy that you can set up any real world Python web app in 1 command, and it will work just the same on Windows, MacOS and Linux.
You also don't need to worry about project isolation because Docker handles that for you. Running apps with Python 2.x or 3.x along with each app having its own dependencies is no problem. Virtualenv won't even be used.
Installing external services are a breeze too. You won't be installing them directly on your main OS. They will run locally on your computer, but they will be ran through Docker. For example, getting Postgres or Redis up and running requires running 1 command and no advanced configuration.
Lastly, you won't need a tool like Honcho and you don't need to worry about external services running when you're not developing your app because Docker has its own tool to spin up and shut down all of your app's services in a few seconds with 1 command.
You can even run multiple versions of Postgres and Redis without conflict. All of this sounds great, but it does come at the cost of a tiny bit of complexity.
You'll first need to install Docker. That's the easy part and it only takes about ~7 minutes.
The complexity is figuring out how Docker works. It's pretty simple once you get it, but to get to the point where everything clicks can be a lot to take in if you're new to Python development or web development in general.
I have a whole tutorial on running a real world Flask application with Docker if you want to read it. It doesn't go into the gory details of everything, but you will have a real world Flask app running in Docker in about 15 minutes.
If you want the gory details on how Docker works and learn how you can start using it on your own projects then I recommend checking out my Dive into Docker course. It's 5 hours of premium video along with 30+ guided labs and challenges to get you going with Docker for Python apps (or any language really).
I know, this is where you're thinking "omg, this whole thing was a sales pitch for buying a Docker course", but it's really not. You can totally do it the long way without Docker using the tools I mentioned above, or you can learn Docker on your own by reading the tutorial I linked and then scouring the internet for dozens of articles to fill in the gaps.
The value of my course is that I've done all of the research for you and I've been using Docker since 2014. I've ran it in development and production for years. Along the way I've came up with a bunch of best practices and it's all in the course.
Plus I offer lifetime updates and support. If you have any questions, I'm only an email away. I've answered over 3,500 questions from 20,000+ people since 2015. Support is something I take really seriously! Hop on board and join us in the Dive into Docker course.
Top comments (0)