Today I Learned: Golang Live Reload for Development using Docker Compose + Air

Today, I’m trying to create a live-reload for my Golang application. It’s just a simple application of REST API.

For your information, Live-Reload is a mechanism that will reload our application on every file change. So it will keep up to date with your code. I’m not sure it’s the same as a hot reload. But people said live-reload is reloading the application when a file changes. And hot reload, will only refresh the changed file without losing the state of the application, so it’s not restarting the entire application.

Also based on this Stackoverflow answer,

Live reloading reloads or refreshes the entire app when a file changes. For example, if you were four links deep into your navigation and saved a change, live reloading would restart the app and load the app back to the initial route.

Hot reloading only refreshes the files that were changed without losing the state of the app. For example, if you were four links deep into your navigation and saved a change to some styling, the state would not change, but the new styles would appear on the page without having to navigate back to the page you are on because you would still be on the same page.

I guess, for this article, I can say this a live-reload because I’m not sure we’re able to use hot reload for Golang. Because for Golang, for every change, we need to restart the application from re-compiling it and re-run it. So the most possible way to do it is to use live reloading instead of hot reloading.

Background

But before telling the details, I want to talk about the background, why I just learned this live-reload after so many years?

I’ve been working with Golang for 3 years now, but I never use live-reload when working on my projects. It’s not because I’m an ignorant person, I do want to use live-reload for development.

But for the Golang case, I have a few reasons back then, why I haven’t used it.

• Golang is a compiled programming language. My mindset to think like it’s impossible to have live-reload for compiled programming language. Because we need to compile the application and run it. Well, technically I wasn’t wrong, even we can like to create a live-reload tool, to watch the changes and re-compile the application (in away, it’s live-compile-run)
• Golang run/build is quite fast compared to any compiled programming language in the market, also one of my reason not to consider live-reload yet, at that time.
• Making my own live-reload tools. Well, it’s just reinventing the wheel, not to mention the time that needed to build it is not worth it, back then. Even sometimes when I’m working a larger project, I’ll feel regretted not to started making my own live-reload tools.
• I’ve tried a few tools but disappointed by the performance. Actually, I’ve tried once back then, I forgot what’s the tool, but back then, it consumes a lot of my laptop resources. It making my laptop lagging. Been trying to some frameworks as well that have a live-reload feature, but again it consuming a lot of resources every time I save my file. Instead of increasing my productivity, it’s making me more stressed because of the lagged laptop.
• My projects relatively depend on small dependencies (libraries, framework, database, any extra layer), so compiling and running the application is not an issue yet at that time.

So then, now I’m looking for a live-reload feature because my projects getting bigger and also the dependencies as well. Like it depends on Google Pubsub, Firebase, Mongo, Postgres, Redis, all other libraries. Just to start the app, it will take time. So when developing new features, it will really take time only to run it locally. That’s the time when I try to consider to use live-reload for my productivity. Well at least, reducing the time needed for manual intervention like compile and building the application. Even milliseconds matter when you are in zone LOL.

So then, I have a few criteria for the live-reload,

• The most important is, it’s not making my laptop lagging. Performance is the most important, I don’t want to get stressed by a lagged laptop. I got a bad experience when using one of the Go frameworks that have a live-reload feature, it frustrates me. In the beginning, it just works fine, but later when I make frequent changes, it made my laptop lagging like a hell. Even when typing in the editor, it’s got delayed.
• Configurable! I want those tools can be configurable with my own settings for the live-reload.
• Easy to use and portable. So everyone can use it no matter what’s the environment. Generally speaking, I would say, it’s dockerized at least. Since I know docker has become a standard tool for engineers now.

And based on that criteria, I tried to find many tools on the internet. I found a few good tools, but then, I remember, my colleague use a live reload for our internal project in the company.

Out of curiosity, I then try to explore that tool. It’s named, “Air”. The repository can be found here, Air.

Live Reload With Air

The first impression, I feel skeptical, because it’s just like yet another live-reload tool. I have to install it, then run it in my project. But, when I check the Github repo, the winning solutions that they provide is,

• Configurable. I can configure my settings based on my need.
• Portable. Since it just a binary, I can use it to docker, and make it portable.

Setting Up My Live Reload Environment

To use the Air live-reload tools, you see in the Github repository. But for me, I use Docker compose to manage my live-reload development environment.

So what I will need to have,

• Application the I will develop.
• Installed docker on my laptop.
• Dockerfile for development.
• Docker-compose file for development.
• Customize the configurations

So the idea is, I will use docker-compose to manage the live-reload with using Air.

1. Making the Dockerfile for Development

The first step is to make the docker file for development.

The docker file is basically only to download and install the Air binary and make it as the entry point for the docker.

If you see from the above docker file, you will see this line

RUN curl -fLo install.sh https://raw.githubusercontent.com/cosmtrek/air/master/install.sh \ && chmod +x install.sh && sh install.sh && cp ./bin/air /bin/air

This is only to get the installation script, then run it, and making it /bin folder.

After that, then we will make the docker-compose file for the development.

2. Making the Docker-Compose file for Development

The next step is to create a docker-compose based on the previous dockerfile. As you can see, in that docker-compose file, I set the dockerfile to dev.Dockerfile

web:
    build:
      context: .
      dockerfile: dev.Dockerfile

Also, the other important thing is, I set the volume link to my current directory,

volumes:
      - ./:/app

It means the container will use my current directory and attached it to /app in the container. So If there are any changes in my directory, it will also change the file in the container.

3. Set your application config

Since we will do a live-reload, so, we also need to provide a config file for our application to be able to run. The config file can be different, usually, people use ENV variable or .env file. Or just a config file, like config.jso n or config.toml etc.

You need to define it if you want to make a live-reload the application. Just assume you will need to run the application, what config you need.

In my case, on my local, I will create a file named config.toml

title="Configuration File for Menekel"
debug=true
contextTimeout="2"
[server]
  address= ":9090"
[database]
  host="mysql"
  port="3306"
  user="root"
  pass="root"
  name="article"

And this config will be copied to the container as well, so we will be able to run it in the container.

4. Set the Air configurations

The next step is, create the Air config. Create a new file with named .air.toml and then configure it based on your needs. I’ll try to break down any important configuration terms from the config here.

a. CMD syntax

[build]
//...
cmd = "go build -o ./tmp/app/engine app/main.go"
//....

The bold one from above is for the command that you need to compile your application. In my case, I just use go build -o ./tmp/app/engine app/main.go . So if you guys want to copy this to your application, make sure the build command is correct.

b. BIN syntax

[build]
//....
bin = "tmp/app" 
//....

The bold text above is only the directory of the compiled binary exist.

c. FULL_BIN syntax

[build]
//...
full_bin = "./tmp/app/engine http"  
//...

The next one from above is how your application will be started after compiled. In my case, to run the compiled application I will need to pass the argument http to determine I want to run the HTTP server.

So if your application doesn’t need any argument, you don’t have to add any additional arguments.

If your application using the ENV variable, you will also need to pass it like,

full_bin = "ENV1=mysql ENV2=localhost . /tmp/app/engine http"

Tricks:

• Make sure your application to be able to read .env file, so you don’t have to list all the ENV in the Air config.
• Use a config file, but bare the consequences.

d. INCLUDE_EXT and EXCLUDE_DIR syntax

[build]
//...
include_ext = ["go", "yaml"]
exclude_dir = ["tmp"]
//...

The include_ext will watch for every file that has an extension as listed. If there are any changes in any file in the project, it will trigger Air to re-compile the application.

The exclude_dir is to tell, any changes on that directory won’t trigger the Air to recompile the application.

e. DELAY syntax

[build]
//...
delay = 1000 # ms
//...

Sometimes, we may make too frequent changes, so to reduce the instant build every saved file, we can add a delay. So it won’t directly compile the application for frequent changes. This one is pretty helpful since compiling and re-running the application will consume the CPU, if we compile to frequent it will make our laptop lagged. And with this, we can configure the delay to reduce the pace for compiling the app.

5. Final Steps: Making It All Running.

The final step is, you can run the application by using docker-compose,

$ docker-compose up -d

And check for the log in your terminal,

$ docker-compose logs -f

See the demo here,

Conclusions

After trying this, at least I can get work faster. If the changes are too frequent, I just need to add the delay in the config, so it won’t affect my laptop resources.

The example repository can be seen here, Menekel, or you check this PR if you want to see what changes I made for adding the live-reload.

Also, if you guys have any better ideas, tools, let me know, put in the comment below so the other people also can try it.

If you guys found this useful, like, and share so other people can reach this, spread the love, and knowledge.