DX: bind mounting node_modules are not work properly

Overview

Nodejs

Nodejs have couple of file to look at :

• package.json
• node_modules

These file are the location of package that used at nodejs application

Docker

Docker is a platform designed to help developers build, share, and run modern applications https://hub.docker.com

I will not explain more here.

Problem

Development using docker

Usually we develop nodejs app using docker and use Docker Volume to able bind directory host to container directly, so we able to update and restart the app whenever we do development. But there is some problem, We actually can't do that because node_modules [at host] are not sync with node_modules [at container].

This is problem since many years, I as developer always face this problem every year. So I will wrap all resource here to make you easy to track all discussion.

Report

Is there a way that is not too impractical to Dockerize a Node.js app without mounting the node_modules folder of your host machine to the container when developing ?

• stackoverflow question by Ewaren

Ewaren (the person who ask that) explain what we expect when using nodejs + docker, this is what he (also we as developer) expected :

1. able to setup dev environtment on new machine without do manually install
2. The app easy to run only use docker compose up
3. The runtime environtment is consistent across all machines.

Thanks to him, he explain so much and give some "good solution" to implement, but each solution have plus and minus. You can follow up some solution at here. Here is list of his solution.

1. mount all files to host folder and run npm install first before running the docker compose. This is classic solution, but we do not want this because not plassform-spesific.
2. mounta all files to host folder and mount node_modules container to anonymous volume. This will prevent node_modules collide with host folder. But we have to re-build the container everytime package added.
3. mount all files to named volumes. This is clean, but we can't do development at host, but use vscode container instead.

Node docker repository

Finally !!, someone make template to easy to start !. Node.js + Docker for Showing Good Defaults in Using Node.js with Docker.

This repository have a lot feature !, dev as close to prod as you can, edit locally while code runs in container, production-minded features, and more.

They using dirty solution, but valid. They do different folder node_modules. node_modules container will saved in container at /opt/node_app and mount project folder host to /opt/node_app/app (sub folder). This make 2 different node_modules, they claim we are still able to development and restart app like usual.

But, when I run it only using docker compose up (of course use sudo for linux). folder node_modules are empty !. I don't know what happen.

Someone have similar problem, Bind mountings not working as expected #66.

Bread Fisher have 2 solution. He also mention this from his docker for node.js course

Solution 1:

• Don't move node_modules up a directory. Leave everything in the standard single dir and do a single bind-mount in the compose file.
• Before doing a docker-compose up the first time, we'll need to do a docker-compose run npm install to see the node_modules that are bind-mounted to the host. This is the main disadvantage of this Solution.
• This also means on macOS/Windows that you can't "dual develop" by sometimes using native host Node.js and sometimes using Docker-based Node.js because the binaries in node_modules (if you have them) would be incompatible. This is a disadvantage if you don't always use docker to run this code.

Solution 2:

• Similar to this repo's setup, you would move node_modules and package*.json up a level in the file path, so they can be separated from host node_modules. This adds complexity to the setup but also provides the flexibility of developing with a different node_modules on the host than what is in the container.

Ok, there is no different with early solution that we found. Similar solution, only this solution that we can rely on.

So, basically you have to Install the package on host manually, to make package are sync with container system, we have to use docker compose exec or docker compose run to run the npm install.

## use exec when container already run
sudo docker compose exec node npm i

## use run before run container
sudo docker compose run node npm i 

You see node at that script, that refer to name of service in docker-compose.yml file.

He also mention at other issue

docker compose exec -w /opt/node_app node npm install

Conclusion

Not good ending.

There is no solid solution for node.js environtment to fit with our expectation, you have to choose some method and face the fate. You have to install the package first !.

Thanks.