Reverse engineering Microsoft's dev container CLI

Dev containers allow you to open up projects and have them running in a fully pre-configured environment with one click!

... is how I introduced the concept of dev containers in my last article.

I also mentioned that I have been working on a small utility named vscli, which enables you to launch Visual Studio Code (vscode) from the command line (CLI) or a terminal user interface (TUI) like this:

Now this does not look very complicated, right? The cool thing is (and actually the main problem I want to solve with this tool), that it can also open devcontainers. So if it detects that the folder you are trying to open, with e.g. vscli dev/myproject, contains a dev container configuration file, it will open it in a vscode dev container instead.

Now, you might think it is as easy as calling a command like code dev/myproject --devcontainer behind the scenes. You couldn't be more wrong!

Before explaining how this actually works, I want to let you know that it will get a lot weirder when we try to add support for a new dev container feature later.

How everybody is doing it

This is not a problem that hasn't been solved before. Lots of people want to start their dev containers right from the CLI, without navigating menus in vscode. So if you look through various bash scripts and some issues on GitHub, you will find that most people solve this by executing the following command:

code --folder-uri vscode-remote://dev-container+<some_weird_hex_string><a_path>

Which then could look like this:

code --folder-uri "vscode-remote://dev-container+5c5c77736c2e6c6f63616c686f73745c417263685c686f6d655c6d696368695c6465765c7673636c69/workspaces/vscli

The last part (/workspaces/vscli in this case) is the path we want vscode to open inside the container. It is /workspaces/project_folder_name by default, but it can be overwritten inside the dev container configuration file.

The hex value is the path on the host, encoded in hex. Decoded could look like this (when the dev container was launched from WSL):

\\wsl.localhost\Arch\home\michi\dev\vscli

Note: We used WSL here, so most paths will be WSL paths. However, it works quite similarly on Windows.

How did we figure that out?

Well, I got the hint from this GitHub issue. Also, it seems like at one time a devcontainer open command existed in the old dev container CLI (it does not exist anymore since the CLI wants to be editor-agnostic).

There is also another version of the dev container CLI, which is included in the proprietary vscode dev container extension. This version actually includes the devcontainer open utility, but we only have access to the minified JavaScript code.

So if this problem is already solved, why implement our own CLI tool? Well, there are multiple reasons: closed-source, sending telemetry by default, and cannot be installed with a proper packet manager.

But we want multiple containers

It was brought to my attention, that the dev containers spec/vscode released a new feature, which would allow having multiple dev container definitions inside one project (by creating multiple configuration files in different places). I needed to try it out immediately, and actually found several use cases in my projects - so vscli needs to support it as well!

But how would I tell vscode which container to open? Using the strategy mentioned earlier, there is no way to point to some specific config. I did some research, and I haven't found a single codebase or shell script containing code to open dev containers in any other way than the command above. So nobody either managed or cared enough to figure this out. Is it even possible? Well, it has to, since vscode does it too!

Down the rabbit hole...

So I sat down together with my buddy and did a late-night investigation. Here is how it went (spoiler: we actually figured it out in the end).

First, we had a look at the closed-source version of dev container CLI again, maybe it supports this? Turns out it does!

So we downloaded the source code of the dev container extension from the official marketplace: https://marketplace.visualstudio.com/items?itemName=ms-vscode-remote.remote-containers. This gives us the ms-vscode-remote.remote-containers-0.320.0.vsix file. Since .vsix is basically just a .zip file, after extracting it we could explore the extension.

Luckily they not only included the compiled binary but also the Node JavaScript "source code": extension/dev-containers-user-cli/dist/node/devContainersUserCLI.js. However, it's not real source code, since the source application is probably written in TypeScript, and what we get to see is the minified, transpiled JavaScript:

This code is optimized for minimal size, so most of the functions and variable names are renamed to one-letter names and everything is put into one line without spaces.

Making sense of the code

Since this code is unreadable, it is very hard to get the control flow. We started by searching for strings we know, like vscode-remote://dev-container. And we had a match! The extracted and formatted function which uses this string looks like this:

function Z$(t, e, n, r, i, o) {
  t = moe(t);
  let s =
    r || o
      ? JSON.stringify({
          hostPath: e,
          localDocker: i,
          settings: r,
          configFile: o,
        })
      : e;
  return `vscode-remote://dev-container+${Buffer.from(s, "utf8").toString(
    "hex"
  )}${t ? `@${t}` : ""}${n}`;
}

We can see that whatever is put behind the + (variable s) is converted into hex, as we already know. n is the workspace path and the final part of the URI. t is an optional parameter I am not quite sure of. But if we look into how s (the hex-encoded part) is defined, we can see that it is either e or some JSON string depending on how r || o evaluated.

So this seems to be the solution to the problem! It is possible to pass in more data than just the project path! We don't really care for r and o at this point, but if we look into how the JSON string is constructed, we can even read what they are: r seems to be some additional settings and o the configFile? e is the hostPath, which is also used when we don't use the JSON string. So this is the path to the project we also used in the old approach. i which is put into the localDocker field is probably some boolean that describes whether to use a local or remote Docker host.

Did we solve it?

The configFile parameter is probably the file path of the dev container config file we are trying to load. So let's build up the following JSON and try to open vscode:

{
    "hostPath": "\\wsl.localhost\Arch\home\michi\dev\vscli",
    "configFile": "\\wsl.localhost\Arch\home\michi\dev\vscli\.devcontainer\testContainer.json"
}

Which opens vscode, but it will complain with the following error: \[UriError\]: Scheme contains illegal character.

We tested a few options to debug this:

• Only using hostPath without configFile: works (but we cannot choose a dev container config)

• Directly putting the dev container config file contents into configFile: UriError

• Using Windows style file paths: UriError

So configFile expects some special URI format? It's time to look into the code a bit more: Z$ is called only once:

  let q = Z$(
      void 0,
      p,
      w.workspaceFolder,
      void 0,
      void 0,
      e ? qn.file(Ze.resolve(process.cwd(), e)) : void 0
    ),

Most parameters are void 0, which is the shorter equivalent of undefined. This shows that r, i and t of Z$ are actually not used (probably for legacy reasons), so we always use the JSON format nowadays. We also learn how the input to the hostPath is constructed: qn.file(Ze.resolve(process.cwd(), e)).

At first look, it looks like it combines the path of the current folder (process.cwd()) and the relative path to the dev container config file and then passes it into a function that builds the file representation that the URI expects. After some investigation and finally ended up with this code:

var LP = ae(require("os")),
 Ze = ae(require("path"));

We concluded that Ze.resolve() is part of the node.js path library and indeed combines path segments.

The last mystery: .file()

This one was difficult. We really could not make sense of this method, since it was part of a bigger library:

var { URI: qn, Utils: awe } = AF;

// and then somewhere else in the code
// ...
        (D.extname = function (A) {
          return F.extname(A.path);
        });
    })(R || (R = {}));
  })(),
    (AF = r);
})();

Luckily, somewhere in this library, we found an error message: The "pathObject" argument must be of type Object. Received type .... I was never as excited before to find an error message. But this is something we can Google and maybe find out which library it is!

And it worked! We found a GitHub issue in the vscode GitHub repo, which referenced the exact file it is implemented in. Turns out this is the internal URI library of vscode!

The code is still quite complex, but there was an easier way anyways. I just opened up the developer tools in vscode and called the .file() function directly, passing in the path to my dev container config:

So the .file() method returns an object that is directly serialized into JSON. The UriError message from before probably hinted at the "scheme":"file", property missing. This does not look like a proper URI interface and is probably an accident, where the developers forgot to properly serialize the path - but hey, we figured it out!

So now we can use the following JSON to open our dev containers with multiple config files:

{
  "hostPath": "\\wsl.localhost\Arch\home\michi\dev\vscli",
  "configFile": {
    "$mid": 1, // optional
    "path": "/Arch/home/michi/temp/multi/.devcontainer.json",
    "scheme": "file",
    "authority": "wsl.localhost"
  }
}

There is still some work to do with getting the paths in the proper format and to make this work on Windows, but this should get you started if you want to implement this yourself. Check out the implementation of this in vscli here: https://github.com/michidk/vscli/blob/main/src/uri.rs

Now, if you try to open a project with more than one dev container using vscli, the following dialog will appear:

Want to use vscli yourself? Check it out on GitHub: https://github.com/michidk/vscli