A few months ago, a colleague reposted a Gitpod blog post in my employer's slack, mentioning that if you're a member of some reputable GitHub organization, you get a plan with unlimited hours. Thank you, the past me who spent a weekend packaging an app for nixpkgs!
I knew these kinds of cloud IDEs have been around for some time, but that message inspired me to give it a bit of try, who can resist free lunch after all? Well, I'm now using it on a daily basis, so this post will record some notes and thoughts along the way. The content is not quite well structured, but I hope they can provide a bit of insight for other Gitpod users as well.
The first Gitpod use case I tried is hacking GHC. And that's quite a roadbump because GHC is hosted on their own GitLab instance!
Yes, you probably can fiddle with tokens and such, to make Gitpod work with self-hosted GitLab. It's mentioned in Gitpod docs, I haven't tried, since there's another problem ahead:
Gitpod requires you to check in
.gitpod.yml into the repo you're working with. That's reasonable for most single-repo use cases, but I don't have any mental energy to upstream a Gitpod config, nor do I want to pollute my GHC working branches with this config file.
So I made a dedicated GitHub repo for hacking GHC. Gitpod allows you to specify scripts to run during workspace initialization, so I can just clone GHC to
/workspace and open it later, profit.
All looks good except when I need to push. For GitHub repos I own, pushes to
https remotes should work out of the box, but well, this doesn't work for GHC's self-hosted GitLab.
This hasn't been a problem when I used my employer's remote build machine, since I can always enable SSH agent forwarding, so I can do the pushes on that machine. Gitpod allows you to SSH into an active workspace, but honestly I don't want to open a separate terminal window just for git operations, I prefer to stay inside that browser tab all the time.
So. Given Gitpod allows specifying secret environment variables, I generated a passwordless SSH private key and encoded it as a secret. The
gitpod-ghc init script will start an
ssh-agent and automatically add that key, so future
git operations will work out of the box.
Yes, doesn't sound like a good security story, other than permuting the key frequently and limiting the key's scope, I still need to trust Gitpod not to leak my secret, no less than they trust me not to abuse their machines to run crypto miners.
There are multiple ingredients in making a Gitpod workspace:
- The base
Dockerfileor a Docker Hub image. Use whatever you like, but given the Gitpod runtime will yolo a bunch of Linux binaries at runtime, it better be a typical glibc-based FHS distro.
.gitpod.ymlconfig, which allows you to specify init scripts.
Having two places to write scripts may seem redundant, but the
.gitpod.yml scripts have access to the
/workplace directory. Across restarts of the same Gitpod workspace, only changes in
/workplace is persisted!
It's quite an annoyance since for a long-lived workspace, you often accumulate implicit state (e.g. language-specific caches) not only in your project directory, but also in the home directory.
Luckily, there's a limited workaround for that: bind mounts. In
gitpod-ghc config, I bind mount a lot of home directory contents from
/workplace instead, things like
.cabal. Even the entire Nix store!
It's a pity I can't bind mount the entire home directory though, likely because of busy resources during workplace initialization. And of course, all system package manager invocations are forgotten after workplace restart.
To think of it in a more positive way, Gitpod is enforcing the erase your darlings philosophy. It doesn't strictly prohibit state, it forces you to make your state explicit.
Gitpod prebuilds is a CI service that runs on the same infra to power the workspaces. When a prebuild is triggered, the Docker image gets built, a fresh
/workplace with a checkout of that commit is mounted, and the expensive (
init) step of
.gitpod.yml scripts get run. The updated
/workplace is saved and reused upon workplace start up, so the
init step can be skipped at that time.
For a large project like GHC, enabling Gitpod prebuilds is essential to ensure a sane startup time for new workspaces. Here's a non-comprehensive list of things done during
gitpod-ghc prebuild time:
- Install latest release versions of
- Compile and install
- Check out GHC tree into
- Go through the
hadrian, build up the
.hie-bioscache, so the language server shall work nearly instantly for a fresh workspace
Oops. I've written a lot in this post and haven't even mentioned VSCode even once. Yes, Gitpod supports multiple IDEs, but the one I'm using on a daily basis is the in-browser VSCode. Some caveats to keep in mind:
- Most default key bindings work. ^W closes the browser tab instead of the editor tab though, luckily there'll be a prompt about unsaved work. Don't silence that prompt.
- Flaky connection is fine, but don't drop offline for more than 3 minutes.
- Settings do get synced properly. A minor annoyance is you can't declare editor settings in
.gitpod.yml, apart from the extension list.
- The default extension store is OpenVSX instead of the Microsoft-hosted one. There do exist extensions that are severely outdated in OpenVSX (like
rust-analyzer), but as a fallback measure, you can specify URLs to
There are things that work, and things that don't work yet. I didn't really spend time to dig into Gitpod's issue tracker, so I'll be more than happy to be corrected in the comments shall the list requires some update!
sudoworks. Of course you're still in a container jail, so you can't really mess with
sysctlconfig and such.
systemddoesn't work. You need to bring your own userspace supervisor if you need such a thing.
nixworks with single-user installation and
sandbox = false. Multi-user mode with
nix-daemonshould still be possible, but not worth the effort if sandboxing won't work anyway. So don't ever push to binary caches from a Gitpod workspace.
dockerworks out of the box as rootless mode, and the state directory is in
podmandoesn't, despite unprivileged user namespace is on.
prootlatest version works.
tailscaleuserspace mode works, but running the daemon as
rootseems quite fragile. Haven't tried other VPNs yet.
htopkinda works. The CPU/RAM stats don't reflect the workspace, but reflects the entire metal below it.
Gitpod development happens on GitHub, they allow self-hosting and the sausages are made in the open.
Apart from that, there's an official discord server. Not a super active one, but the staff are there, and questions do get answered within a day or two, regardless of whether you're a paid user or not.
I believe I've accumulated enough experience to say: Gitpod is nice, and it's worth a try.