How I manage my dotfiles across hundreds of machines

Almost all of the command line tools I use on my Mac workstation, on my development server in the cloud, and on my company’s Linux server infrastructure store their configuration in so-called “dotfiles” — files (or even whole directories) in my user home directory that are hidden from plain sight by prefixing their name with a period, for example .vimrc. (Like this example, actually quite a few of these files do end in rc. That’s why they’re sometimes also called “rc files”.)

On dev.to, Jonathan Carter asked “How do you manage and synchronize your dotfiles across multiple machines?” Since “multiple” in my case means “more than 500 servers” (we operate a managed high-performance hosting platform for Drupal and WordPress), I thought I’d answer his question in a short blog post.

dotfile management

Most of the dotfiles I use come from our shared team dotfiles repository. Having a big overlap in configuration settings between our SRE allows us to easily share and take over a terminal session via tmate without having to struggle with individual tmux or vim keybindings. Doing pair work in a terminal session has the huge advantage that it takes up much less bandwidth than screensharing via Zoom or Slack.

Some tools still allow a certain degree of individuality. For example, the shell prompt details can vary without causing confusion. You can also have your own magic shell and git aliases. That’s why I layer my personal dotfiles repository on top of the team one.

Another complication is that the BSD subsystem on the Mac sometimes behaves a bit differently from Linux. That’s why, for a few configuration files, I’m maintaining alternative versions. They’re stored in subfolders starting with tag-. The dotfile deployment then installs the right version depending on the host OS.

dotfile deployment

So how do you maintain a set of dotfiles in your home directory on hundreds of machines if they come from two overlapping, even conflicting, git repositories? The trick is to clone both repositories and then symlink the configuration files to where they are expected by their application. The key to keeping this process simple and error-free is rcm. It can handle multiple dotfile directories, each with its own precendence. You can also provide it with tag names telling it from where to source certain dotfiles.

I install rcm manually when I set up a new development machine (which only happens once or twice a year). On our servers, it gets installed automatically via our configuration management software Chef.

Here’s the script that I use to deploy my dotfiles:

#!/bin/bash

if ! which rcup >/dev/null; then
  echo "Fatal: rcm is not installed on this machine."
  exit 1
fi

os_type=$(uname -s)

# Remove oh-my-zsh
rm -rf ~/.oh-my-zsh

if [[-d ~/.dotfiles]]; then
  cd ~/.dotfiles
  git pull
else
  git clone https://github.com/geewiz/dotfiles.git ~/.dotfiles
fi

if [[-d ~/.freistil-dotfiles]]; then
  cd ~/.freistil-dotfiles
  git pull
else
  git clone https://github.com/freistil/dotfiles.git ~/.freistil-dotfiles
fi

rcup -f -t $os_type -d ~/.dotfiles -d ~/.freistil-dotfiles

As you can see, it clones the dotfiles repositories (after a bit of cleanup) and finally calls rcup using an OS-specific tag (“linux” or “darwin”). Thanks to a a post-up hook, it even launches vim to update its plugins from Github.

You can find the current incarnation of the deployment script in my personal repository as bin/dotfiles. Since its directory gets symlinked as ~/.bin and added to my shell search path, I can execute the dotfiles command at any time to update my local configuration.

As for the initial deployment on a new machine, I simply curlbash the script:

curl -L https://raw.githubusercontent.com/geewiz/dotfiles/master/bin/dotfiles | bash

That’s how I manage and deploy my dotfiles to have a consistent setup across all my work machines and our hosting infrastructure.

If you’d like to watch me put these nifty dotfiles to good use, join me on Twitch for my next live coding session!