One hosts file to block them all

As a field expert, people often ask me how to improve websites' performance. Sometimes, they also ask me how to improve the Web's performance on their own machine, for their own browsing experience. In that case, my answer is always the same: the lowest hanging fruit is most certainly the hosts file.

Keeping the dirt away

Today's web is full of dirt. Most sites are full of trackers, ads, and lots of other nasty stuff penalizing websites' loading. To avoid this, AdBlockers are blooming. They offer a quick and simple solution (most of the time, a browser extension) that blocks most of the unwanted content.

Unfortunately, as for the UX, AdBlockers don't deliver. They often increase the amount of memory and CPU cycles used by your web browser, slowing your browsing experience instead of boosting it. Some are doing better than others, and whole browsers¹ have been conceived over the idea of blocking unwanted content, and are doing an incredible job. But the web is not confined to your browser, is it?

The Web is requested from everywhere in your computer. Most mainstream applications are as crammed with trackers as your next media website. Sometimes, you can even see the ads displayed in your UI (Hello, Skype. Yes, I'm talking about you, you naughty boy).

The AdBlockers can't do anything about that. But a simple hosts file does.

Hosts… what?

The computer file hosts is an operating system file that maps hostnames to IP addresses. It is a plain text file.
"hosts (file)", on Wikipedia

Whenever your browser interacts with a website, it actually requests a server, located through its IP address, like 185.31.40.11 (IPv4) or 2a00:b6e0:1:20:2::1 (IPv6). It's pretty similar to a postal address.

Now think about accessing a webpage, like https://borisschapira.com. How does your browser know which IP address to contact? Well, the browser simply asks the Internet phone book. At least, one of them. And the phone book, called a DNS, resolves the IP address for the browser.

I don’t know about you, but I never use a phone book. Most of the time, I’ve collected the address of the people to whom I want to write in my personal address book. That’s the hosts file for you. Every time a process on your computer needs to access a resource on the Internet, it first goes through the hosts file to find out where to find it.

Now, let’s say that you don’t want this process to find the resource on the badthings.com domain. Easy peasy, throw it on the wrong track by associating the badthings.com domain to an unspecified address like 0.0.0.0.

So basically, if someone makes a list of all the domains where bad things happen, we can redirect them all to 0.0.0.0 in our hosts file, and make our own Web a much cooler place.

One project to gather them all

There is nothing new in what I am describing here. Cool people have been doing this for years, sharing their host files. Fake news websites, gaming platforms, pornographic hubs, encryption pages, fraud attempts, and scams are all patiently identified and listed in open access files.

I use Steven Black's "hosts" project, a python script, to cram them into a single 2MB hosts file containing more than 70k domains. In case you're wondering, I can and do alter the result with my own whitelist (otherwise blocked domains whose requests I don't want to prevent) and redirections (which allows me to write this article and check the result on https://borisschapira-dev.com:10443/, which actually points to my own machine).

If you're not familiar with the command line and use Windows 10, the hostsman app will help you achieve the same goal (unfortunately, as I just published the post, the website looks down. Fortunately, a portable version exists). I will not recommend modifying your host file on previous versions of Windows even if I did it, before 2015 (FR), unless you like to run ipconfig /flushdns every 30 minutes. On Linux, I've heard people talk about using dnscrypt-proxy… but I've never tried myself.

Some tips from my daily life

Sometimes, I need to temporaly disable this hosts file. For example, when I need to help Dareboost clients understand the impact of third-party scripts on their pages (thus, I need to load these 3ps myself).

To enable or disable my hosts file, I use a command line function that I've develop and put in my ~/.profile file²:

function togglehost() {
    if [ -e /etc/host.bak ]
    then
        sudo mv /etc/host.bak /etc/hosts
        echo "Hosts file is active again"
    else
        sudo mv /etc/hosts /etc/host.bak
        echo "Hosts file is set aside"
    fi
}

Once this function is loaded, activating or deactivating my hosts file is very simple:

$ togglehost
Hosts file is set aside
$ togglehost
Hosts file is active again

But as I sometimes forget to put it back in place, I have set up a routine that checks every minute that the file is in place and, if it is not the case, warns me by using Jaime Piña's noti to trigger a notification. To perform this regular check, I use crontab.

* * * * * if [ ! -e /etc/hosts ]; then /usr/local/bin/noti -t "Host file" -m "does not exists"; fi

I have been using this technique for years now and, sometimes, I forget that my hosts file protects me so much. Whenever I need to use someone else's computer, or temporarily disable my hosts file, I realize the level of comfort it provides me.

I'm well aware that tweaking its own hosts file is a good but technical solution. It doesn't completely replace an AdBlocker (or I haven't aggregated enough files yet) but it's an incredible performance gain, which I highly recommend for everyday browsing.

Try it for yourself, and tell me what you think!

Summary

Hosts files are your computer internal address books that guide Web requests to the right servers. Fill your hosts file with domains pointing to nothingness, and those requests will quickly and surely fail. People are sharing their hosts file for years. Solutions now exist to concatenate them, and crowdsource a solution to the dirty Web we've to deal with everyday.

1. If you've never tested Brave Browser, I can only encourage you to do so, and join the 4 million people that trust it to fix the web. ↩

2. Actually, I put this function in my ~/.zprofile, as I use Oh My ZSH. ↩