Securing our internal tools

Using Cloudflare Access to lock down our self hosted internal tools

At ente we self host a bunch of open source tools. While it takes a bit of effort on our part to set up and maintain these, it helps reduce the amount of data we share with third parties. We use Sentry for error reporting, Grafana for monitoring and Metabase for product metrics (to help folks like me who don't speak SQL).

While we have systems in place to secure the credentials to these services, we wanted to introduce an additional barrier to entry. The obvious answer was to restrict the visibility of these services to an internal network. But looking at Wireguard's landing page made me realize that we'd need a full-time dev-ops person to go down that path.

I started wondering "how has Cloudflare not solved this already", and decided to scrape their maze of a website. And duh-doy, they had already solved this indeed.

Introducing, Cloudflare - Zero Trust Access.

Simply put, Zero Trust Access places a login page in front of your endpoint that authenticates users (across a wide variety of vectors), and stores the result in a cookie, that is from there on passed to Cloudflare with every subsequent request.

Email verification with a "One-time PIN" is provided out of the box, that can be coupled with domain-specific rules to serve as a starting point.

In our case, it also made sense to add Github as an authentication vector, with a rule that verified the presence of the user within our org.

We went ahead and placed all our internal tools behind Access, and all seemed hunky dory.

Until.

We realized that this had broken error reporting to Sentry. All POST requests from clients to report errors started failing due to the newly enforced authentication requirement.

To work around this, Cloudflare provides Service Authentication. Which is just a fancy way of saying "pass us these secrets in headers and we'll let you through". We setup a Cloudflare Worker as a proxy to attach these headers for us, and configured the Sentry SDK to use this Worker as a tunnel, and all has been well since.

We found Zero Trust Access to be a simple way to protect internal endpoints. We later on learned that Google has also been in this game for a while with Beyond Corp.

If you have more insights on what we could do better, let us know by writing to security@ente.io. If you simply want to follow our trysts with dev-ops, follow us on Twitter, or come say hello on Discord.