Background - Why might you need a reverse proxy?

Web development tooling has come a long way, and most frameworks and SDKs have made previewing your app as you work a breeze; often it is as simple as running something like npm start from your CLI. But an unexpected hurdle that many web developers hit, and can seem much more complicated, is "how do I use this outside of my local network?". Things like screen-sharing, screen recording, and VSCode's Live Share can solve this issue if all you need to do is show your localhost demo to your remote coworker. But if you need to actually expose your code to the outside world, those tools fall short.

For example, a very common scenario is trying to test a third-party service (Slack, Github, Twilio, etc.) that talks to your codebase, maybe through a webhook. In this scenario, you can't run the third party code locally (e.g. running all of Github's platform on your computer), and if you put something like "localhost:3001/webhook" as the webhook, that third party is going to be unable to reach the code running inside your local network (for good reason!).

How do we solve this?

Reverse-Proxies

The common solution to this issue is a reverse proxy combined with ssh tunneling. You have a public URL, which points to a public server, which uses a reverse proxy to hand-off the request to a different port and/or server, which in turn gets proxied / tunneled via a SSH tunnel between the server and our local development computer. This is much easier to show visually, so here is a simplistic diagram showing this:

There is a lot that goes into this solution. You need to run a public server, set up SSH support on it, have a domain name, DNS entries, SSL certificates; the list goes on and on. It's not all that complicated (here is a simple guide), but it is also not free or quick.

Quick, Simple, and *Free - Ngrok

Given the complexity of setting up a full reverse-proxy through a SSH pipeline, services have sprung up to simplify this for developers.

One of the most popular and easiest to use is Ngrok. With a single command, you can instantly get a public URL that you can use to access your localhost from anywhere - no public server or complicated setup necessary!

For example, to relay HTTP requests from outside your home network to your localhost on port 3001, you could use ngrok http 3001 and be up and running in seconds!

Ngrok basically takes the place of both the SSH tunnel setup (with its CLI client), and the public server with a reverse proxy - the second and third blocks in my diagram up above. It even comes with built-in tools, such as a dashboard, request inspector, and replay functionality.

Ngrok is also recommended by several well-known third-party platforms (Twilio, Github).

I am not affiliated with Ngrok in any way; I just think they have an impressive service.

The downside to Ngrok

Although Ngrok has a very generous free tier, one major restriction is that the public URLs that are generated are both random and not assigned - meaning that each time you start it up, you will get a different random subdomain, like https://df12f52a.ngrok.io. If you are trying to develop webhooks or something similar, this means that each time you need to restart Ngrok, you would have to login into your third-party platform, find the admin dashboard, and copy and paste your new Ngrok URL into the GUI. Or, if you are sharing URLs with coworkers, you would need to keep sending them updated URLs.

Ngrok does offer custom subdomains or personalized domains, but these come under the paid option, which starts at $5 a month. Not much, but this is basically the same price as running Ngrok or Nginx reverse proxy yourself on a paid server, like Digital Ocean.

My Solution - Add Another Proxy with Cloud Functions!

This is the kind of solution I am simultaneously both proud and ashamed of. Because I am cheap, and didn't feel like buying a paid Ngrok plan just to test webhooks a few times a year, or setting up a dedicated server for it, I started thinking of how I might use free, or very low cost workarounds. What I came up with is kind of silly, but it works - just add another proxy, with a fixed URL, in front of the changing Ngrok URL!

Let me elaborate. As I researched options, I noticed that cloud functions (Google Cloud Functions or AWS Lambda) were often the cheapest to use, in that they start at a scale of zero by design (they only run when requested). At first I was thinking of replacing Ngrok with them entirely, but this quickly was squashed; they are designed to execute quickly and exit, and would not work for maintaining a SSH tunnel connnection.

However, I realized two important things; cloud functions often get a semi-permanent PUBLIC url to invoke them, and they can handle both incoming and outbound requests. With this in mind, I realized I could setup a cloud function with a public URL, that simply proxies inbound requests to a re-configurable Ngrok address. Here is what that looks like:

In this scenario, my Cloud Function public URL stays static, even as the Ngrok address it points to changes!

The code required to build this is very simple too; it can be assembled with just a little NodeJS, Express, body-parser middleware, and, most important, http-proxy-middleware.

The only caveat here is that the cloud function has to be redeployed each time that the Ngrok URL changes. However, this is very easy to automate; in fact, I already have a command set up in my package.json, so if my Ngrok URL has changed, all I need to do to redeploy with the update is type npm run ngrok-deploy.

GCP

I ended up going with Google Cloud Platform to provide the serverless function that proxies requests. Their free tier includes 2 million (!!!) executions per month, plus 5GB of network traffic. This is WAY more than enough for some simple webhook testing.

You can find all my code here, which proxies inbound requests to a configurable destination: github.com/joshuatz/gcp-proxy-func.

Another cheap alternative

Besides the obvious routes of paying for a pro Ngrok plan or paying for a server to run your own reverse-proxy, there are some other alternatives I have yet to explore.

One option could be to wrap a reverse proxy in a container, and deploy it on a platform that supports "scale to zero" container scaling, and have it "wake up" on an SSH connection. Then, you would only have to pay for it while you are actively connected via the SSH tunnel and using it for developing.

This post first appeared on: https://joshuatz.com/posts/2019/using-google-cloud-functions-permanent-url-to-proxy-ngrok-requests/