How to make Cloud Run talk to Cloud Run - The private way

Cloud Run is no longer to be presented among the GCP container services. It is generally an efficient way to deploy web API applications without the overwhelming need of deploying and managing a Google Kubernetes Engine (GKE) cluster.

When it comes to accessing a Cloud Run instance it is also really easily configurable to be accessed from the internet. But if you are in a production environment, or you just don't need your instance to be accessed from the outside, there are some private networking solutions.

Like any other services, Cloud Run offers some out of the box features but some use cases may still require multiple options or services to be combined together.

One of our clients that uses Cloud Run for internal purposes only, wanted one of his Cloud Run instances to access another one, still without going over the internet but using private internal paths.

In this blog post, I will go over the Cloud Run networking options and details about how we finally did make a source Cloud Run service access a destination Cloud Run service using a private network.

Note: Cloud Run can be accessed with authenticated or unauthenticated requests, this aspect will not be developed in this article.

A bit of Cloud Run mechanics

First of all, we could ask ourselves: why would it be tricky to make two Cloud Run instances communicate with each other?
Whereas it is child's play to achieve it with two Compute Engine VM instances!

Well, the main difference is that by default, Cloud Run is not a resource that is "attached" to a VPC the way a GCE instance is.

You may know that every Cloud Run service deployment comes with a (ugly?) default URL that looks something like: https://hello-g6bc2cfcrq-uc.a.run.app. Which is the only endpoint that is given by GCP when we create a service.

As mentioned, one available option is to configure the instance to be accessible from the internet and use that URL from our source. But it is not what we want.

At this point our puzzle looks like this :

So, how is it possible to make a destination Cloud Run service to be accessed from private connections ? And a source Cloud Run service to access it ?

Let's find out.

Cloud Run Ingress control

On one hand we want to make our destination service accessible only from the internal network.

Two options are available when it comes to configure a Cloud Run service inbound traffic:

1. Allow ALL traffic: which means our service is accessible from the internet.

2. Restrict the access to "internal" requests only, with the option to also allow requests from GCP external Load Balancers.

In our case, the internal option is the one that is used by our client to satisfy his requirements.

Now we know that we only allow "internal" access to our destination service, we need to find a way for our source service to send requests through a VPC to be considered as internal ones.

Cloud Run egress options

The idea is to direct our source service traffic to the VPC, and for that Google has implemented Cloud Run "networking options":

Outbound options
1. Serverless VPC Access Connector: backstage this method creates a VM which is used to proxy requests from Cloud Run and forward them to the VPC.

2. Direct Access: this is the new option available since a few months in Cloud Run and still in "Preview". Direct Access avoids the creation of additional resources (and so the associated costs), and is supposed to have more throughput, and lower latency.

Be aware that when setting up any of those, you need to attach it to a VPC subnet, and both will consume private IP addresses.

Additionally we have to choose between two routing options that will allow us to route our source service traffic in different ways:

1. Route only requests to private IPs to the VPC

2. Route all traffic to the VPC

Choosing either option will depend on some more parameters that we will see a bit later.

That starts to make quite a few options combinations, but is that enough to access our destination service ?
Not yet...

Access Cloud Run endpoint

We saw that by default, to access a Cloud Run service, the only endpoint is its (ugly!) URL (*.run.app).

So we need either to be able to reach this URL from our VPC, or potentially to create a different endpoint.

If using the default URL is ok for you, the first option you have is Private Google Access (PGA).

To make it simple Private Google Access is a one box option to check on your subnet configuration, and it enables resources with solely a private IP address to access GCP public APIs by staying inside the GCP's network.

Source: https://cloud.google.com/vpc/docs/private-google-access#example

Magical isn't it ? Yes, but there are counterparts.

1. As you use the URL of your destination service, which is by default resolved as a public URL, you must configure the routing options of your source service to send "ALL TRAFFIC" to your VPC. That could be an issue if you want your Cloud Run instance to access the internet and configured your it to use Direct VPC. Indeed, one Direct VPC limitation is that it is not compatible with Cloud NAT, which is the only way to give private instances a way to reach the internet. If your source service needs an internet access and you enable PGA, you have to use a Serverless VPC Access connector instead.

2. The IP address logged for the incoming request will be 0.0.0.0/32, so it could be difficult to do specific network tracing.

3. You still use the ugly default URL to access your Cloud Run service.

What about the other options ?
Even if I will not go into the details, just know that if you have specific needs like accessing Cloud Run without directing all traffic to the VPC, or using an IP address or a custom DNS name, it is possible by:

1. Using an internal Load Balancer, with the Cloud Run service as a backend
2. Using a Private Service Connect (PSC) endpoint with an associated Cloud DNS entry.
3. Configure DNS to use the public URL as an internal endpoint so that it is routed to the VPC.

If you want me to detail those other options in a dedicated article, let me know in the comments section.

Our final configuration

Putting all of those pieces together lead us with the following final solution for our client use case:

• We used PGA and routed all traffic to the VPC.
• As the source Cloud Run instance needed an internet access, we used a VPC Access connector coupled with a NAT gateway.

We finally came to this final configuration:

I hope you enjoyed this article and that helped you to understand a bit more about Cloud Run private networking.

Thanks for reading!

---

I’m Kevin, Cloud architect at Stack Labs.
If you want to join an enthusiast Infra team, feel free to contact us.