Access inaccessible: SSH tunnelling 🔥🔥🔥

My mum always told me to avoid going to the dark and scary tunnels.

But SSH tunnels are fun and interesting! 🔌

SSH (stands for Secure Shell) is a way/protocol of connection and interaction with remote machine in a secure way. A very typical situation would be a cloud instance, which we need to access from our local device. For this purpose we can either use a public/private key pair or directly login with account credentials like this:

ssh -i ~/.ssh/some_key nick@remote_host

Now let’s imagine a different scenario:

We are developing our super-awesome NodeJS backend application which uses a database. And one of the integration tests requires a real connectivity to a database located on some remote instance.

Let’s imagine we are connecting to MongoDB:

mongoose.connect('mongodb://remote_host:27017/db’);

What to do? 🤔

Well, there is also an option to expose a database port on a remote machine, however it is definitely not very secure. Plus let’s pretend we don’t have ANY access to security groups and any other configuration, BUT we are able to SSH to that instance…

There is a way!

We can create a tunnel to that instance, and forward local port to the one on our remote machine!

Starting a tunnel is fairly easy:

ssh -i ~/.ssh/some_key nick@remote_host -L 127.0.0.1:27017:remote_host:27017 -N

What above code tries to do, is to establish SSH connection to remote_host using ~/.ssh/some_key, and forward any local connection (127.0.0.1) which goes to port 27017 to be redirected to the one on remote_host.

If there are any issues, you can also add -vvv flag for getting more logs.

-N in the end states to not execute any command on remote. (What man page says)

And now if we configure our NodeJS application to connect to our local machine localhost:

mongoose.connect('mongodb://127.0.0.1:27017/db’);

We should be able to reach our remote database in a secure way! Yes!

Sounds cool?

What about we have a little bit different situation:

Let’s imagine we still have our NodeJS application and remote instance (let’s call it instance A) we are able to SSH to. But this time, our database will be located on another instance (let’s call it instance B with hostname remote_host_2), and there is no way our local machine can reach instance B (since it has different security rules), BUT instance A can reach it. What now?

We can build a… longer tunnel! Just like this:

ssh -i ~/.ssh/some_key nick@remote_host -L 127.0.0.1:27017:remote_host_2:27017 -N

The only difference with our previous tunnel, is that we are redirecting traffic not to remote_host, but to remote_host_2 (which is our instance B).

When we make SSH tunnel to instance A, we are able to reach any other instance which is accessible by instance A (even if our local machine can’t reach it).

Awesome, right? But there is a little bit more tricky scenario…

Let’s imagine you are working with a client application which communicates to the master of multiple slave nodes. Master, as well as all slaves, has its own host name. We can access master via SSH, however slave instances are inaccessible for us. Master is able to connect to slaves. Forgot to mention: client application can’t be modified :)

Our client application sends HTTP request to the master, which responds with a list of slave nodes to be connected. Let’s say there are 3 slaves around, so we get 3 host addresses:

a.slave.host:2020
b.slave.host:2020
c.slave.host:2020

(Yes, ports are the SAME!)

And a master:

master.host:2020

We can build a tun… wait a second.

If our client application receives a list of slave hosts (which we don’t have an access to) from master, how can we build a tunnel for that?

1. Create virtual network interface on your local machine

What we are going to do is to create a dedicated virtual network address for each of the hosts on our local machine.

We can make a range of addresses from 192.168.1.201 to 192.168.1.203.
(But firstly check if these addresses are not already occupied on your machine!)

We need to create both IPv4 and IPv6

Just Google it for Mac or any other platform.

2. Associate DNS resolution of created local addresses with remote host names

We created three dedicated local addresses for each of slave nodes, but how would it help us? A lot, just follow up.

Now we need to declare a DNS networking rule: If there is a request pointing to a.slave.host, it should go to 192.168.1.201 (and the same for each node).

To declare DNS address resolution, let’s have a look at /etc/hosts file (on Mac and Linux):

##
# Host Database
#
# localhost is used to configure the loopback interface
# when the system is booting.  Do not change this entry.
#
127.0.0.1   localhost
255.255.255.255 broadcasthost
::1             localhost

This file represents an association of IP addresses with hostnames.

Check out the first line after comments:

127.0.0.1   localhost

It explicitly says that “localhost” host name will point to 127.0.0.1

And in our scenario we need to associate every created IP address with node hostname in the same fashion, like this:

##
# Host Database
#
# localhost is used to configure the loopback interface
# when the system is booting.  Do not change this entry.
#
127.0.0.1   localhost

a.slave.host 192.168.1.201
b.slave.host 192.168.1.202
c.slave.host 192.168.1.203

255.255.255.255 broadcasthost
::1             localhost

# These are IPv6 addresses which were declared during virtual network interface creation: 

1::2 a.slave.host
1::3 b.slave.host
1::4 c.slave.host

Now, if you try to navigate to any of those hostnames it will point to virtual address we created!

And now the final part: our tunnel on steroids!

3. The Tunnel

Now, once we have all pieces together, we can create our unbelievable tunnel:

ssh -i ~/.ssh/some_key nick@remote_host 
-L 192.168.1.201:2020:a.slave.host:2020
-L 192.168.1.202:2020:b.slave.host:2020
-L 192.168.1.203:2020:c.slave.host:2020 -N

What’s happening now?

Well, now when our client application will receive a list of hostnames to connect to, it will try to make a request to each of slave node hostnames, which are associated to dedicated local virtual network interface address which is tunnelled to an actual node!

IT WORKS! WE DID IT! 🔥🔥🔥

Special thanks to my colleagues who shared their knowledge regarding tunnels:

https://github.com/fxbonnet