SSH setup and tunneling via Bastion host

Overview

Follow the below steps to setup your SSH and enable tunneling via Bastion host or jump servers.

Generate SSH keypair

• Launch git bash from your laptop/pc in your home directory.
• Create a .ssh folder in your home directory and generate your ssh keypair in the .ssh folder by running the following commands(if you do not have any ssh keypair generated before)

mkdir .ssh
cd .ssh
ssh-keygen -t rsa -b 4096 -C <ID>@mailid

• The above will produce a public key ./.ssh/id_rsa.pub

• Add your SSH key to the ssh-agent in .bash_profile (can be found in your home directory, else you can create it using vi .bash_profile) and allow private key to be forwarded to AWS bastion server.
  

eval `ssh-agent -s`
ssh-add ~/.ssh/<private_key_name>
alias ssh=”ssh -A”

• To list how many keys you had in memory, you can run the following command. Take note max is 3 keys in memory.

ssh-add -L

Using PuTTY for tunnels

Use PuTTY Gen to create PPKs

• If you had ssh public/private keypairs, convert your private keys into putty’s ppk files, for use in Pageant.
• (Find your id_rsa and id_rsa.pub files)
• Recommended to place these in your USERHOME\ssh (C:\Users<YOURNAME>\ssh directory)
• Load an openssh private key, then save a putty private key to USERHOME\ssh

Run Pagent
Right-click on the icon in the taskbar, then add your keys. Click Add Key and navigate to USERHOME/ssh and select the .ppk you created earlier.

General process, for each hop:

Right-click on pagent icon → New Session
Add user_name@host 22 in Session tab
Connection Tab
-> Proxy
-> Select the radio button Local (port change from default 80 to 22 at the below), (for nonprod-jumpbox, leave it as None)
-> Configure the telnet text box Command (as per below)
Save each session and then create another. Careful not to overwrite your sessions!

• Create a new session and SAVE each session.

• Given the above setup, you can now test connecting to each hop: corp jumpbox, aws bastion, aws application instance.

Using PuTTY plink to tunnel RDS/etc to localhost
Use cases:

• Use local clients/tools to connect to RDS, other apps, over any port.
• Forward a web server running on a non-standard port to your local laptop. Steps:
• Setup the PuTTY sessions as per above steps.
• Use Command Prompt to run the command below (example is an Oracle RDS server)
• Use any client tool to connect to localhost:9001 (or whatever port you setup)

Oracle on RDS

• Open command prompt and run something like this.

• The command is broken down like the following
  

plink -L <LOCAL PORT>:<HOSTNAME OF RDS INSTANCE>:<REMOTE PORT> -N <HOST TO PROXY VIA>

LOCAL PORT: the port you want to connect to locally.

HOSTNAME OF RDS INSTANCE: the dns/fqdn or IP address of the hostname in AWS.

REMOTE PORT: the port exposed by the instance

HOST TO PROXY VIA: the bastion or instance that you want to use as a proxy.

Using bash for tunnels
The general process, after setup, is:

• Open bash on your local machine
• Start sshagent (see below)
• Add ssh keys as required (usually just your private key that you’ve previously shared)
• Setup ssh config (see below)
• Then you’re able to SSH to remote servers in AWS.

SSH Agent Setup and Adding Keys

• Before you can ssh through to some destination server, for bash shells you need to have your sshagent running, and your keys loaded. I use an alias in my ~/.bashrc

alias sshagent='eval $(ssh-agent -s)'

• Now we can start up the agent, ready for our keys. Here’s an example of me doing this:

ak@sys  ~
$ ps -ef | grep ssh
ak@sys  ~
$ sshagent
Agent pid 7552
ak@sys  ~
$ ps -ef | grep ssh
ak    7552       1 ?        10:56:38 /usr/bin/ssh-agent
ak@sys  ~
$ ssh-add ~/.ssh/id_key
Enter passphrase for /c/Users/ak/.ssh/id_key:
Identity added: /c/Users/ak/.ssh/id_key (/c/Users/ak/.ssh/id_key)
ak@sys  ~
$ ssh-add -l
4096 SHA256:aRY7RpFoA3Q4Mbgb343d2e05RaKTVzp66IoL+qedfW /c/Users/ak/.ssh/id_key (RSA)

SSH Config Setup

• Using git bash on my laptop, my ssh config looks like this:

ak@sys  ~
$ cat ~/.ssh/config
Host *
    ServerAliveInterval 60
    ForwardAgent yes
    StrictHostKeyChecking no

# nonprod host
Host <ip>
    User ec2-user
    IdentityFile /c/Users/ak/.ssh/id_key
    ProxyCommand ssh -qW%h:%p nonprod-bastion
    UserKnownHostsFile=/dev/null
# Application server
Host appserv
    HostName <host>
    User ak
    IdentityFile /c/Users/ak/.ssh/id_key
    ProxyCommand ssh -qW%h:%p nonprod-bastion
    UserKnownHostsFile=/dev/null

# Bastion
Host nonprod-bastion
    HostName <host>
    User ak
    IdentityFile /c/Users/ak/.ssh/id_key
    #User ec2-user
    #IdentityFile /c/Users/ak/.ssh/id_key
    ProxyCommand ssh -qW%h:%p jumpbox
    UserKnownHostsFile=/dev/null

# Jump Box
 Host jumpbox
    Hostname <ip>
    User ak
    IdentityFile /c/Users/ak/.ssh/id_key
    UserKnownHostsFile=/dev/null

What is SSH Agent Forwarding?
If you look at above ~/.ssh/config, you’ll notice “ForwardAgent Yes” at the very top.

Read more about that, here:

[http://www.unixwiz.net/techtips/ssh-agent-forwarding.html]

Using bash to SSH into a remote instance
With the above all setup, I can simply do:

ssh appserv
ssh <ip> (some ec2 server for my app)

Using bash to tunnel to RDS
Using the previous ssh config, I can therefore use local port forwarding to connect to the RDS instance:

ssh -nNT -L 9000:<dbhost>:3306 nonprod-bastion

Note: If there is a need to ssh tunnel via your application server (replace nonprod-bastion with the host which you configure under SSH config setup, see example if your app is appserv.) Example:

ssh -nNT -L 9000:<dbhost>:3306 appserv

Troubleshooting

Removing a host from known_hosts (bash)

Happens because you’ll sometimes ssh to an instance using a domain name, and then a new build gets deployed with a new IP. Remove from your local known_hosts by:

ssh-keygen -f "/home/ak/.ssh/known_hosts" -R <host>