Install Debian 10 on Netgate SG-2220 via Serial Port with iSCSI Disk

This post is originally published on yoursunny.com blog https://yoursunny.com/t/2020/Debian-serial-iSCSI/

I get my hands on a Netgate SG-2220 network computer.
It comes preinstalled with either pfSense firewall software, or CentOS in the case of DPDK-in-a-box edition.
However, I'm more familiar with Debian based operating systems.
Can I install Debian on the Netgate SG-2220?

The Console Port

Hardware specification of the Netgate SG-2220 includes:

• Intel Atom C2338 processor, dual core at 1.7 GHz
• 2GB RAM
• 4GB eMMC storage
• one USB 2.0 port
• two 1 Gbps Ethernet adapters

Notably missing is a VGA or HDMI port to connect to a monitor.
Instead, this computer offers a mini-USB "console port".
It is a UART serial port with a USB-UART bridge chip already included, unlike the C.H.I.P $9 computer that only provides serial over pin headers.

I plugged a USB cable to connect the "console port" to my Raspberry Pi, and a /dev/ttyUSB0 device showed up on the Raspberry Pi.
Then, I can type the following command to open the console screen:

sudo screen /dev/ttyUSB0 115200

iSCSI Target

Netgate SG-2220 has only 4GB storage.
This may be enough for a basic installation, but is insufficient for my use case.
Not wanting to permanently attach a USB hard drive, I decide to try something different: an iSCSI disk.

I learned (a little about) iSCSI in 2009 when I interned at Microsoft MSN.
iSCSI is a network protocol that allows one machine (called the target) to export a disk drive for use in another machine (called the initiator).
The initiator would see the disk drive as a block device (e.g. /dev/sdc), which can be partitioned using standard tools like fdisk or parted then formatted.

I followed How to Setup iSCSI Storage Server on Ubuntu 18.04 LTS to setup an iSCSI target on my Raspberry Pi.
The backing store is a 16GB file on an old 160GB USB hard drive.

Steps to Install Debian 10 on Netgate SG-2220

Before these steps, open the console screen and have the iSCSI target ready, according to last two sections.

1. Prepare a USB flash drive with Debian 10 "tiny CD" mini.iso image.
   Plug the USB flash drive on the SG-2220, and connect the power cord.

2. BIOS screen should show up on the console screen, with a prompt "Press F12 for boot menu".
   Press F12 and select "1. USB MSC Drive USB Flash Disk 1100".

   

3. When Debian installer menu appears, use arrow keys to highlight Install, then press TAB key.
   The kernel command line should show up at the bottom of the screen.
   Delete -- quiet at the end, then type console=ttyS1,115200n8 (this has to be typed one key at a time, pasting does not seem to work).

   

4. The next few steps are same as any other Debian installation.

5. Upon reaching the "Partitioning" step, select Manual partitioning method.
   Then, select "Configure iSCSI volumes".

   

   The installer would ask for the IP address, username, and password of the iSCSI target.
   It then discovers available iSCSI targets.
   Select our iSCSI target, and continue.

   

   After that, it's time to create partitions.
   The root partition (mount point /) should be placed on the local drive "3.8 GB Generic Ultra HS-COMBO".
   The /home mount point goes on the iSCSI disk "IET VIRTUAL-DISK".

   

6. It would take about 20 minutes to get to the "Install the GRUB boot loader" step.
   Select the local drive /dev/sda only.

   

7. The last screen of the installer is "Finish the installation".
   Do not get too excited and hit the "Continue", because we still need to do a few adjustments.
   Select Go Back, and then in the installer main menu select Start a shell.

   

8. Now we get a shell, in the installer's context.
   To enter the "destination context" (installed Debian system), type these commands:
   

   mount --bind /proc /target/proc
   mount --bind /sys /target/sys
   chroot /target
   

9. Change the kernel command line in the GRUB boot loader to use ttyS1:
   

   vi /etc/default/grub
   # change this setting:
   #  GRUB_CMDLINE_LINUX="console=ttyS1,115200n8"
   
   update-grub
   

10. Configure the iSCSI initiator to login automatically, and mark /home as a network drive:
    

    vi /etc/iscsi/nodes/*/*/default
    # change these settings:
    #  node.startup = automatic
    #  node.conn[0].startup = automatic
    
    vi /etc/fstab
    # for /home mount point, change mount options from "defaults" to "_netdev"
    

11. Move /usr/local and /usr/share to the iSCSI disk:
    

    cd /usr
    mv local ../home/usr-local
    ln -s ../home/usr-local local
    mv share ../home/usr-share
    ln -s ../home/usr-share share
    

    /usr/local is where I need more capacity in my use case.
    /usr/share weighs 190MB and is not needed by the iSCSI initiator.

12. Type exit to return to the installer shell, type reboot to reboot into the installed Debian system, and quickly unplug the installer USB drive.

If everything went right, you'll see the login prompt in about a minute.

Mistakes and Lessons

I attempted the installation at least 5 times before getting the procedure right.
I'd share some mistakes I made, so that you can avoid them.

• Ubuntu does not support installation via the serial port. Although I like Ubuntu more than Debian, I have to settle for Debian.
• The mini-USB console port on the Netgate SG-2220 is ttyS1 as recognized by the Linux kernel. Thus, it's necessary to modify the kernel command line with console=ttyS1,115200n8. Without this setting, the kernel would use ttyS0 serial port, which might exist somewhere inside the computer but not accessible from USB-UART. This modification is needed both during installation and in the GRUB configuration of the installed system.
• Debian's initial ramdisk does not support iSCSI initiator. Thus, putting the root partition (/ mount point) on the iSCSI disk will not work. The root partition has to remain on the local disk.
• While the Debian 10 installer would write iSCSI initiator configuration to the installed system, it's configured with node.startup = manual. This default setting would cause "iSCSI login failed due to authorization failure" error. Both node.startup and node.conn[0].startup should be changed to automatic.
• In /etc/fstab, partitions on the iSCSI disk must be marked _netdev. Without this setting, the system may attempt to mount the partition before the network is ready.