Deploy .NET 6 Web Application With GitHub Actions To Self-Hosted Linux Machine (Virtual Private Server, Raspberry Pi, etc.)

Let's suppose you are that kind of developer that holds code on GitHub but deploys his fantastic private application manually to some Linux based Virtual Private Server (VPS) you pay $5/month, or to home Raspberry Pi (because why not). You are not alone; a lot of us do that. There is no issue there, but it can be a much better experience.

More advanced users have some script to build to the output folder and then copy it to the server using Secure File Transfer Protocol (SFTP). Not bad, but still not perfect. Different branches - different rules (unless you push everything directly to the main/master branch. I won't tell anyone, but I know you, we will do the same here).

GitHub Actions

GitHub Actions are a new type of CI/CD (Continuous Integration, Delivery) server that has been released by GitHub recently. They integrate closely with GitHub, and the GitHub repository can link GitHub Actions, so they will only run when the GitHub repo is updated.

Running applications on self-hosted runners provides more control over the hardware, operating system, and software tools than GitHub-hosted runners. You may use self-hosted runners to build a customised hardware configuration with greater processing power or memory to execute bigger jobs, install programs available on your local network, and select an operating system that GitHub-hosted runners do not offer.

The process

I will be using Raspberry Pi with Ubuntu Server 20.04.3 LTS, but the same process we can perform on many other Linux distributions.

Foreword

If your application uses a local database to persist the data, that part is not covered here, but you can do it independently of this tutorial.

Step 1 - Installing .NET 6.0 SDK

There are always two parts of .NET applications - building and running ** them. That's why when you visit Download .NET 6.0 (Linux, macOS, and Windows), you will see at least two things - **SDK and Runtime. The former includes everything you need to build and run .NET Core applications, while the latter includes everything just to run them.

We will be building and restoring packages for our application on our machine, so we need to install the SDK part. It also includes the Runtime, so we don't need to install it separately.

From Microsoft documentation: Package manager installs are only supported on the x64 architecture. Other architectures, such as ARM, must install .NET by some other means such as with Snap, an installer script, or a manual binary installation.

Case 1 - CPU on your machine IS x64

If you have Linux operating system besides Ubuntu 20.04 LTS, you should visit Install .NET on Linux Distributions | Microsoft Docs documentation and act accordingly. Installing the SDK on Ubuntu 20.04 LTS on x64 architecture will need:

wget https://packages.microsoft.com/config/ubuntu/20.04/packages-microsoft-prod.deb -O packages-microsoft-prod.deb
sudo dpkg -i packages-microsoft-prod.deb
rm packages-microsoft-prod.deb

sudo apt-get update
sudo apt-get install -y apt-transport-https
sudo apt-get update
sudo apt-get install -y dotnet-sdk-6.0

This step installed all the necessary dependencies, and we are ready to go.

Case 2 - CPU on your machine IS NOT x64 (e.g. Raspberry Pi)

There is a slightly different setup for Raspberry Pi because it has an ARM64 CPU. We need to install the Runtime manually or with a script - I prefer the script way (If you are a bit sceptical about the file, you can check the documentation here.

curl -sSL https://dot.net/v1/dotnet-install.sh | bash /dev/stdin -c 6.0

Wait for the process to complete, and the .NET SDK will put files in your current directory ~/.dotnet. To have dotnet in our PATH for a simple resolution, we can run:

sudo ln -s "$HOME/.dotnet/dotnet" "/usr/bin/dotnet"

If you don't want to add dotnet to the PATH, you will need to invoke the dotnet command from the directory $HOME/.dotnet/dotnet.

Check if dotnet installation completed without any issues:

dotnet --version

It is unnecessary to reboot your machine, but as I use Raspberry Pi as a DHCP server, I noticed some problems obtaining the IP address, but reboot solves it.

Since we installed .NET 6 SDK manually (using the script), we need to ensure that we have the required dependencies for the .NET by running:

 sudo apt install libc6 libgcc1 libgssapi-krb5-2 libicu66 libssl1.1 libstdc++6 zlib1g

Our machine is now ready to build and run .NET 6 applications, but this is still not the end. We need to let GitHub Actions know where to deploy, build, and run the code. Back to the GitHub repository.

Step 2 - Create a self-hosted runner

You can add self-hosted runners to a single repository. To add a self-hosted runner to a user repository, you must be the repository owner.

• Navigate to the main page of the repository
• Click Settings
• Select Actions from the left sidebar
• Select Runners submenu
• Click on New self-hosted runner.

In the next step, you need to choose what type of operating system and CPU architecture. If you are using Raspberry Pi 3 and higher versions, you probably have ARM64, but anyways, you can check that quickly by running:

uname -m

If you see something like aarch64, it means an ARM64 CPU. If you are on VPS, it will probably show something like x86_64, which means your machine is running on Intel's 64-bit CPU. GitHub will provide you with the information needed to install the active-runner on your device. Follow it to the tee, but you can skip running the script with ./run.sh and Using your self-hosted runner part because we will cover it later.

The final screen should look like the image below. If you get, unexpected errors like An error occurred while sending the request or Resource temporarily unavailable, retry the command that caused it.

Run a self-hosted runner as a background service

In the previous configuration, we said to skip running ./run.sh because we want it to be a background process. Maybe someone likes it or uses it because of some security considerations, and it's OK, but the background process is my preferred way.

Action-runners come with an option for installing runners as a service in the background. It will enable it by default, so the next time your machine restarts, the service will be up and running.

sudo ./svc.sh install
sudo ./svc.sh start

Step 3 - Setup the application directory

We created the action runner that will listen for the changes from GitHub Actions. Now we need to create a directory that will hold the application files. You can set it anywhere available, but I like to keep everything in /var/www/<name_of_the_app>.

sudo mkdir /var/www/sample-app
# I will give the directory ownership to the user `ubuntu` and its group
sudo chown ubuntu:ubuntu /var/www/sample-app

Now we have a directory where we can output applications build assets.

Step 4 - Create a GitHub Actions workflow

We created a self-hosted runner on our machine; now, we need to create a GitHub Action workflow to perform our deployment.

• Go to your GitHub repository
• Select the Actions tab
• Click on set up a workflow yourself.

GitHub will create a prepopulated file with a default filename which you can change. Replace the content of the file with the following:

name: CI

# Controls when the workflow will run
on:
  # Triggers the workflow on push request event for the master branch
  push:    
    branches: [master]

jobs:
  deploy:
    # Our previously created self-hosted runner
    runs-on: self-hosted

    strategy:
      matrix:
        dotnet: ["6.0.x"]

    # A sequence of tasks that will execute as part of the job
    steps:
      # Checks out repository so our job can access it
      - uses: actions/checkout@v2
      - name: Setup .NET Core SDK ${{ matrix.dotnet-version }}
        uses: actions/setup-dotnet@v1.7.2
        with:
          dotnet-version: ${{ matrix.dotnet-version }}

      - name: Install dependencies
        run: dotnet restore

      - name: Build
        run: dotnet build --configuration Release --no-restore

      # We will output publish files to the folder we previously created
      - name: Publish
        run: dotnet publish -c Release -o /var/www/sample-app

Once we commit the file, GitHub will place it in the .github/workflows/ folder in the root of our repository, and the workflow process will start right away (if we did everything correctly). You can see the output of the workflow in the Actions tab:

Step 5 - Create a background service for the application

You probably already figured out that we don't have the dotnet xxxx.dll command in our GitHub Actions workflow YAML file. Of course, there is a reason. When we start the application using the dotnet command, it is triggered only as a foreground process, and in our case, the GitHub Actions workflow will wait as long as the application is running and won't complete.

We could use something like nohup, but it would create a new process every time we run the application (not to mention that we didn't specify the port on which it should run). Also, if our machine reboots, our application will be down until we start it manually.

# It may look cool, but we don't want this
nohup dotnet SampleApp.dll > /dev/null 2>&1 &

While there might sometimes be better tools for the job, always consider using existing software first, and writing our service can give us a level of flexibility.

To solve the issue, we will create a Linux user service that will:

• Run our application after reboot
• Automatically restart the application if something internally happens and the application fails
• Ability to restart the application from a deployment script (GitHub Actions)

We create services in the /etc/systemd/system directory, but we would need elevated privileges to control them by default. Sure, there are ways to run systemd services without using sudo by tampering with sudoers files or configuring PolKit, but we will approach it differently. In the end, we want to give our GitHub Actions user the ability to run and restart the service without using sudo or PolKit configurations.

We must locate user services in the ~/.config/systemd/user/ directory. Let's create it:

# Using -p argument to create the whole path if it does not exist
mkdir -p ~/.config/systemd/user/

First, you need to enable lingering for the user on which we installed GitHub Actions runner. In our case, that is the ubuntu user, and it is necessary because it allows users who are not logged in to run long-running services. Otherwise, the user services will start on user login instead of boot.

loginctl enable-linger ubuntu

Next, we need to create a systemd unit file for the user (the service). We can name it however we want, but I prefer logical names, such as name-of-the-application.service. Place it under the user's systemd instance.

nano ~/.config/systemd/user/sample-app.service

There are a couple of crucial things for which we prepared in previous steps, and we need to make sure we configure them correctly. The final configuration of sample-app.service should look like this:

[Unit]
Description=Example .NET 6 Application

[Service]
# This is the directory where our published files are
WorkingDirectory=/var/www/sample-app
# We set up `dotnet` PATH in Step 1. The second one is path of our executable
ExecStart=/usr/bin/dotnet /var/www/sample-app/SampleApp.dll --urls "http://0.0.0.0:5000"
Restart=always
# Restart service after 10 seconds if the dotnet service crashes
RestartSec=10
KillSignal=SIGINT
SyslogIdentifier=sample-app-log
# We can even set environment variables
Environment=ASPNETCORE_ENVIRONMENT=Production
Environment=DOTNET_PRINT_TELEMETRY_MESSAGE=false

[Install]
# When a systemd user instance starts, it brings up the per user target default.target
WantedBy=default.target

We set --urls "http://0.0.0.0:5000" in ExecStart for Kestrel to listen on a specific port to test our application. I usually don't use Kestrel as an edge web server, but in a reverse proxy configuration with Nginx. For this tutorial, we will use Kestrel directly for brevity.

If you are interested in the reverse proxy configuration, the following link describes it in great detail.

After we are sure that we have the correct configuration, we can save the file with Ctrl+X, Y. Let's reload systemd and enable our service to run even after the reboot.

# Reload `systemd`
systemctl --user daemon-reload

# Enable the service at the next boot
systemctl --user enable sample-app.service

# Start it right away
systemctl --user start sample-app.service

It is pretty easy to create a service on Linux, but it does not mean that it is the best solution. This tutorial didn't cover security considerations regarding the usage of systemd services.

Step 6 - Update the GitHub Actions workflow file

Once we have our application service ready, we can update the GitHub Actions configuration to include our newly created service to run our application every time we push new code to the deployment branch.

Edit the GitHub Actions workflow file created in Step 4 under the <repository>/.github/workflows/ folder. As the last step of our workflow, we will add:

      - name: Restart the app
        run: |
          export XDG_RUNTIME_DIR=/run/user/$(id -u)
          systemctl --user restart sample-app.service

Our final workflow YAML file should look like this:

name: CI

# Controls when the workflow will run
on:
  # Triggers the workflow on push request event for the master branch
  push:    
    branches: [master]

jobs:
  deploy:
    # Our previously created self-hosted runner
    runs-on: self-hosted

    strategy:
      matrix:
        dotnet: ["6.0.x"]

    # A sequence of tasks that will execute as part of the job
    steps:
      # Checks out repository so our job can access it
      - uses: actions/checkout@v2
      - name: Setup .NET Core SDK ${{ matrix.dotnet-version }}
        uses: actions/setup-dotnet@v1.7.2
        with:
          dotnet-version: ${{ matrix.dotnet-version }}

      - name: Install dependencies
        run: dotnet restore

      - name: Build
        run: dotnet build --configuration Release --no-restore

      # We will output publish files to the folder we previously created
      - name: Publish
        run: dotnet publish -c Release -o /var/www/sample-app

      - name: Restart the app
        run: |
          export XDG_RUNTIME_DIR=/run/user/$(id -u)
          systemctl --user restart sample-app.service

Once you deploy the task, you can track its status in Actions tab on GitHub repository. Now you can access your web application over the assigned URL and port.

Final words

I hope this text will help you complete that 2% of "what’s left of the project" and make deployment easier. If you find anything unclear, please comment here, and we will try to solve it if possible. Thank you for reading!