Base API: Making the self hosted version

In my previous post I announced that Base API can now be self-hosted.

In this post I will show how I managed to compile the same application that is running as a SaaS software to be able to run as a single binary (with 90% of the same functionality).

The goal was to have one code-base for both versions, to keep them in sync feature wise and to make maintaining easier.

Stack

Currently Base API is running in two places:

• On Heroku - the backend application is compiled on deploy and the binary is running in a dyno. The only thing needed there is the Crystal Buildpack
• On Netlify - the frontend of the application is built on Netlify and served as a Progressive Web App

Self-Hosted vs. SaaS

The SaaS version has some features that are not needed in the self-hosted version:

• authentication and user management - since it's a service people are able to sign up, login, etc...
• limits on services - rate-limiting and email sending limits
• error reporting - runtime errors are reported to Sentry

Also the Self-Hosted version contains it's own features:

• status of the server - a dashboard that lists information about the server
• user and password authentication for the interface

Crystal

When I started to research I had some options to handle the two different versions:

• use an environment variable at runtime - this is easiest version to go with, the downside is that all of the application is compiled into the binary and by using the right environment variable the users could host their SaaS version which is not desired.

• use a compile time flag - this is the more complicated version, it has the upside of the binary only containing the code for the self-hosted version, but it's harder to test and compile.

I went with the compile time flag which can be used in macros like this:

{% if flag?(:standalone) %}
  # This will be in self-hosted version
{% else %}
  # This will be in the SaaS version
{% end %}

So in every case there is something present in one version and not the other version a macro like that needed to be added.

At the end I just need to compile with the flag:

crystal build src/base.cr -D standalone

and Crystal would do it's magic.

Mint

Since Mint does not have a macro system, the only way to go is with environment variables.

To check which version we are in I created a computed property on the main store:

store Application {
  get isStandalone : Bool {
    !String.isEmpty(@STANDALONE) 
  }

  ...
}

And everywhere I needed to check I just used that property for deciding what to do, for example on one of the routes:

/download {
  if (Application.isStandalone) {
    Window.navigate("/")
  } else {
    sequence {
      Application.initialize()
      Application.setPage(Page::Download)
    }
  }
}

You can provide the environment variables using a .env file when building:

mint build -e .env.standalone

Putting them together

Crystal has a nice shard for exactly this purpose: https://github.com/schovi/baked_file_system it allows you to "bake" files into the compiled binary.

The frontend after compiling, its baked into the final binary and served through a custom HTTP handler.

The static binary is compiled in a Docker container on Alpine linux.

Licensing

I needed to figure out how to limit people for using the software without permission, and for this I used Public-key cryptography.

There is a key pair - the public key is built into the binary and the private key is supplied to the server.

Every 10 minutes or so the self-hosted version checks if it can keep running or not (stops if any of the steps fail):

• it sends a request to the server with a unique id and a license key (which represents the user)
• the server then checks if the license key is valid or not
• if it's not valid then sends an error
• if it's valid then sings the payload with its private key and sends the payload back with along the signature
• the self-hosted server checks the signatures validity

This way I can check the validity of the license and if it's invalid the self-hosted server will just stop, also it is very hard to circumvent since it needs an active internet connection and the requests are random every time.

UX

So now I had the binary but it was not easy to use so I added a CLI to do migrations and start the server. The configuration can be read from a YAML file or from environment variables. Also logging needed to be unified and configurable.

It took around two days to do the modifications, if you have any questions let me know!