Familiar Day #1 - My new open source project for table-top roleplaying, built with modern technologies.

As part of this series, I'll be (sporadically) walking you through how I built my upcoming open-source project, Familiar, and what difficulties and challenges I (hopefully) overcome along the way.

What is Familiar?

Familiar is an all-in-one Tabletop Roleplaying game assistant, providing campaign tools, a virtual tabletop, and real time chat. It's named after a wizard's familiar, a small magical creature who assists them. As naming things is one of the hardest problems in computer science, I'm already partially there!

Eventually I'll need a cute logo though...

Who are you, and why should I care?

I currently lead a small team building distributed systems at a start-up called Wakelet, serving many hundreds of thousands of people a month and tens of millions of API calls. I've been building cloud native applications pretty much solidly for about 4 years now.

I understand the project is a little niche, but hopefully at the end of it, this series will have a great overview of how to build and architect a web app from nothing.

Enough rambling, on with the blog post!

Staring at a blank piece of paper is always the hardest thing about writing, and software is no different.

I always enjoy setting goals for myself, so let's start with setting some goals, and sketching some high level infrastructure.

Goals

I've added the goals to the README in GitHub, but I'll reproduce it here for posterity:

Modern front end, progressive web app, using best-in-class tools.
Responsive, sleek UI.
Performant, well architected back-end.
Cost effective infrastructure fully within GCP free tier - pay only for cloud DNS.
Take advantage of free tooling for open-source projects.
Full CI/CD.

Goals in depth

UI

Firstly, the UI. I'm no Android or iOS developer, but luckily Progressive Web Apps are here to the rescue. They're definitely not quite ready for primetime, but in my opinion they're the future.

The tools I'm going to be using are LitElement, because WebComponents are cool, and the library is really performant. Unfortunately it doesn't support Server-Side Rendering / isomorphic applications right now, but that's on the roadmap 😎.

I'm probably going to use some pre-made CSS and/or WebComponents, and I've been having a look at Bulma which looks quite cool, and I've used Material Design Components in the past. I'll be the first to admit I'm not the best at CSS, so I'm taking this as a learning opportunity.

For state management, Redux is a godsend for complicated projects. It's really the holy grail of state management, and there's not much more to say except that it's great.

Back End

Quick note on languages

I'm probably going to be using TypeScript, and GoLang.
I have a love-hate relationship with JavaScript, on the one hand it's amazing for quick prototyping, on the other hand you look away for 3 seconds and your new shiny framework is deprecated. Go is fast becoming my favourite language, it's simple, efficient and is amazing for microservice development.

There are plenty of modern languages out there that are great, but sticking to two languages means I don't have to keep context switching, and they're both great for web development.

Infrastructure and API design

The overall goal is to almost entirely fit into the Google Cloud free tier. In terms of raw compute this gives us 28 hours of App Engine to work with, alongside a free Compute Engine instance, and some Cloud Function Invocations.

Each service will be its own docker container, perhaps on Kubernetes, as managed Kubernetes is free on GCP (though I'm not sure how well that will work on one tiny free machine). I'm probably going to use Traefik to route to the internal containers and issue SSL certificates. The basic set up I'm aiming for with Traefik will NOT work in cluster mode, so that's an exercise left to the reader. All the other services will be stateless docker containers, so can scale horizontally with no issues.

For serving the front end, I don't really know what I'll use yet, but there are so many options, I'll probably pick something out of a hat and it'll work well.

The rest of the backend will be a set of microservices, potentially communicating via GRPC if I'm brave enough, otherwise just over JSON/REST, with a GraphQL API stitching them all together into one coherent whole.

For real time communication, I'll try to have it running on a separate server (possibly running on Compute Engine and everything else on App Engine). I'll use GraphQL subscriptions over Server Sent Events now that Edge is moving to Chromium and all other major browsers support it - this is pretty much finished and will probably be the first service to be open-sourced. Ideally I'll be able to get this serving over HTTP/2 as well.

Build & Test Tools

Build tools/dev-ops are invisible to the user, but you miss them if they're not there. I'm going to start here, to force myself into good habits, and make sure that I'm not handcrafting any infrastructure. Cattle not pets!

Infrastructure as Code (IAC) is great, many of the DSLs for IAC are not. I've used Terraform recently, and it's pretty nice, and cloud agnostic, so I'm definitely going to use it again.

For a build system, I have a lot of experience with Jenkins, which is great, especially with their declarative builds with Jenkinsfiles. However, managing a Jenkins server is overkill. I'm not going to be doing lots of builds. I'm going to evaluate a couple of build services like CircleCI and TravisCI, and pick one I find easiest.

For testing, I'm not entirely sure what's the best way to test WebComponents/ShadowDOM is right now. I really like Jest, but for DOM testing it includes JSDOM which doesn't support the Shadow DOM just now. Karma actually runs on real browsers, which do implement Shadow DOM, so looks a good choice. Running on real browsers also exposes some of their quirks which might not be revealed by something else. Hopefully I'll have some better insights into this after writing my first few tests.

To bundle the front end up, I'll try to use ParcelJS, because ideally I want something that just works, but I'll fall back to old trusty Webpack if I run into issues.