My friend is a data scientist and I've helping him start making his first web app during lockdown. It's reminded me that there is a lot of App101 that you might not know even if you're already a programmer. So, I'm going to bundle up some of those discussions into these App101 posts to bring you up to speed on the decisions that you'll have to make when building a web app today.

Tales of Yore: Old Webapps and the Invention of the API

Let's start with a little bit of (oversimplified) history.

For the first 20 years the web existed, there was basically one way to build a web app that did more than just serve static content. You would get a database (often MySQL) and you would write a bit of software (often PHP or Perl) that would receive HTTP requests, interact with the database, and then return HTML with the next page to show the user. Basically the whole internet worked like Amazon.com: you'd click links or submit <form>s, those actions would cause HTTP GET or POST requests, and the server would return fully formed pages back to your browser.

That all started to change around 2000 with the invention of AJAX. AJAX used Javascript to move some of the work off the server and onto the user's computer. Now the browser could perform single actions without requiring a whole HTML page be sent over the wire. Instead, your code would call a special URL to get back just a snippet of HTML to substitute onto the page. This snippet later became structured data, first XML and then JSON, that would be used to calculate the HTML changes locally. This was the evolution of the modern web API - a structured set of URLs for loading or changing data on a server.

Monolith Webapps: The old school architecture

By 2015 the rise of mobile apps, a new class of non-html travelers on the internet, solidified the API as part of standard web app architecture. This meant the way we now build traditional webapps had been set:

This is the way that many modern app frameworks still work: Django, Ruby on Rails, etc. Many of these frameworks exist to try to simplify this setup by making the API and Web codebases easier to manage, or by automatically generating the Javascript for the website so you don't have to code your site twice.

Traditional apps built like this have their advantages. They are well-understood and traditional, they have decades of solid tooling built for them, and they generate websites that are roughly what browsers and search engines expect.

However, this architecture has a couple of clear issues as well. A minor one is that both the API and the web renderer have to interpret the data coming from your database and an inconsistency between those interpretations can cause some pretty weird bugs for your users.

A much more serious risk, though, is that both your web renderer and your API have direct connections to your database. This can create serious headaches because the database is often the most fragile and resource-limited part of your architecture, while your website is often your most exposed. That combination opens up a lot of surface area for bugs or hacks to appear. One of the most common is that heavy traffic might overload your database, as was often the case with Twitter in its early days. Meanwhile, a security issue with one of your pages might allow direct access to the data, as has happened many times with Wordpress over the years. While these risks are today often mitigated by well designed frameworks, the connection to the database remains a vulnerability.

Jamstack: The upstart

Around 2015 engineers began to ask, "what if we solve those issues by cutting the link between the web server and the database?" Since many of the larger apps already had full-featured APIs, it was a pretty natural leap to have the website consume data solely via the API. This takes the spider's web of dependencies above and changes it into a linear conga of data:

Once you do this, you find you get a whole host of benefits. Now, the only code you need to worry about having direct access to your database is your API. That gives you one location where proper escaping and rate limiting can protect your uptime. Plus, your API code is now the one source of truth for all your data, so you no longer need to worry about those pesky interpretation bugs.

What I love about the Jamstack is that from this starting point the community began to find even more advantages. Around this time Node.js was skyrocketing in popularity and having Javascript on the server meant that your server-side renderer could literally run the same code as the browsers would. Going even further, though, people realized the server-side renderer could run no code at all! Instead, you could render all your pages during your deploy and serve purely static files through a service like Netlify or a CDN like Cloudfront. This gave rise to a whole generation of tools like Gatsby, Hugo, and Jekyll.

However, JAMStack does come with its own issues and complexities. The main issue is that you have to now handle your API's code and deployment in addition to your webapp's. For lightweight projects, this is often unnecessary overhead. Even if you ignore the extra code, handling the deploys of the static files, the API, and potentially also the server-side renderer is clearly more work than a traditional single-deploy app would have been. But, for apps that might have to handle a lot of requests for static content (like a blog or marketing site) or web+mobile apps that will be dealing with similar requests from multiple different sources, the advantages you gain in simplifying your architecture outweighs the complexity you add to your deploy.

So which is right for you?

The JAMStack is still a very new development in the web world, even if it has its own conference and fancy .org website now. This means there's still a lot of innovation and improvement to be reaped there. You can see this in the development of frameworks that make generating Jamstack applications easier and in the roll-out of server products that make it seamless to deploy snippets of dynamic server-side rendering code alongside large chunks of static code.

However, the traditional form of webapp is far from dead. Single-deploy frameworks like Rails, Laravel, and Django are often still the right choice if you'd like to keep your code and deploy process simple at the expense of some hidden complexity in your architecture.

As for my friend who's working through App 101 right now, we decided the right choice was to work with Django for his project. It gave him a simple, single codebase in a language he was familiar with where he could build his API, admin pages, and database schema at once. I may love Jamstack, but it wasn't right for him because of the added complexity managing and deploying it. That may be the case for you too, or you might take to it as enthusiastically as I have. I'd love to hear what you decided to use for your projects and why.