Yet Another Tour of an Open-Source Elm SPA

Introduction

About 7 years ago, in the midst of writing Elm in Action, Richard Feldman developed rtfeldman/elm-spa-example, wrote Tour of an Open-Source Elm SPA and graciously shared both of them with the Elm community. The community's response was overwhelmingly positive and it was clear that he had addressed a major need. If you were one of the many web application developers asking "Where can I find an open-source example of an Elm Single Page Application?", then, the Elm SPA Example instantly became the canonical example that everyone was going to point you towards. This was a landmark achievement in the history of Elm.

However, since 2018, Richard set clear expectations on the effort he would be able to devote to the repository. And, nearing the end of 2023 it became clear that his focus shifted towards Roc and raising a family.

I'd really like to go back and update elm-spa-example. ... After a while, it's like, yeah this is never happening.

— Richard Feldman at 3:23

Fast-forward to now, circa April 2024, and unsurprisingly, the repository has been unmaintained for 5 years and the demo is broken in a few places.

In light of all this, it became exceedingly clear that someone else needed to step in and help. Why not me? Well, it can be me. And, after 3 months of development, I am happy to announce (again) dwayne/elm-conduit (demo), an open-source Elm SPA for RealWorld's Medium.com clone.

dwayne/elm-conduit (demo)

dwayne/elm-conduit is built from scratch using the full power of Elm, no holds barred. This is how I would architect and build a reliable, maintainable, and scalable production-ready Elm web application.

It uses devbox, Elm 0.19.1, the latest Elm packages (in particular elm/http 2.0.0), elm-review, Caddy, a sprinkle of Dart Sass, and a handful of Bash scripts (one of them being a deployment script). It uses elm test and features tests for key data structures.

In short, if you were asking:

Where can I find an open-source example of an Elm Single Page Application that is well-maintained, uses the latest Elm libraries and tooling, and has a build and deployment story?

Then, dwayne/elm-conduit is your answer.

Process

I started out with an HTML/CSS prototype, built the views in a Storybook-like sandbox and finally put it all together with domain logic, interactivity, and API requests.

HTML/CSS prototype

This step was made easier by the fact that RealWorld already had frontend specs with templates. So I didn't have to develop the HTML/CSS from scratch. However, I still made a few tweaks to the existing templates.

The source code for the prototype.

Storybook-like sandbox

This is the step where I went from HTML/CSS to Elm. The goal of this step was to realize all the views in Elm. I've found that if I'm able to independently work with the views outside the context of the application then I can be 100% confident that I'm not introducing unnecessary dependencies between my view code and the domain logic.

During this step I also got hints of the supporting data structures, the Data.* modules, that would be needed.

The source code for the sandbox.

Domain logic, interactivity, and API requests

Finally, it's in this step where I started to build out Main. I built the router, Data.Route, the reusable Api module, and the port infrastructure, Lib.Port.Message and Port.Outgoing. From there, I constructed the pages from least to most difficult based on what I learned when building the sandbox. As I put together a page, I would continue to flesh out Main (the page coordinator), Data.* (domain specific data structures), Lib.* (reusable library functions and data structures), and Api.* (the Conduit API) as needed for that page to be completed.

Module structure

When Richard first wrote rtfeldman/elm-spa-example he used a module structure similar to what you see above. However, when he upgraded his application to Elm 0.19 he decided to completely rearchitect everything and move to a different structure. Why? We don't know. But, whatever his reasons, I'd still be partial to the original structure because you end up with a cleaner separation of concerns and a "louder architecture".

The Main module

This is the entry point to the application and the page coordinator.

When you first load the application in the browser, it is Main that determines whether or not you're logged in. It decodes your configuration, passed through flags, and tries to determine who's logging in.

It uses your URL from the browser's location bar to determine which page you're trying to access.

The Page.* modules

Examples: Page.Article, Page.Editor, Page.Home, Page.Login, Page.Profile

These modules contain the logic for the individual pages in the web application. A page is responsible for fetching its own data and for composing its look.

The View.* modules

Examples: View.ArticlePreview, View.AuthErrors, View.Comment, View.FollowButton, View.Tabs, View.TagInput

Each module contains one or more reusable view functions, fully decoupled from the domain logic, which multiple Page modules import.

Some, like View.AuthErrors, are very simple. Others, like View.Navigation, are a little more complex. Each exposes an appropriate API for its particular requirements. All are featured in the sandbox.

Interestingly, no View.* module needed an update function. This correlates exactly with this user's perceptive commentary.

The Api.* modules

Examples: Api.CreateArticle, Api.GetArticles, Api.Login, Api.UpdateUser

These modules expose functions to make HTTP requests to the application server.

There is a nice correspondence between the Api.* modules and the API as described by the Conduit API documentation. Except for Api.GetArticles, each module contains the data structures and logic necessary to deal with a single API endpoint.

Api.GetArticles wraps GET /articles/feed and GET /articles into a single function, getArticles, that makes it easy to get an article in all the ways possible without making a mistake. See its Request opaque type for further details.

The Api.* modules build upon the Api module. The key function in the Api module is the private function, expectJson, that describes how to deal with every possible response, good or bad, we could get from the Conduit API.

The Data.* modules

Examples: Data.Article, Data.Comment, Data.Pager, Data.Route, Data.Timestamp, Data.Validation

These modules describe common data structures used throughout the application. Some of the modules expose type aliases with a few helper functions, for e.g. Data.Article. Other modules expose opaque types with smart constructors and useful functions that work on the data type, for e.g. Data.Pager.

Data.Route exposes a function, fromUrl, to translate URLs from the browser's location bar into a valid route, as well as a function, toString, that converts a valid route back into a valid path used by the application. fromUrl is extensively tested. But, toString is kind of hard to get wrong due to the type system, so I decided not to test it.

Ports

Three things need to be done over in JavaScript land.

1. When a user logs in we need to save their token in localStorage.
2. When a user logs out we need to remove their token from localStorage.
3. When any unexpected errors occur we need to send the details over to an error logging service, for e.g. like Rollbar.

This suggests we need three ports. However, through multiplexing, we can reduce it to one port. How?

In the port module, Port.Outgoing, I create one outgoing port called send that takes, as input, an arbitrary JSON value. Then, I use a general JSON message format, defined in Lib.Port.Message, to communicate with the JavaScript side. Port.Outgoing exposes three commands, deleteToken, logError, and saveToken, whose side-effect is to send a message over the send port.

For e.g. saveToken token might send the following message:

{
    "tag": "saveToken"
    "value": "eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJ1c2VyIjp7ImlkIjoxMjc3Nn0sImlhdCI6MTcxMzc4MzQ4NywiZXhwIjoxNzE4OTY3NDg3fQ.Pqfubt_-KRujNhiXr-vwSvAhvlyu8IhV6eXw2iEdHzF"
}

The approach scales. We do a similar thing at Qoda. A naive use of ports would require 161 total ports. But, through multiplexing, we only ever use two ports, one outgoing and one incoming.

The Lib.* modules

Examples: Lib.Browser.Dom, Lib.Html.Attributes, Lib.Either, Lib.OrderedSet, Lib.Task, Lib.Validation

These modules contain helper functions and data structures that are needed to implement this project but that are general enough to be extracted and reused in other projects.

Lib.Html.Attributes exposes an attrList function that allows you to add both required and optional HTML attributes to an HTML element. The optional HTML attributes can easily be added and removed depending on the boolean value it is paired with.

Lib.Task exposes the dispatch function I use for parent-child communication. For e.g. it is used by Page.Login to tell Main when you've logged in.

Tests

By using Elm and working with the type system I eliminated whole categories of tests but not all. The tests/Test contains the unit and fuzz tests that, I found, were necessary for me to gain confidence in the correctness of the web application. I used elm-explorations/test to write the unit and fuzz tests.

Test.Data.Comments, Test.Data.Pager, and Test.Data.Timestamp contain interesting tests. For e.g. Test.Data.Comments has a fuzz test that ensures when we decode the comments from the API we always get it sorted in reverse chronological order.

Build and deployment

I reused my build and deployment scripts from dwayne/elm-hello.

For this application, Elm controlled the routing. So, I had to adapt the scripts to deploy to Netlify instead of GitHub Pages. Why? Because you need to be able to tell the web server to redirect all relevant requests to the application. GitHub Pages doesn't have support for it.

deploy-production is the deployment script. It makes use of git worktree to commit a production build to a separate production branch. Netlify is configured to deploy from this branch anytime it changes. Read Kris Jenkins' Git for Static Sites to learn more.

Conclusion

dwayne/elm-conduit is a modern open-source example of an Elm Single Page Application that is written purely in Elm with no frameworks, is production-ready, has 100% coverage of the RealWorld spec, provides examples of useful tests used in a web application context, uses the latest Elm libraries and tooling, and has a build and deployment story.

I hope this would be useful to anyone who's learning Elm and trying to write a non-trivial SPA.

Enjoy!