Exploring local-first development

It starts with a problem

I like solving problems. It’s the reason I love working with software. It’s really fun to get some constraints and problems you need to solve, then write code that solves them within the constraints.

This is an inherently creative process, not just in terms of the creativity required to frame, understand, and address problems in the most effective and elegant way, but also in terms of the fact that you’re literally creating code and putting it out into the world, where it lives beyond you. I have code that’s been running in production on some server in the US since 2007. I put a piece of myself out there 17 years ago and it’s still alive. I love that.

A different way of syncing databases

I have been hearing more and more about local-first data syncing and there is a growing movement around it, like the first annual local-first conference, which suggests it's worth checking out. The local-first developer paradigm is an approach to software development that is effectively the opposite of how things have been going recently with the cloud-first paradigm. It involves building apps that treat the user’s local device as the source of truth, so you’re reading and writing data from a local database rather than a central API (in fact, because it’s a more self-contained app, you can generally dispense with writing API functions).

Although local-first prioritizes the creation of applications and systems that operate primarily on local data, it offers strong support for collaboration and synchronization across multiple devices and users. The idea is to give users a seamless and responsive experience while ensuring fine control over their data.

To do this you need a sync layer between your app's local database and a central database in the cloud. Sync layers are necessary to create local-first apps that allow users to back up and share their data. They do this by providing a way to sync data with a backend database even if the network isn’t immediately available.

Learning through building an app

I got to grips with this functionality by creating a travel notes app called NomadNotes, built using PowerSync as a sync layer, Next.js, and Supabase. I used Next.js because I am familiar with JavaScript and React and it works on any device, so it’s accessible. I wanted to understand how PowerSync worked, so I came up with this simple note-sharing app whereby users can share their adventures with loved ones while traveling. Users upload a photo, a note description, and a rating of the location or experience and can do this even when they don’t have a network connection. They can then decide on who they share their travel notes with and this is done once a connection is available again. NomadNotes also includes a private note section where users can capture things like expenses or reminders, which aren’t shared with others.

When working with a local-first app, each user has their own database on their system, and PowerSync allows you to sync to a centralized cloud database. However, you need to determine whose data should be shared with whom. That’s where sync rules come in.
Sync rules essentially let you determine which buckets of data you download again for each user. You can write a host of different queries and rules that give you granular control over what gets downloaded on the user side from the centralized database.

De-risk your build and pick tools you know

Working with new tech can be intimidating, especially with the goal that I set for this project, which was to have something that worked and could be shipped within a day or two. I’ll typically start a project by mapping out all the elements that need to be built, and really think through how well I understand each of those elements. I try to de-risk the project from the outset by focusing my initial efforts on those elements I don’t understand very well.

Obviously, it’s easy to go down a rabbit hole when doing this, so I try to get a feel for how long a project should take overall and stick to that. If I get to the halfway point in terms of time and I’m not halfway done, then I’ll look at what needs to change to get halfway there as quickly as possible. It’s definitely more art than science – sort of like sailing a ship, in that you need to be adjusting the rudder constantly. Ultimately, though, my philosophy is that if something feels too hard, it probably is, and grinding my way through it isn’t the right approach.

To offset the challenge of working with new technology, I elected to build something with which I was comfortable. Building a web app that was accessible to everyone narrowed my choices down to web frameworks and I settled on Next.js. It’s based on React, which I’d used to build a number of apps before, and there were PowerSync starter kits available for it. Because I wanted to expedite things, instead of setting up a fresh database I used Supabase, which is a managed database instance that can be set up in a matter of minutes.

This helped, as the process simply involved setting up the React and Node adapter with my PowerSync account details and creating a local app schema for the tables I was using. I copied an example app that had the necessary adapter details, so I simply input my credentials and was good to go.

My local-first “Aha” moment

I was surprised at how intuitive it felt to load up context for a given React component by writing an SQL query instead of making API calls, transforming the data, and doing a whole lot of JavaScript work. PowerSync effectively removes the server layer component, so instead of building API functions and managing data, you speak directly to the database. That sped things up, liberating me somewhat to add features without a whole lot of background work.

PowerSync also significantly streamlined the process of setting up sharing functionality. NomadNotes includes a sharing page where you can see a list of users whom you have shared and can share with, and choose whether or not you want to share with them. Normally in React, I would make an API query to fetch the users, do a bunch of JavaScript to separate them into lists, and show the buttons based on different rules. That’s a lot more complex than what I did with PowerSync installed, which was to simply run an SQL query for each list. Because I used auto-updating queries, when I added someone to one list they would be removed from the other automatically. This felt like a real “a-ha” moment for me, when I got what local-first was all about.

On the subject of sharing, PowerSync took care of uploading and downloading automatically, which meant I didn’t need to check that the changes I’d made propagated to the other people I was sharing with. That also helped simplify state management, which can get very complicated, especially in frameworks like React where you have a relatively complex tree of states that gets passed in and out of different components. Using PowerSync in a local-first database, each component can have a query that just pulls the data that it needs, and because it updates automatically, any data changes are refreshed.

Evaluating local-first development

Using React, which is generally a front-end framework, and coming from a full-stack background, I initially found it unusual to add SQL queries to the front-end. To me, that was something that belonged on the back-end on a separate API or similar, but I became comfortable with it pretty quickly. I can imagine that this is very useful for front-end developers who aren’t comfortable with service layer components. Generally, there's a lot of overhead in maintaining a server with authentication/authorization and building out API endpoints for each action you want to take. I think being able to work directly with the local database from the front-end app actually made the development process feel less fragmented and much faster.

Approaching the project, I didn’t have enough context on whether local-first was a paradigm that actually worked. Building NomadNotes helped convince me of its merits. For that reason, I’d advise developers who are on the fence about local-first to do the same: build a real-world project to get a feel for it. On the other hand, if you’re already convinced they’d like to work in local-first, I would suggest finding a working starter project and making modifications. It’s a lot easier to change someone else’s app than to start from scratch. And use PowerSync, seriously. It really helps. There are some great demos and tutorials on their website, plus GPT4 is great at bouncing ideas off for local-first development.

My stumbling blocks

One tricky issue I encountered was managing photos on NomadNotes. Setting up a photo database in Supabase was more work than I was willing to commit for a two-day build, so I did a bit of a hack by converting photos to data URLs and storing them in the database that way. It’s something that would likely be frowned upon by database people, but it really expedited the process.

At the beginning, I spent perhaps more time than I needed to on sign-up and sign-in, just to make sure that the user management side of things worked. That was probably unnecessary, and simply hacking it would have given me an extra half a day to add more PowerSync features to the app. A commenting and heart or like feature, for example, would have made the app more social.

The future of local-first

There's a noticeable shift to giving users and developers more control over data. It's not about ditching the cloud but keeping our data closer, on our own devices. It's about having the best of both worlds - using the cloud when it makes sense but keeping our data under our control. For this reason, I think we’ll see local-first mature, and more apps, libraries, tools, and techniques will appear that make it easy to build out complex local-first apps. There’s no doubt it’s an exciting new paradigm.

Further reading

View and star the Nomad Notes repo here
You can view the PowerSync docs here