Better Ways To Handle Data Storage on The Web Client
Whenever you mention data storage and state management on the web client, different people will provide you with different solutions. From the vanilla developers fans who like to mess with the browser's raw storage APIs to those who prefer third-party libraries, there is a lot to consider when deciding how to handle data on web clients.
Native Browser Storage Solutions
First, let’s look at how browsers allow us to manage, track, and store data from our web applications.
LocalStorage — Part of the Web Storage API, localStorage is a data storage solution for in-between sessions. These are data you want the browser to keep for the next time the user visits the website. Such data have no expiration date but are cleared after a private or incognito session end.
SessionStorage — Also part of the Web Storage API, sessionStorage is a data storage solution for a single session. It works just like localStorage but the data is cleared as soon as the tab for the app is closed.
IndexedDB — A low-level API for client storage. Just like localStorage the data is kept in between sessions but unlike localStorage , it handles a larger, more complex, and structured amount of data. It is also built for performance. Another advantage is that it is accessible inside Web Worker where Web Storage API is not available.
WebSQL — Also a low-level API for client storage intended to work like SQLite but it is currently in the process of being deprecated from browsers.
Cookies — This is not, per se, a data storage solution but it is a way to set a small amount of data often used for tracking, web preferences, and session management (like sessionStorage). Nowadays it is mostly used for tracking as more powerful data storage solutions now exist in the browser.
In-Memory (Javascript)— Increasingly we have seen complex applications keeping all their data in memory. This is more flexible as the developer can decide how the API should look like. This is great for data that only needs to exist between specific contexts but it is often used for global state management as well.
As you can see, the browser pretty much provides a lot of solutions depending on your data needs. However, it is more complex than it seems which leads to a lot of confusion and mental overload for developers.
The problem with Browser Storage Solutions
Serialization: The Web Storage API only supports strings which means that for complex data you must use JSON strings or find a good serialization package to handle everything.
API complexity: If you decide to try IndexedDB or WebSQL you will quickly regret it since the API is super complex. You are much better off using a third-party library if that’s the way want to go.
API differences: All of these data storage solutions are different APIs which means you need to learn them all in order to use them. The problem is, one application may have different data storage needs, and switching back and forward between these APIs adds complexity to your application as they behave differently.
Data type support: Not every data type of your application can be stored. Again, you need to concern yourself with serialization and data type when picking solutions.
Asynchronous actions: IndexDB is asynchronous while Web Storage API is not. Being asynchronous is super important so data processing does not block your code.
Storage Limits and Data size: Although it is getting better, different browsers handle storage limits differently. It also depends on the disk space of the user. As a developer, you need to be mindful of this to make sure you dont run into quota issue which will break your app or find flexible solutions to keep data fresh.
Availability: As mentioned above, only IndexedDB is available in Web Worker which means you may need to find ways around this if you need another type of storage solution. In general, all these storage solutions are well-supported in all browsers.
Structure and Validation: Normally you want your data to have some structure otherwise you will spend a lot of time performing validation checks and data mapping. You may also need to handle defaults which is additional complexity. You may look for some sort of schema to guarantee data integrity so you have to worry about these things less.
A lot of these problems can be solved quickly, others (like limits) you just need to be mindful of as a web developer. Let’s look at few of my preferred solutions.
Third-party library solutions recommendations
If Dealing with IndexedDB and WebSQL
- Dexie: this is a wrapper around IndexedDB which removes the complexity and pairs up fine with all your UI Frameworks.
const db = new Dexie('MyDatabase');
// Declare tables, IDs and indexes
db.version(1).stores({
friends: '++id, name, age'
});
// Find some old friends
const oldFriends = await db.friends
.where('age').above(75)
.toArray();
// or make a new one
await db.friends.add({
name: 'Camilla',
age: 25,
street: 'East 13:th Street',
picture: await getBlob('camilla.png')
});
- PouchDB: this is a wrapper around IndexedDB and WebSQL which is compatible with your backend CouchDB setup.
const db = new PouchDB('dbname');
db.put({
_id: 'dave@gmail.com',
name: 'David',
age: 69
});
db.changes().on('change', function() {
console.log('Ch-Ch-Changes');
});
db.replicate.to('http://example.com/mydb');
- JSStore: this is a wrapper around IndexedDB but has a SQL-like behavior.
const dbName ='JsStore_Demo';
const connection = new JsStore.Connection(new Worker('jsstore.worker.js'));
var tblProduct = {
name: 'Product',
columns: {
// Here "Id" is name of column
id:{ primaryKey: true, autoIncrement: true },
itemName: { notNull: true, dataType: "string" },
price: { notNull: true, dataType: "number" },
quantity : { notNull: true, dataType: "number" }
}
};
const database = {
name: dbName,
tables: [tblProduct]
}
await connection.initDb(database);
const insertCount = await connection.insert({
into: 'Product',
values: [{
itemName: 'Blue Jeans',
price: 2000,
quantity: 1000
}]
});
Handles all Storage Solutions, simple to use, and provides additional features
- LocalForage: this is a wrapper around IndexedDB, WebSQL, LocalStorage, and SessionStorage with a way to define more interfaces (called drivers) *for additional storage solutions. It does a great job handling all your serialization needs, it is **asynchronous **and handles a large set of data types. Its **API resembles Web Storage API* and it is ridiculously simple to learn.
const todoStore = localforage.createInstance({
name: "todo",
version: 1,
});
const todoId = crypto.randomUUID();
const todo1 = await todoStore.setItem(todoId, {
id: todoId,
name: "Check LocalForage out",
description: "Try to find the perfect storage solution for my app"
});
todoStore.removeItem(todoId)
- ClientWebStorage: this is a wrapper on LocalForage — which means it inherits all its benefits —, but takes to a whole new level your data storage needs to also be your preferred application state manager. It is asynchronous, event-driven, schema-based, handles data defaults and type checks for you, allows for data action subscription and interception to handle side effects, and integrates nicely with the backend server for continuous data synchronization. It also can be used as a state management solution for any UI framework like React and Angular.
interface ToDo {
name: string;
description: string;
complete: boolean;
}
const todoStore = new ClientStore<ToDo>("todo", {
// define schema
$name: String,
description: "No Description",
complete: false
}, {
// config
type: INDEXEDDB, // LOCALSTORAGE | WEBSQL | MEMORYSTORAGE (default)
version: 1
});
// listen to action events on the store
todoStore.on(EventType.Error, ({error, action, data}) => {
trackErrorJS.track(error); // track errors
console.log(`Action "${action}" failed with error "${error.message}" for data`, data);
})
todoStore.intercept(EventType.Created, ({data}) => {
// intercept create action to call API
// and return API response to update todoStore with
return todoService.createTodo(data);
})
todoStore.intercept(EventType.Removed, ({id}) => {
// intercept delete action to call API for same action
todoService.deleteTodo(id);
})
const todo1 = await todoStore.createItem({
name: "Check ClientWebStorage out"
});
/* Creates
{
_id: "123e4567-e89b-12d3-a456-426614174000",
_createdDate: "January, 4th 2022",
_lastUpdatedDate: "January, 4th 2022",
name: "Check ClientWebStorage out",
description: "No Description",
complete: false,
}
*/
await todoStore.removeItem(todo1._id);
If you are looking for a single full data storage and application state management solution, I recommend taking a look at* ClientWebStorage*. If you just need a data storage solution for everything, LocalForage is the one.
If you are just looking for a solution for your IndexedDB needs I find Dexie to be one of the best but depending on other needs, the others in the list are also good to consider.
Conclusion
The web client is a great platform and as the complexity of web applications increases, a solution for our data storage and management follows. This is a very sensitive topic that needs careful consideration.
You must understand your data well to decide which solutions to go for. There are no silver bullets here but in general, I love swiss knife solutions like ClientWebStorage and LocalForage which offer everything out of the box in a very simple and powerful API that still allows me to configure and extend as needed.
YouTube Channel: Before Semicolon
Website: beforesemicolon.com
Top comments (1)
nicely explained!