First Steps With TinyBase

So TinyBase was one of those thing that came across my feed, and I wondered what the point was. It's a reactive client-side data store, which takes a little parsing to understand.

What exactly is a data store? Simply, it is a structure we use to store data. An array, an object, a Map, a Set... these are data stores.

What exactly do we mean by a "client-side data store"? It's a library that lets us build objects or arrays in the browser. Which, on the face of it, seems silly. We can always just create an array or object, right?

The key lies in the "reactive" bit. Not only does this allow us to construct a data store, but it allows us to respond to changes in that data store. We can observe those changes, and respond appropriately.

Reactive Data Store

What do we mean by respond appropriately? Any sort of action we might like. Think of the event listeners in the DOM, only with data.

• We might want to update the DOM, say a grid of employee <card> elements, when one is added or removed. Or we might need to update the contents of a card if one is edited.
• We might want to store data locally when a change happens (localStorage, sessionStorage or IndexedDb). Tinybase provides for that.
• We might want to handle some remote call, say passing a message to a Socket.
• We might not want to store data locally - when something changes, we might want to handle that change via a debouncer, so when the user stops editing for a given period, we handle a remote save request. While Tinybase provides for the remote storage, we would write our own debouncer.

Note that last, it is one of the strengths of this library. It doesn't dictate how we might use it or encapsulate it or consume it, it simply does this one thing, and does it quite well.

Note that TinyBase actually provides two storage mechanisms: we can store values by key, or we can store tabular data. That is, we can do either or both of these things:

// we can set key/value pairs, just as in localStorage:
const store = createStore()
  .setValue('currentProject', projectId)
  .setValue('colorMode', 'dark');

console.log(store.getValues()); 
// {
//   currentProject: '36b8f84dd-df4e-4d49-b662-bcde71a8764f',
//   colorMode: 'dark'
// }

// or we can store tabular data:
store.setTable('todos', {
  crypto.randomUUID(), {
    title: 'Look into TinyBase',
    description: 'A pretty darn okay reactive local data store.',
    priority: 'normal',
    done: false
  }
});

So with the first part of that, we defined two keys: currentProject and colorMode. We defined values for them, and we're able to use either store.getValue(key) or store.getValues() to view those things.

In the second, we are defining a row of data in the todos table in our store. Note that we didn't specify any particular ordering or columns to that table, we simply went ahead and created the object. We could (and in future parts we will) define schemas or relations between tables - this project will let us do this, and more.

It's a data store powerhouse, but it is intentionally limited to that domain. It does one thing well, without needing to do all things.

A Quick Idea...

To begin, let's consider how we might create a Todos data model, first with plain javascript and then by leveraging TinyBase.

// Todos-service.js
let todos = [];

const isAMatch = (id) => (obj) => Object.keys(obj)[0] === id;

export const add = (todoObject) => {
  const id = crypto?.randomUUID();
  todos = [...todos, {[id]: todoObject}];
  return {[id]: todoObject};
}

export const update = (todoId, todoObject) => {
  todos = todos.map( todo =>
    isAMatch( todoId)(todo) ?
      {[todoId]: {...Object.values(todo)[0], ...todoObject} :
      todo
    )
}
export const remove = (todoId) => {
  todos = todos.filter( todo => !isAMatch(todoId)(todo));
}
export const findById = (todoId) => todos.find( isAMatch(todoId) )
export const findAll = () => [...todos];

All our basic functionality for Todo things to be collected, in a nice tidy package. Later, when we want to consume it, we simply

import * as Todos from './services/Todos-service';

Todos.add({
  title: 'Learn about TinyBase', 
  descripton: 'Client-side reactive data stores!',
  due: '2023-02-17',
  priority: 'normal',
  done: false
});
console.log(Todos.findAll() );

And that's great. Let's see the same basic functionality with TinyBase:

import { createStore } from 'tinybase';
const store = createStore();

export const add = (todoObject) => {
  const id = crypto?.randomUUID();
  store.setRow('todos', id, todoObject);

  return {[id]: store.getRow('todos', id) };
}

export const update = (todoId, todoObject)=>
  store
    .setPartialRow('todos', todoId, todoObject)
    .getRow('todos', todoId);

export const remove = (todoId) =>
  store.delRow('todos', todoId);

export const findById = (todoId) =>
  store.getRow('todos', todoId);

export const findAll = () => 
  store.getTable('todos');

This is giving us all the same functionality as an array, but it is abstracting that internal array into an external data store. And if this was all we were doing with it, we really haven't gained anything.

But as a first step, we can see that, basically, the use is very similar - only rather than using array methods, we're using TinyBase store methods.

Getting Started

1. Setting up shop

To build this one, we'll use a bundler and a few packages. Of late, my preferred bundler is Vite - it is quick, clean, and minimal. So we'll open the console and:

yarn create vite vanilla-todo-app --template vanilla

That will create a directory, vanilla-todo-app, and set up the package.json and dependencies for us. Then we will cd vanilla-todo-app to get in there, and

yarn add tinybase

And that will both install the package.json, as well as adding the TinyBase package for us. If we have other dependencies we might like, we can add them - but for what we're about to do, that's everything we'll need.

At this point, we can open this in our editor. I'll be using VS Code, simply because it's fairly universal:

code .

This opens the editor with the current directory as the workspace. We will be able to clean up the template quite a bit, we don't need any of the content in the main.js, or the files to which it refers - so we can delete javascript.svg and counter.js, and remove everything but the import './style.css' from the main.js. At that point, we're ready to start!

2. Creating a Store

Now we need to create a data store. And we'll want to place it into its own file, importable by others - we might want to allow for multiple data sets (for example, we might want a "todos" and a "projects"). Let's start there.

1. Create a src directory to keep the root tidy, we'll work in there for the most part. Within there, we'll create a services directory.
2. Inside that services directory, we'll create a store.js file.

// /src/services/store.js
import { createStore } from 'tinybase';

const store = createStore();

export store;

And there we go. We have our datastore defined! At this point, that's all we'll need in the store.js, though later we'll add a few other useful things to this file.

While we could interact directly with the store wherever we might need, a better approach might be to define an interface that can consume a particular service. With that, if we choose to swap out our data provider later, it would only require editing one place rather than scattered throughout our code.

3. Defining an Abstract Model

The interface methods for each are fairly standard: add, update, remove, byId, and all will do. We'll start by defining a generic Model.js:

// src/models/Model.js
import { store } from '../services/store';


const Model = (table) => {
  const add = (object) => {
    const id = crypto.randomUUID();
    store.setRow(table, id, object);
    return {[id]: object };
  }
  const update = (id, object) =>
    store
      .setPartialRow(table, id, object)
      .getRow(table, id);  
  const remove = (id) => store.delRow(table, id);
  const byId = (id) => store.getRow(table, id);
  const all = () => store.getTable(table);

  return {
    add,
    update,
    remove,
    byId,
    all
  }
}

export default Model;

We've defined this as a factory function. To consume it, we could simply call const Todos = Model('todo'), providing it with the name of the data table we wish to use.

Now, when we are adding an object, we are getting a uuid for the project, and we are using the TinyBase setRow method to create a row in the given table.

Side note, when I refer to the projects table, it may be easier to think of the store as an object, and the projects as a key on that object that contains an array of {[id]: object} things.

When we update a project, TinyBase provides the setPartialRow method. With that, we provide a table id, a row id, and an updater object. That updater object doesn't need to redefine all the properties in our original object, only the ones we might want to update. And the setPartialRow method returns the TinyBase store instance, so we can chain that and call getRow() to get and return the updated value of our object.

To delete a row, we simply call delRow with the table and row ids.

Now, if we look at that code, there is nothing there that is unique to the Project model. In fact, it's fairly abstract - the only thing identifying it is the const table = 'projects' line. And that is deliberate.

To create the Todos.model.js, we can use the same code. Simply by changing that one line to const table = 'todos', we will be performing those same CRUD operations elsewhere in our store.

And this is a pretty good abstraction, to my mind. We can reuse this as a template for each of our data models.

4. Creating Instance Models

// src/models/Todos.model.js
import Model from './Model'; 

const Todos = Model('todos');

export default Todos;

And that's all we need at this point. We can do the same for projects:

// src/models/Projects.model.js
import Model from './Model';

const Projects= Model('projects');

export default Projects;

At that point, we have two functional tables of data we can stuff into our store service.

Let's Consider Structure

Up to this point, we haven't really considered how we should structure things. We set up a data store, we defined an interface to consume that data store, and we created a couple of basic data models. But we would do well to step back and think about how our data should be structured.

Let's examine the Todos model first, in isolation.

// Todo:
{
  [id]: {
    title: 'Default Todo Title',
    description: 'Default description',
    priority: 'normal', // ['low','normal','high','critical']
    created: '2023-02-02',
    due: '2023-02-09',
    done: false
  }
}

So note that we have that priority key, which can be one of four values. Now, if we wanted to get all the high priority todos, we could simply get them all and use .filter, but TinyBase gives us an option. This would be a good candidate for an index.

With an index, we can select all rows from a table that match a given index value. When we query the index, we get back an array of keys, all of which meet that index condition. And that array we get back is reactive - so as we add/edit/remove rows, the indexes are dynamically updating.

So we have a basic structure - the priority key will be indexed, and we want to be able to get two key pieces of information back from our Todos: all currently-used indexes, and sets of ids that meet a given index. So we'll be adding two methods to the Todos object: priorities and byPriority. The first will get an array of priority keys, while the second will get a complete list of the Todos with a given priority value.

But we're using the Model to generate the Todo.model - can we somehow add to or compose that?

We Have the Power!

We can, actually. We want to first add an export to the store service, allowing for indexing:

// src/services/store.js
import { createStore, createIndexes } from "tinybase/store";

export const store = createStore();
export const indexes = createIndexes(store);

That will let us set up an index in the Todos.model.js:

// src/models/Todos.model.js
import Model from './Model';
import { store, indexes } from '../services/store';

indexes.setIndexDefinition(
  'byPriority',
  'todos',
  'priority'
);

const Todos = Model('todos');

export default Todos;

At that point, we have defined our index column. setIndexDefinition is a method on the indexes instance, and we tell it to create an index with the id of byPriority (so we can retrieve it later), that is working on the todos table, and in particular is indexing the priority field in that table.

Extending the Data Models

In the above Todo.model.js, we now have a good index, but we aren't actually using it yet. And what we'd like to do, if possible. But what that means is, we want to take in the basic Model functionality, and add to that.

// src/models/Todos.model.js
import Model from './Model';
import { store, indexes } from '../services/store';

indexes.setIndexDefinition(
  'byPriority',
  'todos',
  'priority'
);

const Todos =(()=>{
  // our base functionality...
  const baseTodos = Model('todos');

  return {
    ...baseTodos
  }
})();

export default Todos;

So we have changed Todos from simply being an instance of our Model factory to being an IIFE that is returning all the methods of that baseTodos, which is still the instance. We're composing the functionailty from Model with some custom methods.

Within that Todos IIFE, let's add this:

  const baseTodos = Model('todos');

  // get all the todo ids associated with this project
  const priorities = () =>
    indexes.getSliceIds('byPriority');
  const idsByPriority = (priority) =>
    indexes.getSliceRowIds('byPriority', priority);
  const byPriority = (priority) =>
    idsByPriority(priority).map( baseTodos.byId )

  return {
    ...baseTodos,
    priorities,
    byPriority
  }

So we've added two methods to the Todos model: priorities and byPriority. The first gets all the currently-used priority values, while the second gets the todos themselves with a given priority.

To get the array of priority values, we use the Index module's getSliceIds method. That gets us all possible key values for the indexed cell (all the possible values for priority currently used in our data store).

The Indexes module also gives us the getSliceRowIds method, which simply gets the id for each row that meets its condition. In our case, the condition is a matching priority.

And we can leverage that in the byPriority function - we get the ids for each row, and then use those to get each individual row for the project.

Finally, we spread the ...baseTodo object, exposing its methods on the Todos returned object as references to this inner baseTodos thing. And we compose that interface, by adding two more methods to the Todos returned object.

Indexes vs Relationships

The next bit of structuring to consider is the relationship between the Project and the Todo. Each thing has a unique id assigned to it, and that is used as the key of its row in the data table.

And a project can contain any number of todo elements, but each todo can only belong to one project. This is, in relational database terms, a one-to-many relationship.

Typically, in a relational database, we might give the Todos.model a field for the projectId, some way of identifying with which project it is associated. So, for example, to find all the Personal project's todos, we could select all the todo elements with that projectId.

So note how, in the model of the Todo, we show a projectId:

// Project:
{
  [id]: {
    title: 'Default Project Title',
    description: 'Default Description',
    created: '2023-02-02',
  }
}

// Todo:
{
  [id]: {
    projectId: [projectId], // <-- the key from `projects`
    title: 'Default Todo Title',
    description: 'Default description',
    priority: 'normal', // ['low','normal','high','critical']
    created: '2023-02-02',
    due: '2023-02-09',
    done: false
  }
}

Indexes vs. Relationships:
We discussed indexes above, and now we're discussing relationships. They are similar, and if you've worked with relational data, you may likely already know the difference, but here it is in a nutshell:

• indexes are used within the context of a table, to facilitate searching for a particular property value within that table (for example, priority==='high').
• relationships are used within the dynamic of multiple tables, indicating a connection between one (a local table) and the other (the remote). In this case, the relationship would be todo.projectId === project.id. We're still comparing to something - but with indexes, we're typically getting information about a table while a relationship is giving us information about multiple tables through a common point.

In order to support relationships between data tables, we will need to provide the Relationships module:

import { 
  createStore,
  createIndexes,
  createRelationships  
} from 'tinybase';

export const store = createStore();
export const indexes = createIndexes(store);
export const relations = createRelationships(store);

So we now have a relations module in which we can define our many-to-one relationship. As this is primarily the domain of the Project, we'll put it in the Project.model.js for now.

// src/models/Projects.model.js
import { relations } from '../services/store';

import Model from './Model';
import Todos from './Todos.model';

relations.setRelationshipDefinition(
  'projectTodos',  // the id of the relationship
  'todos',         //  the 'many', or local side of the relation
  'projects',      //  the 'one', or remote side of the relation
  'projectId'      //  the local key containing the remote id
);

const Projects = (()=>{
  const baseProjects = Model('projects');

  return {
    ...baseProjects,
  }
})();

export default Projects;

Note that we import just the relations export, as we don't need an instance of the store itself. All we need to define the relationship is the relationship module itself. Also, we import the Todos module, as we want to add an array of todos to the project.

relations.setRelationshipDefinition defines the relationship between projects and todos, and gives that relationship the id projectTodos. The parameters for that function are:

• relationshipId: a unique string to identify this relationship.
• localTableId: the id of the local table for the relationship. Note that the local table is the 'many' side of the many-to-one relationship.
• remoteTableId: the id of the remote table for the relationship. This is the 'one' side of that many-to-one, representing the unique project that can relate to zero or more todo rows.
• getRemoteRowId: the name on this threw me for a bit, but it is the cell in the local row that contain the unique id of the remote row. So, in our case, this would be projectId, as that is the todos row reference to projects.id

Finally, we can consume that relationship within the definition of the Projects model itself:

// src/models/Projects.model.js
const Projects = (()=>{
  const baseProjects = Model('projects');

  const byId = (projectId) => {
    const project = baseProjects.byId(projectId);
    project.todos = relations.getLocalRowIds('projectTodos', projectId)
      .map(Todos.byId);

    return project;
  };

  return {
    ...baseProjects,
    byId
  }
})();

Again, we expose the interface of the baseProject, and replace the stock byId method with a custom one.

That's Nice and All, But... Why?

This post is about how to interface with and consume TinyBase's store, using plain vanilla javascript. And to this point, it's pretty darn okay. But if that was all there was, it wouldn't have much going for it.

In the next post, we will explore the reactive aspect of that store. We can set listeners on tables, rows, cells or data values, and we can respond when those points change.

We will also look at data storage and persistence. TinyBase, in itself, includes some great mechanisms for both local and remote storing, and also describes how to write your own storage "hooks."

This is something I'm still playing with, something I'm still learning as I go - if y'all find something neat (or something I missed), lemme know!