Federated Widgets: Frontend Components and Backend Endpoints in One Module

TLDR: This post outlines the idea around open source modules that contain both frontend components and backend endpoints (FCBE), that mimic the embed-able html and scripts from popular third-party services but self-hosted. Skip the summary and go down to the Technical Ideas.

There are tons of companies that offer embed-able features for a month fee. The typical service usually provides a way for you to configure the "seen settings" through some sort of backend dashboard or admin panel. Some offer ways to customize the look and feel minimally. These embeds usually provide some sort of interactivity, some popular examples include audio (spotify), video (youtube), maps (openstreetmap), and things like input forms (calendly) or newsletter signups (substack). The one thing in common with these types of services is that the data all lives on their servers. Services like these range in complexity and price.

Today many companies, apps, or websites are single-purpose. One app for sharing photos, one for microblogging, one for newsletters, one for short-form video, most have timeline feeds, one for e-commerce. Most do one thing well and when you need to accomplish something else you need to continuously find a new service to provide that thing.

I'm interested in a more decentralized way of doing things. Through hosting your own web server you can "own" your data. Why pay $10 a month for something when you could pay less to just host the server and the data yourself? I believe we can do better as programmers to bridge this gap and offer something better to people who want it. A cheaper option with more customizability.

I believe that the future of the web is to provide low-code solutions that inspire people to have ownership of the tools they use if they choose to do so. The web is built on standards, people should learn those and not some companies no-code ui, their way of doing things.

When open-source options do exist they're usually released at a server level and include a hefty framework, things aren't intended to run along side each other.

The way I think about this is I imagine an app that has posts or blocks, but the post / block types themselves being open-source ui components, they can be anything. A map? A blog post? A calendar scheduler? A poll? A product card with a buy button? Why are we limited to what kinds of things we can post?

I'm really impressed with apps like mastodon. However I'm disappointed that the decentralized / federated tools are designed to look like the social media platforms they were meant to replace and that they don't offer any customizability other than basic posts.

I like the idea of people in the future not creating full web-apps, but individual ui "widgets" a collection of UI Components and data with interactivity. I think the testament to this is the slow roll-out of companies copying individual ui features, such as stories.

My inspiration comes from Wordpress. Wordpress was a thriving ecosystem of themes and plugins that allowed anyone to create their own hosted website and customize it how they wanted. I think as a CMS for writing Wordpress was great, however as a platform for doing anything and everything it had some shortcomings, it was still designed for blogging, and authoring text content and doing anything other than that was a pain.

Technical Ideas

One opinionated goal is to highly-couple the component props as the source of truth for the system. No need to duplicate this definition for the form inputs, validation, database definition. Props drive the entire widget.

I like to think of an individual Widget as a noun, a thing. The props are stored in the database and are given a UUID. The Component is the visualization of that noun.

I'm interested in having a way to serialize the component and pass it from the client to the server arbitrarily. Ideally with the UUID of the props you can render the widget directly via permalink. You should also be able to arrange several widgets on a page through Page and Column components. The form is defined via JSON using react-jsonschema-form, I've also tinkered around with generating this from the typescript interface for Props itself as well. In an ideal world the props also define what gets saved to the database whether it's a table specific for one component or single table for all components using a jsonb props column, and what gets sent back down to the client.

Here's an example playground page that allows me to test a Widget and the props passed into it, think of this as a storybook page, the Widget on the right is made up of two components, the wrapper Widget which handles the metadata, things like owner information, and uuid, privacy and the YoutubeEmbed is the component inside with one property embed.

When you start to think in widgets, every component seems to create or interact with a specific noun in the database, either creating, reading or editing it. In the case above it's creating a YoutubeEmbed. But in the case of Signup or Login it's interacting with the User noun. The User noun itself also is a component allowing someone on the platform to view and possibly edit their username or settings. The User component props somewhat would map to the user in the database, omitting server-data like hashed passwords or personal info.

One radical design choice I had was that the backend would dictate which components would show on the page. The page would have columns and interacting with a widget could make new widgets appear within the UI and position them anywhere on the page. The response from a submitted widget has the potential to do literally anything to the page itself, sending an array of actions like deleting / moving / editing the originating widget or creating new arbitrary widget anywhere on the page (column matrix).

The issue with having the database and backend would dictate which components show is that it's inefficient to have the front-end contain code for every widget if there are 100+ widgets that's a lot of bloat on every page, especially when not every widget is rendered. Next.js does provide a way to give a map and use dynamic imports.

import dynamic from 'next/dynamic';

export const FormDynamic = {
  SignupForm: dynamic<any>(() => import('modules/signup/SignupForm').then(m => m.SignupForm)),
  UserForm: dynamic<any>(() => import('modules/user/UserForm').then(m => m.UserForm)),
}

export default FormDynamic;

Then you can serialize a component via the key here, and it's props and pass that to the frontend which can render it in real time (obviously there can also be SSR for page load). But the general idea here is that there's no front-end code base. The database contains pages -> columns -> widgets all using UUID's to describe what widgets show on which page.

Other Ideas / Features & Struggles

Deno

I'm really inspired by deno and deno deploy, and while my first iteration of this was written in Next.js I'd like to push using deno so that people ultimately don't need to use NPM to utilize this system.

Astro

I'm also really inspired by Astro, which allows a mixture of any component library. I like the idea of not being locked into using react forever, and allowing widgets to be framework-agnostic.

Light weight system design

The real question is: How do you design a system that is agnostic and also interconnected?

I like the idea of someone customizing the youtube widget so that when you press submit, along with the update a Youtube Profile widget pops up to the right of it. How do you keep YoutubeEmbed and YoutubeProfile separate but also coordinate it so they can reference each other?

Customization (CSS)

Customizing the look and feel. I'm a little obsessed with tailwind, and react-form-ui doesn't provide a tailwind scheme. I'd also be interested in allowing consumers to edit the classes on these components. That's a hard problem to solve as well.

Providing a centralized option?

The wordpress model? Perhaps the platform allows people to signup on a .com option where you can create an account try it out and also allows a roll your own option?

Federation?

Federation, actually allowing cross-talk between individual servers, logging in, pubsub, cookies, and liking and sharing things, interacting with user data, this is all a bit overwhelming but would be rad.

IPFS 👀

I like the idea of a widget being able to accept an json src from any location. This would theoretically allow you to store JSON props on IPFS and provide a blog post using a deno module for the widget and ipfs endpoint for the data.

import { BlogPost } https://deno.land/x/reggi@1.0.0/mod.ts

export default BlogPost({
   src: "https://ipfs.io/ipfs/QmT5NvUstoM5nWFfrQdVrFtvGfKFmG7AHE8P34isapyhCx"
});

Editing content "on the web" versus "in-code"

I struggle a bit with how "in-code" I want this. For example do I need the ability to create and edit via the web? This involves permissions, roles, ownership, perhaps this feature comes later?

For example you could have a server render a article like this where the content itself is in code:

import { Article } from 'https://deno.land/x/reggi@1.0.0/widgets/Article.ts'
import { server } from 'https://deno.land/x/reggi@1.0.0/server.ts'

export default server({
  widgets: [Article],
  pages: {
    _: Article({
      title: 'Hello World',
      body: `Hi there!`
    })
  }
});

Or you can supply a reference key, that will act as an upsert. If the post exists, it will be pulled and be editable from the browser, If it's not you can create it from the web.

import { Page } from 'https://deno.land/x/reggi@1.0.0/widgets/Page.ts'
import { Article } from 'https://deno.land/x/reggi@1.0.0/widgets/Article.ts'
import { server } from 'https://deno.land/x/reggi@1.0.0/server.ts'
import { Postgres } from 'https://deno.land/x/reggi@1.0.0/store/postgres.ts'
import { Google, Login, Logout } from 'https://deno.land/x/reggi@1.0.0/iam/google.ts'

export default server({
  widgets: [Article],
  store: Postgres(),
  iam: Google(),
  widgets: [ Login, Logout ],
  pages: {
    login: Page(Login)
    _: Page(Article({ ref: 'reference-editable-from-browser' }))
  }
});

Module Data store, Modular login

Tinkering here with a possible spec.

import { Create } from 'https://deno.land/x/reggi@1.0.0/widgets/Create.ts'
import { Page } from 'https://deno.land/x/reggi@1.0.0/widgets/Page.ts'
import { Article } from 'https://deno.land/x/reggi@1.0.0/widgets/Article.ts'
import { YoutubeEmbed } from 'https://deno.land/x/reggi@1.0.0/widgets/YoutubeWidget.ts'
import { CalendarScheduler, CalendarSchedulerId } from 'https://deno.land/x/reggi@1.0.0/widgets/CalendarScheduler.ts'
import { Postgres, WidgetByUuid } from 'https://deno.land/x/reggi@1.0.0/store/postgres.ts'
import { Google, Login, Logout } from 'https://deno.land/x/reggi@1.0.0/iam/google.ts'
import { server } from 'https://deno.land/x/reggi@1.0.0/server.ts'

const creatableWidgets = [Article, YoutubeEmbed]
const CreateWidget = Create(widgets) 

export default server({
  store: Postgres(),
  iam: Google(),
  widgets: [Login, Logout, ...creatableWidgets, CreateWidget],
  pages: {
    login: Page(LoginWidget),
    logout: Page(LogoutWidget),
    new: {
        _: Page(CreateWidget),
        '[widget]': Page(CreateWidget),
    },
    widget: {
        '[uuid]': WidgetByUuid,
    },
    calendar: Page(CalendarScheduler({ ref: CalendarSchedulerId })),
  }
});

No API?

I'm realizing now that with this sort of architecture there's no real API. Obviously endpoint with some custom code would exist to signup / logout, but the bulk of content-related modules would simply be autogenerated. The goal may be to hide away the api by generating most of it for you in most cases.

Some adjacent thoughts

Conclusion

I love the idea of providing frontend components and backend endpoints in one module. One piece of encapsulated software that serves one purpose.

This is very much a half-baked idea and not fully flushed out. I'd love to chat more about it and brainstorm ideas. Feel free to reach out!