The Vision Behind the "Widget" Site: A Deep Dive into Component-Based Design

I've been working with different meta-frameworks, front-end frameworks, and servers over the past year. I've been really interested in the technical challenge of building modular and open components and shifting the focus from building full apps to building domain-specific components. However, I've never really spoke about the true vision which I've coded in parts and never really achieved. This is a deep dive into a different way of thinking about components and websites.

The ideas behind this "Widget" site are as follows:

1. The design layout is rigid. There exists a grid of boxes, rows, and around 3-4 columns depending on screen width (1 column on mobile).
2. The entire site/application is comprised of "widgets" - small boxes that consist of two parts:
   1. A widget wrapper. This allows you to see metadata about the widget, such as a date, edit and save buttons, widget type, etc.
   2. The inner widget, which can be changed between an "edit" view and a "presentation" view. Think of the edit view as the text representation of an address, and the presentation view as the map.
3. All component "props" are stored in the database. The props for a component map 1:1 with the API endpoints and database schema.
4. Some "widgets" are editable and have two "views", such as a blog post. Others, like a signup or login form, just have one.
5. Each component has a single API endpoint to do CRUD or auth, and the payload determines the action.
6. The component API response has access to do a couple of front-end things, such as creating entire new "widgets" on the page. For instance, if you had an "Add" widget which was two form inputs and you clicked "submit," the API can return a response that is a new "number widget" and say put it on the page to the right of the originating widget.

Here's an example of a "YouTube Widget" in a test environment. The widget wrapper includes a border and a wrench icon that leads to a dropdown menu with "edit" and "make private" buttons. In the "edit" view, there is a single input for the YouTube link, and in the "presentation" layer, the embedded video is displayed. At the bottom of the wrapper, you can see a truncated UUID that references this item in the database.

There are many technical challenges for creating this type of site. For one, because you can potentially have an unlimited number of unique components, they all don't need to be loaded on every page. Only relevant components should be included in the page-load bundle. I've used Next.js's dynamic imports and had a good prototype where components could be loaded dynamically by the front-end on an as-needed basis. Because the site structure is very rigid, it needs to be well thought out and designed from the start to accommodate anything a widget may need to do, which can be challenging. The database layer is also a bit of a pain. I've experimented with one massive table and using the Postgres JSON type to store props. I've also experimented with custom tables for each component type. Versioning components is also challenging, as changing the props or data structure over time can break things.

I've been really interested in defining the component props type, edit form, database schema, form, and database validation all in one definition. I've used a myriad of different methods to achieve this. I've used react-jsonschema-form which uses the JSON Schema format to create a form dynamically, and I've been using AJV for validating the JSON schema against input data. This combo means that if I have an address widget with address, zip, city, state, country, etc., I can get the "edit" view and all the input components for free. All I'd need to do is create a Google Maps or OpenMaps "presentation" view. It's been daunting to get all of these things from one definition without some tradeoffs or fragility.

My personal goal for this sort of project is to have a one-stop-shop framework to create and build a library of widgets and then build on top of them, combining them in new and interesting ways. I'm sick of creating new full apps and would like to focus on widgets.

I believe everything I've outlined here is pretty doable within most metaframe works. The key parts you need are front-end components, dynamic bundling, and server-side rendering. The main issue I've been struggling with is how to create a more "open" flavor of this, consisting of components, servers, and modularity, to prevent framework and vendor lock-in. Is it possible to do this without heavy JavaScript frameworks? Some options to consider are HTMX, Deno, AstroJS, iFrames, micro front-ends, and streaming HTML. I've been exploring the fringes of different servers and technologies, but nothing has fit quite right. That's where I'm stuck at the moment. Could there be a "standard" format for this type of component modularity? I'd love to hear feedback and thoughts, feel free to reach out!