Unlocking the frontend - a call for standardizing component APIs pt.1

#tldr We're still wasting massive amounts of valuable development cycles in the frontend world by working in silos, or by to at least some extent reinventing the wheel for every project. Results suffer in the process, impacting real world results for users and content creators alike.
How did we get here, and how could a way forward look like? How we've already come a long way, and why still (so far) even Jamstack hasn't been the sole answer, either...

So what's wrong?

This is something I've been thinking about for a while now.
Way back in the day, when you wanted to build a website or digital product, you built everything yourself. You had no choice, you had to! This began with defining models and relationships, adding logic on top of it to extract the correct data to display, and implementing actions that should be triggered upon user interaction. Typically mutating server state of some sort. And then, finally, you purpose-built your frontend accordingly, for those exact circumstances.

All in all, oftentimes a very interlocked process between all parties involved. Be it between software specification and backend development, design creation & specification and frontend development or frontend and backend development.
Changes of every kind typically being costly as a consequence, leading to everyone trying to over-spec even more... a vicious cycle of ultimately missed expectations

Classic architectures

We've moved away from this by adding frameworks and new, more flexible development approaches into the mix. Mostly adopting a Model-View-Controller (MVC) approach in the process, doing some domain driven design, and as a result agreeing to a way more manageable amount of shared best practices. This happened for almost every programming language (e.g. Symfony for PHP, Rails for Ruby, Express for Node.js, Spring for Java).

In combination with adopting more agile processes to structure our development, this enabled everyone to be more resilient to often changing requirements. Because changes could be made in a more predictable fashion, and were better aligned with real needs, the resulting positive feedback loop kept converging everyone to adopting those shared best practices, fostering further exchange and standardization. Often designed into the respective frameworks from the start, as plugins, extensions or addons. Associated marketplaces further incentivizing this.

What about websites?

In parallel those conventions took hold for website development, too. Instead of general purpose frameworks (like Rails), which were still to expensive as a basis for the often repeated customer needs, content management systems (CMS solutions like WordPress, TYPO3, Drupal) took that place.

Adding already established models, and thus a specific domain for content, to the open approach of working with "just" a MVC framework. This also enabled them to create much more tailored interfaces for editors working with said content, and layering additional functionality on top; as you just knew much more about what everyone was working with.

But you still had to fine-tune it for every customer to create a truly fitting editor experience, and the frontend still got painstakingly re-implemented with every major overhaul. Additionally you had to fit everyone into the same monolithic system, increasing complexity and cognitive overhead for all different types of editors involved (news, career, marketing, blog, internal communication, the list goes on, all with different editorial needs).

The first wave of ecosystems for the web

Both ways of building web interfaces (website vs application, or custom vs CMS) succeeded in establishing flourishing ecosystems around their approaches, though, allowing developers to start exchanging real code instead of just ideas or framework best practices... getting more efficient in the process, while generally also creating better results. But you were still locked into that specific ecosystem, trying to build a monolithic jack-of-all-trades, even more so if your goal was to have a consistent experience over all the different brand touchpoints a user may encounter on his customer journey.

Emancipating the frontend

For websites, lately, this mental model has been challenged again by the notion of headless content mangement systems, and more generally speaking by completely detaching the frontend from the backend in an approach now often called the Jamstack (in comparison to the LAMP stack traditionally found in a CMS).

This decoupling enables establishing a shared contract between both sides, an API. Making teams working on them more independent of each other, while also increasing stability because everyone can test against it in isolation.

With generators (Gatsby, Next.js, 11ty) being a central part of the Jamstack approach, one additional super power that's unlocked by them is the possibility to use multiple sources to feed into one resulting interface or website. Ultimately mix-and-matching backend and API solutions into one consistent experience, with everyone using the tool most suited for his daily job. The marketing team may work with WordPress to create long form content and something like Storyblok to build engaging landing pages. The main website could be built with Sanity, or other structured content-oriented solutions, as the headless CMS to really leverage connected, long-lasting content where it matters. Even other data can be spliced into the mix by adding third-party APIs as needed.

Frontend frameworks like React embrace that way of working by optimizing for composability, while tech like GraphQL helps with enabling that same composability for APIs. In combination with a lot of SaaS offerings being API based, or at least supplying really rich, well documented APIs as par of their open source strategy, this unlocked a huge potential in the space for re-use, specialization and standardization. And I'm sure we've just started experiencing the paradigm shift associated with that!

But... where's my frontend ecosystem? 😓

Unfortunately though, we still often build frontends completely isolated from each other. Creating and reinventing one-off work that suffers as a consequence, because you just can't invest the time that's needed to make something really good (in regards to performance, accessibility, responsiveness, cross browser-compatibility) in the time allotted for such frontend work. All while additionally trying to stay on top of the current trends, and changes inside our own ecosystems because of the rapidly changing landscape.

Way too often still you're confronted with some form of digital design (be it Sketch, Figma or Adobe XD) on the one hand, and data structures that are the result of decisions that were made in your absence, when choosing and building the generator and APIs on the other hand. It's your job then to create a component from that, hopefully creating something re-usable, which is expected to be universally understood by everyone to be the same thing. And did you know that the same component is supposed to work on the blog, too? But it should look and behave slightly different? A thankless job at best.

Man stands face to face with the irrational. He feels within him his longing for happiness and for reason. The absurd is born of this confrontation between the human need and the unreasonable silence of the world.

• Albert Camus, The Myth of Sisyphus and Other Essays

This all, unfortunately, still makes frontend development a pain. While valuable engineering time goes into chasing problems that have been solved a thousand times before. Often even by oneself, just two projects before the current one...

Why just creating a Design System is not the answer, either

For product teams tasked with starting a Design System today, this also way too often feels like just another added thing on top, that's expected to be delivered, en passant. Neglecting that for that to work the focus would have to shift dramatically, adjusting the way frontends are architected, and enabling frontend developers to realise the potential demanded from them. And also fulfill those Design System promises like "automatic" consistency, more efficient development cycles or magically aligned stakeholders.

There needs to be a better way to foster the same exchange, longevity and interchangeability between frontend developers and the results of their work! Unlocking the frontend potential, like it has already been unlocked for other parts of the products and pages we're building! In a world of Design Systems this also means letting the frontend somewhat dictate the rules of engaging with components more, in some places inverting the way we work today.

And the attentive reader could have already spotted: we haven't even really spoken about content strategy, connected content and the influence content creators and editors should have in those architectural decisions. Which for most just reflects the way digital interfaces are built today... with the people working with the resulting software unfortunately being an afterthought. And the results suffering in the process. Only in uniting all those stakeholders in the decision making process will the end result, a tangible page or application that is generating true value for both the user and the creator, be successful.

So what now?

What was behind every major evolution of the web in the past was people rallying around shared principles, doing something in a more coordinated and repeatable way. I'm pretty convinced that for the frontend this has to mean some kind of standardization. Standardizing the way components can be compared, the way they expect and consume data, and effective ways of mapping between all of those. I'll explore those thoughts in part 2, and try to showcase how we're trying to solve for this today inside kickstartDS, for our users. And maybe, just maybe, this could someday align with a greater approach to all of this, unlocking not only the true potential of something like kickstartDS, but the potential of the wider frontend development world.

How this has impacted our work over the last 15 years? How possible solutions to this could look like? And why we think to be having some potential answers at the core of kickstartDS, today?

Read more in part 2, very soon!

Also, hit me up on Twitter at @tsnmp to exchange ideas (or to tell me where I'm possibly getting it all wrong 😋). Would love to hear others takes on this!