DEV Community

Cover image for Building JavaScript Frameworks to Conquer eCommerce
Ryan Carniato for This is Learning

Posted on • Updated on

Building JavaScript Frameworks to Conquer eCommerce

There was a time that I would have never imagined the need to write an article like this. If you were to ask someone how a website worked, even 10 years ago, the answer would have been pretty simple. A site consists of a collection of HTML documents that reside at locations (URLs), that each describe how a page is displayed and provide links to navigate to additional pages. A web browser is used to request and display these pages.

But in the past 10 years how we build for web has evolved significantly. The paradigm has flipped so much that it is the traditional Multi-Page Application (MPA) that requires explanation now that Single Page Apps (SPA) are becoming the ubiquitous product.

When I talk about this I find many JavaScript developers don't understand the difference and how profound an impact it has. How Frameworks like Marko, Astro, Elder, or Qwik are a completely different architecture than Next.js, Nuxt.js, or SvelteKit.

While SPA's bring a lot to the table, I'm going to talk today about where they are the less optimizable solution, and how that has been the motivation for a whole different sort of JavaScript framework.


The State of Frontend JavaScript in 2021

Alt Text

The vast majority of JavaScript Frameworks are designed to help you make what we call Single Page Apps(SPA). React, Vue, Ember, Preact, Svelte, Solid, you name it. A SPA is simple an app where the whole experience is served from a single page sent from the the server(or CDN). This characteristic carries on in Metaframeworks built on top of these like Next, Nuxt, Gatsby, SvelteKit, Remix, Blitz, etc..

The defining trait is that these are built around client-side routing. That is that the browser handles navigation after initial page load without sending HTML page requests to the server. The JavaScript then re-renders portions page. They can opt into using server-side routing but the application runs through a single entry.

These frameworks are really amazing to use and their use case has grown from their origins in admin dashboards and highly interactive apps, to branching into things like blogs, content sites, and eCommerce.

However, for these sites where SEO is important as well as initial page load we face a problem. We need to have the pages rendered on the Server so that content is present when the page first appears.


Server Side Rendering to the Rescue?

Alt Text

Yes and no. Server Rendering is not free. No one wants to maintain multiple conceptual applications all of sudden because things are on the server now. Projects have been working at creating a universal JavaScript environment where your single application code base seamlessly works on both server and browser.

It can also be complicated to configure and host for different deployment environments. One easy solution is Static Site Generation. We can use the framework's server rendering to render static HTML pages ahead of time.

Now when the user requests the page it can send the already pre-generated page to the browser. Since it is static it can be hosted in a CDN and this loads really quickly. A lot of solutions in this space even advertise how they have this quick initial render and then afterwards the client navigation takes over.

But there are still a couple problems. First Static Generation doesn't lend to dynamic content. Sure nothing beats a pre-rendered page but if the page needs to customizable per person, and involves A/B testing different products etc.. the combinatorics get prohibitively expensive quickly. There are situations where this is fine, and solutions are looking at pre-rendering 10's of thousands of pages in parallel but for dynamic content it just can't keep up to date without great cost.

Even if that doesn't apply to your site the bigger issue is, Frameworks and libraries require a lot of JavaScript and that is expensive to load and parse even when the application is server rendered. Additionally, in order to make the application interactive in the browser JavaScript Frameworks need to hydrate or walk over their component tree in the browser to create the initial framework scaffolding and wire up event listeners. This all takes time and directly impacts end user experience.

Now we've seen these frameworks let you turn off JavaScript for certain pages but it's basically all or nothing. This is serviceable but we can do much better if we know that we are optimizing for First Paint and Time to Interactivity.

It really does beg the question. Are we ok with this?


Return of Multi Page Applications

So what does viewing apps as a collection of separate pages have going for it? Most of the content on the page never needs to be rendered in the browser.

How much of your page actually needs to be re-rendered? The answer is probably very little. How many points on the page can the user interact with? Probably not as many as you think, when you remove all navigation from from the picture. How about if you can remove all the async loading too?

This isn't necessarily no JavaScript(although it can be), just way less of it. You can see this is difficult for an application written as if it were all one big application. Code splitting doesn't really save you here. If the page shares a single root that renders top-down how can we view this thing independently? We can prune unused branches but not the trunk.

Very few frameworks optimize for this since they aren't setup to build this way. When you have chains of props running down through a component tree it's hard to break this apart. You really only have 3 options:

  1. Don't. Manually break your page into a bunch of micro-apps or Islands. (Astro)
  2. Do all data passing through dependency injection. Every part of your page is independent and ship as needed. (Qwik)
  3. Have a compiler smart enough to understand the statefulness of your application and output optimized bundles. (Marko)

These all require special consideration. The first requires you to identify the islands and only scales as well as you are diligent. The second forces you to push state outside of your components which puts a lot of pressure on DX, like can you pass props.children? Are there limits to what can be serialized? The 3rd is immensely complicated and requires specialized language and years of R&D to pull off.

But the results are obvious. Here's a simple example of the impact the Marko team saw when toggling this optimization off some eBay pages.

Alt Text

The optimization has 60%-84% savings in JavaScript bundle size!

Why so much? Marko is not a huge library weighing in at 13kb minified and gzipped. Obviously you are saving on the component code but there is more. Having components only on the server also means certain API wrappers, and formatters like Moment and Lodash just never need to reach the browser.

Marko no-bundle Streaming also helps in this case since it can serve the page immediately without waiting for async calls. It can stream content into server rendered placeholders in real-time all without pulling that code into the bundle.


To the Point

If you need the cutthroat performance for that initial load like you do in eCommerce where milliseconds mean potential lost sales; Where you can't be guaranteed the network or the device power of your customers; You aren't reaching for a framework like Next.js. It just isn't optimized for that. Even if you are using it with a smaller library like Preact here you still are doing way too much in the browser.

You might be thinking, what about things coming in React 18 like Server Components and Streaming SSR? These can help but they don't change the physics alone.

Alt Text

Streaming SSR is incredibly powerful as seen already in Marko and Solid as it removes the initial delay on async data. You can remove most of the overhead of on-demand server rendering over static site generation this way, but it alone doesn't reduce the amount of JavaScript sent.

Server Components make it much easier to write customized APIs. This saves sending the Lodash and Moment to the browser, but you are still running client side diffs, the template is getting sent via API. You can view this as lazy loading/hydration of sorts, but it actually increases the core library size to handle it. If you think about it a different way, given Server Component rules these would just be the static parts an MPA would never be sending to the browser anyway!


Conclusion

Right tool for the job. Yada yada. In all seriousness though, while I dream of a point in the future where this is all the same thing today, MPA frameworks can optimize in ways that are just not available to those building with SPA architecture in mind.

It doesn't take a different language or platform. I'm not saying pull out Rails or Django. You can still get that modern single application JavaScript execution and feel already with the tools available. But if you care about the greatest performance on your initial page loads you aren't going to find that with the likely candidates.

Next time you come across a new solution for eCommerce that promotes its speed. Ask if it is optimized for MPAs, because most likely if not, it is more of the same. There is a reason eBay, Alibaba, and Builder have invested in building their own JavaScript frameworks.

This isn't new but revisiting web foundations. But it's been a decade so maybe it's time. Don't get me wrong. I'm an author of one of those SPA frameworks. One that prides itself on being the fastest of them all on client and server. But architecture trumps raw speed almost every time when comes to delivering the best user experience. So depending on your use case Maybe you don't need that SPA?

Oldest comments (35)

Collapse
 
ianmcburnie profile image
Ian McBurnie

"The paradigm has flipped so much that it is the traditional Multi-Page Application (MPA) that requires explanation now that Single Page Apps (SPA) are becoming the ubiquitous product."

I’m often baffled (and I'd be lying if I didn't say sometimes "frustrated" too) when I have to explain traditional web development foundations, but you're right - it's been a decade - so thank you for a timely and important article.

Collapse
 
chrisczopp profile image
chris-czopp

I think in nearly all the applications I developed in my experience there was no need for SPA. We were doing it because front-end frameworks went this direction. I think MPA built with a framework which allows for writing JS declaratively is the actual way forward. And we've been seeing this realization quite recently. But still, it's unbelievable seeing fresh start-ups deciding to build SPA for their web-based businesses without thinking about SSR or at least static prerendering.

Collapse
 
ryansolid profile image
Ryan Carniato

Yeah my experience was opposite. I worked almost exclusively in scenarios where SPAs are beneficial. Administration and internal web applications, private social media, education apps behind logins. It's why I created Solid since I wanted to improve that, but I also recognize I have to be in the minority.

But I just keep seeing React powered Isomorphic Frameworks (and I don't mean Astro, which is clearly unique) that claim they are somehow improving performance or changing the game. And sadly maybe not so much. I've already explored the limits to performance a long this vector, and I suspect these frameworks won't even be taking things that far. For many use cases it really is not the optimal approach. And this isn't specific to React, but applying SPA patterns where they aren't needed.

Collapse
 
trusktr profile image
Joe Pea

The issue is, people want to write apps one way. Many people don't want to know how to write web apps 3 or 4 different ways and have to pick each time; that is a big decrease in developer experience. These systems are so complex, many people don't want to know all the specific implementation details of all of them, let alone even one in many cases. Many people aren't even front end devs, and they need to pick something and roll with it (React is that at the moment).

Thread Thread
 
ryansolid profile image
Ryan Carniato

Yeah I'd contend though if I were looking at realworld cases. MPAs would probably be the better default for the vast majority of sites. This was a difficult perspective for me to come around to being such a big proponent of SPAs. But let's face it. Just writing some HTML with sprinklings of JavaScript probably still covers the vast majority. We need better tools here for sure, but we haven't successfully bridged the gap yet.

@swyx article dev.to/swyx/svelte-for-sites-react... touches on this. But in the spectrum even Svelte isn't that far on the site side. Elder which the article touches on is like the frameworks I'm talking about today though. And from that perspective maybe Elder and Astro are on to something letting you use familiar tools. But ones optimized for this case can do even better.

Thread Thread
 
trusktr profile image
Joe Pea

Maybe that's true, but there's something magical about SPAs: they can be dynamic, like a game or other real-time morphing experiences. Starting with an MPA architecture then switching to an SPA (when it is decided that dynamic updating of the experience is needed) isn't as easy or convenient as simply starting with an SPA.

How do we let everyone write in the SPA paradigm, but have their applications be MPAs or SPAs as needed depending on use cases, while keeping the performance up?

Thread Thread
 
ryansolid profile image
Ryan Carniato

I have a few ideas on how to achieve that. But nothing exists today meets what would be needed. Marko is closest, Qwik has promise. But you need to take aspect of basically both approaches and fast forward about 3 years. I can picture what it looks like mechanically but to actually bring it to where it needs to be, to be consumable is still years away realistically.

Thread Thread
 
redbar0n profile image
Magne

Could it be done at all, starting from an existing SPA approach? Or would it need a fundamental re-architecting from scratch?

Thread Thread
 
ryansolid profile image
Ryan Carniato

It would be hard. I'm looking at this with Solid to a degree. But realistically it is much easier from MPA. I think Qwik is the closest to a SPA friendly approach and even Misko acknowledges the approach isn't compatible with any existing SPA framework because of the way you need to approach the parent child relationship. In Qwik all data is dependency injected. props.children is fundamentally awkward to deal with. I think a smart enough compiler could reconcile these differences and we are incredibly close to doing so with Marko but it takes starting from different core assumptions/restrictions. For instance all data passed between components needs to be serializable. That isn't exactly 100% the way it needs to be (that assumes components are the boudaries like in Qwik) but same category of considerations apply.

I'm hoping as we learn more a path will form here. But right now between Marko and Qwik MPAs are leading the way on this. They will push the boundaries here further before SPA frameworks can catch up to what Marko was doing in 2014. Which is fine since this isn't what SPAs were made for. It's just going to take some time.

Thread Thread
 
luizz4ugusto profile image
Luiz Augusto Schneider da Silva

Take the reversed way, add a serializable tag/props in solidjs.

Collapse
 
simeydotme profile image
Simon Goellner

I work with e-commerce and this article is lovely! Thanks!

We struggle a lot with server performance due to the backend being a monolithic oracle Java app from over a decade ago, but I've been revamping the front-end and mid-layer. One of the targets was performance and responsiveness. Our competitors used to be far ahead on load times, then sightly ahead after the front-end revamp... But recently they've all moved to react/angular for their front-ends and I saw we are now outperforming them in load time as they've all degraded by moving to client side rendering. 🤷‍♀️ I'm sure their DX is much better, but I doubt their customers, or CEO would care 🙍

Collapse
 
drsensor profile image
૮༼⚆︿⚆༽つ

I've been wondering if there is an SSR framework that doesn't tied to specific back-end engine 🤔 (e.g node, deno)
I see most of SSR frameworks require Nodejs. Hope someday this trend will change.

Collapse
 
ryansolid profile image
Ryan Carniato

Well there are platform required tools. It's a matter of not to be tied to them. We are seeing the adoption of pluggable adapters in things like SvelteKit which support a variety of targets like node or various serverless environments (Vercel, Cloudflare, Begin).

Marko has recently refactored to allow for use on non-node environments like Cloudflare workers. I think we are already seeing the shift.

Collapse
 
drsensor profile image
૮༼⚆︿⚆༽つ • Edited

That's a good news. I heard SvelteKit move away from Vite. So I'm guessing it is because some parts of Vite rely on Rollup which need to be run inside Node 🤔

Thread Thread
 
drsensor profile image
૮༼⚆︿⚆༽つ

Nevermind. I confuse it with svite 😅

Thread Thread
 
ryansolid profile image
Ryan Carniato • Edited

I haven't seen that. To be fair there is a difference between development and deploy environments. I think given the tooling moving off Node is a huge effort and I expect things to stay there for the time being. But just because you use Node to build your solution doesn't mean it requires Node to run it. An app built with Vite is deployed without Vite. My Cloudflare Workers Hacker News demo for Solid is built with Vite but runs using that non-Node platform including streaming SSR leveraging Web (not Node) streams.

Collapse
 
mtyson profile image
MTyson

Just to throw in some complexity - it seems like the per-page routing for Next/SvelteKit would lend itself to MPA... so you get a kind of best of both worlds there (SPA-like for a given pageset, MPA-like for the overall app).

Collapse
 
ryansolid profile image
Ryan Carniato

I mean a SPA could be built on any page hence we see SPA in MPA too.

But this per page routing is what I meant by all in. What I wanted to call out. It isn't equivalent. The problem is MPA style optimizations essentially deconstruct the app. If you model it like a SPA and then just turn off client side routing you still have an app written in a way that is difficult to optimize for MPA. You would need to apply one of the 3 approaches mentioned in the article and the popular solutions don't have tools for that. I just want to clarify we are not near a best of both worlds solution in those.

Collapse
 
mtyson profile image
MTyson

Cool, yeah, that makes a lot of sense.

Collapse
 
johnwarner profile image
John Warner

Good article. I've never been sold on using an SPA for larger projects due to the large upfront loading time. I primarily work with .NET (5, Core, Framework) on the backend which handles the routing and server-side rendering seamlessly, so I primarily develop MPAs.

The SPA centric front end frameworks can be troublesome with an MPA so I ended up developing my own library, Modstache. It works on DOM fragments or HTML strings, so can be used with components, prerendered or dynamic HTML or shadow DOMs. I like to use the Fetch API to get page components/partial updates, JSON objects and to submit forms. Modstache can be applied to these as needed.

I don't mean for this to sound like an ad for Modstache. It's just that I use it to help solve issues brought up in this post. Feel free to use it if it will help solve your problems.

Collapse
 
ryansolid profile image
Ryan Carniato

Yeah there are a few different ways to attack this. SPAs can be made pretty slim, and slim enough that if the use case merits running everything in JavaScript it isn't a problem. Even with huge sites. It's just a question of if the type of site really benefits from it or cares about the upfront cost. And you can always have multiple SPAs.

I talk about how React Server Components don't handle the ruthless case, but they are more than adequate for the large site that would be a SPA. Some parts of the app probably work better server rendered. For a framework like Next, it means maybe removing the need for an explicit API service. Forget GraphQL just author your site. This is powerful technology.

But this can't account for the bottom-line case. If your business is built on page loads and secondary navigations mean they are already in the funnel, it's worth considering how much more streamlined this can be.

Collapse
 
aftabbuddy profile image
Aftab Alam

After doing a lot of SPAs since the last few years, I can't help but appreciate the effectiveness and simplicity of purely server-rendered solutions, and conditionally sprinkle just sufficient enough javascript to add the beloved flair and interactivity we have come to expect from modern experiences. I was trying to actually use Fastify and Pugjs, for a fully server-rendered solution, with client-side JS only available on few pages(as per the need) and was amazed to clearly see how for we could go just with the classic web app approaches + modern vanilla JS.

Though, we wouldn't have probably reached here, without JS frameworks taking the plunge, trying to optimise, embrace what browser offered to replicate what server did, breaking the browser shell and going to the server taking its expressiveness, thinking universal-ity, SEO, aspiring to bank on the hype-train, enabling full-stack universal experiences, and then realising oh shit, we're back at what we did, with more fragmentations on the approaches to solve the same `ol problems. But Astro, Svelte, Solid (I don't get Qwik yet), all seem like superb choice to craft the next-gen web experiences.

After all, when you have chains of props running down through a component tree it's hard to break this apart :-)

Thank you for producing such great content around crafting mindful experiences, and SolidJS.

Collapse
 
brokenthorn profile image
Paul-Sebastian Manole

What about WASM and WASI. How do these new technologies play into the future of SPAs and MPAs?

Collapse
 
ryansolid profile image
Ryan Carniato

WASM has a lot of potential. But in terms of shipping less code I don't think it is particularly compelling. I think it means potential of bringing some really rich performant applications to the browser. I think it opens up new ways to run code of the server(or serverless) in more performant ways.

I expect we will see serverless WASM platforms to offer optimal performance using things like AssemblyScript. This might be a competitive edge. Maybe even new WASM based server platforms.

In the browser as we see more of the barriers ironed out WASM should get more performant. Which it will make it an easier choice for offloading heavy computational work. Still unclear whether it can compete with a SPA framework for rendering, all indications so far suggest not for a while but it will definitely be a key role player.

For traditional sites and MPA which are trying to shave overhead I don't expect it to play much of a role outside of powerful widgets you lazy load.

For the foreseeable future, while I see a lot of value in WASM I don't expect it to be game changer in terms of the core experience. But rather a really nice addition to the arsenal.

Collapse
 
brokenthorn profile image
Paul-Sebastian Manole

Magnificent.

What about if they develop a specific runtime or sandbox similar to how JavaScript runs in the browser, only made specifically for WASM?

Could that happen and would that spell doom for JavaScript?

I mean could something like Rust or AssemblyScript with full access to the browser DOM and browser APIs result in massively more powerful and more performant PWAs?

Could all the above translate to less bytes shipped to the client especially if the code can be sent already compiled for the specific binary interface of that runtime?

PS - Sorry if I'm misunderstanding how some of these technologies work or are supposed to play together. I'm new to the field of web development.

Thread Thread
 
ryansolid profile image
Ryan Carniato

Hard to speculate when they get access to the DOM API if they'd be massively more powerful. I do think that if they could avoid the overhead of going through JavaScript bindings there are good improvements to be made. I wouldn't be surprised if it could be really good for app like experience. However, if I were guessing I wouldn't expect this reality for at least 5 years.

In the next couple years there will likely be some big improvements but to fully realize this is going to take a while. I wouldn't be surprised if we don't see this fully happen this decade. That sounds callous but like Web Components have been in the works for 10 years. WASM has been going for about 4 years now. Which is why I'm unclear if it is more likely JavaScript becomes more dominant than ever during this time and WASM ends up just a way to continue for it to take over traditional backend services. I'm basically saying I'm more likely to bet on the speed proprietary solutions(like a new backend platform) move at than that of the Browser platform which basically dictates the speed WASM can be adopted.

In terms of size I imagine WASM doesn't end up being smaller. However it's possible there is less parsing/execution overhead that makes up for that. I'd be surprised if it has any meaningful impact on this initial load scenario. So for traditional websites/MPAs I wouldn't really hold my breath. We might see exploration there but likely only after what I wrote about above.

Thread Thread
 
redbar0n profile image
Magne

I find this discussion interesting. I want to share my findings on the future of WASM, but I realised that they are just a tad too much for a comment here (bullet points and all). So here are my Notes on the future of WASM and JS (2 min read, most important facts, with sources). FWIW.

avoid the overhead of going through JavaScript bindings

See esp. the Web IDL bindings proposal which would allow WebAssembly to directly access the native web API’s (which allow operating on the DOM). Imagine the possibilities...

Thread Thread
 
ryansolid profile image
Ryan Carniato

Yeah that's pretty much what I've seen. A good summary. I'm aware of these proposals but these things take years. So what I was getting at is I give it about equal chance that JavaScript to WASM compilation gets in a good place before those proposals get implemented. To be performant enough to push JavaScript further into the backend before it's practical to take WASM approaches mainstream in the browser.

Collapse
 
layzee profile image
Lars Gyrup Brink Nielsen

Are you aware of WASI? Interesting potential future for serverless services.

Thread Thread
 
ryansolid profile image
Ryan Carniato

Yeah I didn't realize this had a name. This is an example of what I was referring to in using WASM not in the browser being a potentially more interesting application. It's cool that they are attempting to standardize this approach.

Thread Thread
 
layzee profile image
Lars Gyrup Brink Nielsen

It's been an ongoing effort for a few years. I would love to see vendors support this. It seems like it could be an OCI (Docker) container killer.

Thread Thread
 
layzee profile image
Lars Gyrup Brink Nielsen

A few interesting projects:

Collapse
 
redbar0n profile image
Magne

I'm not saying pull out Rails or Django.

"Pull out" as in "throw away", or as in "bring forth"? Might be good to disambiguate.

Collapse
 
ryansolid profile image
Ryan Carniato

I meant bring forth. I suppose if you were already using those technologies it could be seen as ambiguous. Although in context of someone using JavaScript framework I meant it isn't time to bring back those server technologies.

Collapse
 
gadrawingz profile image
Gad Iradufasha

Great article, many thanks 😊