Frontend Platform use case - Enabling features and hiding the distribution problems

Like other similar products, Hasura is distributed and can be consumed in a lot of different ways, due to different client’s needs and the evolution of the product during the last years.

The lack of a centralized way to identify the server type and version highly impacts working on the Hasura Console (the most important Hasura’s front-end product) and enabling a feature or not. Things get even worse when the distribution matrix crosses the pricing tiers and their frequent changes.

Here is a walkthrough the plan to tackle the problem by offering new React APIs to the Hasura frontenders.

Photo by John Barkiple on Unsplash

How the Hasura Console identifies the server "type"

The Hasura Console "assets" (CSS, JS, static files) are in charge of the Console build tools but the HTML serving them unfortunately is not. The various servers were used to add some logic and some environment variables in the global scope (populating a window.__env object), load the Console assets, and the Console can consume the window.__env object and decide what to show to the users and what not.

How the abovementioned situation became such a big problem for the Console developers? Because of the Hasura distribution options:

1. The Hasura server embeds the Console assets in order to allow the clients to work offline

2. At the same time, the Hasura server always tries to load the assets from the CDN before fallback to the embedded ones (to fix hotfixes on the fly)

3. The Hasura CLI acts as a server that in turn talks to the Hasura server. When served by the Hasura CLI, the Console talks with the Hasura CLI and the Hasura server based on the actions to do

4. There is no way to force the clients to update their Hasura CLI, resulting in a lot of outdated distributions

5. In either the Hasura Server or Hasura CLI, the client could be logged as an "Enterprise" user

6. The Hasura Console can be served by Lux too, the server behind Hasura Cloud

7. Hasura Cloud has different pricing tiers that evolved over the years

8. Hasura was introducing a new "Enterprise" distribution and licensing model that differs from the previous one

If you mix it with the fact that

1. The window.__env object is not documented nor typed

2. The domain names for the plans themselves are not clear (what a "Pro" Console is is a source of confusion too)

3. How to launch the application in all the modes/types (they turned out to be 24 different combinations) is hard

4. Some "new" server implementations presented some "old" domain names with the goal of easing the Console's developer life

you will not be surprised if there were a lot of creative ways to identify the current mode/plan, including a lot of duplication, a lack of centralized management, and a lot of subsequent PRs needed to adjust enabling features after the release.

The feature-first APIs

During one of the internal Front-end Office Hours, we discussed the generic idea to stop dealing with plans/modes/types, and creating some feature-first APIs that hide the implementation details of identifying the plans and that allows the developers to simply tell something like "I need to show this feature in Cloud, that's all". Since this was one of the most reported problems by the frontenders (if you are curious about how we identified the problems to work on, read Frontend Platform use case - Creating a roadmap without a Product Manager, we (the Platform team) immediately jumped on it.

The generic plan was something like:

1. Quickly creating a POC to be discussed internally and understand if it would have fulfilled the desired needs

2. Identifying all the existing modes/plans/types/tiers impacting the Console/features and documenting them

3. Analyzing the environment variables and APIs served by the different servers could have helped us

4. Implementing a new React wrapper (<LoadHasuraPlan />) around the application that identifies the "Hasura plan"

5. Dealing with just one/two Hasura plans to later iterate and all the other plans

6. Creating the "feature-first" APIs: some React components and hooks that allow to easily deal with when a feature is enabled or not (and if not, also telling "why" to the Hasura developers to allow them to propose some alternative or upselling UIs)

7. Creating some testing and, especially, Storybook utilities to help the Hasura developers to simulate all the edge cases straight from their working tool of choice

8. Easing the Hasura developers to add and configure new features

9. Easing the Hasura developers to add and configure new ways to detect the user/license properties

10. Refactoring the existing implementations of the most important one/two plans to dogfood the new APIs

11. Presenting the APIs to Hasura developers to gain their feedback and keeping them updated about the current progress of the new APIs

12. Going back to point 5 to deal with all the Hasura plans one by one

13. (maybe) Sitting at a table with the back-end developers to expose a new and documented API dedicated to moving the implementation details from the Console to the server

Let me elaborate on some of the above steps.

The high-level APIs

At a very high level, this diagram summarizes the new APIs and how to use them from the App.

(since dev.to compresses the images, here it the link to the original Excalidraw)

1. A <LoadHasuraPlan /> wrapper that identifies the Hasura plan and wraps the app with a React Context Provider whose value is a Zustand store that stores the Hasura plan details

2. A useIsFeatureEnabled React hook usable everywhere in the app that received the name of the feature, retrieves the Zustand store from the React Context and passes both the store and the feature object to a checkCompatibility function

3. All the implementation details are just a sort of state machine to identify the plan and a lot of if statements to check if the feature is enabled for the current Hasura plan (the TypeScript part includes more complexity, you can find more info about it later in the article)

Please note: despite Zustand allowing us to expose Vanilla JS APIs, we decided from the very beginning to expose only React APIs. The big advantage is that if we offer only React APIs, we can count on React's reactivity system! That means that managing dynamic/reactive cases like "the user activated the license during the Console lifetime" is a no-brainer because we update the store and all the component/hook consumers automatically re-render.

Otherwise, exposing Vanilla JS APIs

• Would have forced us to expose some subscription-like APIs to be sure the API consumers always work with the latest data

• Would have opened the doors to a lot of freedom and creativity for all the Console's developers. While speaking about working on big codebases, and speaking from the standpoint of who usually have to later deal with a lot of refactors due to the mentioned creativity... It is way better to limit the options

The initial POC

You can find the initial POC here but the main goal of creating a POC was

1. To validate the fallback/upselling APIs

2. To validate the idea with the Console developers and to collect their feedback since they are the real users of the new APIs and they have all the feature-related complexity in mind

At the end of the POC's README, you can find all the feedback we collected (asking for different API shapes, asking about the Feature Flags, etc.).

Identifying all the existing modes/plans/types/tiers impacting the Console/features

This step required a lot of back and forth with a lot of other stakeholders to identify the available options, all the differences, some of the design choices behind the current state, etc. Then, I spent some time on a lot of trial and error to document all the cases and elaborate some steps to correctly identify all the Hasura plans. It turned out that the window.__env object plus a series of three XHR requests in a row (in the worst-case scenario) are enough to correctly identify all 24 possible distribution combinations.

Implementing the new <LoadHasuraPlan /> React wrapper

The main characteristic of the wrapper are:

1. It can be completely disabled through a passThrough prop. This is important to get it merged on the main branch as soon as possible way before the new APIs are ready. Putting everything on main instead of long-living branches is crucial in terms of maintainability and size of PRs, and it's the logic at the base of Feature Flags for instance.

2. It can be enabled through some secret, in-app, developer tools to test it out everywhere at any time. You can refer to the great Make your own DevTools article if to learn how to effectively create something similar.

3. The internal list of static (like the window.__env object) and dynamic (like the license and the pricing tiers) resources consumed to identify the current Hasura plan must be extended easily by everyone to cover future needs.

4. It must be tested thoroughly to be sure it covers all the existing and eventually unknown edge cases. This is crucial because the crazy number of combinations leads for sure to some unmanaged cases and if the wrapper does not handle them could make the Hasura customers block from using the Console!

That's why I spent a good amount of time writing a lot of unit tests like this

it('When the server lacks the server_type, the EE license API fail, and the consoleType is pro, then set the Hasura plan as EE Classic and render the children', async () => {
  // Arrange
  const versionApiResponse: VersionApiResponsePayload = {
    version: '',
  };
  const rawServerEnvVars: ServerEnvVars = { consoleType: 'pro' };

  // Act
  server.use(
    rest.get('*/v1/version', (req, res, context) =>
      res(context.json(versionApiResponse))
    ),
    rest.get('*/v1/entitlement', (req, res, context) =>
      res(context.status(404))
    )
  );

  render(
    <LoadHasuraPlan rawServerEnvVars={rawServerEnvVars}>
      <>
        <RenderHasuraPlanName />
        <RenderEeLicenseStatus />
      </>
    </LoadHasuraPlan>,
    { wrapper: TestWrapper }
  );

  // Assert
  expect(
    await screen.findByText('Hasura plan: eeClassic')
  ).toBeInTheDocument();
  expect(
    screen.getByText('EE license is not expected in this environment')
  ).toBeInTheDocument();
});

Creating the "feature-first" APIs

The most interesting part here is not about the APIs themselves but about the hidden TypeScript gymnastics. Let me explain what was the goal we had in mind: think about a feature like OpenTelemetry integration, which compatibility is defined by the following object

const openTelemetry = {
  ce: 'disabled',
  ee: {
    withLicense: 'enabled',
    withoutLicense: 'disabled',
  },
} as const satisfies Compatibility;

When the developer calls isFeatureEnabled('openTelemetry') and the current Hasura plan is ce the result must be, from a TypeScript perspective:

{
  status: 'disabled,
  doMatch: {}, // ← empty object
  doNotMatch: { ee: { withLicense: true } }, // ← does not include `withoutLicense`
  current: { hasuraPlan: { name: 'ce' } },
}

Why have I said, "from a TypeScript perspective"? Because TypeScript means autocompletion for known properties and error for unknown ones.

Why is it so much important to ensure, at the type level, that the doNotMatch object does not include the ee.withoutLicense property? Because the developer must be prevented from doing something like this!

function OpenTelemetry() {
  const {
    status,
    doNotMatch,
  } = useIsFeatureEnabled('openTelemetry');

  if (status === 'disabled') {
    if (doNotMatch.ee.withLicense) {
        return (
          <div>Activate your license to use OpenTelemetry!</div>
        );
      }

    if (doNotMatch.ee.withoutLicense) {

        // WHAAAT? This edge case does not exist!!!!!! 😱😱 But the developers could try to cover
        // all the edge cases without realizing some of them are impossible!
        // TypeScript could prevent this error by throwing an error that `ee.withoutLicense` does
        // not exist for the `OpenTelemetry` feature! 🎉​​
        return (
          <div>Deactivate your license to use OpenTelemetry!</div>
        );
      }
    }
  }

  return <div>Enjoy OpenTelemetry!</div>;
}

To be honest: I am not sure that, in the end, the TypeScript gymnastics we implemented (customizing a couple of utility types of the great type-fest) are worth it, but in the first implementation we kept them.

Creating some testing and Storybook utilities

The Hasura Console developers spend most of their time working in Storybook, hence offering some ad-hoc utilities to write all the component stories for all the different Hasura is crucial in terms of Developer Experience.

We initially opted for exposing some Storybook Decorators that allow simulating the Hasura plans, without also offering some in-Storybook UI addons to switch between plans at runtime. The goal was to offer something very practical without implementing something expensive before validating the overall idea. So we just:

1. Expose a hasuraPlanDecorator from the new IsFeatureEnabled directory

2. The hasuraPlanDecorator accepts the name of the Hasura plan to simulate

3. Based on the name of the plan, hasuraPlanDecorator returns a React Context Provider whose value is an externally-initialized Zustand store

4. The basic plan is used for every story

5. Every story can import and customise its own decorators passing a different plan name

The whole implementation is something like

type MockedPlan = {
  name:
    | 'ce'
    | 'eeWithoutLicense'
    | 'eeWithActiveLicense'
    | 'eeWithExpiredLicense'
    | 'eeWithDeactivatedLicense'
    | 'eeWithLicenseInGracePeriod';
};

// --------------------------------------------------
// MOCK PROVIDERS APIS
// --------------------------------------------------

const HasuraPlanMockProvider: React.FC<{
  mockedPlan: MockedPlan;
}> = props => {
  const { mockedPlan, children } = props;

  switch (mockedPlan.name) {
    case 'ce':
      return (
        <MockStoreContextProvider
          params={createNewStore({
            hasuraPlan: { name: 'ce' },
            serverEnvVars: {}, // The server env vars (aka window.__env) is not mandatory
          })}
        >
          {children}
        </MockStoreContextProvider>
      );

    // etc.

  }
};

// --------------------------------------------------
// STORYBOOKs APIS
// --------------------------------------------------

export const hasuraPlanDecorator =
  (mockedPlan: MockedPlan) => (Story: React.FC) =>
    (
      <HasuraPlanMockProvider mockedPlan={mockedPlan}>
        <Story />
      </HasuraPlanMockProvider>
    );

And every story can simulate a different plan with something like this

export const OpenTelemetryStory: ComponentStory<typeof OpenTelemetryEeRequired> = () => {
  return <OpenTelemetryEeRequired />;
};

OpenTelemetryStory.decorators = [hasuraPlanDecorator({ name: 'eeWithoutLicense' })]; // <-- simulating a custom Hasura plan

OpenTelemetryStory.storyName = '💠 Primary';

The same APIs can be leveraged for the unit tests too.

Refactor the existing implementations of the most important one/two plans to dogfood the new APIs

This is the longest part of the first iteration. The fast way to do that is to simply change the core of all the abstractions that have been created to tackle the problem. But this is suboptimal and does not allow properly dogfooding the new APIs. The right thing to do, instead, is to deeply understand the existing abstractions, go back to the problem that led to creating the abstractions and then solve the problem in an easier way thanks to the new API.

More: it also means updating all the tests and stories involved to test and document the refactored components and functions.

The result is hundreds of lines of code removed and a lot of functional (manual) tests required to be sure no features are broken. Unfortunately, it also means digging into some existing bugs that now arise thanks to the full understanding of the different distribution combinations.

You could wonder: why not just using Feature Flags?

Because a hypothetical feature flag like "Enable OpenTelemetry for the EE users" is not helpful if you are not able to properly identify who are the EE users. The whole problem presented in this article is specific to identifying plans and hence users.

Conclusion

Many thanks to N. Beaussart and N. Inchauspé (the frontenders of the Platform team along with me) for supporting me designing and discussing the plan presented in this article 😊