Creating a Simple Generator for Nx Plugins

Introduction

Welcome to the second blog post in our series on creating custom Nx plugins! This post targets developers who are familiar with the basics of the Nx ecosystem but have never built a plugin for Nx before. In this article, we'll dive deep into generators, learn how to set up a development environment, and create a simple generator for an Nx plugin. Let's get started!

Table of Contents

1. Deep Dive into Generators
2. Setting Up the Development Environment
3. Creating a Simple Generator
4. Working with the Nx Devkit Tree
5. Summary

Deep Dive into Generators

Nx Generators are an integral part of the Nx ecosystem, providing an interface for code generation and modification. They operate by reading and altering the Abstract Syntax Tree (AST) of your codebase, enabling precise and flexible modifications. This fine-grained control fosters creation of tailored development environments that reflect the unique needs of your project.

At its core, a generator is a function that accepts a Tree and an options object. The Tree represents your filesystem, and you interact with it to make file changes. The options object carries the values provided by the user when invoking the generator.

A fundamental concept in understanding Nx generators is their immutability. All changes you make to the Tree within a generator are staged, and not immediately applied. This staged approach allows changes to be grouped into an atomic commit, ensuring your filesystem stays in a consistent state even if the generator fails partway through execution.

To create a custom generator, you define a generator function in a file named generator.ts. The function receives a host Tree and an options object. For complex generators, Nx provides the generateFiles utility. This powerful utility employs schematics-like template syntax to interpolate variables into file contents and filenames.

For input control, Nx uses JSON Schema, allowing generators to accept complex inputs, validate them, and provide interactive prompts to the users. Defining a schema makes your generator self-documenting, as Nx can automatically generate descriptions, validations, and prompts based on it.

Generators can invoke other generators, allowing code reuse and composition. Nx provides helper functions like runTasksInSerial or runTasksInParallel to manage the lifecycle of these generator executions.

Testing generators is crucial, and Nx's virtual file system makes this easy. During tests, you can verify that the right changes are being applied without affecting your actual filesystem.

In conclusion, Nx Generators provide a powerful way to automate your development tasks, creating a robust, repeatable, and efficient development workflow.

Setting Up the Development Environment

Creating an Nx workspace and a custom plugin begins by using the create-nx-workspace utility. Let's create a new workspace named nx-tools.

npx create-nx-workspace@latest nx-tools

Select "empty" when prompted for a preset to create a bare workspace.

Now, incorporate the @nx/nx-plugin into your workspace. The @nx/nx-plugin furnishes essential tools for creating and managing Nx plugins.

nx add @nx/nx-plugin

With the Nx Plugin established, we'll now generate our custom plugin, named nx-cdk, in our workspace. This plugin will be designed to generate, build, and deploy AWS CDK projects, facilitating efficient development in our Nx monorepo.

nx g @nx/nx-plugin:plugin nx-cdk

This command creates a new plugin with the name nx-cdk and provides the preliminary code and configuration.

Finally, open your freshly created workspace in your preferred code editor. For Visual Studio Code users, this can be done with:

code nx-tools

With these steps completed, your development environment is primed for crafting your nx-cdk plugin.

Creating a Simple Generator for our nx-cdk Plugin

Now that the development environment for nx-tools is set up, we can begin with creating a simple generator for our nx-cdk plugin. The generator will help scaffold AWS CDK applications.

Nx Devkit provides a built-in schematic to help generate the boilerplate for our generator. Run the following command to create the generator:

nx g @nrwl/nx-plugin:generator NxCdk --project=nx-cdk

This command does several things:

1. It adds a new generator named nx-cdk in the nx-cdk plugin.
2. It creates a new nx-cdk directory inside the generators directory of our nx-cdk plugin.
3. Inside the generators/nx-cdk directory, it creates a generator.ts file which is the main file for our generator, a generator.spec.ts file which contains the test cases for our generator, a schema.json file which will be used to define the options for our generator and a schema.d.ts file containing the interface definition for our schema.

Open the schema.json and modify the content as per your requirements:

  "$schema": "http://json-schema.org/schema",
  "$id": "NxCdk",
  "title": "",
  "type": "object",
  "properties": {
    "name": {
      "type": "string",
      "description": "",
      "$default": {
        "$source": "argv",
        "index": 0
      },
      "x-prompt": "What name would you like to use?"
    },
    "tags": {
      "type": "string",
      "description": "Add tags to the project (used for linting)",
      "alias": "t"
    },
    "directory": {
      "type": "string",
      "description": "A directory where the project is placed",
      "alias": "d"
    }
  },
  "required": [
    "name"
  ]
}

Next, update the generator.ts file with the code to scaffold a new AWS CDK project in your workspace:

import { readFileSync } from 'fs';
import * as path from 'path';
import { resolve } from 'path';

import {
    GeneratorCallback,
    Tree,
    addDependenciesToPackageJson,
    addProjectConfiguration,
    formatFiles,
    generateFiles,
    getWorkspaceLayout,
    names,
    offsetFromRoot
} from '@nx/devkit';
import { jestProjectGenerator } from '@nx/jest';
import { Linter, lintProjectGenerator } from '@nx/linter';
import { runTasksInSerial } from '@nx/workspace/src/utilities/run-tasks-in-serial';

import {
    awsCdkLibVersion,
    awsCdkVersion,
    constructsVersion,
    sourceMapSupportVersion,
    tsJestVersion
} from '../../utils/versions';
import { addJestPlugin } from './lib/add-jest-plugin';
import { addLinterPlugin } from './lib/add-linter-plugin';
import { NxCdkGeneratorSchema } from './schema';

interface NormalizedSchema extends NxCdkGeneratorSchema {
    projectName: string;
    projectRoot: string;
    projectDirectory: string;
    parsedTags: string[];
}

function normalizeOptions(tree: Tree, options: NxCdkGeneratorSchema): NormalizedSchema {
    const name = names(options.name).fileName;
    const projectDirectory = options.directory ? `${names(options.directory).fileName}/${name}` : name;
    const projectName = projectDirectory.replace(new RegExp('/', 'g'), '-');
    const projectRoot = `${getWorkspaceLayout(tree).appsDir}/${projectDirectory}`;
    const parsedTags = options.tags ? options.tags.split(',').map((s) => s.trim()) : [];

    return {
        ...options,
        projectName,
        projectRoot,
        projectDirectory,
        parsedTags
    };
}

function addFiles(tree: Tree, options: NormalizedSchema) {
    const templateOptions = {
        ...options,
        ...names(options.name),
        offsetFromRoot: offsetFromRoot(options.projectRoot),
        template: ''
    };
    generateFiles(tree, path.join(__dirname, 'files'), options.projectRoot, templateOptions);
}

export default async function (tree: Tree, options: NxCdkGeneratorSchema) {
    const tasks: GeneratorCallback[] = [];
    const normalizedOptions = normalizeOptions(tree, options);
    addProjectConfiguration(tree, normalizedOptions.projectName, {
        root: normalizedOptions.projectRoot,
        projectType: 'application',
        sourceRoot: `${normalizedOptions.projectRoot}/src`,
        targets: {
            bootstrap: {
                executor: '@myorg/nx-cdk:bootstrap'
            },
            deploy: {
                executor: '@myorg/nx-cdk:deploy'
            },
            destroy: {
                executor: '@myorg/nx-cdk:destroy'
            },
            diff: {
                executor: '@myorg/nx-cdk:diff'
            },
            ls: {
                executor: '@myorg/nx-cdk:ls'
            },
            synth: {
                executor: '@myorg/nx-cdk:synth'
            }
        },
        tags: normalizedOptions.parsedTags
    });
    addFiles(tree, normalizedOptions);
    tasks.push(addJestPlugin(tree));
    tasks.push(addLinterPlugin(tree));
    tasks.push(addDependencies(tree));
    await lintProjectGenerator(tree, { project: options.name, skipFormat: true, linter: Linter.EsLint });
    await jestProjectGenerator(tree, {
        project: options.name,
        setupFile: 'none',
        skipSerializers: true,
        testEnvironment: 'node'
    });
    await ignoreCdkOut(tree);
    await formatFiles(tree);
    return runTasksInSerial(...tasks);
}

function addDependencies(tree: Tree) {
    const dependencies: Record<string, string> = {};
    const devDependencies: Record<string, string> = {
        'aws-cdk': awsCdkVersion,
        'aws-cdk-lib': awsCdkLibVersion,
        constructs: constructsVersion,
        'source-map-support': sourceMapSupportVersion,
        'ts-jest': tsJestVersion
    };
    return addDependenciesToPackageJson(tree, dependencies, devDependencies);
}

async function ignoreCdkOut(tree: Tree) {
    const ignores = readFileSync(resolve(tree.root, '.gitignore'), { encoding: 'utf8' }).split('\n');
    if (!ignores.includes('cdk.out')) {
        ignores.push('# AWS CDK', 'cdk.out', '');
    }
    tree.write('./.gitignore', ignores.join('\n'));
}

Here's the explanation of what the generator.ts file does:

1. NormalizedSchema Interface: A TypeScript interface is defined that extends the NxCdkGeneratorSchema interface from schema.d.ts. It adds properties like projectName, projectRoot, projectDirectory, and parsedTags.
2. normalizeOptions Function: It accepts a Tree and generator options and returns the options after performing transformations like naming conventions, file path conventions, and tag parsing.
3. addFiles Function: This function generates files from the provided template and places them in the project directory.
4. The Main Generator Function: It's an async function that performs several tasks:
   • Normalize the options using the normalizeOptions function.
   • Add project configuration using the addProjectConfiguration function.
   • Generate files using the addFiles function.
   • Add Jest and Linter plugins to the Nx tree.
   • Add dependencies to the package.json file using the addDependenciesToPackageJson function.
   • Setup Jest and ESLint for the newly generated project.
   • Ignore the cdk.out directory in .gitignore.
   • Finally, format all the generated files. 5 addDependencies Function: Adds dependencies required for the AWS CDK project, like aws-cdk, aws-cdk-lib, constructs, source-map-support, and ts-jest to the devDependencies of package.json.
5. ignoreCdkOut Function: Reads the .gitignore file and checks if 'cdk.out' is included in the ignore list. If not, it adds 'cdk.out' to the list, which prevents the generated AWS CDK output files from being tracked by git.

The schema.d.ts file is where we define the TypeScript interface for our generator options. This interface helps TypeScript to understand the structure of the options and provide type checking. In our case, the interface for the NxCdkGeneratorSchema should look like this:

export interface NxCdkGeneratorSchema {
    name: string;
    tags?: string; 
    directory?: string; 
}

This TypeScript interface aligns with the schema.json file and provides a typed way of accessing the generator options.

Testing is a critical part of any software development, and Nx generators are not an exception. Nx provides a built-in way to test generators using Jest. When we created our NxCdk generator, Nx also created a generator.spec.ts file in the nx-cdk directory. This is a test specification file for our generator.

import { Tree } from '@nrwl/devkit';
import * as testingUtils from '../../../utils/testing';

import generator from './generator';
import { NxCdkGeneratorSchema } from './schema';

describe('myCdkApp generator', () => {
  let appTree: Tree;
  const options: NxCdkGeneratorSchema = { name: 'test' };

  beforeEach(() => {
    appTree = testingUtils.createEmptyWorkspace(Tree.empty());
  });

  it('should run successfully', async () => {
    await expect(
      generator(appTree, options)
    ).resolves.not.toThrowError();
  });
});

In this test case, we first create an empty Nx workspace. We then run our generator with a given set of options. The expect function is then used to assert that our generator should run without throwing any errors.

Working with the Nx Devkit Tree

The Nx Devkit Tree object represents the workspace's file system. It allows you to read, write, and manipulate files and directories. In our simple generator, we use the generateFiles function to create files from templates located in the ./files directory.

Registering the Generator

Register the generator in nx-cdk/generators.json:

{

  "$schema": "http://json-schema.org/schema",

  "name": "nx-cdk",

  "version": "0.0.1",

  "generators": {

    "nx-cdk": {

      "factory": "./src/generators/nx-cdk/generator",

      "schema": "./src/generators/nx-cdk/schema.json",

      "description": "nx-cdk generator"

    }

  }

}

Summary

The blog post provides an in-depth guide on creating a simple generator for Nx plugins. It introduces the Nx ecosystem and the concept of generators, functions that manipulate your codebase via the Abstract Syntax Tree (AST). The key idea is the immutability of generators, where changes are staged and grouped into atomic commits for filesystem consistency.

The post presents a step-by-step tutorial on setting up an Nx workspace and crafting a custom Nx plugin. This involves creating a new workspace, incorporating the @nx/nx-plugin, generating a custom plugin, and opening the workspace in your code editor.

The main tutorial focuses on creating a generator for an nx-cdk plugin designed to generate, build, and deploy AWS CDK projects. This process includes running commands to create the generator, modifying the schema.json file to fit your requirements, and updating the generator.ts file with code to scaffold a new AWS CDK project.

Alright, you've made it this far and hopefully enjoyed the ride. The next stop on our journey will be implementing an executor. So grab some popcorn and stay tuned - I promise it will be just as fun, if not more! See you next time, folks!

Hey there, dear readers! Just a quick heads-up: we're code whisperers, not Shakespearean poets, so we've enlisted the help of a snazzy AI buddy to jazz up our written word a bit. Don't fret, the information is top-notch, but if any phrases seem to twinkle with literary brilliance, credit our bot. Remember, behind every great blog post is a sleep-deprived developer and their trusty AI sidekick.