Effective CRUD and Validation with NestJS and JOI

When writing any API it's paramount that the data you expect to receive is the data you receive.

I have seen on so many occasions where even senior developers fail to apply any validation allowing anything into the app only to cause bugs and problems further down the line.

In this tutorial i want to share some insight on options you have available to help improve and secure your API a little more .

Let's start with a new project and Create a new resource (Nest Docs for CLI) using the command below where the theme will be a space-ship

nest g resource space-ship

? What transport layer do you use? REST API
? Would you like to generate CRUD entry points? Yes
CREATE src/space-ship/space-ship.controller.spec.ts (608 bytes)
CREATE src/space-ship/space-ship.controller.ts (992 bytes)
CREATE src/space-ship/space-ship.module.ts (277 bytes)
CREATE src/space-ship/space-ship.service.spec.ts (482 bytes)
CREATE src/space-ship/space-ship.service.ts (679 bytes)
CREATE src/space-ship/dto/create-space-ship.dto.ts (35 bytes)
CREATE src/space-ship/dto/update-space-ship.dto.ts (190 bytes)
CREATE src/space-ship/entities/space-ship.entity.ts (26 bytes)
UPDATE package.json (2023 bytes)
UPDATE src/app.module.ts (330 bytes)
✔ Packages installed successfully.

As you can see this give us a nice structure to get up and running.

The main area of focus will be the controller as this is the entry point and the data should be validated here before we progress anywhere.

Failing to prepare is preparing to fail

As you can see below we have our basic CRUD operations that would simply allow us to pass in any value through our end points and has no validation whats so ever.

@Controller('space-ship')
export class SpaceShipController {
  constructor(private readonly spaceShipService: SpaceShipService) {}

  @Post()
  create(@Body() createSpaceShipDto: CreateSpaceShipDto) {
    return this.spaceShipService.create(createSpaceShipDto);
  }

  @Get()
  findAll() {
    return this.spaceShipService.findAll();
  }

  @Get(':id')
  findOne(@Param('id') id: string) {
    return this.spaceShipService.findOne(+id);
  }

  @Patch(':id')
  update(@Param('id') id: string, @Body() updateSpaceShipDto: UpdateSpaceShipDto) {
    return this.spaceShipService.update(+id, updateSpaceShipDto);
  }

  @Delete(':id')
  remove(@Param('id') id: string) {
    return this.spaceShipService.remove(+id);
  }
}

We'll start with the easy stuff and then build up more complex techniques.

Let's get the app up and running so we can call the endpoints and quickly see the problems.

@Get(':id')
  findOne(@Param('id') id: string) {
    return this.spaceShipService.findOne(+id);
  }

This should raise alarm bells straight away:

• potentially any value can be passed in
• no validation
• passed straight into the service

Let's try it out:

http://localhost:3000/space-ship/one-million-char-string

returns:

This action returns a #NaN spaceShip

We shouldn't even call the service if the number is invalid.

One of the in-built validation pipes can be used here and is simple as this:

  @Get(':id')
  findOne(@Param('id', ParseIntPipe) id: number) {
    return this.spaceShipService.findOne(id);
  }

using the ParseIntPipe allows us to check at entry level for the correct type.

When calling the end point again the same way as before we see the error throws straight away:

{
    "statusCode": 400,
    "message": "Validation failed (numeric string is expected)",
    "error": "Bad Request"
}

We can now add this pipe to the delete and patch functions. These have all of a sudden becomes secure and have prevented the passing on of data that aren't numbers.

Most of the primitives have (built-in pipes)[https://docs.nestjs.com/pipes#built-in-pipes] that can used and added to the API endpoint easily.

While this is convenient, most of the complexity will be using PUT and POST request as there many more fields that come in various structures.

Custom Pipes / Validators

Now the more fun part.

export class SpaceShipValidatorPipe implements PipeTransform<MyTypeIn, MyTypeOut> {
  public transform(query: MyTypeIn, metadata: ArgumentMetadata): MyTypeOut {
    return {} as MyTypeOut;
  }
}

The new class implements PipeTransform<MyTypeIn, MyTypeOut> which requires the transform function. The interface simply takes in a class and returns another class, in this case MyTypeIn would be the raw data from the client and then the MyTypeOut is the class we will send downstream. The function simply allows us to carry out the logic.

So let's create a new class that represents the JSON passed from the client.

class SpaceShipRequestDto {
  public id: number;
  public name: string;
  public onMission: boolean;
  public dateCreated: string;
  public captain: {
    id: number;
    name: string;
  };
}

or

{
    "id": 1,
    "name": "FTL 1",
    "onMission": true,
    "dateCreated": "2021-09-13T09:37:43.130Z",
    "captian": {
        "id": 10,
        "name": "Dale"
    }
}

We can send this in as a POST and it still gets through to the service. As before we want to validate this earlier and fail fast.

Also, a lot of the time the data coming into the system is different from what it receives.

Let's say out space-ship.dto.ts was this:

export class CreateSpaceShipDto {
  public spaceShipId: number;
  public shipName: string;
  public onMission: boolean;
  public dateCreated: Date;
  public captainId: number;
}

With the main differences are some field names, the dateCreated is a Date and we just need captain id. But it generally maps to same thing.

Update the validator pipe to now use the correct classes:

export class SpaceShipValidatorPipe implements PipeTransform<SpaceShipRequestDto, CreateSpaceShipDto> {
  public transform(query: SpaceShipRequestDto, metadata: ArgumentMetadata): CreateSpaceShipDto {
    return {} as CreateSpaceShipDto;
  }
}

Then update the controller to use this new pipe. it's as simple as creating a new instance in the @Body() annotation. The dto is console logged just to see what the output is and as expected, it is {}, as that is what we are returning from the pipe:

  @Post()
  create(
    @Body(new SpaceShipValidatorPipe()) createSpaceShipDto: CreateSpaceShipDto,
  ) {
    console.log(createSpaceShipDto);
    return this.spaceShipService.create(createSpaceShipDto);
  }

Now our pipe is wired up, we can now start the validation and transformation. To do this we will use JOI.
npm i joi
npm install --save-dev @types/joi

update tsconfig.json:
"esModuleInterop": true

Restart the server

We create a schema that has the same structure as the incoming request but with the validation we want to have on the fields.

export const captainSchema = Joi.object({
  id: Joi.number().required(),
  name: Joi.string().max(20).optional(),
}).options({ abortEarly: false, allowUnknown: true });

export const spaceShipSchema = Joi.object({
  id: Joi.number().required(),
  name: Joi.string().max(20).required(),
  onMission: Joi.boolean().required(),
  dateCreated: Joi.date().required(),
  captain: captainSchema.required(),
}).options({ abortEarly: false, allowUnknown: true });

This reads quite self explanatory, the functions all return the instance so it's easy to build up the query. Check the docs for all the options.

Update the pipe to validate the request using the schema. We can also throw an error should the validation not match.

export class SpaceShipValidatorPipe
  implements PipeTransform<SpaceShipRequestDto, CreateSpaceShipDto>
{
  public transform(
    query: SpaceShipRequestDto,
    metadata: ArgumentMetadata,
  ): CreateSpaceShipDto {
    const result = spaceShipSchema.validate(query, {
      convert: true,
    });

    if (result.error) {
      const errorMessages = result.error.details.map((d) => d.message).join();
      throw new BadRequestException(errorMessages);
    }

    const validSpaceShip = result.value;
    return {
      spaceShipId: validSpaceShip.id,
      onMission: validSpaceShip.onMission,
      shipName: validSpaceShip.name,
      captainId: validSpaceShip.captain.id,
      dateCreated: validSpaceShip.dateCreated,
    } as CreateSpaceShipDto;
  }
}

Now if we run the POST with this request

{
    "id": 1,
    "name": "FTL 1",
    "onMission": true,
    "dateCreated": "2021-09-13T09:37:43.130Z",
    "captain": {
        "id": 10,
        "name": "Dale"
    }
}

We will see the request has been converted into a SpaceShip.dto:

{
  spaceShipId: 1,
  onMission: true,
  shipName: 'FTL 1',
  captainId: 10,
  dateCreated: 2021-09-13T09:37:43.130Z
}

In industry we can be writing tests along side this doing TDD.

We can try remove a field or change a type in the json request and see an error is thrown:

{
    "statusCode": 400,
    "message": "\"name\" is required,\"onMission\" is required",
    "error": "Bad Request"
}

The good things as well is the the dates will auto parse into a Date() from string, string auto cast to numbers.

Next steps would be to test the living daylights out of it and get it rock solid.

Conclusion

Adding the validation hook in controllers secures the API as it doesn't progress into the service.

The logic is extracted out keeping our controllers and services free of validation checking.

There are also various other ways to do this such the class-transformer package. I would generally use this to convert between the classes.

All in all there are more classes required but when doing doing TDD it helps to separate the concerns of each section and add tests for particular uses cases.

Thanks for Reading :)