Afonso Barracha

Posted on Sep 3, 2022 • Updated on Sep 22, 2022

How to solve the GraphQL N+1 problem in NestJS with Dataloaders and MikroOrm (for both Apollo and Mercurius)

#graphql #nestjs #node #typescript

Introduction

In this article I will give a brief explanation on what is the GraphQL n+1 problem, on how to solve them with Dataloaders, and how to implement them in NestJS with a Code First Approach.

GraphQL n+1 problem

One of the biggest advantages of using GraphQL over REST is that it reduces the amount of requests that you need to do to the server, as you can request all you need in one go. For example, for a simple SQL entity diagram like this:

In an REST API you would need three route to fetch a post with its author and comments:

/api/post/:id to get a single post by a given id;
/api/post/:id/author to get the post author;
/api/post/:id/comments?first=10 to get the post first 10 comments.

While in GraphQL you could not only choose what data you need from each entity but query all the related fields in a single query:

query PostById($postId: Int!, $commentsFirst: Int = 10) {
  postById(postId: $postId) {
    title
    body
    createdAt
    updatedAt
    author {
      name
      username
    }
    comments(first: $commentsFirst) {
      edges {
        node {
          id
          body
          author {
            name
            username
          }
        }
        cursor
      }
      pageInfo {
        hasNextPage
      }
    }
  }
}

To be able to get all of this information in a single query you would need to JOIN all those relations together, but that would be wasteful for when they are not necessary.

A better option is to resolve the data only when you need it with the @ResolveField decorator:

^{NOTE: this is only an example.}

import {
  Args,
  Context,
  Mutation,
  Parent,
  Query,
  ResolveField,
  Resolver,
} from '@nestjs/graphql';
import { PostEntity } from './entities/post.entity';
import { CommentsService } from '../comments/comments.service';
import { PaginatedCommentsType } from '../comments/entities/gql/paginated-comments-type';
import { FilterRelationDto } from '../common/dtos/filter-relation.dto';
import { UsersService } from '../users/users.service';
import { UserEntity } from '../users/entities/user.entity';

@Resolver(() =>  PostEntity)
export class PostsResolver {
  constructor(
    private readonly postsService: PostsService,
    private readonly usersService: UsersService,
    private readonly commentsService: CommentsService,
  ) {}
  //...

  @ResolveField('author', () => UserEntity)
  public async resolveAuthor(
    @Parent() post: PostEntity
  ) {
    return this.usersService.userById(post.author.id);
  }

  @ResolveField('comments', () => PaginatedCommentsType)
  public async resolveComments(
    @Parent() post: PostEntity,
    @Args() dto: FilterRelationDto,
  ) {
    return this.commentsService.postComments(post.id, dto);
  }
}

This will work when you are only fetching a single post, but what happens if you are fetching multiple posts, such as:

query UsersPosts($userId: Int!, $first: Int = 10, $after: String) {
  usersPosts(userId: $userId, first: $first, after: $after) {
    edges {
      node {
        title
        body
        createdAt
        updatedAt
        author {
          name
          username
        }
        comments(first: 10) {
          edges {
            node {
              id
              body
              author {
                name
                username
              }
            }
            cursor
          }
          pageInfo {
            hasNextPage
            hasPreviousPage
            startCursor
            endCursor
          }
        }
      }
    }
  }
}

You would end up doing multiple queries to the database for both the author and the comments, to be precise, one for each post leading to the n+1 problem. Your database interaction would look something close to this:

SELECT * FROM posts WHERE user_id = 1 LIMIT 3;
SELECT * FROM users WHERE id = 2;
SELECT * FROM users WHERE id = 3;
SELECT * FROM users WHERE id = 4;
SELECT * FROM comments WHERE post_id = 2 LIMIT 10;
SELECT * FROM comments WHERE post_id = 3 LIMIT 10;
SELECT * FROM comments WHERE post_id = 4 LIMIT 10;

How dataloaders solve this problem

In essence dataloaders cache and batch your requests, basically they cache all the parent entities (or only the id) and do a single query to get the necessary related data.

So the previous queries would be transformed to something close to this:

SELECT * FROM posts WHERE user_id = 1 LIMIT 3;
SELECT * FROM users WHERE id IN (2, 3, 4);
-- ...

^{NOTE: I omitted the paginated query as it is long and more advance. Advance dataloaders are covered later in this article.}

NestJS Dataloader Implementation

The way you set up dataloaders on NestJS depends on the adapter you choose: Apollo or Mercurius.

Initial Set Up (common for both)

On your project folder create a new module (and service) for your loaders:

$ nest g mo loaders
$ nest g s loaders

^{NOTE: Do not forget to export the loaders service from the loaders module.}

Move the spec file to a tests folder, and create an interfaces folder with the Loader Interface (loader.interface.ts):

export interface ILoader<T extends Object, P = undefined> {
  obj: T;
  params: P;
}

After this go to your loaders service, add the entity manager as a dependency and create a single public method called getLoaders:

import { EntityManager } from '@mikro-orm/postgresql';
import { Injectable, Type } from '@nestjs/common';

@Injectable()
export class LoadersService {
  constructor(
    private readonly em: EntityManager,
    private readonly commonService: CommonService,
  ) {}

  public getLoaders() {}
}

Since a lot of the loaders will have the same logic I recommend creating the following helper static methods:

^{NOTE: IBase is the base of all entities with common fields like id and createdAt, I recommend having one on your project for type inference.}

Get Entities to get the entities out of the loader object:

// ...
@Injectable()
export class LoadersService {
  constructor(
    private readonly em: EntityManager,
    private readonly commonService: CommonService,
  ) {}

  /**
   * Get Entities
   *
   * Maps the entity object to the entity itself.
   */
  private static getEntities<T extends IBase, P = undefined>(
    items: ILoader<T, P>[],
  ): T[] {
    const entities: T[] = [];

    for (let i = 0; i < items.length; i++) {
      entities.push(items[i].obj);
    }

    return entities;
  }

  // ...
}

Get Entities IDs to get the entities' IDs of the loader object:

// ...
@Injectable()
export class LoadersService {
  // ...

  /**
   * Get Entity IDs
   *
   * Maps the entity object to an array of IDs.
  */
  private static getEntityIds<T extends IBase, P = undefined>(
    items: ILoader<T, P>[],
  ): number[] {
    const ids: number[] = [];

    for (let i = 0; i < items.length; i++) {
      ids.push(items[i].obj.id);
    }

    return ids;
  }

  // ...
}

Get Relation IDs gets the ids of a many-to-one relationship:

// ...
@Injectable()
export class LoadersService {
  // ...

  /**
   * Get Relation IDs
   *
   * Maps the entity object many-to-one relation to an array of IDs.
   */
  private static getRelationIds<T extends IBase, P = undefined>(
    items: ILoader<T, P>[],
    relationName: string,
  ): number[] {
    const ids: number[] = [];

    for (let i = 0; i < items.length; i++) {
      ids.push(items[i].obj[relationName].id);
    }

    return ids;
  }

  // ...
}

Get Entity Map, turns an entity array (normally from the findAll/getResult method) into a JavaScript HashMap:

// ...
@Injectable()
export class LoadersService {
  // ...

  /**
   * Get Entity Map
   *
   * Turns an array of entity objects to a map of entity objects
   * with its ID as the key.
   */
  private static getEntityMap<T extends IBase>(entities: T[]): Map<number, T> {
    const map = new Map<number, T>();

    for (let i = 0; i < entities.length; i++) {
      const entity = entities[i];
      map.set(entity.id, entity);
    }

    return map;
  }

  // ...
}

Get Result takes the relation id array and map and returns the final result array:

// ...
@Injectable()
export class LoadersService {
  // ...

  /**
   * Get Results
   *
   * With the IDs of the relation id array, gets the results of the map.
   */
  private static getResults<T>(
    ids: number[],
    map: Map<number, T>,
    defaultValue: T | null = null,
  ): T[] {
    const results: T[] = [];

    for (let i = 0; i < ids.length; i++) {
      const id = ids[i];
      results.push(map.get(id) ?? defaultValue);
    }

    return results;
  }

  // ...
}

Complex Loaders

The following private methods are completely optional and only work with SQL, but If you are using SQL I recommend them, they are generics for complex relations like paginated relations, counting related entities and checking existence (boolean cases).

To be able to use these generics first you need to follow these steps in your project first:

Remove all many to many relationships and manually create all pivot tables (annoying I know, but it is necessary to be able to create the pagination sub-query) that implement from a creation interface:
```
export interface ICreation {
  createdAt: Date;
}
```
Create a cursor pagination function on your common service like the one in my other article, you will need to add a optional virtual numeric field called count on your base entity to be able to use this;

Create the following dto for paginating relations on your common dtos folder (filter.dto.ts):

import { ArgsType, Field, Int } from '@nestjs/graphql';
import { IsEnum, IsInt, Max, Min } from 'class-validator';
import { QueryOrderEnum } from '../enums/query-order.enum';

@ArgsType()
export abstract class FilterRelationDto {
  @Field(() => QueryOrderEnum, { defaultValue: QueryOrderEnum.ASC })
  @IsEnum(QueryOrderEnum)
  public order: QueryOrderEnum = QueryOrderEnum.ASC;

  @Field(() => Int, { defaultValue: 10 })
  @IsInt()
  @Min(1)
  @Max(50)
  public first = 10;
}

For counts create the following interface on the loaders interfaces folder (count-result.interface.ts):
```
export interface ICountResult {
  id: number;
  count: number;
}
```
Finally for checking for existence create this interface (existence-result.interface.ts):
```
export interface IExistenceResult {
  id: number;
  existence: number;
}
```

Since we can have two types of relations, there are always two versions for each generic, one for many-to-many relationships (with pivot tables) and one for one-to-many relationships, so the generics are thus:

Paginator Loaders:

// ...
import { Collection } from '@mikro-orm/core';
import { Injectable, Type } from '@nestjs/common';
// ...

@Injectable()
export class LoadersService {
  // ...

  /**
   * Basic Paginator
   *
   * Loads paginated one-to-many relationships
   */
  private async basicPaginator<T extends IBase, C extends IBase>(
    data: ILoader<T, FilterRelationDto>[],
    parent: Type<T>,
    child: Type<C>,
    parentRelation: keyof T,
    childRelation: keyof C,
    cursor: keyof C,
  ): Promise<IPaginated<C>[]> {
    if (data.length === 0) return [];

    const { first, order } = data[0].params;
    const parentId = 'p.id';
    const childAlias = 'c';
    const childId = 'c.id';
    const knex = this.em.getKnex();
    const parentRef = knex.ref(parentId);
    const parentRel = String(parentRelation);
    const ids = LoadersService.getEntityIds(data);

    const countQuery = this.em
      .createQueryBuilder(child, childAlias)
      .count(childId)
      .where({
        [childRelation]: parentRef,
      })
      .as('count');
    const entitiesQuery = this.em
      .createQueryBuilder(child, childAlias)
      .select(`${childAlias}.id`)
      .where({
        [childRelation]: {
          id: parentRef,
        },
      })
      .orderBy({ [cursor]: order })
      .limit(first)
      .getKnexQuery();
    const results = await this.em
      .createQueryBuilder(parent, 'p')
      .select([parentId, countQuery])
      .leftJoinAndSelect(`p.${parentRel}`, childAlias)
      .groupBy([parentId, childId])
      .where({
        id: { $in: ids },
        [parentRelation]: { $in: entitiesQuery },
      })
      .orderBy({ [parentRelation]: { [cursor]: order } })
      .getResult();
    const map = new Map<number, IPaginated<C>>();

    for (let i = 0; i < results.length; i++) {
      const result = results[i];

      map.set(
        result.id,
        this.commonService.paginate(
          result[parentRelation].getItems(),
          result.count,
          0,
          cursor,
          first,
        ),
      );
    }

    return LoadersService.getResults(
      ids,
      map,
      this.commonService.paginate([], 0, 0, cursor, first),
    );
  }

  /**
   * Pivot Paginator
   *
   * Loads paginated many-to-many relationships
   */
  private async pivotPaginator<
    T extends IBase,
    P extends ICreation,
    C extends IBase,
  >(
    data: ILoader<T, FilterRelationDto>[],
    parent: Type<T>,
    pivot: Type<P>,
    pivotName: keyof T,
    pivotParent: keyof P,
    pivotChild: keyof P,
    cursor: keyof C,
  ): Promise<IPaginated<C>[]> {
    if (data.length === 0) return [];

    // Because of runtime
    const strPivotName = String(pivotName);
    const strPivotChild = String(pivotChild);
    const strPivotParent = String(pivotParent);

    const { first, order } = data[0].params;
    const parentId = 'p.id';
    const knex = this.em.getKnex();
    const parentRef = knex.ref(parentId);
    const ids = LoadersService.getEntityIds(data);

    const countQuery = this.em
      .createQueryBuilder(pivot, 'pt')
      .count(`pt.${strPivotChild}_id`, true)
      .where({ [strPivotParent]: parentRef })
      .as('count');
    const pivotQuery = this.em
      .createQueryBuilder(pivot, 'pt')
      .select('pc.id')
      .leftJoin(`pt.${strPivotChild}`, 'pc')
      .where({ [strPivotParent]: parentRef })
      .orderBy({
        [strPivotChild]: { [cursor]: order },
      })
      .limit(first)
      .getKnexQuery();
    const results = await this.em
      .createQueryBuilder(parent, 'p')
      .select([parentId, countQuery])
      .leftJoinAndSelect(`p.${strPivotName}`, 'e')
      .leftJoinAndSelect(`e.${strPivotChild}`, 'f')
      .where({
        id: { $in: ids },
        [strPivotName]: {
          [strPivotChild]: { $in: pivotQuery },
        },
      })
      .orderBy({
        [strPivotName]: {
          [strPivotChild]: { [cursor]: order },
        },
      })
      .groupBy([`e.${strPivotParent}_id`, 'f.id'])
      .getResult();
    const map = new Map<number, IPaginated<C>>();

    for (let i = 0; i < results.length; i++) {
      const result = results[i];
      const pivots: Collection<P, T> = result[strPivotName];
      const entities: C[] = [];

      for (const pivot of pivots) {
        entities.push(pivot[strPivotChild]);
      }

      map.set(
        result.id,
        this.commonService.paginate(entities, result.count, 0, cursor, first),
      );
    }

    return LoadersService.getResults(
      ids,
      map,
      this.commonService.paginate([], 0, 0, cursor, first),
    );
  }

  // ...
}

Counter Loaders:

// ...

@Injectable()
export class LoadersService {
  // ...

  /**
   * Get Counter Results
   *
   * Maps the count result to a number array
   */
  private static getCounterResults(
    ids: number[],
    raw: ICountResult[],
  ): number[] {
    const map = new Map<number, number>();

    for (let i = 0; i < raw.length; i++) {
      const count = raw[i];
      map.set(count.id, count.count);
    }

    return LoadersService.getResults(ids, map, 0);
  }

  // ...

  /**
   * Basic Counter
   *
   * Loads the count of one-to-many relationships.
   */
  private async basicCounter<T extends IBase, C extends IBase>(
    data: ILoader<T>[],
    parent: Type<T>,
    child: Type<C>,
    childRelation: keyof C,
  ): Promise<number[]> {
    if (data.length === 0) return [];

    const parentId = 'p.id';
    const knex = this.em.getKnex();
    const parentRef = knex.ref(parentId);
    const ids = LoadersService.getEntityIds(data);

    const countQuery = this.em
      .createQueryBuilder(child, 'c')
      .count('c.id')
      .where({ [childRelation]: { $in: parentRef } })
      .as('count');
    const raw: ICountResult[] = await this.em
      .createQueryBuilder(parent, 'p')
      .select([parentId, countQuery])
      .where({ id: { $in: ids } })
      .groupBy(parentId)
      .execute();

    return LoadersService.getCounterResults(ids, raw);
  }

  /**
   * Pivot Counter
   *
   * Loads the count of many-to-many relationships.
   */
  private async pivotCounter<T extends IBase, P extends ICreation>(
    data: ILoader<T>[],
    parent: Type<T>,
    pivot: Type<P>,
    pivotParent: keyof P,
    pivotChild: keyof P,
  ): Promise<number[]> {
    if (data.length === 0) return [];

    const strPivotChild = String(pivotChild);
    const parentId = 'p.id';
    const knex = this.em.getKnex();
    const parentRef = knex.ref(parentId);
    const ids = LoadersService.getEntityIds(data);

    const countQuery = this.em
      .createQueryBuilder(pivot, 'pt')
      .count(`pt.${strPivotChild}_id`, true)
      .where({ [pivotParent]: { $in: parentRef } })
      .as('count');
    const raw: ICountResult[] = await this.em
      .createQueryBuilder(parent, 'p')
      .select([parentId, countQuery])
      .where({ id: { $in: ids } })
      .groupBy(parentId)
      .execute();

    return LoadersService.getCounterResults(ids, raw);
  }

  // ...
}

Existence Loader:
^{NOTE: in this one you need to pass the entire from statement.}

// ...
import { IExistenceResult } from './interfaces/existence-result.interface';
// ...

@Injectable()
export class LoadersService {
  // ...

  /**
   * Get Existence
   *
   * Finds if the entity relation exists and returns a boolean array
   */
  private async getExistence<T extends IBase>(
    data: ILoader<T, FilterRelationDto>[],
    parent: Type<T>,
    fromCondition: string,
  ): Promise<boolean[]> {
    if (data.length === 0) return [];

    const ids = LoadersService.getEntityIds(data);
    const caseString = `
        CASE
          WHEN EXISTS (
            SELECT 1
            ${fromCondition}
          )
          THEN 1
          ELSE 0
        END AS 'existence'
      `;
    const raw: IExistenceResult[] = await this.em
      .createQueryBuilder(parent, 'p')
      .select(['p.id', caseString])
      .where({ id: { $in: ids } })
      .execute();

    const map = new Map<number, boolean>();

    for (let i = 0; i < raw.length; i++) {
      const { id, existence } = raw[i];
      map.set(id, existence === 1);
    }

    return LoadersService.getResults(ids, map);
  }

  // ...
}

Apollo Specific

For Apollo, firstly we need to install the dataloader package.

Subsequently you need to create an interface called Loaders (loaders.interface.ts) with all the loaders that you need in your project, for the example above it would look something like this:

// or import * as Dataloader if you don't have "esModuleInterop": true
import DataLoader from 'dataloader';
import { ILoader } from './loader.interface';
import { IComment } from '../../comments/interfaces/comment.interface'
import { IPaginated } from '../../common/interfaces/paginated.interface';
import { FilterRelationDto } from '../../common/dtos/filter-relation.dto';
import { IPost } from '../../posts/interfaces/post.interface';
import { IUser } from '../../users/interfaces/user.interface';

export interface ILoaders {
  author: DataLoader<ILoader<IPost | IComment>, IUser>;
  comments: DataLoader<ILoader<IPost, FilterRelationDto>, IPaginated<IComment>>;
}

Now we need to add the loaders logic inside the loaders service, so create two new private methods that instantiate a new Dataloader with the batching logic inside:

Author Relation Loader:
Firstly I would create an interface for every entity that has an author, as that is very common:

import { IBase } from './base.interface';
import { IUser } from '../../users/interfaces/user.interface';

export interface IAuthored extends IBase {
  author: IUser;
}

And finally the loaders logic would be as follows:

// ...
import DataLoader from 'dataloader';
import { IAuthored } from '../common/interfaces/authored.interface';
// ...

@Injectable()
export class LoadersService {
  // ...

  /**
   * Author Relation Loader
   *
   * Gets every author relation.
   */
  private authorRelationLoader<T extends IAuthored>() {
    return new DataLoader(async (data: ILoader<T>[]): Promise<UserEntity[]> => {
      if (data.length === 0) return [];

      const ids = LoadersService.getRelationIds(data, 'author');
      const users = await this.em.find(UserEntity, {
        id: {
          $in: ids,
        },
      });
      const map = LoadersService.getEntityMap(users);
      return LoadersService.getResults(ids, map);
    });
  }

  // ...
}

Post Comments Loader:

// ...

@Injectable()
export class LoadersService {
  // ...

  /**
   * Post Comments Loader
   *
   * Get paginated comments of post.
   */
  private postCommentsLoader() {
    return new DataLoader(async (
      data: ILoader<PostEntity, FilterRelationDto>[],
    ): Promise<IPaginated<CommentEntity>[]> => {
      return this.basicPaginator(
        data,
        PostEntity,
        CommentEntity,
        'comments',
        'post',
        'id',
      );
    });
  }

  // ...
}

Inside the loaders service complete the getLoaders method with the newly created loaders:

// ...
import { ILoaders } from './interfaces/loaders.interface'; 
// ...

@Injectable()
export class LoadersService {
  // ...

  public getLoaders(): ILoaders {
    return {
      author: this.authorRelationLoader(),
      comments: this.postCommentsLoader(),
    };
  }

  // ...
}

Finally inside your GraphQL Config Class you need to add the loaders to the context:

^{NOTE: Do not forget to add the LoadersModule to the GraphQL.forRootAsync() imports array.}

import { ApolloDriver, ApolloDriverConfig } from '@nestjs/apollo';
import { Injectable } from '@nestjs/common';
import { ConfigService } from '@nestjs/config';
import { GqlOptionsFactory } from '@nestjs/graphql';
import { ICtx } from './interfaces/ctx.interface';
import { LoadersService } from '../loaders/loaders.service';

@Injectable()
export class GqlConfigService implements GqlOptionsFactory {
  constructor(
    private readonly configService: ConfigService,
    private readonly loadersService: LoadersService,
  ) {}

  public createGqlOptions(): ApolloDriverConfig {
    return {
      driver: ApolloDriver,
      context: ({ req, res }): ICtx => ({
        req,
        res,
        loaders: this.loadersService.getLoaders(),
      }),
      // ...
    };
  }
}

To use the loaders just take them out of the the context when you need them, so the example resolver would now look something like this:

import {
  Args,
  Context,
  Mutation,
  Parent,
  Query,
  ResolveField,
  Resolver,
} from '@nestjs/graphql';
import { PostEntity } from './entities/post.entity';
import { PaginatedCommentsType } from '../comments/entities/gql/paginated-comments-type';
import { FilterRelationDto } from '../common/dtos/filter-relation.dto';
import { UserEntity } from '../users/entities/user.entity';
import { ILoaders } from '../loaders/interfaces/loaders.interface';

@Resolver(() =>  PostEntity)
export class PostsResolver {
  constructor(private readonly postsService: PostsService) {}
  //...

  @ResolveField('author', () => UserEntity)
  public async resolveAuthor(
    @Context('loaders') loaders: ILoaders,
    @Parent() post: PostEntity
  ) {
    return loaders.author.load({ obj: post, params: undefined });
  }

  @ResolveField('comments', () => PaginatedCommentsType)
  public async resolveComments(
    @Context('loaders') loaders: ILoaders,
    @Parent() post: PostEntity,
    @Args() dto: FilterRelationDto,
  ) {
    return loaders.comments.load({ obj: post, params: dto });
  }
}

Mercurius Specific

Working with dataloaders in Mercurius can be weird. Loaders are already built in to the adapter, so the logic needs to go inside the GraphQL Config Class loaders parameter, and we no longer need the dataloader package.

First we need to make some changes to our resolver:

Delete the resolveAuthor as we only need the @Field decorator on the entity classes;
Delete all arguments apart from @Args() from the resolveComments and return nothing.

Thus the resolver would look something like this:

import {
  Args,
  Context,
  Mutation,
  Parent,
  Query,
  ResolveField,
  Resolver,
} from '@nestjs/graphql';
import { PostEntity } from './entities/post.entity';
import { PaginatedCommentsType } from '../comments/entities/gql/paginated-comments-type';
import { FilterRelationDto } from '../common/dtos/filter-relation.dto';

@Resolver(() =>  PostEntity)
export class PostsResolver {
  constructor(private readonly postsService: PostsService) {}
  //...

  // The logic will go to the loaders object
  @ResolveField('comments', () => PaginatedCommentsType)
  public resolveComments(
    @Args() dto: FilterRelationDto,
  ) {
    return;
  }
}

You might be asking why do we even need the empty method? We need the empty method so the comments field is generated on the GraphQL Schema.

The loaders service is not that diferent from the Apollo one, we just return the arrow function without the DataLoader instatiation:

^{NOTE: we still need to return arrow functions because we need to bind the this keyword.}

Author Relation Loader:

// ...
import DataLoader from 'dataloader';
import { IAuthored } from '../common/interfaces/authored.interface';
// ...

@Injectable()
export class LoadersService {
  // ...

  /**
   * Author Relation Loader
   *
   * Gets every author relation.
   */
  private authorRelationLoader<T extends IAuthored>() {
    return async (data: ILoader<T>[]): Promise<UserEntity[]> => {
      if (data.length === 0) return [];

      const ids = LoadersService.getRelationIds(data, 'author');
      const users = await this.em.find(UserEntity, {
        id: {
          $in: ids,
        },
      });
      const map = LoadersService.getEntityMap(users);
      return LoadersService.getResults(ids, map);
    };
  }

  // ...
}

Post Comments Loader:

// ...

@Injectable()
export class LoadersService {
  // ...

  /**
   * Post Comments Loader
   *
   * Get paginated comments of post.
   */
  private postCommentsLoader() {
    return async (
      data: ILoader<PostEntity, FilterRelationDto>[],
    ): Promise<IPaginated<CommentEntity>[]> => {
      return this.basicPaginator(
        data,
        PostEntity,
        CommentEntity,
        'comments',
        'post',
        'id',
      );
    };
  }

  // ...
}

The major difference from Apollo is how the getLoaders method is formated, it needs to follow the GraphQL Schema as seen in the Mercurius documentation.

Hence the getLoaders will look something like this:

// ...

@Injectable()
export class LoadersService {
  // ...

  public getLoaders(): ILoaders {
    return {
      Post: {
        author: this.authorRelationLoader<IPost>(),
        comments: this.postCommentsLoader(),
      },
    };
  }

  // ...
}

Finally on the GraphQL Config Class there is a loaders paramenter:

^{NOTE: Do not forget to add the LoadersModule to the GraphQL.forRootAsync() imports array.}

import { Injectable } from '@nestjs/common';
import { ConfigService } from '@nestjs/config';
import { GqlOptionsFactory } from '@nestjs/graphql';
import { MercuriusDriver, MercuriusDriverConfig } from "@nestjs/mercurius";
import { ICtx } from './interfaces/ctx.interface';
import { LoadersService } from '../loaders/loaders.service';

@Injectable()
export class GqlConfigService implements GqlOptionsFactory {
  constructor(
    private readonly configService: ConfigService,
    private readonly loadersService: LoadersService,
  ) {}

  public createGqlOptions(): MercuriusDriverConfig {
    return {
      driver: MercuriusDriver,
      loaders: this.loadersService.getLoaders(),
      // ...
    };
  }
}

Conclusion

With this you can now lean your database request while still being able to optimize queries with relationships.

Top comments (2)

Sushil Jaiswar • Oct 28 '22

I followed and implemented this solution but it doesn't work for federation.
__resolveReference of DataloaderService is never called. I am using @Nestjs+Mercurius+Federation

In Resolver File

  @ResolveReference()
  public async resolveReference(reference: IReference): Promise<User> {
    // console.log(reference);
    return;
  }

 public getLoaders() {
    return {
      User: {
        __resolveReference: this.userRelationLoader(),
      },
    };
  }

Afonso Barracha • Sep 12 '22 • Edited

Just noticed I had an error on the pivot paginator, it has to be a for of as written in mikro-orm docs and not a normal for loop, as it actually returns a collection and not an array.

Already fixed it.

DEV Community

How to solve the GraphQL N+1 problem in NestJS with Dataloaders and MikroOrm (for both Apollo and Mercurius)

Introduction

GraphQL n+1 problem

How dataloaders solve this problem

NestJS Dataloader Implementation

Initial Set Up (common for both)

Complex Loaders

Apollo Specific

Mercurius Specific

Conclusion

Top comments (2)

Read next

Using Tailwind plugins to create custom utility classes

Migration - Module query with TypeORM version 0.3.x

Angular Event Modifier

Coder's Journey