DEV Community

Cover image for Pagination & Sorting with AWS Amplify
Rakan Nimer
Rakan Nimer

Posted on

Pagination & Sorting with AWS Amplify

#aws #amplify #javascript #graphql

In my previous post, Server-Side Rendered Real-time Web App with Next.js, AWS Amplify & Next.js we went into the details of setting up a Next.js React app and connecting it to Amplify on both the server and client-side.

In this more high-level post we'll be going through how to build a channel-oriented group chat app with Amplify that needs to fetch, listen to, paginate and sort lists of data.

Feel free to skip the write-up and check the final code here and a demo here built and deployed with the AWS Amplify Console.

Project Requirements

The group chat app should consist of 3 routes :

  • channel?id={id} A chat room identified by an id. A user can send and receive messages in real-time. Messages are sorted by descending message creation data (newest at the bottom).

What the channel page should look like

  • channels A list of channels sorted by descending last update date (newest always at the top). A user can add channels and see channels added in real-time.

What the channels page should look like

  • me The current user's profile route, used to edit the user's basic information. A form with 3 fields username, url, bio

What the profile page should look like

The color palette is taken from @dabit3's excellent conference-app-in-a-box

Defining Local State

We won't go much into the implementation details of the app's offline functionality but you can do that by trying it here, or by checking the final code here. If you have any questions about the code, please leave a comment here or ping me on twitter and I'll be happy to answer them !

A single state object is being used for the whole app.

Our state without pagination data looks like this :



export type State = {
  me: {
    id: string;
    name?: string;
    bio?: string;
    url?: string;
  };
  channels: Array<{
    id: string;
    messages: Array<{
      id: string;
      text: string;
      createdAt: string;
      senderId: string;
    }>;
    name: string;
    createdAt: string;
    updatedAt: string;
  }>;
};
Enter fullscreen mode Exit fullscreen mode

To be able to paginate through the data we will be needing to store the list of elements and an additional piece of data, the nextToken string that our API returns when fetching a list of items.

We can store that token at the same level as the list (e.g. { channelsNextToken:string, messagesNextToken: { [channelId]: string } }).

However it's easier to follow the format our API uses and instead of having channels and messages as arrays we can define them as a custom List.

A list has 2 fields : items and nextToken.

The type of elements in the items array depends on the list we're querying.

So the state becomes :



type List<T = unknown> = { items: T[]; nextToken: string };

export type State = {
  me: {
    id: string;
    name?: string;
    bio?: string;
    url?: string;
  };
  channels: List<{
    id: string;
    messages: List<{
      id: string;
      text: string;
      createdAt: string;
      senderId: string;
    }>;
    name: string;
    createdAt: string;
    updatedAt: string;
  }>;
};
Enter fullscreen mode Exit fullscreen mode

Defining the Data Model with GraphQL

We want the messages in a channel to be sorted by createdAt and the channels in our ChannelList to be sorted by updatedAt.

To do that we assign a sortField to our connection directive.



type Message @model {
  id: ID!
  text: String!
  createdAt: String
  senderId: String
  channel: Channel @connection(name: "SortedMessages")
  messageChannelId: String
  # Because every message belongs to a channel
  # a messageChannelId field is added when we create an instance
  # of Message to know to which channel this message belongs.
  # Note that when doing 1-to-n relations using connections with GraphQL Transformer 
  # between 2 types typeA & typeB where typeA has many typeB
  # then a field typeBtypeAId is added to typeB to link it to the right typeA instance
}

type Channel @model {
  id: ID!
  name: String!
  createdAt: String!
  updatedAt: String!
  messages: [Message]
    @connection(name: "SortedMessages", sortField: "createdAt")
  channelList: ChannelList @connection(name: "SortedChannels")
  channelChannelListId: String
  # Same as in message, channel will have the id of its owner (ChannelList)
}

type ChannelList @model {
  id: ID!
  channels: [Channel]
    @connection(name: "SortedChannels", sortField: "updatedAt")
}

type User @model {
  id: ID!
  name: String
  bio: String
  url: String
}

# Note here our custom subscriptions.
# Amplify will generate subscriptions by default but the generated ones are too "broad".
# For example we don't want to listen to every new message created if we're in a channel,
# we just need messages that belong to the current channel.

type Subscription {
  # Used when the user is in the channels route to see channels added by others in real-time
  onCreateChannelInList(channelChannelListId: ID!): Channel
    @aws_subscribe(mutations: ["createChannel"])

  # Used when the user is in the channels route to re-sort channels when their updatedAt timestamp changes
  onUpdateChannelInList(channelChannelListId: ID!): Channel
    @aws_subscribe(mutations: ["updateChannel"])

  # Used in 2 places :
  # 1. In the channels route to listen to new messages added to the channel (We need to display the latest message in every channel)
  # 2. In the channel route to receive new messages in real-time

  onCreateMessageInChannel(messageChannelId: ID!): Message
    @aws_subscribe(mutations: ["createMessage"])
}
Enter fullscreen mode Exit fullscreen mode

With this GraphQL schema, Amplify will :

  1. Setup all the cloud resources we need for our app to work on any scale.
  2. Generate code to CRUD the data

For customizing our data pagination and sorting, we will need to do a bit of extra work ourselves but for the rest we will just be using code generated by Amplify.

Mutations

We won't need to write any query for our mutations, the ones Amplify created for us are all we need.

In src/graphql/mutations.ts we'll find all the different possible mutations we can do.

We will be using :

  • createUser
  • createMessage
  • createChannel
  • updateChannel
  • createChannelList

For example when a user sends a message :



import { API, graphqlOperation } from "aws-amplify";
import { createMessage as createMessageQuery } from "../graphql/mutations";
import { MessageType, Dispatcher } from "../types";

const addMessage = async (
  content: string,
  dispatch: Dispatcher,
  me: State["me"],
  channelId: string
) => {
  const message = {
    text: content,
    createdAt: `${Date.now()}`,
    id: nanoid(),
    senderId: me.id,
    messageChannelId: channelId
  };
  dispatch({
    type: "append-message",
    payload: message
  });
  setScrollDown(Date.now());
  try {
    await (API.graphql(
      graphqlOperation(createMessageQuery, { input: message })
    ) as Promise<unknown>);
  } catch (err) {
    console.warn("Failed to create message ", err);
  }
};
Enter fullscreen mode Exit fullscreen mode

Creating our Custom Queries

getChannelList

Let's create a new file in src/models/ and call it custom-queries.ts.

Inside it, we will add functions that return a GraphQL query when called.

In custom-queries.ts:



export type GetChannelListInput = {
  channelLimit?: number;
  channelNextToken?: string;
  messageLimit?: number;
};

export const getChannelList = ({
  channelLimit = 5,
  channelNextToken = "",
  messageLimit = 1
}: GetChannelListInput) => `
query GetChannelList($id: ID!) {
  getChannelList(id: $id) {
    id
    channels(

      # Number of channels to fetch on each request

      limit: ${channelLimit},

      # sorting direction by the sortField we specified in our schema: updatedAt

      sortDirection: DESC,

      # nextToken is a long string that our API sends back that we can use to
      # retrieve the next batch of entries (older channels in this case)
      # When nextToken is null, then we reached the end of the list

      ${channelNextToken !== "" ? `nextToken:"${channelNextToken}"` : ``}
    ) {
      items {
        id
        name
        createdAt
        updatedAt
        messages(

          # How many messages per channel to retrieve, in our case 1
          limit: ${messageLimit},

          # To get the latest first

          sortDirection: DESC,
          # No need for nextToken here
        ) {
          items {
            id
            createdAt
            senderId
            text
          } 

        }
      }
      nextToken
    }
  }
}
`;
Enter fullscreen mode Exit fullscreen mode

Looking more closely at our query we'll notice that we are using 3 optional arguments to the channels and messages list fields, limit, sortDirection & nextToken explained above in the comments.

getChannelMessages

This one should be straight-forward to understand, it also uses limit, sortDirection & nextToken



export type GetMessageListInput = {
  messageLimit?: number;
  messageNextToken?: string;
};

export const getMessageList = ({
  messageLimit = 10,
  messageNextToken = ""
}: GetMessageListInput) => `
query GetChannel($id: ID!) {
  getChannel(id: $id) {
    id
    name
    createdAt
    updatedAt
    messages(
      limit: ${messageLimit},
      sortDirection: DESC,
      ${messageNextToken !== "" ? `nextToken:"${messageNextToken}"` : ``}
    ) {
      items {
        id
        text
        createdAt
        senderId
      }
      nextToken
    }
  }
}
`;
Enter fullscreen mode Exit fullscreen mode

updateChannel

The result of a GraphQL subscription with AppSync is the mutation selection set.

In our case, the mutation is updateChannel and the subscription onUpdateChannel

The generated updateChannel looks like this : 



mutation UpdateChannel($input: UpdateChannelInput!) {
  updateChannel(input: $input) {
    id
    name
    createdAt
    updatedAt
    creatorId
    messages {
      items {
        id
        text
        createdAt
        senderId
        messageChannelId
      }
      nextToken
    }
    channelList {
      id
      channels {
        nextToken
      }
    }
    channelChannelListId
  }
}
Enter fullscreen mode Exit fullscreen mode

When a conversation is updated we want to receive the last message and some information about the channel.
However, by default, lists are sorted in ascending order, so we need to tell our AppSync API that we want them in descending order, and we'll limit the messages in the set to just one, because we're only interested in the last one.

So we write a custom update query (in src/models/custom-queries.ts) based on how we want the data to look like when a subscription fires an event.



mutation UpdateChannel($input: UpdateChannelInput!) {
    updateChannel(input: $input) {
      id
      name
      createdAt
      updatedAt
      creatorId
      messages(limit: 1, sortDirection: DESC) {
        items {
          text
        }
        nextToken
      }
      channelChannelListId
    }
  }
Enter fullscreen mode Exit fullscreen mode

Using our Custom Queries

The queries above should give us everything we need to fetch both our messages and channels as lists in chunks of 10 or whatever we specify above in the limit.

For example in the channel route, when the component receives a valid channelId we run our query for the first time :



import * as React from "react";
import { Flatlist, ActivityIndicator, View } from "react-native-web";
import { API, graphqlOperation } from "aws-amplify";

import { DispatcherContext } from "../state";

const getChannelMessages = (channelId: string, nextToken: string) => {
  try {
    const query = getMessageList({
      messageLimit: 10,
      messageNextToken: nextToken
    });
    const messages = await API.graphql(
      graphqlOperation(query, { id: channelId })
    );
    return {
      messages: messages.data.getChannel.messages,
      channel: messages.data.getChannel
    };
  } catch (err) {
    console.warn("Failed to get messages ", err);
    return { messages: { items: [], nextToken: "" }, channel: {} };
  }
};

const Channel = ({ channelId, messages }) => {
  const dispatch = React.use(DispatcherContext);
  const [isLoading, setIsLoading] = React.useState(false);
  React.useEffect(() => {
    let isMounted = true;
    if (!channelId) return;
    setIsLoading(true);
    // We start by sending an empty string as nextToken
    getChannelMessages(channelId, "")
      .then(({ messages, channel }) => {
        if (!isMounted) return;
        setIsLoading(false);
        // We store the new messages that contain the next batch of messages and update the nextToken giant string
        dispatch({ type: "append-messages", payload: { channelId, messages } });
        // And update the channel's updatedAt field
        dispatch({ type: "update-channel", payload: channel });
      })
      .catch(err => {
        console.warn(
          "Failed to retrieve channel messages for channel ",
          channelId
        );
        setIsLoading(false);
      });
    () => {
      isMounted = false;
    };
  }, [channelId]);
  return (
    <FlatList
      inverted={true}
      ref={flatlistRef}
      ListFooterComponent={() =>
        isLoading ? (
          <ActivityIndicator
            animating={true}
            color={colors.highlight}
            style={{ marginTop: 15, marginBottom: 15, height: 30 }}
          />
        ) : (
          <View style={{ height: 30 }} />
        )
      }
      keyExtractor={item => item.id}
      data={messages.items}
      renderItem={({ item }) => <Message key={item.id} message={item} />}
      onEndReached={() => {
        if (messages.nextToken === null) return;
        setIsLoading(true);
        // When the end is reached we fetch the next batch of messages if they exist
        getChannelMessages(channelId, messages.nextToken).then(
          ({ messages }) => {
            setIsLoading(false);
            dispatch({
              type: "append-messages",
              payload: { channelId, messages }
            });
          }
        );
      }}
      onEndReachedThreshold={0.01}
    />
  );
};
Enter fullscreen mode Exit fullscreen mode

Subscriptions

For our subscriptions we won't need to write any GraphQL queries. Amplify will generate all the ones we need.

In the GraphQL Schema input for the GraphQL Transformer we defined some subscriptions: 



type Subscription {
  # Used when the user is in the channels route to see channels added by others in real-time
  onCreateChannelInList(channelChannelListId: ID!): Channel
    @aws_subscribe(mutations: ["createChannel"])

  # Used when the user is in the channels route to re-sort channels when their updatedAt timestamp changes
  onUpdateChannelInList(channelChannelListId: ID!): Channel
    @aws_subscribe(mutations: ["updateChannel"])

  # Used in 2 places :
  # 1. In the channels route to listen to new messages added to the channel (We need to display latest message in every channel)
  # 2. In the channel route to receive new messages in real-time

  onCreateMessageInChannel(messageChannelId: ID!): Message
    @aws_subscribe(mutations: ["createMessage"])
Enter fullscreen mode Exit fullscreen mode

This will generate the queries in src/graphql/subscriptions.ts and the types we need in src/API.ts

For example let's look at the code needed to listen to new messages on a channel : 



import { API, graphqlOperation} from 'aws-amplify'
import {
  onCreateMessageInChannel,
} from "../graphql/subscriptions";

const Channel = () => {
  React.useEffect(() => {
    let isMounted = true;
    if (!channelId) return;
    API.graphql(
      graphqlOperation(
        onCreateMessageInChannel,
        { messageChannelId: channelId }
      )
    ).subscribe(message => {
      const newMessage = message.value.data.onCreateMessageInChannel;
      if (newMessage === null || newMessage.senderId === me.id || !isMounted) return;
      // prepend instead of append because they are sorted in descending order by createdAt 
      dispatch({ type: "prepend-message", payload: newMessage });
    });
    () => {
      isMounted = false;
      onCreateListener.unsubscribe();
    };
  }, [channelId]);

  // Rest of the code
}
Enter fullscreen mode Exit fullscreen mode

Simple enough, listening to a graphql subscription and turning it off on unmount.

Deploying

The code is built and deployed by the AWS Amplify Console. To deploy your own, you can click on this button amplifybutton or just connect your repository to the Amplify Console and that's it.

As you can see in the picture below, Amplify builds and deploys every commit on the master branch.

Wrapping it up

Most applications will need to handle lists of data and fetch progressively from it (chat, comments, history, feed).

This post goes through the challenging parts of doing that with React & Amplify and provides a good starting point to build one yourself !

Top comments (1)

Collapse
 
undef_obj profile image
Ricardo •

Loved this post!

Read next

sarvar_04 profile image

🚀 Amazon Nova: AWS's New Foundation Model for GenAI🤖

Sarvar Nadaf -

franckpachot profile image

Aurora Limitless - Resharding

Franck Pachot -

j_dev28 profile image

OKX DEX API Essentials: Single and Cross-Chain Swaps on Avalanche C-Chain

Julian Martinez -

nikdyankov profile image

State of JS 2024

Nik Dyankov -