Rakan Nimer

Posted on Aug 9, 2019

Pagination & Sorting with AWS Amplify

#aws #amplify #javascript #graphql

In my previous post, Server-Side Rendered Real-time Web App with Next.js, AWS Amplify & Next.js we went into the details of setting up a Next.js React app and connecting it to Amplify on both the server and client-side.

In this more high-level post we'll be going through how to build a channel-oriented group chat app with Amplify that needs to fetch, listen to, paginate and sort lists of data.

Feel free to skip the write-up and check the final code here and a demo here built and deployed with the AWS Amplify Console.

Project Requirements
Defining Local State
Defining the Data Model with GraphQL
Deploying
Wrapping it up

Project Requirements

The group chat app should consist of 3 routes :

channel?id={id} A chat room identified by an id. A user can send and receive messages in real-time. Messages are sorted by descending message creation data (newest at the bottom).

channels A list of channels sorted by descending last update date (newest always at the top). A user can add channels and see channels added in real-time.

me The current user's profile route, used to edit the user's basic information. A form with 3 fields username, url, bio

The color palette is taken from @dabit3's excellent conference-app-in-a-box

Defining Local State

We won't go much into the implementation details of the app's offline functionality but you can do that by trying it here, or by checking the final code here. If you have any questions about the code, please leave a comment here or ping me on twitter and I'll be happy to answer them !

A single state object is being used for the whole app.

Our state without pagination data looks like this :



export type State = {
  me: {
    id: string;
    name?: string;
    bio?: string;
    url?: string;
  };
  channels: Array<{
    id: string;
    messages: Array<{
      id: string;
      text: string;
      createdAt: string;
      senderId: string;
    }>;
    name: string;
    createdAt: string;
    updatedAt: string;
  }>;
};

To be able to paginate through the data we will be needing to store the list of elements and an additional piece of data, the nextToken string that our API returns when fetching a list of items.

We can store that token at the same level as the list (e.g. { channelsNextToken:string, messagesNextToken: { [channelId]: string } }).

However it's easier to follow the format our API uses and instead of having channels and messages as arrays we can define them as a custom List.

A list has 2 fields : items and nextToken.

The type of elements in the items array depends on the list we're querying.

So the state becomes :



type List<T = unknown> = { items: T[]; nextToken: string };

export type State = {
  me: {
    id: string;
    name?: string;
    bio?: string;
    url?: string;
  };
  channels: List<{
    id: string;
    messages: List<{
      id: string;
      text: string;
      createdAt: string;
      senderId: string;
    }>;
    name: string;
    createdAt: string;
    updatedAt: string;
  }>;
};

Defining the Data Model with GraphQL

We want the messages in a channel to be sorted by createdAt and the channels in our ChannelList to be sorted by updatedAt.

To do that we assign a sortField to our connection directive.



type Message @model {
  id: ID!
  text: String!
  createdAt: String
  senderId: String
  channel: Channel @connection(name: "SortedMessages")
  messageChannelId: String
  # Because every message belongs to a channel
  # a messageChannelId field is added when we create an instance
  # of Message to know to which channel this message belongs.
  # Note that when doing 1-to-n relations using connections with GraphQL Transformer 
  # between 2 types typeA & typeB where typeA has many typeB
  # then a field typeBtypeAId is added to typeB to link it to the right typeA instance
}

type Channel @model {
  id: ID!
  name: String!
  createdAt: String!
  updatedAt: String!
  messages: [Message]
    @connection(name: "SortedMessages", sortField: "createdAt")
  channelList: ChannelList @connection(name: "SortedChannels")
  channelChannelListId: String
  # Same as in message, channel will have the id of its owner (ChannelList)
}

type ChannelList @model {
  id: ID!
  channels: [Channel]
    @connection(name: "SortedChannels", sortField: "updatedAt")
}

type User @model {
  id: ID!
  name: String
  bio: String
  url: String
}

# Note here our custom subscriptions.
# Amplify will generate subscriptions by default but the generated ones are too "broad".
# For example we don't want to listen to every new message created if we're in a channel,
# we just need messages that belong to the current channel.

type Subscription {
  # Used when the user is in the channels route to see channels added by others in real-time
  onCreateChannelInList(channelChannelListId: ID!): Channel
    @aws_subscribe(mutations: ["createChannel"])

  # Used when the user is in the channels route to re-sort channels when their updatedAt timestamp changes
  onUpdateChannelInList(channelChannelListId: ID!): Channel
    @aws_subscribe(mutations: ["updateChannel"])

  # Used in 2 places :
  # 1. In the channels route to listen to new messages added to the channel (We need to display the latest message in every channel)
  # 2. In the channel route to receive new messages in real-time

  onCreateMessageInChannel(messageChannelId: ID!): Message
    @aws_subscribe(mutations: ["createMessage"])
}

With this GraphQL schema, Amplify will :

Setup all the cloud resources we need for our app to work on any scale.
Generate code to CRUD the data

For customizing our data pagination and sorting, we will need to do a bit of extra work ourselves but for the rest we will just be using code generated by Amplify.

Mutations

We won't need to write any query for our mutations, the ones Amplify created for us are all we need.

In src/graphql/mutations.ts we'll find all the different possible mutations we can do.

We will be using :

createUser
createMessage
createChannel
updateChannel
createChannelList

For example when a user sends a message :



import { API, graphqlOperation } from "aws-amplify";
import { createMessage as createMessageQuery } from "../graphql/mutations";
import { MessageType, Dispatcher } from "../types";

const addMessage = async (
  content: string,
  dispatch: Dispatcher,
  me: State["me"],
  channelId: string
) => {
  const message = {
    text: content,
    createdAt: `${Date.now()}`,
    id: nanoid(),
    senderId: me.id,
    messageChannelId: channelId
  };
  dispatch({
    type: "append-message",
    payload: message
  });
  setScrollDown(Date.now());
  try {
    await (API.graphql(
      graphqlOperation(createMessageQuery, { input: message })
    ) as Promise<unknown>);
  } catch (err) {
    console.warn("Failed to create message ", err);
  }
};

Creating our Custom Queries

`getChannelList`

Let's create a new file in src/models/ and call it custom-queries.ts.

Inside it, we will add functions that return a GraphQL query when called.

In custom-queries.ts:



export type GetChannelListInput = {
  channelLimit?: number;
  channelNextToken?: string;
  messageLimit?: number;
};

export const getChannelList = ({
  channelLimit = 5,
  channelNextToken = "",
  messageLimit = 1
}: GetChannelListInput) => `
query GetChannelList($id: ID!) {
  getChannelList(id: $id) {
    id
    channels(

      # Number of channels to fetch on each request

      limit: ${channelLimit},

      # sorting direction by the sortField we specified in our schema: updatedAt

      sortDirection: DESC,

      # nextToken is a long string that our API sends back that we can use to
      # retrieve the next batch of entries (older channels in this case)
      # When nextToken is null, then we reached the end of the list

      ${channelNextToken !== "" ? `nextToken:"${channelNextToken}"` : ``}
    ) {
      items {
        id
        name
        createdAt
        updatedAt
        messages(

          # How many messages per channel to retrieve, in our case 1
          limit: ${messageLimit},

          # To get the latest first

          sortDirection: DESC,
          # No need for nextToken here
        ) {
          items {
            id
            createdAt
            senderId
            text
          } 

        }
      }
      nextToken
    }
  }
}
`;

Looking more closely at our query we'll notice that we are using 3 optional arguments to the channels and messages list fields, limit, sortDirection & nextToken explained above in the comments.

`getChannelMessages`

This one should be straight-forward to understand, it also uses limit, sortDirection & nextToken



export type GetMessageListInput = {
  messageLimit?: number;
  messageNextToken?: string;
};

export const getMessageList = ({
  messageLimit = 10,
  messageNextToken = ""
}: GetMessageListInput) => `
query GetChannel($id: ID!) {
  getChannel(id: $id) {
    id
    name
    createdAt
    updatedAt
    messages(
      limit: ${messageLimit},
      sortDirection: DESC,
      ${messageNextToken !== "" ? `nextToken:"${messageNextToken}"` : ``}
    ) {
      items {
        id
        text
        createdAt
        senderId
      }
      nextToken
    }
  }
}
`;

`updateChannel`

The result of a GraphQL subscription with AppSync is the mutation selection set.

In our case, the mutation is updateChannel and the subscription onUpdateChannel

The generated updateChannel looks like this :



mutation UpdateChannel($input: UpdateChannelInput!) {
  updateChannel(input: $input) {
    id
    name
    createdAt
    updatedAt
    creatorId
    messages {
      items {
        id
        text
        createdAt
        senderId
        messageChannelId
      }
      nextToken
    }
    channelList {
      id
      channels {
        nextToken
      }
    }
    channelChannelListId
  }
}

When a conversation is updated we want to receive the last message and some information about the channel.
However, by default, lists are sorted in ascending order, so we need to tell our AppSync API that we want them in descending order, and we'll limit the messages in the set to just one, because we're only interested in the last one.

So we write a custom update query (in src/models/custom-queries.ts) based on how we want the data to look like when a subscription fires an event.



mutation UpdateChannel($input: UpdateChannelInput!) {
    updateChannel(input: $input) {
      id
      name
      createdAt
      updatedAt
      creatorId
      messages(limit: 1, sortDirection: DESC) {
        items {
          text
        }
        nextToken
      }
      channelChannelListId
    }
  }

Using our Custom Queries

The queries above should give us everything we need to fetch both our messages and channels as lists in chunks of 10 or whatever we specify above in the limit.

For example in the channel route, when the component receives a valid channelId we run our query for the first time :



import * as React from "react";
import { Flatlist, ActivityIndicator, View } from "react-native-web";
import { API, graphqlOperation } from "aws-amplify";

import { DispatcherContext } from "../state";

const getChannelMessages = (channelId: string, nextToken: string) => {
  try {
    const query = getMessageList({
      messageLimit: 10,
      messageNextToken: nextToken
    });
    const messages = await API.graphql(
      graphqlOperation(query, { id: channelId })
    );
    return {
      messages: messages.data.getChannel.messages,
      channel: messages.data.getChannel
    };
  } catch (err) {
    console.warn("Failed to get messages ", err);
    return { messages: { items: [], nextToken: "" }, channel: {} };
  }
};

const Channel = ({ channelId, messages }) => {
  const dispatch = React.use(DispatcherContext);
  const [isLoading, setIsLoading] = React.useState(false);
  React.useEffect(() => {
    let isMounted = true;
    if (!channelId) return;
    setIsLoading(true);
    // We start by sending an empty string as nextToken
    getChannelMessages(channelId, "")
      .then(({ messages, channel }) => {
        if (!isMounted) return;
        setIsLoading(false);
        // We store the new messages that contain the next batch of messages and update the nextToken giant string
        dispatch({ type: "append-messages", payload: { channelId, messages } });
        // And update the channel's updatedAt field
        dispatch({ type: "update-channel", payload: channel });
      })
      .catch(err => {
        console.warn(
          "Failed to retrieve channel messages for channel ",
          channelId
        );
        setIsLoading(false);
      });
    () => {
      isMounted = false;
    };
  }, [channelId]);
  return (
    <FlatList
      inverted={true}
      ref={flatlistRef}
      ListFooterComponent={() =>
        isLoading ? (
          <ActivityIndicator
            animating={true}
            color={colors.highlight}
            style={{ marginTop: 15, marginBottom: 15, height: 30 }}
          />
        ) : (
          <View style={{ height: 30 }} />
        )
      }
      keyExtractor={item => item.id}
      data={messages.items}
      renderItem={({ item }) => <Message key={item.id} message={item} />}
      onEndReached={() => {
        if (messages.nextToken === null) return;
        setIsLoading(true);
        // When the end is reached we fetch the next batch of messages if they exist
        getChannelMessages(channelId, messages.nextToken).then(
          ({ messages }) => {
            setIsLoading(false);
            dispatch({
              type: "append-messages",
              payload: { channelId, messages }
            });
          }
        );
      }}
      onEndReachedThreshold={0.01}
    />
  );
};

Subscriptions

For our subscriptions we won't need to write any GraphQL queries. Amplify will generate all the ones we need.

In the GraphQL Schema input for the GraphQL Transformer we defined some subscriptions:



type Subscription {
  # Used when the user is in the channels route to see channels added by others in real-time
  onCreateChannelInList(channelChannelListId: ID!): Channel
    @aws_subscribe(mutations: ["createChannel"])

  # Used when the user is in the channels route to re-sort channels when their updatedAt timestamp changes
  onUpdateChannelInList(channelChannelListId: ID!): Channel
    @aws_subscribe(mutations: ["updateChannel"])

  # Used in 2 places :
  # 1. In the channels route to listen to new messages added to the channel (We need to display latest message in every channel)
  # 2. In the channel route to receive new messages in real-time

  onCreateMessageInChannel(messageChannelId: ID!): Message
    @aws_subscribe(mutations: ["createMessage"])

This will generate the queries in src/graphql/subscriptions.ts and the types we need in src/API.ts

For example let's look at the code needed to listen to new messages on a channel :



import { API, graphqlOperation} from 'aws-amplify'
import {
  onCreateMessageInChannel,
} from "../graphql/subscriptions";

const Channel = () => {
  React.useEffect(() => {
    let isMounted = true;
    if (!channelId) return;
    API.graphql(
      graphqlOperation(
        onCreateMessageInChannel,
        { messageChannelId: channelId }
      )
    ).subscribe(message => {
      const newMessage = message.value.data.onCreateMessageInChannel;
      if (newMessage === null || newMessage.senderId === me.id || !isMounted) return;
      // prepend instead of append because they are sorted in descending order by createdAt 
      dispatch({ type: "prepend-message", payload: newMessage });
    });
    () => {
      isMounted = false;
      onCreateListener.unsubscribe();
    };
  }, [channelId]);

  // Rest of the code
}

Simple enough, listening to a graphql subscription and turning it off on unmount.

Deploying

The code is built and deployed by the AWS Amplify Console. To deploy your own, you can click on this button or just connect your repository to the Amplify Console and that's it.

As you can see in the picture below, Amplify builds and deploys every commit on the master branch.

Wrapping it up

Most applications will need to handle lists of data and fetch progressively from it (chat, comments, history, feed).

This post goes through the challenging parts of doing that with React & Amplify and provides a good starting point to build one yourself !

Top comments (1)

Ricardo • Aug 30 '19

Loved this post!

DEV Community