Converting a custom EC2 websockets chat to use serverless

I have been maintaining a custom, on-server websockets chat for about 5 years as part of the broader ninjawars.net web app. Today, I got tired of maintaining the uptime of the chat server, and decided to move it over to some form of serverless backend. So I decided to go through converting my custom EC2 server-based websockets chat over to the new(ish) apigateway serverless websockets, so I wouldn't have to keep the chat server "up" and the chat could still work. Here is what I found.

• Read time: 3 minutes
• Development time: 3 hours (your mileage may vary, much of mine was spent debugging AWS arn permissions :) )

Findings first:

• Don't handcode your websocket realtime chat, (except as a learning exercise for how to deal with other realtime websocket clients). There are quite a few subtle obstacles that make a custom coded websocket finicky, and require custom coding. The one I ran into was the “ack” system needed to deal with keep alive. Otherwise the chat may experienced a loss of connection that the serverless websocket api gateway does after a time of inactivity, disconnecting the client.

The Repo with the chat tools

I have not yet extracted the aws lambda code down into that repository, nor converted the quick prototype over to something like aws cdk. That may be a future exercise.

The path

Ok, so what overall pieces are there to create a serverless realtime chat with websockets?

AWS API Gateway's websockets api is relatively new, and due to the potential need to do "ack" (keepalive of the connection), it may be necessary to convert over to aws appsync in the future (see this article: full realtime websockets client via appsync ). However, in the meantime I just spun up some custom resources: A dynamodb table, three lambdas, and an apigatway. Oh, and I connected it up to my pre-existing domainname *.ninjawars.net. Websockets is weird. It's like an echo chamber, you send a message and it broadcasts it out to the various connected client connections.

Let's walk through a few of the necessary parts to get a quote unquote "simple" web chat up in a realtime way (without having to maintain a full EC2 server for the websocket)

Step 1: Create an API Gateway WebSocket API

The first step is to create an API Gateway WebSocket API. To do this, navigate to the API Gateway console and select "Create API." Then, select "WebSocket API" and follow the prompts to create your API.

Step 2: Create a Lambda Function or Three

The next step is to create a Lambda function that will handle the WebSocket connections and messages. In my case, I used Node.js and the AWS SDK to handle the WebSocket connections and messages. I am using the guide here: websockets-api for the general websockets mechanism.

Step 3: Configure the WebSocket Routes

Once you have created your Lambda function, you need to configure the WebSocket route to use the Lambda function. To do this, navigate to the API Gateway console and select your WebSocket API. Then, select "Routes" and assign each route ($connect, $disconnect, and the custom "onMessage", or perhaps just the fallback route of $default) to point at their respective lambdas. For the "Integration target," select your Lambda function.

Step 4: Deploy Your API

After configuring your WebSocket route, you need to deploy your API to make it available for use. To do this, navigate to the API Gateway console and select your WebSocket API. Then, select "Actions" and "Deploy API." Follow the prompts to deploy your API.

Step 5: Test Your API from the lambdas and externally

Once your API is deployed, you can test it using a WebSocket client. I used the wscat command-line tool to test my API. You connect to the wss: url, then just paste in any arbitrary text, though probably it should in the end be a json object.

I hear there are also browser-based clients such as the "Simple WebSocket Client" Chrome extension.

Test Event For AWS Lambda

{
  "requestContext": {
    "accountId": "123456789012",
    "resourceId": "123456",
    "domainName": "000000xxxx.execute-api.us-east-2.amazonaws.com",
    "stage": "production"
  },
  "body": "{\"message\":\"test from test event in aws console\",\"uname\":\"xxxxxxusername\",\"date\":\"1681948677627\",\"sender_id\":\"128274\"}"
}

Test message from client

{"message": "test from test event in aws console", "uname": "xxxxxxusername", "date": "1681948677627", "sender_id": "128274"}

Takeaways

Beware!
Because there is the assumption that websockets is the mechanism for the realtime, there's a pitfall that you don't get any long-term storage out of it! So if you want to show a "history" for the chat (and you do, trust me you do), then you have to have a separate backend source that will handle the posting, saving, and retrieving of the chats anyway, and that will need to be pulled from for the initial chat history. So the websockets part of the chat is actually only half of the equation, a different api and database is needed for long-term storage of the chats that were sent in a non-realtime way.

Overall, the reason I recommend going with a framework like amplify is because the complexity of getting it right in a bespoke way is pretty high. The initial "prototype" is simple with a websockets serverless backend, plus some UI, but there are hidden gotchas that come up invisibly later, like the need to ack the connection, and handle reconnections to the websocket server.

Even this minor task ("just create a realtime chat!") is actually much harder than it looks on the face of it.