How I designed an abuse-resistant, fault-tolerant, zero cost, multiplayer online game

Nearly a year ago I deployed a multiplayer feature for Farmhand, an open source and web-based farming game that I created. Since that initial deployment, the multiplayer system has experienced no downtime or service degradation. And best of all, I've paid nothing to host the service and therefore I am able to allow others to play for free. This article is an overview of how I designed this system from the ground up.

The game

In short, Farmhand is a game that mixes farming and market mechanics. The goal is to buy seeds for a low price, plant and harvest them, and then sell the crops at a high price. Prices fluctuate from day-to-day, so you'll have to be smart about your buy/sell decisions.

Farmhand was initially designed as a single-player game and seed/crop values were randomly generated at the start of each game day. One day I thought it would be cool to create a shared, online market that players around the world could participate in together. My vision was for one player's buy/sell decisions to affect the global market that determines the seed/crop values for all connected players.

In order for this market system to be fun, it needed to be simple and reliable. I gave myself the following constraints:

• Zero hosting costs. I'm not making money from Farmhand, so I don't want to spend money to host it.
• Minimal devops involvement. Farmhand is just a hobby for me, and I have a day job. I don't want to be dealing with managing service outages during the work day (or the middle of the night).
• Fault-tolerance and abuse-resistant. If you're putting a service online, expect people to abuse it. I wanted this system to not only be highly-available, but resistant to griefing as well.

In the end I was able to ship a fun and functional multiplayer system that adhered to all of these constraints.

The tech

There are a few pieces to this system:

The client

Farmhand is implemented as a PWA that runs in a web browser. The client's overall architecture is outside the scope of this article, but for the purposes of online multiplayer it uses Trystero with the WebTorrent matchmaking strategy to connect peers to each other. It interacts with the central market server via a REST API.

The server

Farmhand's API is hosted on Vercel's Hobby tier. Vercel provides an excellent Serverless platform that offers scalable runtime performance, as well as static file hosting, automatic preview builds (great for testing out PRs), and more.

The Vercel-based API is backed by a Redis instance for data "persistence." "Persistence" is in quotes because the data only ever lives in memory, so a system failure would result in complete data loss. However, the application logic is designed such that this kind of failure would be a feature and not a bug. The Redis instance is hosted on Redis Labs' free tier.

For Farmhand, both Vercel and Redis Labs are configured to run in AWS.

System architecture

At any given time, a player can go to https://www.farmhand.life/, switch the "Play online" toggle and join a room of their choosing (global by default). When this happens, two things occur:

1. A request is made to GET https://farmhand.vercel.app/api/get-market-data?room=global, which is a Serverless function. This retrieves the latest market data and also informs the server of the player's presence. This request is repeated to serve as a heartbeat until the player leaves the room to maintain an "active" session with the API.
2. A WebSocket connection to a WebTorrent tracker is made. The tracker then connects the client to any other clients in the requested room. The peer-to-peer connection is persistent until the player leaves the room. This complexity is abstracted away via Trystero.

The API manages room data that is stored in Redis. When a GET https://farmhand.vercel.app/api/get-market-data?room=global request (source) is made, the API checks to see if a value associated with the key room-global exists. If not, it is initialized. Here's an example of a room object:

{
  "activePlayers": {
    "4a793fe2-9eb1-4041-935b-5caf55177dde": 1640727668293,
    "58f90cc1-1089-4394-a7e7-2f079f87ed4d": 1640727669934,
    "b26b2d59-79f5-40f3-bc91-cfc0554bb994": 1640727674791,
    "d1e34686-925e-4344-b7cb-e15ce6d7dad3": 1640727667860
  },
  "valueAdjustments": {
    "asparagus": 0.6798235686529905,
    "asparagus-seed": 0.9797840434970977,
    "carrot": 0.5382522777963925,
    "carrot-seed": 1.1233740954422615,
    "corn": 1.1524067154896047,
    "corn-seed": 1.2309158460921086,
    ...
  }
}

activePlayers is a map of unique player IDs (determined by clients via uuid to timestamps of when they last made a GET https://farmhand.vercel.app/api/get-market-data?room=global request. Each time the function is invoked, it examines the map to see which timestamps are older than the HEARTBEAT_INTERVAL_PERIOD (currently 10 seconds) and deletes any that are expired. This data is returned to the client and also written back to Redis to be persisted across function invocations. This is how the active room participants are tracked.

valueAdjustments is the current state of the room's market. The map's keys refer to an item ID in the game and the values represent their respective in-game market value. Market values are bound between 0.5 and 1.5 and go up or down based on individual player activity. When a player ends their in-game day, an API request to POST https://farmhand.vercel.app/api/post-day-results is made with a payload that looks something like:

{
  "positions": {
    "carrot-seed": 1
  },
  "room": "global"
}

positions represents all the items that the player either increased or decreased inventory of in their most recent in-game day. 1 means that they increased their inventory of the associated item ID (either by buying seeds or harvesting crops), which increases the item's market value. -1 means they decreased their inventory (typically by selling the item), which decreases the item's market value. Here's the source for that logic:

const applyPositionsToMarket = (valueAdjustments, positions) => {
  return Object.keys(valueAdjustments).reduce(
    (acc, itemName) => {
      const itemPositionChange = positions[itemName]

      const variance = Math.random() * 0.2

      const MAX = 1.5
      const MIN = 0.5

      if (itemPositionChange > 0) {
        acc[itemName] = Math.min(MAX, acc[itemName] + variance)
      } else if (itemPositionChange < 0) {
        acc[itemName] = Math.max(MIN, acc[itemName] - variance)
      } /* itemPositionChange == 0 */ else {
        // If item value is at a range boundary but was not changed in this
        // operation, randomize it to introduce some variability to the market.
        if (acc[itemName] === MAX || acc[itemName] === MIN) {
          acc[itemName] = Math.random() + MIN
        }
      }

      return acc
    },
    {
      ...valueAdjustments,
    }
  )
}

The updated market data is again persisted back to Redis.

Abuse mitigation

One of the nice things about this server-side logic is that it naturally mitigates abuse. There is nothing stopping people from crafting custom POST https://farmhand.vercel.app/api/post-day-results requests to manipulate the market however they want. However, once the adjusted value for any given item reaches the upper or lower bound (1.5 or 0.5, respectively), the item's value is randomized within those bounds. So while nefarious people can manipulate the market, it will reset itself and balance out before long. Even in those cases, it only presents as normal (if somewhat volatile) market dynamics to players.

Fault tolerance

Farmhand room data is only stored in memory via Redis and never written to a disk. Because of this, it is inherently ephemeral. The worst case scenario with this design is that room data gets lost either due to the Redis server shutting down or something like a FLUSHALL command being issued. However, since the API initializes requested room data that doesn't already exist, this would only present to the user as a bit of market volatility that would likely go unnoticed.

Peer-to-peer interaction

The Vercel-based API effectively manages the shared market data, but I wanted players to have a sense of who else is playing with them and how they are affecting the experience for everyone else. This is where the peer-to-peer communication comes into play.

Instead of a server and logic to act as a broker between clients, they connect to each other directly via Trystero and WebTorrent as explained previously. As players perform various actions such as buying or selling items, messages are broadcast to all connected peers to display in the "Active Players" modal:

In an effort to mitigate abuse, player names are the result of a simple hashing algorithm based on player IDs (which are stable).

export const getPlayerName = memoize(playerId => {
  const playerIdNumber = playerId
    .split('')
    .reduce((acc, char, i) => acc + char.charCodeAt() * i, 0)

  const adjective = adjectives[playerIdNumber % adjectives.length]
  const adjectiveNumberValue = adjective
    .split('')
    .reduce((acc, char, i) => acc + char.charCodeAt() * i, 0)

  const animal =
    animalNames[(playerIdNumber + adjectiveNumberValue) % animalNames.length]

  return `${adjective} ${animal}`
})

Retrospective analysis

This system has been effective so far. Vercel and Redis Labs have provided excellent performance and availability since this system launched nearly a year ago, which is impressive given that I'm using the free tier of both services. The fault-tolerance and abuse mitigation measures have resulted in the minimal maintenance burden I was hoping to achieve. The only manual intervention that's been required from me so far to keep things running is having to log into Redis Labs every couple of months to indicate that my account is still active.

I'm quite pleased with how this multiplayer system has turned out so far. I'd like to expand on Farmhand's multiplayer features and further develop its online market mechanics. I'd like to know what others think as well, as I've never designed a full stack system before this and I would like to learn how it can be improved. Let me know what you think via the comments below! And if you're up for some easygoing farming fun, give Farmhand a try sometime. 🙂