(This is a part of my cloudretro.io series. Introduced in https://dev.to/giongto35/i-write-a-cloud-gaming-service-for-retro-games-in-go-lkn)

This part of the series will introduce the challenges of cloud gaming technology and describe how I design the cloud gaming system.

Challenges and technology

Cloud gaming is innovative not only in its application but also in the technical aspect of game streaming. If Google or Microsoft succeeds with Stadia and XCloud, it will mark the new milestone of technology.

Behind Cloud gaming is the art of extremely low latency streaming to ensure user interaction is as smooth as possible. There is no other application in history requiring this tight delay. Taking comparison with live stream platforms like Twitch or Youtube, it is acceptable for them to have a few seconds delay as long as they can broadcast an undisruptive stream. On the other hand, the goal of Cloud Gaming is to keep the gap between input from users and media unnoticeable while maintaining media quality. In this tight gap, Game logic + Heavy Video/Audio Encoding Decoding + Network Round Trip... must run with 60 FPS, less than 20ms per frame.

To a Github trending POC

I found cloud gaming is very challenging, so I decided to make a Cloud Gaming POC to test how good the system I can achieve. I loosen the requirement to make it more feasible to start.

• I target Retro Games (NES/GBA/PS) instead of modern AAA games. These games don't require much computation resources and have less image entropy.
• The service runs on the browser directly to demonstrate the spirit of cloud gaming: bring the most comfortable gaming experience without any initialization.

Design cloud gaming service: Tech stack

1. CloudGaming like Stadia hosts offline single-player games, not online games.

This requirement sounds not relevant and simple but it’s one of my key findings. It makes cloud gaming different from trivial streaming service. If we focus on a single-player, we don’t need to distribute the game stream and get rid of a centralized server or CDN.

Therefore, instead of uploading a stream to ingest server or passing media packets to a centralized WebSocket server, the server will set up a peer to peer communication with the user. Multiple streaming servers will be managed by a central coordinator. You can check the infrastructure diagram below for more understanding.

2. Full control of Game Emulator internal for easier state manipulation.

I need full control from Game Emulator, so I can hook the image/audio out directly from the emulator and manage game state easier. I picked libretro using its GoLang Wrapper

3. Low Latency media stream

A compression algorithm for media is a must for these latency-sensitive projects. The compression algorithm needs to perform fast encoding and decoding speed per frame. H264/VP8/VP9 Video Codec and Opus Audio Codec are famous in this field and I applied them to my project. These compression algorithms are designed to run on GPU and optimized for network transmission.

As I know, Google Stadia has some other optimization on this compression flow such as

• Researching a better Video Encoding algorithm dedicated to cloud gaming use case.
• Applying distributed GPU to empower and scale compression flow. This distributed GPU infrastructure likely also comes from Google's existing machine learning foundation.

4. Reduce network latency

No matter how good your compression and how optimal your code is, the network still contributes the most to latency. To deal with that, we distribute the streaming servers around the world and the infrastructure has a mechanism to pick the closest server to the user.

In Google Stadia's introduction, Google built an infrastructure to distribute their stream server around the globe for Stadia.

5. Browser Compatible

One of cloud gaming service goals is to make the gaming experience as comfortable as possible. Being able to run on the browser as a web service can get rid of any gaming software or hardware, and achieve cross-platform flexibility on Mobile, Desktop.

6. Horizontal scaling

And Last but not least, as every SAAS nowadays, it must be designed to be Horizontally scalable. The infrastructure is self-implemented to distribute and match game sessions properly.

Infrastructure

More on CloudRetro.io

This is the end of the first part in my series. You can check my presentation in GopherCon Vietnam 2019 about this topic: Video and Slide here. I attached my speaker note in the slide as well. The blog series will have more detailed and clearer explanation, so please still keep following.

Please give me more motivation to write by upvote this post or staring Github repo