DEV Community

Cover image for Using Video Games to Improve Machine Learning

Using Video Games to Improve Machine Learning

couch profile image S🅰️Ⓜ️ 🛋 ・1 min read

One application for machine learning that I've been excited about is in gaming. Not only useful for game developers, but also many other applications as well! One area of interest to me is the possibility to use video games to simulate real-world challenges and create solutions that can then be implemented outside of the video game. Here I will walk you through how I built a custom object detection model trained for Fortnite.

What are we building a model for?

In machine learning its best to start with deciding on what question you want to answer. In my case, I decided to build a model that could track the unique vehicle in Fornite called The Baller. The goal will be to identify and track when a baller is in a player's field of view, including when a player is using the vehicle.

What's the point?

The primary thesis of this exercise is to demonstrate how we can use simulated environments to solve real-world, general-purpose, AI problems. Rather than have to gather real-world data, we can capture video game data as a testbed to build, test, and refine AI systems. In the case of Fortnite, and specifically, the model demonstrated here, imagine being able to create the framework for object-avoidance, decision-optimizations, or planning the best route to travel. The possibilities are endless!

Hopefully, this post spurs your imagination of what's possible using video games. In the next post, I'll share a guide on how to replicate the model above to detect and track ballers.


Editor guide