This week I've been playing around with neural networks, or more specifically neuroevolution of augmenting topologies (NEAT) which is nowhere near as complicated as it sounds. I'll go over the ins and outs of NEAT in a future blog. For now, just know that NEAT is a matter of combining the strengths of a genetic algorithm and a neural network to form something quite powerful, as this blog will show!

Starting simple

As with anything I know nothing about, I like to try and implement it in the simplest way possible and then iterate and expand on the implementation. I did this in my previous blog where I developed a simple genetic algorithm for a game where the goal was for the protagonist to move up, and that's it. This week was no different.

Flappy bird

One of the simplest games I could think of to get the ball rolling is Flappy bird as the rules and actions are quite simple, the bird must go through the pipes, and the bird does that by choosing one of two actions: flap wings or don't flap wings.

Luckily, I coded my own flappy bird for a YouTube video I did a while back. I'll grab that and attempt an implementation of NEAT.

Generation 1

I spawn a population of 100 birds with a random neural network. The neural network takes as input the x/y of the bird and the x and y of the top pipe.

As you can see, the first generation only 1 bird makes it through the first set of pipes, this means that bird is more likely to pass on its genetic material.

Generation 20

By generation 20 the bird is getting through a few pipes now and seems to have developed an understanding of the game and its objective.

Generation 30

By generation 30 the bird has mastered the game and continues on indefinitely.

The bird likes to live on the edge though, look how close it gets to the top pipes!

Self-driving car

Okay, so now I've got a pretty good idea of the implementation let's bring this code into my car game. If you're interested in how the reward mechanism works I cover it in detail in this blog, but basically there are checkpoints, and if the car drives over a checkpoint their score increments.

Inputs

So I think the only inputs the neural network really needs are some sensors, similar - but more primitive - than the sensors your car might have. So I'm going to choose 3 points that'll represent the sensors. The value of each sensor will be either 0 or 1, where 0 represents on track and 1 represents you're about to die.

I do this by taking the pixels of the track and then determine the RGB of the pixel at the location of the sensor, and if that RGB is not the same colour as the track, then it's safe to assume that going that way is probably not a good idea, but can my neural network figure that out.

The green dots represent the sensors.

The code for determining whether a sensor is still on the road looks like the following:

    let infrontCollision = 0
    const index = 4 * (floor(this.s1.y) * width + floor(this.s1.x));
    if (pixels[index] != 110 && pixels[index+1] != 111 && pixels[index+2] != 114) {
      infrontCollision = 1;
    }

Generation 0

Let's create a population of 100 with a mutation rate of 5% and see how it fairs:

I must admit, when I first ran this I was blown away. To get this far using just a traditional genetic algorithm took way over 100 generations. It's as though the car knew its purpose from the get-go!

Generation 2

By generation 2 we already have cars that can complete a full circuit of the track! Which took over 250 generations in my previous blog, quite remarkable.

Can it race another track?

One of the drawbacks of using just a genetic algorithm was that it adapted to the environment it was in, each generation had to learn how to get around a corner and then pass on that genetic information so that its children could get around the corner and attempt the next. Here though, we're using live information and teaching the car to stay on the track, and hopefully any track - provided the grey is the same 😁

Exporting the model

The neural network is essentially the brains of the car, when a car is successful I can essentially export their brain and inject it into another car. In the example below, I inject the brain of a successful car from the other track onto this brand new track and it already knows what it's got to do!

Conclusion

Well, that concludes this week's hacking! If you enjoyed my writing please sign up for my newsletter! All the code written in this blog can be found here although I very likely will have edited it by the time this blog is live.

As mentioned in my previous blog thanks to Daniel Shiffman for the inspiration and great tutorials opening my eyes to this mysterious world!

I hope you've enjoyed this blog, if you do by some miracle enjoy my blabbering then head over to my blogging site at codeheir.com where I write weekly blogs about whatever in the world of programming has my attention!