My new company flew me to Sydney 🛩️ to attend a 2-day team building event which included half of the day putting our AWS Deep Racer model into action by competing in an actual physical race.
Despite the workload, I spent time outside working hours to develop our team's model. Since I am a very competitive person (yes, I trash talk my kids when I win a Mario Kart game 🏎️ over them - teaches them resilience 👀), the motivation to win was multi-fold: bragging rights, winning cool prizes (we each got Echo dot and AWS notebooks!) and most importantly, my inclination to learn "real" machine learning (I used to be part of a team "selling" AI/ML but it's all just big data 🤷).
The awesome AWS team had a couple of workshops with everyone on how to prepare your models (creating accounts, training, simulation, racing with the company-specific community race). Non-technical or newbies don't need to be scared as the AWS console for Deep Racer is quite a user-friendly UI. We only had one track to train on which is the re:Invent 2018 so it was easier to tweak your models.
I trained multiple models (and iterations) after deep-diving into github repositories and browsing through multiple online articles
Each models were trained between an hour and three hours based on different reward functions. One of the things I learned is that longer training would not mean a better time (as it might pick up some bad habits too as you can see in the graph below where it shows a bit of downward trend at the end)
We were allowed to submit any models we have (as many times as we want) and enter into the simulation virtual race. Overall, my entry came 3rd with a time of 34.724 seconds having no lap going "off-track"
You can see above that I tried to iterate my superpumped model (sorry, I just watched the Amazon Prime series so that's the first thing that came to my mind, although I'm not a fan of Uber's toxic culture) by tweaking the values in the hyper parameters and reward functions but the version 2 did the best time (not bad considering there were 20 entries). Each team then had to download the physical car model then submit to the AWS team the day before the actual physical race.
During "race" day 🏁 (thanks AWS team for preparing the track and all the set-ups), we had to drive our models on the track one team at a time for 3 minutes each (so the car can go as many laps as possible and the best lap will be recorded). We were the last team to drive the vehicle (a.k.a. RC model with a camera) and as they say, "saved the best for last". Our car clocked 11.04 seconds, while the runner-up was 8 seconds slower (yep, big gap which validated the hard work!). AWS team also said the fastest ever was 7 seconds so our result was not bad for a first timer 😎. The other thing to note is that there is a "speed function" introduced while driving the car. The AWS team initially set it to 41% and you would need to adjust it accordingly while the car is moving (e.g. turn up the speed when it's a straight path while try to decrease it when it's curving). There were a couple of times where the car went off-track so I realised, I had to slow it down.
I know you might have skipped the babble above so for those who are just here to check how I did it, click the repository link below containing my reward function:
my awesome aws deeprace model ⬅️ ⬅️ ⬅️
... with training configuration below:
I ran it for 100 minutes (1 hour and 20 minutes) because when I iterated and changed the duration to 3 hours, it just went ballistic and did no good.
The model focused on accuracy instead of speed where it tries to stay at the centre as much as possible. It rewards the model the closer it is while it penalises if it veers away.
By no means I am taking full credit for this as I checked out a number of articles and tried out a dozen of models (hence, used up all my training credits). I used the Example 4 from the reference below with only changing the Loss type from Mean Square to Huber (by accident... but it worked out well in the end):
https://www.linkedin.com/pulse/samples-reward-functions-aws-deepracer-bahman-javadi?trk=public_profile_article_view
Top comments (0)