It is with a great deal of satisfaction that I can introduce my latest coding/cloud/hobby project, the Skyboy App!
Skyboy is an FPV quadcopter post-flight telemetry visualization tool, providing key flight metrics, a map for GPS-equipped quads, and graphs for analyzing related subsets of data. It brings together my love for flying FPV quadcopters with my passion for coding and cloud infrastructure.
FPV stands for First-Person View, and in the context of "drones," is shorthand for referring to a certain type of quadcopter (a.k.a. "quad") that has a live video link between a camera on the quad and goggles worn by the pilot. Quad operators can be hobbyists, content creators, or paid professionals filming for commercials, television, and movies. Among the various styles of FPV flying are racing, freestyle, cinematic, and long range; regardless of how they're flying, the pilot gets a quad's eye view of the action and is typically in full control of the aircraft's movement. This is in contrast to how, for example, a commercially-produced DJI drone flies (slow and steady, or autonomously along a pre-defined flight path).
The flight controller board on the quad can be configured to send telemetry data about the quad and its systems to the pilot's handheld radio transmitter, where the data can be logged for future reference. I initially set up telemetry logging on my radio to aid in recovery if my quad crashes. After downloading my first few flight logs, I started thinking about ways to use the captured data. I looked at several existing visualization and dashboard applications, but they were all geared towards DJI drones. So... what if I made something myself?
A search for data visualization tools led me to Streamlit, a Python browser-based application with the flexibility to display data in multiple ways. It supports multiple Python graphing modules, as well as mapping with Mapbox.
I began by taking the imported telemetry log - a comma-separated values file - and transforming it using Pandas. With the data loaded into a dataframe, I built multiple plotly charts with selected columns. The GPS data was manipulated and loaded into a layer on top of satellite imagery to produce a traced flight path in an interactive map widget. I also extracted and calculated key flight metrics, such as time, distance, and battery consumption.
Once the app was running from an IDE, it was time to decide how best to package it for cloud-based deployment. I settled on creating a Docker container for the application so that it could be run in a browser on any platform. Building the image went fairly smoothly, considering it was my first Docker project (after guided lessons and tutorials). I automated the build and repository upload process using GitHub actions, which I wrote about in this blog post.
The final step to this initial product rollout was getting the application deployed to the cloud. I started by manually deploying the necessary infrastructure: a Virtual Private Cloud (VPC) environment; an Elastic Container Service (ECS) cluster, service, and task definition; a load balancer; and associated security groups and service roles. I also registered a domain and created a Route 53 hosted zone for DNS routing.
This was my first time working with AWS's container orchestration services, so I kept it simple by using a serverless Fargate deployment (meaning no instances that I have to manage). Getting everything set up was fairly straightforward, with a big shout-out to this tutorial series that I used as a guide. The manual deployment configuration became my reference as I started to build out the Terraform code to automate my infrastructure.
I plan to write a post about the Terraform development process shortly, so I won't go into much detail here, except to say that this was a more complex project than I've previously tackled on my own. There was a lot to figure out on the Terraform side, as well as on the GitHub Actions side to create another set of branch-dependent workflows. After quite a bit of work - and a few this-is-driving-me-crazy moments - I tore down the manual deployment and my Terraform configuration spun up all of the necessary resources in the Skyboy production account to successfully redeploy the application.
Skyboy is, at this point, an MVP (minimally viable product). It currently only handles telemetry logs with one specific set of headers. After changing some of the hardware on my latest quad build, I've found that some of the telemetry has either changed or is omitted from the log file. This throws errors in the application, and handling those errors to accommodate a variety of data sets is next on the roadmap.
Other to-do items include:
- Storing telemetry logs on S3 instead of in memory
- playing with mapping options
- layout and aesthetics
I have a lot more learning to do about cloud container orchestration with ECS and monitoring with one of the many available options (Datadog, perhaps). The process of developing and deploying my first home-grown application has been humbling and thrilling, and I am excited to see where I can take this project!
All code (Python, Terraform, and GitHub Actions workflows) can be found in the Skyboy GitHub repository.