Finding the Right "Brain" and Software for Civilian Drone. Part 1

This is like a beginning. See part 2 here.

Introduction

Unmanned Aerial Vehicles (UAVs), colloquially known as drones, have revolutionized industries ranging from agriculture and construction to surveillance and logistics.

This transformative potential hinges on the intricate interplay between hardware components and software systems. At the heart of this symbiotic relationship lies the Microcontroller Unit (MCU), an indispensable component orchestrating the UAV's operations. However, the efficacy of UAVs is contingent upon the selection of an appropriate MCU and the deployment of a robust operating system tailored to their unique operational exigencies.

Specialized Operating Requirements of UAVs

UAVs operate in dynamic and resource-constrained environments, necessitating operating systems that diverge from traditional paradigms. Unlike desktop computers or laptops, UAVs prioritize power efficiency to maximize flight time, embodying stringent constraints on memory and processing capabilities. Real-time responsiveness assumes paramount importance to ensure flight stability, thereby demanding operating systems that offer deterministic timing and swift reaction to dynamic environmental stimuli.

Real-Time Operating Systems: A Tailored Solution

Real-Time Operating Systems (RTOS) have emerged as a tailored solution to address the specialized operational requirements of UAVs. By offering deterministic timing and efficient resource utilization, RTOSes facilitate the seamless execution of critical tasks such as flight control and sensor management. Popular RTOS choices, including FreeRTOS, NuttX, ChibiOS, Zephyr and RT-Thread offer varying strengths in terms of size, security, and hardware support, catering to diverse UAV projects.

STM32: A Preferred MCU for UAV Development

The STM32 family of MCUs has garnered widespread acclaim in the realm of UAV development due to its versatility and performance capabilities. With a plethora of options catering to diverse application scenarios, STM32 MCUs offer seamless integration with various RTOS options, thereby enhancing flexibility and scalability in UAV development endeavors.

Hardware Abstraction Layers: Facilitating Portability and Focus

Hardware Abstraction Layers (HALs) play a pivotal role in bridging the firmware and hardware components of UAV systems. By encapsulating low-level hardware details and providing a standardized interface, HALs facilitate portability, allowing firmware such as Ardupilot to run seamlessly across different MCUs and RTOSes. Furthermore, HALs enable firmware developers to focus on high-level tasks such as navigation and sensor fusion, thereby enhancing development efficiency and code maintainability.

Vendors' Role in HAL Provisioning

Semiconductor vendors, exemplified by STMicroelectronics for the STM32 MCUs family, play a crucial role in providing and maintaining Hardware Abstraction Layers (HALs). One of the pivotal reasons for the widespread adoption of STM32 microcontrollers in civilian drone applications is 🗎 their robust support for various communication buses and protocols, which are essential for facilitating seamless integration with a wide range of peripheral devices and external sensors.

STM32 MCUs boast extensive support for industry-standard communication interfaces, including Universal Asynchronous Receiver-Transmitter (UART), Serial Peripheral Interface (SPI), Inter-Integrated Circuit (I2C), Controller Area Network (CAN), and USB. These communication interfaces enable UAV developers to establish reliable and high-speed data exchange with external components, such as GPS modules, inertial measurement units (IMUs), cameras, and telemetry systems.

Pic source: st.com 🔎

Furthermore, STM32 microcontrollers feature built-in support for popular communication protocols commonly used in UAV applications. This includes protocols such as MAVLink, which facilitates communication between onboard flight controllers and ground control stations, as well as protocols like I2C and SPI for interfacing with peripheral sensors and devices.

By offering comprehensive support for communication buses and protocols, STMicroelectronics empowers drone-makers to build highly interconnected and interoperable UAV systems. This enables seamless integration of diverse sensor arrays, payload systems, and communication modules, thereby enhancing the functionality, versatility, and performance of civilian drones.

With STM32 MCUs as the backbone of UAV development, drone-makers can involve a rich ecosystem of communication interfaces and protocols to realize their vision of advanced and mission-critical drone applications.

The Zynq UltraScale+ platform, developed by Xilinx (now AMD)

This solution offers several distinct advantages for drone making, particularly in scenarios that demand high computational power, flexibility, and integration of custom hardware accelerators. Below are some key advantages of the Zynq platform for drone development:

Zynq UltraScale+ combines the processing capabilities of Cortex-A53 application processors with the programmable logic of FPGA (Field-Programmable Gate Array) fabric on a single chip. This hybrid architecture enables developers to implement custom hardware accelerators in the FPGA fabric to offload computationally intensive tasks from the CPU, thereby enhancing overall system performance and efficiency.

Being built on the industry success of the Zynq 7000 SoC family, the UltraScale+ MPSoC architecture extends AMD SoCs to enable true heterogeneous multi-processing with ‘the right engines for the right tasks’ for smarter systems

The ARM Cortex-A53 processors integrated into the Zynq UltraScale+ provide significant computational power, enabling drones to execute complex algorithms for tasks such as image processing, computer vision, sensor fusion, and autonomous navigation. This high computational capability is crucial for enabling advanced drone functionalities, including obstacle detection and avoidance, object tracking, and environmental mapping.

The programmable logic fabric of the Zynq platform allows developers to implement custom hardware accelerators tailored to specific drone applications. The flexibility of FPGAs allows developers to customize algorithms and processing pipelines, unlocking unique capabilities for specialized drone missions.

Additionally, the reconfigurability of FPGAs enables rapid prototyping and iteration, facilitating agile development processes in the fast-paced UAV industry.

The Zynq platform offers low-latency processing capabilities, making it suitable for real-time applications in drone control and navigation. By deploying critical control and decision-making algorithms on the ARM processors and offloading latency-sensitive tasks to the FPGA fabric, developers can achieve high responsiveness and stability in drone flight operations.

Pic source: linuxgizmos.com

The platform features a rich set of peripheral interfaces, including GPIOs, UARTs, SPI, I2C, USB and MIPI facilitating seamless integration with a wide range of sensors, actuators, communication modules, and external devices. This extensive peripheral support simplifies the development of fully integrated drone systems and enables interoperability with existing UAV components and standards.

The Zynq platform is available in various configurations with different processing capabilities and FPGA resources, allowing developers to choose the optimal combination of performance and cost for their drone applications. Whether designing lightweight drones for aerial photography or high-end UAVs for surveillance and reconnaissance missions, developers can select the Zynq device that best aligns with their performance requirements and budget constraints.

Xilinx (the maker of Zynq UltraScale+ devices) provides a comprehensive HAL as part of their software development tools and frameworks. The primary HAL access point is usually through the Xilinx SDK. It includes libraries and drivers for the various hardware blocks within Zynq UltraScale+ SoCs, including the ARM processor cores, programmable logic, and peripherals.

A talented Javanese engineer Ichiro Kawazome has composed the repository which provides a Linux boot image (U-boot bootloader, kernel, RootFS) for Zynq UltraScale+ MPSoC.

Target Applications

STM32 microcontrollers are ideal for lightweight drones, flight controllers, and embedded systems where power efficiency, real-time responsiveness, and cost-effectiveness are paramount.

Zynq is suitable for drone applications that require high computational power, flexibility, and customization, such as autonomous navigation, advanced image processing, and real-time decision-making.

The choice between Zynq and STM32 depends on the specific requirements of the drone application, including computational complexity, power efficiency, real-time performance, and system integration needs. While STM32 MCUs excel in low-power operation and real-time responsiveness, making them suitable for a wide range of embedded applications, including drones, Zynq offers higher computational power and hardware customization capabilities.

Zynq offers significantly higher computational power compared to STM32 microcontrollers, thanks to its ARM application processors and FPGA fabric.

Developers can create customized Linux distributions tailored for Zynq-based drones, integrating necessary drivers, libraries, and applications for drone control, sensor interfacing, and communication. This approach offers flexibility and control over the software stack but requires significant development effort.

Delineating OS and Firmware: A Conceptual Framework

In the architectural hierarchy of UAV systems, the RTOS serves as the foundational layer, orchestrating task scheduling, resource allocation, and timing management. RTOSes can be remarkably small, especially ones designed for the strict memory constraints of embedded systems.

The HAL, influenced by the MCU vendor, provides a standardized interface for firmware interaction, encapsulating hardware intricacies and enhancing portability.

The most common structure of the existing flight controller firmware, which consists of layers and modules. Pic source: Gyeongsang National University, Rep. of Korea

Firmware, typified by Ardupilot or PX4, operates at a higher level, focusing on flight control, navigation, and hardware interfacing, thereby delineating the boundaries between OS and application-specific functionalities.

ChibiOS is hidden under the hood of Ardupilot, while what's in the same role for PX4 is NuttX.

A high level overview of a typical "simple" PX4 system based around a flight controller. Pic source: docs.px4.io

Is There a Place for Linux Above the Ground?

While Real-Time Operating Systems (RTOS) like FreeRTOS, NuttX, and others have traditionally dominated the realm of civilian drone development due to their suitability for real-time responsiveness and resource optimization, the use of Linux-based platforms, particularly the Zynq platform, has also gained traction in certain niche applications within the UAV ecosystem.

Linux offers several advantages for civilian drone applications, especially in scenarios where complex computational tasks, such as image processing, data analysis, or machine learning, are required. The Zynq platform, which combines the flexibility and ease of use of an ARM processor with the programmable logic capabilities of an FPGA, presents an attractive option for UAV developers looking to leverage the power of Linux in conjunction with customizable hardware acceleration.

One area where Linux-based platforms like Zynq have found utility is in high-level mission planning, data processing, and decision-making tasks. For instance, Linux enables developers to deploy sophisticated algorithms for autonomous navigation, obstacle avoidance, and environmental mapping, leveraging the extensive software libraries and development tools available within the Linux ecosystem.

Additionally, Linux-based platforms offer robust networking capabilities, allowing drones to communicate with ground control stations, cloud services, and other drones in a distributed manner. This facilitates collaborative missions, swarm behavior, and real-time data sharing, thereby expanding the scope of civilian drone applications beyond individual flight operations.

However, it's essential to acknowledge that Linux-based solutions may not be suitable for all drone applications, particularly those that prioritize real-time responsiveness, low latency, and deterministic behavior. In such cases, RTOSes remain the preferred choice due to their ability to guarantee timing constraints and optimize resource utilization.

Saying briefly, while RTOSes continue to dominate the civilian drone landscape, Linux-based platforms like Zynq offer a compelling alternative for certain niche applications that require advanced computational capabilities, networking features, and flexibility.

As the UAV industry evolves and the demand for more sophisticated autonomous capabilities grows, the integration of Linux-based solutions alongside traditional RTOSes is likely to become increasingly prevalent, driving innovation and expanding the horizons of civilian drone technology.

The Spirit of Open-Source

The following projects have become very significant within the discussed area of development and have gained a wide community over the past ten years:

ArduPilot is a widely used open-source autopilot system for UAVs. It offers similar features to PX4, including flight control, mission planning, and telemetry support. ArduPilot supports a variety of airframes and can run on microcontroller-based flight controllers like Arduino boards.

PX4 Autopilot is another popular open-source autopilot system for UAVs. It provides a complete set of flight control algorithms, mission planning tools, and communication protocols. PX4 supports a wide range of airframes and is highly customizable, making it suitable for both hobbyist and commercial UAV applications.

MAVLink is an open-source communication protocol used for exchanging telemetry and control messages between UAVs and ground control stations. It provides a lightweight, efficient messaging format and supports various transport protocols, including serial, UDP, and TCP/IP.

MAVSDK (MAVLink Software Development Kit) is a cross-platform, open-source SDK for accessing MAVLink-based UAV systems programmatically. It provides APIs in multiple programming languages, including C++, Python, and Swift, making it suitable for a wide range of UAV applications.

OpenCV (Open Source Computer Vision Library) is a popular open-source computer vision library that is widely used in UAV applications for tasks such as object detection, tracking, and image processing. It provides a comprehensive set of algorithms and tools for working with visual data. The reader definitely have to take a look at Kenni Nilsson's project!

ROS (Robot Operating System) is an open-source robotics middleware framework that provides libraries and tools for building robotic systems. It includes packages for tasks such as sensor integration, localization, mapping, and path planning, which are applicable to UAVs as well.

LibrePilot, an open-source flight control software for UAVs that offers a range of features including stabilization, navigation, and telemetry. It is designed to work with a variety of hardware platforms and offers extensive configurability and customization options.

ExpressLRS (ELRS) is a rapidly growing open-source radio control link designed to surpass the performance limitations of traditional systems. It prioritizes long range, low latency, and high update rates for applications like FPV drone racing and long-range aircraft. Users have extensive control over the firmware, allowing them to fine-tune parameters like transmission power, frequency bands, telemetry options, and much more. This level of customization caters to a variety of use cases.

Future Trajectory of Drone Software

The trajectory of drone software is poised for significant advancements, propelled by burgeoning research endeavors in secure software engineering, artificial intelligence (AI), and programming language design.

Anticipated advancements encompass the integration of AI algorithms for onboard decision-making, adoption of safer programming languages to mitigate system vulnerabilities, and enhancements in software security mechanisms to fortify UAV resilience against adversarial threats.

Conclusion

In conclusion, the selection of an appropriate MCU or SoC and the deployment of a tailored operating system constitute foundational pillars underpinning the efficacy and reliability of UAV systems.

By embracing Real-Time Operating Systems (RTOS) and leveraging Hardware Abstraction Layers (HALs), developers can navigate the intricate landscape of UAV software development with confidence, ushering in a new era of innovation and transformative potential across diverse industry verticals.

It is also difficult to overestimate the contribution of the open source community in popularizing drone-centric development!

Soft landings and clear skies to everyone!

Most sources are quite fully identified with links given inside the article.
Cover pic by Marian A. Juwan, Pixabay.

See also: 'Getting Started with STM32 - Introduction to FreeRTOS', a blog post by Shawn Hymel @ maker.io / DigiKey.