From Zero to Hero, My Qt/Python Journey

There's a lot of hype and misinformation surrounding large language models, from claims of gross inaccuracy to fears of robot overlords taking over and making us their house pets. In this article I will explain how an LLM took me from "zero to hero" in just four months with a software programming project using tools I knew almost nothing about. And then I made some interesting discoveries using the software application itself.

I wrote a diagnostic utility that plots disk I/O activity on a per-process basis in an interactive pie chart. I couldn't figure-out what to call it, so I nicknamed it QtΠ for now (Cutie Pie), drawing inspiration from my lovely wife.

Why though?

My motivation for building such an application stems from frustration with software applications that won't leave the hard drive alone to rest. I'm sitting here waiting for something to load and my hard-drive LED has been on steadily for the past 10 minutes, and now my coffee is cold. Did making drives work hard become an international sport in the past 15 years? Is there a conspiracy by "big hard drive" to stimulate more product sales? I wanted to find out why, and spy on the files these rogue applications were interacting with. If you're using a solid state drive then it won't be obvious when it's busy. You'll just wonder why your drive keep wearing out prematurely. I still prefer to use mechanical drives for this reason. If some process takes too long depending on a mechanical drive, then it takes too long, period. And with the help of this diagnostic utility I discovered that the perpetrators were guilty of some heinous crimes.

How it Started

At first I thought about using strace to produce text output of disk activity, but then I would still need to parse it and put it in an excel spreadsheet or database. It was a lot of manual work that I knew I could automate. So then I thought about building a GUI using the Qt framework. It was an opportunity to learn something new while diagnosing frustrating disk-related problems with rogue processes. Besides, I'd have to parse strace and load it into a database anyway. Caveat: I knew nothing of Qt other than it looks pretty, and I had only some experience with Python. I had no idea how a Python/Qt app would take shape, but my confidence was high and my adventurous spirit was ready to take a chance on the LLM. And with its help I got started very quickly. I found the most effective and rewarding approach to the LLM was to treat it like an instructor. Rather than asking it to write turnkey code for me, I wanted it to teach me so that I could write the code myself. I asked it questions like the following:

1. Write Python/Qt code that puts a pie chart in a window.
2. Explain how QPieSeries works and the various ways of using it.
3. I need a multi-threaded producer and consumer in Python/Qt that separates data processing from the UI.
4. Why is my Qt for Python app randomly crashing with SIGSEGV?

If I disclosed all my prompts we would be here all night, and I want to keep some trade secrets of course. I just wanted to make the point that treating the LLM as an instructor rather than a personal slave was both rewarding and efficient. Most of the examples could be run as is and produced something visual:
Code examples produced by the LLM were not end points but starting points to ask further questions, elaborate on each of the components used, and explore the various ways to leverage them. Using the LLM this way felt a lot like a much more efficient and targeted search engine. The pie chart implementation I ended-up going with looks nothing like the code example produced by the LLM. It had grown well beyond that, many prompts later.

How it's Going

It wasn't long at all before I was asking the LLM to show me how to exploit the wide variety of Qt widgets available. I was having so much fun! It had been several years since I found so much reward in software development. This infusion of confidence lead me to spread my wings a little further, and I decided it was time to try my hand at making SVG icons for the app, using Inkscape, another tool I knew nothing about.

Inkscape is a rich and clean application for creating and modifying vector graphics, and good for creating icons. I was looking forward to applying my (very limited) art skills to create icons for the application. I fired up Inkscape and began throwing prompts at the LLM:

1. How would I draw a filled polygon? and how would I edit the vertices?
2. Is there a way to cut a shape into pieces using another shape? like computing the subtraction of one from the other?
3. How do I apply a translucent gradient? I want to make something that looks like a magnifying glass.

I was having fun, not only because I was pumping out icons quickly, but they looked decent! (at least I thought so, but then I would've been content with some crudely filled shapes) This was my first time with Inkscape and my first time creating vector graphic icons. The most challenging exercise was figuring out how to merge some shapes into a whole, then chop it up into cubes. The LLM took me through the whole process and I came-up with the icon for my app that perfectly reflected my vision: a mechanical drive breaking-up into pieces, under a magnifying glass. I wanted to connote the destructive nature of rogue software on hardware, and I am pleased with the result.

Reflections

As I iterated on the project, I found I really enjoyed working with Qt and Python, but garbage collection was a thorn in my side. Qt and Python have different ways of cleaning up objects, which this leads to attempts to deallocate objects twice and ultimately causing the application to crash without warning. There are ways to work around this, but it is tricky and very difficult to troubleshoot. I managed to make headway using the development version of Qt, core dumps, and GDB for examining the backtrace just before the crash. That exercise is worth its own article, and it was painful, so I won't vex any further.

There are many things about Qt I appreciate, and I dreamt-up new features to add to the app as an excuse to learn more about Python and Qt, from customizing widgets to building an embedded help system. I particularly enjoyed working with slots and signals, Qt's way of passing data between threads. I imagined it like people, otherwise beyond reach, passing messages to each other through pneumatic tubes. It was an enduring source of reward that kept me motivated, bringing the project to completion after four months. This is incredibly important because without reward the work is a slog. The application relies heavily upon slots and signals because I wanted to ensure a perfectly smooth user experience, without any lagging or freezing while interacting with it. All data processing and long-performing tasks are run in threads isolated from the main UI thread. While the more technical reader might think this resembles an actor model, it's only a crude reflection. I was still making use of mutual exclusion locks in a few places. Yet I would consider the overall design elegant and Qt lends itself well to clean and organized code.

So Who Was The Culprit?

There are three applications (that shall remain nameless) I was frustrated with. The first two would take an upwards of 10 minutes to load due to the excessive disk activity. One of the benefits of using Linux is that there are some ways around it, but you first need to understand what the process is trying to access.

Concerning the first application, let's call it Magnification Groups, whenever I would "share my screen" and select an application window to share instead, my hard drive would thrash constantly, and scrolling through the list of options was chunky and laggy if not unresponsive. Here's what I discovered using I/O Cartographer: the application was constantly writing bitmap thumbnail images of each and every window I had open on my system, to the disk. One wonders who signed off on that design decision. The work-around I attempted first was removing the directory where these thumbnails were accessed and replacing it with a symbolic link to a temporary in-memory file system. In hindsight I realize I should've bind-mounted it rather than use symbolic links. The point is moot because this feature was removed from Magnification Groups entirely since.

The second application, let's call it Flaming Canine, would take up to 10 minutes to start. The only indication you have of something gone wrong are endless "loading" spinners. What I found is that the application was loading the entire universe of all the offline storage it had ever collected, at startup. It didn't matter whether it was used recently or several years ago. I suppose lazy loading is not a virtue appreciated by everyone. I'm still dreaming-up ways to tackle this one, without involving one of my solid state drives. But right now I'm distracted by my game engine.

The third application was the most mysterious, but with the help of my new toy I was able to isolate the problem quickly. This was a game that we can call "Dirtylandia" and it would periodically freeze for a few seconds. This seemed to be random. I fired-up my I/O scanner and found that there was a timing gap consistently implicating /dev/bus/usb/001/007. On Linux, this means the game was trying to query a USB device. I ran a command-line utility commonly known on Linux, lsusb -D /dev/bus/usb/001/007, which does the same thing, it queries a specific USB device. It froze for several seconds there too! I decided to change the permissions on this USB device and make it root access only: chmod 600 /dev/bus/usb/001/007. Voila! the freezing problem went away completely. It only took 20 minutes to diagnose and fix a completely mysterious problem, that's something worth writing about. Upon doing further research, it looks like I have a bad keyboard USB cable, and since it's a mechanical RGB keyboard, that's not something I'm replacing any time soon. Groceries are more important.

Wrapping Up

Using an LLM as a guide and an instructor for building an application proved to be, not just beneficial, but essential. Without an LLM I am confident it would have taken me no less than two years to build what took me only four months. It wasn't just useful as a software engineering instructor, but also an effective instructor for graphics tools. One could build a business startup with very little capital and use the LLM to teach one how to get up to speed and get noticed, especially for LLM work that can be easily verified for correctness such as compiling and running code or using graphics tools. I found the experience so effective and efficient that it might be time to put the conventional search engine out to pasture.

I decided to name the application "I/O Cartographer" and it has proven ability to isolate disk I/O problems caused by individual applications and processes, including one USB I/O problem. I am tempted to continue working on it, but it does what I originally wanted. I think it's time to solicit feedback.

If you're on Linux, download I/O Cartographer, give it a try, and let me know what do you think!