For the longest time, software ecosystems have been dominated by one concept. Libraries. There is nothing wrong with the concept of redistributable packages. Such an idea is what allows any developer to build almost any application...
but is there a better way? That's what I wish to share my thoughts on: the idea that we can build any application with ease, as long as we think about our ecosystems the right way.
Abstractions
Abstractions are one of the most core concepts when it comes to software, the idea of taking complex, sometimes unintuitive logic and providing a view of it which is simple and intuitive.
This is most prevalent as programming languages, which give us developers the ability to fluently, easily express our logic and data types, minimizing errors, oversights, and difficulties.
I believe there are multiple types of abstractions in today's environment.
- Hardware Abstractions: Abstracting over low-level hardware calls. Examples include graphics drivers, kernels, etc.
- Platform Abstractions: Abstracting over the user's platform elements, e.g. architecture and operating system. Examples include graphics APIs (DirectX, Metal), windowing libraries (SDL, winit), programming languages, etc. These can allow developers to write cross-platform software.
- Software Abstractions: Abstracting over platform abstractions or, in a recursive fashion, software abstractions. Examples include GUI libraries (Qt, GTK), HTTP servers, etc. These can allow developers to write simple software, and also serve cross-platform software.
Monolithic Libraries
Abstractions are often monolithic. This means they are singular units that cover a large surface area. The most prevalent form of monolithic library is a framework.
Problems
Say you wanted to write your own HTTP server that supports HTTP/1, HTTP/2, HTTP/3, TLS, etc.? This is quite the feat! You would need to write much code yourself, and HTTP grows larger and larger by the day, so your library would eventually grow too outdated if left unmaintained.
It is much more convenient to work with libraries that provide a fully-fledged, albeit somewhat complex HTTP server, and to build a software abstraction over that server.
This is the approach followed nowadays by libraries like Express (built on top of Node.js's http module), and Rust's own HTTP ecosystem, most of which is built on the Tower middleware system and Hyper client/server implementation.
But... what if we decided we wanted to customize the HTTP parser?
Depending on the library, this is possible, but wouldn't it just be easier to tweak the existing parser a little, or perhaps replace it with one that is more suited to our needs? Unfortunately, this means we would either need some type of plugin system (which can cause our program to become slower and possibly more complex) or we would need to fork the project which incurs a higher maintenance cost as we would need to be up-to-date with upstream!
Component Libraries
Component libraries are a concept that would help us achieve this goal. Rather than create an HTTP framework that includes parsing, writing, transport, routing, etc. We split each feature into a component library, i.e. a non-monolithic software abstraction, meaning it does one thing, does it well, and can be swapped out by a library consumer.
Rather than creating frameworks, we are essentially delegating the work of combining each component to the library consumer, i.e. a person developing a monolithic library (monolithic software abstraction) or an application developer.
This may sound complicated, as we are requiring our library consumers to put in more work, but it's actually the contrary. As implied in the last sentence, monolithic libraries exist for general-purpose needs, i.e. when someone just needs something to work.
Additionally, the combination of these component libraries should be simple enough! Almost comparable to just using a regular monolithic library, even. If we need more complexity, i.e. the existing options just don't cut it in terms of features, we can replace a component library with a lower level abstraction or even our own.
As software abstractions can be abstractions of software abstractions, we can even make a component more simple rather than more complex if we really want to.
In theory, you could create a lightweight, simple, and perfect (for your needs) HTTP server by just importing the required components, e.g. router, parser, writer.
Swappable Libraries
In the Linux ecosystem, there exists multiple different sound server projects (ALSA, JACK, PulseAudio, and PipeWire). It's common for a user to need multiple of these sound servers as different programs require different APIs, although...
It is possible to replace ALSA, JACK, and PulseAudio with a PipeWire-based implementation of their APIs. This unifies all APIs to one sound server, meaning your system is more cohesive and you can benefit from PipeWire's improved latency regardless of the program.
This is all possible due to C/C++'s "header" system, something dropped in more modern programming languages. This is, in a way, an example of the facade design pattern. Facades are also commonly used by logging libraries, i.e. Java's SLF4J and Rust's log.
Logging facades allow developers to use one API while swapping out the implementation dependent on their program's needs, like how SLF4J has a "no-op" logger that doesn't actually log messages, a "simple" logger for those who don't need a heavily customizable logger, and etc.
It is inevitable that we will run into a situation where you want to use a monolithic library but it uses an abstraction you can't because it just conflicts with another library in your dependencies. So, rather than have a library dependent directly on that abstraction, why not have it depend on an interface? This would allow program developers to select which implementation they want for that interface across all dependencies, keeping the project cohesive.
Applications
Nowadays, we run into an issue where software we're using is lacking features we want... or has features we really don't want. There is no perfect software, just good enough, and to get to something that is perfect (a subjective matter), we need to write our own.
But this is not always an easy task. Look at digital audio workstations, integrated development environments, browsers, game engines, command-line shells/terminal emulators, and operating systems! These are all hard projects to work on because there exists many features that a developer is forced to implement, sometimes complex features (especially with OSes), even though they exist in other projects anyway.
It is simply not possible for someone to make a set of software that is perfect for them within a reasonable set of time... but what if it was?
The ideal software ecosystem is one where we can have each feature for each of these different applications all existing as their own component libraries, so developers can take what they need, avoid what they don't, reimplement what they require, and perhaps even make monolithic libraries so other developers can benefit.
Closing Notes
What if we lived in a world where it was possible to make your own Unix-like within a week? Where you could have a program that perfectly fit your workflow, needs, wants, ideals, etc. because you made it?
As time goes on, we become more dependent on the simple options regardless of whether or not they are the right ones. We also become more dependent on software that belongs to organizations that we do not agree with, which strive for profit rather than consumer happiness. Things go closed-source out of nowhere, horrible pricing models are introduced, and management conflicts with community. We need to take back control of our software.
I do not suggest that this "alternative view" is perfect, but I do think it's a step in the right direction.
Top comments (0)