Determining what protocols to carry in our source code tree never really had a standard or process until recently. In this post I'm going to describe the process we're currently adopting and the events that led to it.
Way back when we used to accept protocols into the tree without much hesitation. For example, we added a protocol Plugin for GroupWise that was gifted to us by Novell even though we couldn't actually test it because we didn't have a GroupWise server.
We had similar situations with SameTime and Zephyr where we also had no access to a server which meant we couldn't test anything. It should come as no surprise that these protocols started to bit rot immediately only seeing updates when a technical user with enough patience would work with us to fix issues.
Other times new protocols would get added by existing Pidgin developers because of their interest in it. This was generally okay because if someone reached Developer status, they had gained enough trust from the other developers and were expected to be around for quite a while. See Pidgin Developer Tiers for a more in-depth description.
As you may have guessed, over time developers would lose interest in the protocols they brought in which would then of course start bit rotting as well. This happened with the popular protocols too, but sometimes that was just due to their complexity like the OSCAR protocol that AIM and ICQ used.
While bit rot is a huge issue, the bigger issue with proprietary protocols especially, is keeping up with changes. With proprietary protocols changes are never expected and must be reversed engineered. This of course is all expected as these are proprietary protocols and there aren't supposed to be third party implementations by design but it's a huge burden to stay on top of.
But the unexpected changes in proprietary protocols still aren't nearly as bad as dubious legality of the reverse engineering processes. While I believe we have always followed the law here, that doesn't mean our contributors have.
I recall one contributor, a very long time ago, that was boasting about de-compiling an official client and using that to write patches for us. We of course had to reject those patches as they would have put the rest of the project as risk as that method of reverse engineering is to the best of my knowledge, illegal.
This dubious legality even led to us losing a contributor a few years ago. This contributor had immigrated to the United States on a work visa, and the legal department at the company they were working for decided that this legal dubiousness wasn't an acceptable risk and stated they could no longer work on Pidgin while employed there.
We were of course understanding of the situation, and as you expect, the contributor found something else to do with their free time and hasn't returned to the project. There's no ill will here, but we did learn an important lesson here about how others see some of the code that we carry.
So with all that history out in the open, lets move on to how we're moving forward.
To put it succinctly, Pidgin 3 will only have Open Source/Open Specification protocols in its source code tree that we can actually test. In doing so we will remove all legally questionable source code from our tree and thus avoid the previous situation where we lost a contributor.
This means that the core protocols we'll be aiming to have are Bonjour, IRCv3, and XMPP. We're not aiming to have these all done by 3.0, but these are acceptable for us to have in tree.
You might have noticed that Zephyr is not on this list even though it's an open protocol. We actually already removed the Zephyr implementation because we could not test it and because some of the code still use K&R functions as well as other pre-ansi C stuff.
Similarly, SILC has been removed because we can't connect due to a crash. We're not sure if it's the client library or the server that is the cause and looking deeper into it didn't seem like a good use of time.
When it comes to proprietary protocols they are going to have to live as separate repositories. I know this has been problematic for users especially when it comes to stuff like Facebook, Discord, and others, but this is just the way it has to be.
For starters, when something changes in these protocols, updates need to come as soon as they're ready. Releasing Pidgin is, complicated to say the least, and is not quick at all. So if these protocols were in the main tree, we'd have to force them into micro releases to get them out as fast as possible because the protocol would be broken but that importance is hidden expectations of a minor version number. They'd also be subject to string freezes and other quirks about our release process.
But the real problem that users have with protocols not being in the main distribution is one of discovery. Sure we have the plugins list to help people find them, but most users don't go to our website.
The plan to solve this is to add an "addon store" inside of Pidgin, similar to the Chrome Web Store and FireFox Add-ons.
This has been a concept we've been designing for a while but haven't actually built yet. A few interested parties have put together a few proof of concepts, but we haven't quite gotten to an minimum viable product yet. If this is something you might be interested in, please reach out!!
So there you have it. Our new policy of including protocol plugins into the main source tree is that it must be Open Source/Open Specification. That means stuff like SIP, Mastodon, and Matrix could one day land in tree, but due to resources constraints, aren't on our road map.
Proprietary protocol plugins will all be moved out of tree (only Gadu Gadu remains at the moment) but we plan to help users discover them via an add-on store that will also keep them up to date.
I hope you're enjoying these posts! Remember they go live for patrons at 9AM CST on Mondays and go public at 12AM CST on Thursdays! If there's something specific you'd like to see me cover here, please comment below!
Top comments (2)
On the other side the Pidgin is literally a platform to create clients for proprietary networks.
This is how I started with Pidgin: I switched to Linux and needed a client for a FChat (UDP based local chat).
I thought it would be quickly to reuse the Pidgin and libpurple.
I was ok that it's UI is bad (sorry) but I needed to solve the most important problem: keep communication.
Today the libpurple is more important than the Pidgin itself. E.g. on the PostmarketOS the Chatty messenger is based on the libpurple.
As you noted, the good (and bad) thing is that FOSS protocols aren't evolve so quickly.
But they all have own specialized clients that outperform the Pidgin.
So the Pidgin can peak only one most feature rich i.e. XMPP to have a testbed for all features.
The IRC doesn't really need to be supported as good as the XMPP but instead spend time for proprietary protocols,
If something is "out of tree" then chances that it will be installed by an average user are very low.
Plugins are also something that is not so easy to find and install. Users simply may not know that they can use Telegram in Pidgin.
So ideally the proprietary protocols needs to be installed too.
It doesn't really mater how, maybe as a separate package (Debian packages have a Recommends for such packages).
But a user should see the protocols in a select.
Your samples with GroupWise, SameTime (IBM Lotus), Zephyr and SILC are more corporate solutions.
But for regular users it was always more important to have a Skype and even used the plugin even though the Skype had a decent Linux client.
Today we have about 2 billion users of WhatsUp. And we have a plugin for it, so it very important if you can review it and add to core.
iMessage 1.3 billion: has no plugin, but it should be possible.
WeChat 1.26 billion: there was an attempt to create a plugin.
Telegram 900 million (and highly growing now): there is a more or less working plugin that needs for some help. The protocol itself is published.
Viber 260 million: no any plugins, no protocol.
LINE 217 million and KakaoTalk 57m: Korean networks, no any plugins, no protocol.
Signal 40 million: it has a plugin but also has an official Linux client which users are probably better to use for advanced security.
Summary: we already have WhatsUp, TG and Signal plugins that can and should be supported by the Pidgin officially.
By the "support" I mean at least:
There's a lot to touch on here but I'll try to keep this simple.
Pidgin is an open source project built by volunteers to do the things that they want. Quoting user numbers to volunteers doesn't work because there's no incentive for volunteers to take on that immense amount of work. Again we're doing this for ourselves, we're not trying to make a business out of this. In fact, there's a good chance that if we tried we'd be litigated out of existence.
As far as chatty goes, it's not quite based on libpurple, or at least it wasn't last time I checked. When I looked they used libpurple as an interface to our xmpp protocol plugin which is horribly out of date. Also, as far as I could tell, it didn't look possible to load another libpurple protocol plugin into chatty.
As far as proprietary protocols go, we will no longer be shipping them in the main release. This was covered extensively in this post and I'm not going to rehash it. A solution was even provided in this post that you may have glossed over?
Finally, using Pidgin 2 as an example of anything we're doing is foolish, especially as I've been posting here all the things we're doing in Pidgin 3. Also those corporate protocols you mention became huge burdens that we have finally deleted in Pidgin 3 because we have no way to test them.
Anyways, the long and the sort of it is that we do not have the resources, (people, time, money, etc) to support any proprietary protocols in any "official" capacity.