Stability

It's been three years since I announced the Rakudo CI bot project (then still called the Raku CI bot) I am working on and almost two years since I last published a grant report. It's high time to publish another. But in this post I want to focus on a milestone that's not part of the grant: Stability.

In my last two reports (6, 7) I started added a paragraph dedicated solely to bugs I found and fixed in third party components. Quoting one of those reports:

I do hope that instead of evading bugs in third party software by utilizing workarounds or switching the library, but instead golfing, reporting and fixing bugs, the quality of our software ecosystem will improve and future users will be able to enjoy a more reliable ecosystem.

I had less time working on the CI bot than I liked (one of the happy reasons being my families head count increasing). But I did not expect that getting the bot stable would take me two more years, 39 pull requests, two problem-solving issues and a dozen more bug reports that were solved by others. In the last year when I was asked what my go-to programming language of choice was, I would say "Raku. But I honestly can't recommend using it yet. It's lacking in stability."

Over the course of the last years the CI bot always somehow worked, but within a few minutes, at most one or two hours after starting it, it would:

• Segfault in multiple ways
• Deadlock in multiple ways
• Infini-loop
• Error out in various ways (and - depending on the error class - recover. Remember that stability was one of the key goals of the CI bot.)

Well, I'm happy to report, that since last week the CI bot manages to run for multiple days without dying and without logging a single unexpected error.
To be honest it still manages to hit an out of memory condition roughly every two days on a system with 4GB of RAM. This is worth looking into, but not the kind of failure that makes me not recommend using Raku in general.

Lets look at the dependencies of this project (showing only the larger ones):

• Cro::HTTP
  • IO::Socket::Async::SSL
• Cro::WebApp
• LibXML
• OO::Monitors
• Red
  • DB::Pg
• Log::Async
• JSON::JWT
  • OpenSSL
• YAMLish
• WebService::GitHub
  • HTTP::Tiny
  • IO::Socket::SSL

So when you want to build an application that

• has a web frontend,
• uses PostgreSQL,
• sends webrequests
• including uploading larger files and
• using XML and JSON APIs and
• has things happening concurrently,

I can now recommend Raku as a language to do so.

Yay! \o/

The Bug Rooster

Do note that not all of the below PRs are merged yet. I'll keep pushing to get the remaining lot merged in due time.

Cro

• Do not send the ENO_OF_STREAM flag twice #160: merged 2021-11-19
• Implement remote window handling #161: merged 2022-10-17
• Implement support for sslkeylogfile #162: merged 2022-10-17
• Fix cleanup of timed out connections #168: merged 2022-10-17
• Enable logic to prevent connection timeout #169: merged 2022-10-17
• Fix parsing media types with '.' in them #36: merged 2022-11-14
• HTTP2: Implement GoAway handling for the non-error case #180: merged 2023-04-12
• Correct initial dynamic table limit #5: merged 2023-04-12
  • Bump HTTP::HPACK dep #188: merged 2023-11-1
• Implement a retry mechanism when GoAways are received #187: merged 2024-01-22
• Fix HTTP1.1 pipeline teardown #190: merged 2024-01-22
• Fix a connection reuse race #191: open
• Fix HTTP1.1 connection caching with explicitly given version #192: open
• Fix HTTP2 connections with multiple streams stalling #193: open
• Fix using template parts via render-template #88: open

Red

• Remove duplicate operator #535: merged 2021-11-21
• Lock accesses to the alias-cache of Model #536: merged 2021-11-21
• Extend alias lock to also cover the read accesses #545: merged 2022-02-10
• Fix DateTime roundtrip on Pg #548: merged 2022-02-10
• Fixes constucting an enum column from another column #553: merged 2022-06-15

Rakudo

• Fix IO::Path::parent #4795: merged 2022-02-19
  • Add more IO::Path::parent tests #801: merged 2022-02-19
  • Change parent to always just remove the last element #4800: merged 2022-02-26
  • Change .parent behavior to "stupid" resolving #802: merged 2022-02-26
• Supply blocks may reorder flow of execution #364: solved
  • Deadlock in supply / whenever chain #5141: fixed
  • Simplify Supply / whenever processing order #5158: superseeded
  • Fix rare deadlocks during supply setup #5202: merged 2023-11-24
  • Switch Supply.zip to a watermark approach (https://github.com/rakudo/rakudo/pull/5211rework) #5211: merged 2023-03-09
  • Test supply setup with a blocking recursion #833: merged 2023-11-24

MoarVM

• LEAVE phaser with return #417: open
  • Implement return prioritization #1786: superseeded
  • Don't lose return value with LEAVE phasers #1782: open
  • Fix returning from LEAVE to surrounding scope #1785: open
  • Test return prioritization #851: open
  • Implement return prioritization (https://github.com/MoarVM/MoarVM/pull/17882nd try) #1788: open

Other

• IO::Socket::Async::SSL: Implement support for sslkeylogfile #64: merged 2021-11-19
• OpenSSL: Fix OpenSSL::RSAKey sometimes failing to create a key #95: merged 2022-01-13
• DateTime::Timezones: Fix depends spec #7: merged 2023-20-21
• OpenSSL: Fix segfaults when double closing #106: merged 2024-02-18
• IO::Socket:SSL: Rewrite this module as a IO::Socket::Async frontend #24: open
  • IO::Socket::Async::SSL: Make .writes Promise result return the written bytes #73: merged 2024-03-03
• HTTP::Tiny: Fix connection reuse: open
• HTTP::Tiny: Fix segfault: open