The matrix fail-fast interaction is an important cost and reliability detail: a spot interruption can amplify work far beyond the interrupted job. I would publish a small workload-level metric beside per-minute price—median wall time and rerun minutes after an interruption—so runner comparisons expose that amplification instead of treating every billed minute as equivalent.
I agree that there are quite a few nuances here Alex!
In practice over the last month interruptions weren’t a problem. Our CI spends most of the time running test suites. Test suites are naturally many short (~3min), parallel jobs (since you want them to finish asap).
It fits the AWS spot model perfectly - if anything is interrupted, it is a cheap short job that can be re-run very quickly. AWS actually gives a 2min headsup when it wants to take back the spot, so usually your suite has the time to finish (and SuperCI takes that into the account).
That workload shape explains why the interruption cost is small in your month of observations. It would be useful to report job-duration distribution and retry minutes alongside the spot rate: longer integration jobs or correlated interruptions could change the economics even when the short test shards behave well.
Backend engineer in Oslo. Mostly Go and Postgres, with a bias for simple systems that are easy to operate. Learning SQLite internals; building a CLI for tracking side projects.
DevOps engineer in Portland. Mostly Kubernetes YAML, Terraform and making deploys boring. I write about CI/CD and cloud cost. Learning Rust; building a home lab dashboard.
Creator of GithubStarsManager — an open-source, local-first tool that organizes GitHub stars with AI: auto-categorization, semantic search, release tracking, and an MCP server for star library.
The part I will take back to my own CI setup is that per-minute price is only half the sum. Your points about disk throughput, cache locality, and macOS carrying 59% of the cost despite 15% of the minutes are a good reminder to measure job duration and cache-restore time, not just the runner rate, before switching. For short, easily retried jobs the spot-interruption trade-off looks reasonable; for long or stateful jobs I would want a clearer retry and checkpoint story first.
Yeah, I guess key learning is that you can get CI 8x cheaper than GitHub actions if you run it on AWS spot. For long jobs, you can just run it on AWS on-demand (which SuperCI supports aswell, even per-workflow) - this ensures its not interrupted but AWS on-demand machines cost about 3x more than spot. Hope it helps!
Creator of GithubStarsManager — an open-source, local-first tool that organizes GitHub stars with AI: auto-categorization, semantic search, release tracking, and an MCP server for star library.
That 8x number is wild — spot really is the cheat code when the job can tolerate being interrupted. The per-workflow on-demand fallback is the part I'd actually use: keep the long jobs safe without paying on-demand prices for everything. Do you find the interruption rate on spot stays low enough in practice, or do you end up re-running enough jobs that some of the savings evaporate?
Pricing every family in every zone is the detail most runner comparisons skip, and the disk note — 125 MiB/s flat on EBS until you pay for provisioned throughput — is the one half the table owner never mentions. The metric still missing from every row is minutes lost to reclaim retries: you retry on on-demand so a job is never caught twice, but a spot reclaim at minute 9 of a 10-minute matrix job still re-runs nearly all of it. Did the October runs give you a count of how often that retry path fired? That number would settle hosted-vs-spot for long test suites more decisively than any per-minute price.
ML engineer focused on cost-efficient AI infrastructure. I benchmark GPU clouds and share practical guides for running LLMs in production.
📬 Telegram: https://t.me/qisuancloud
The comparison that matters most here is not the per-minute table alone, but how much of the bill can actually move to spot. With macOS taking 59% of the money on only 15% of the minutes, a Linux-only saving leaves the largest cost block untouched. I would also want to see the interruption cost counted explicitly: for jobs that cannot be split into short pieces, a reclaimed instance plus a retry on demand could erase the spot discount, so the break-even point probably depends more on job length and retry policy than on the headline rate. How do you decide which jobs are short and safe enough to put on spot, and which ones stay on demand?
Useful table. One hosted row worth adding at the same 4 vCPU size: Sengi Basic is $0.002 a minute ($0.001 at 2 vCPU, $0.004 at 8), so 100,000 minutes comes to about $197 after its monthly free allowance. That puts it under every hosted option in the table and within about $50–60 of the spot-in-your-own-account rows, without a cloud account, spot interruptions or a licence to manage: it is a one-line runs-on change and every job gets a fresh VM.
The limit is scope: Linux x64 only, 2 to 8 vCPU, so it does nothing for the macOS share you called out (59% of your list-price bill).
Disclosure: I work on Sengi.
We're a place where coders share, stay up-to-date and grow their careers.
The matrix fail-fast interaction is an important cost and reliability detail: a spot interruption can amplify work far beyond the interrupted job. I would publish a small workload-level metric beside per-minute price—median wall time and rerun minutes after an interruption—so runner comparisons expose that amplification instead of treating every billed minute as equivalent.
I agree that there are quite a few nuances here Alex!
In practice over the last month interruptions weren’t a problem. Our CI spends most of the time running test suites. Test suites are naturally many short (~3min), parallel jobs (since you want them to finish asap).
It fits the AWS spot model perfectly - if anything is interrupted, it is a cheap short job that can be re-run very quickly. AWS actually gives a 2min headsup when it wants to take back the spot, so usually your suite has the time to finish (and SuperCI takes that into the account).
That workload shape explains why the interruption cost is small in your month of observations. It would be useful to report job-duration distribution and retry minutes alongside the spot rate: longer integration jobs or correlated interruptions could change the economics even when the short test shards behave well.
self-hosted on spot is the only part of that comparison that moves the bill.
The cost math here matches what we saw. Self-hosted runners were cheaper only after we accounted for maintenance time.
The 100k minutes framing is useful. Are you including the cost of the runner VM in that number?
The part I will take back to my own CI setup is that per-minute price is only half the sum. Your points about disk throughput, cache locality, and macOS carrying 59% of the cost despite 15% of the minutes are a good reminder to measure job duration and cache-restore time, not just the runner rate, before switching. For short, easily retried jobs the spot-interruption trade-off looks reasonable; for long or stateful jobs I would want a clearer retry and checkpoint story first.
Yeah, I guess key learning is that you can get CI 8x cheaper than GitHub actions if you run it on AWS spot. For long jobs, you can just run it on AWS on-demand (which SuperCI supports aswell, even per-workflow) - this ensures its not interrupted but AWS on-demand machines cost about 3x more than spot. Hope it helps!
That 8x number is wild — spot really is the cheat code when the job can tolerate being interrupted. The per-workflow on-demand fallback is the part I'd actually use: keep the long jobs safe without paying on-demand prices for everything. Do you find the interruption rate on spot stays low enough in practice, or do you end up re-running enough jobs that some of the savings evaporate?
Pricing every family in every zone is the detail most runner comparisons skip, and the disk note — 125 MiB/s flat on EBS until you pay for provisioned throughput — is the one half the table owner never mentions. The metric still missing from every row is minutes lost to reclaim retries: you retry on on-demand so a job is never caught twice, but a spot reclaim at minute 9 of a 10-minute matrix job still re-runs nearly all of it. Did the October runs give you a count of how often that retry path fired? That number would settle hosted-vs-spot for long test suites more decisively than any per-minute price.
The comparison that matters most here is not the per-minute table alone, but how much of the bill can actually move to spot. With macOS taking 59% of the money on only 15% of the minutes, a Linux-only saving leaves the largest cost block untouched. I would also want to see the interruption cost counted explicitly: for jobs that cannot be split into short pieces, a reclaimed instance plus a retry on demand could erase the spot discount, so the break-even point probably depends more on job length and retry policy than on the headline rate. How do you decide which jobs are short and safe enough to put on spot, and which ones stay on demand?
Useful table. One hosted row worth adding at the same 4 vCPU size: Sengi Basic is $0.002 a minute ($0.001 at 2 vCPU, $0.004 at 8), so 100,000 minutes comes to about $197 after its monthly free allowance. That puts it under every hosted option in the table and within about $50–60 of the spot-in-your-own-account rows, without a cloud account, spot interruptions or a licence to manage: it is a one-line runs-on change and every job gets a fresh VM.
The limit is scope: Linux x64 only, 2 to 8 vCPU, so it does nothing for the macOS share you called out (59% of your list-price bill).
Disclosure: I work on Sengi.