Powershell and Ryzen - a build server

This being my first article I’ve ever written, I expect it will be awkward, a little dense and potentially hard to read.

But I thought this was an interesting experiment, so I’m putting the information out into the web!

I’m amidst building a companion application for the community edition of SonarQube, a static code analysis tool. After a few light weekends, I finally completed the project and deployed it into my local environment. Feeling successful, I turned my attention to a new project around Kubernetes!

However, as soon as I tried to deploy my application in a large enterprise, with 1500+ projects configured within the SonarQube tool, my application started to break down. I realized I needed to recreate this environment on my home server to be able to development effectively for enterprise size.

The fun thing about SonarQube, is that the simplest way to push a project into it for static code analysis, is by running a build of some code. In my case, this is building using Maven. So I wrote a script that will perform Maven builds in parallel on my development desktop, just to populate the SonarQube instance. This brings us to the purpose of this article.

I figured it would be interesting to see how builds can scale in parallel (at least when triggered within Powershell) on a Windows machine, so I began recording the data. It’s worth noting that these builds do not perform a clean, and part of the build is copying the source directory to a new location.

Here is the script that performs the builds:

param(
[Parameter(Mandatory=$true)][string]$ParentFolder,
[Parameter(Mandatory=$true)][string]$OutputFolder
)
Clear-Host
$ScriptBlock = { param($ParentFolder,$OutputFolder)
    $Guid = [Guid]::newGuid()
    Write-Output "Writing SonarQube project-$Guid from generated folder $OutputFolder/$Guid"
    Copy-Item -Path "$ParentFolder" -Destination "$OutputFolder/$Guid" -Recurse
    & mvn package -q -DskipTests versions:set -D:"newVersion=$Guid" -f "$OutputFolder/$Guid/pom.xml" sonar:sonar -D"sonar.host.url=http://192.168.2.155:19100" -D"sonar.projectKey=project-$Guid"
}
for($i = 0; $i -lt <someNumberOfExecutions>; $i++)
{
    Start-Job $ScriptBlock -ArgumentList $ParentFolder,$OutputFolder
}
Measure-Command {
    While (Get-Job -State "Running") {Start-Sleep 2}
}
Get-Job | Receive-Job
Remove-Job *

The specs of the machines are below:
+------+----------------------------------------+
| Part | Desktop |
+------+----------------------------------------+
| CPU | Ryzen 7 3700X (Resting CPU util = ~5%) |
| Mem | 16GB DDR4 |
| Disk | Samsung NVMe |
+------+----------------------------------------+
+------+----------------------------------------+
| Part | "Server" |
+------+----------------------------------------+
| CPU | i7 3610QM (Resting CPU util = ~1%) |
| Mem | 16GB DDR3 |
| Disk | 7200 RPM |
+------+----------------------------------------+

During any builds (above 2 in parallel), the machine reported itself at 100% CPU utilization. (Around 50 parallel builds is when my machine locked up and things started crashing)
java.nio.file.FileSystemException: C:\Users\Usr.sonar: Insufficient system resources exist to complete the requested service
AND
WARNING: A PSScheduledJob job source adapter threw an exception with the following message: Insufficient system
resources exist to complete the requested service.
And here is the data!

These aren’t the most interesting results, but I think this shows exactly how Powershell is scaling it’s tasks under the hood. While the machine is getting bogged down, the amount of time that builds take to complete is surprisingly linear.

Once builds start to cross over the 35/40ish in parallel threshold, individual builds begin failing. As I mentioned above, when pushing 50/60 builds in parallel, windows begins to struggle to function properly. Not unlike during a stress test!

The takeaway from this experiment is if you’re planning on using a Ryzen 7 3700x in a build server, pay attention to how you’re going to execute the builds. Triggering naive powershell tasks is going to create an invisible boundary that, when crossed, will cause builds to mysteriously start failing without warning.

Edit Notes:
I transitioned this article from my Medium account, and it's pretty old. I'll be the first to admit it's not the most scientific test.