Locust on Azure: an end-to-end experience

Locust.io is a simple and powerful load testing framework, based on python, perfectly suited for developers and APIs.

Locust can be installed locally on your dev workstation, deployed on a cloud VM or on a Kubernetes cluster.
IMO the best scenario, perfectly fitting into the idea that load testing should be an easy and recurring task, is enabled by the excellent work by Davide Mauri to deploy a master-slave Locust configuration on Azure Container Instances which was my starting point.

Running Locust on Azure

Davide Mauri for Microsoft Azure ・ Feb 17 '20

#python #testing #azure #webdev

Deployment options

This project provides you with as many slaves hatching as many locust users you configure: a deployment script setup the Azure Storage shared between the Azure Container Instances, upload the test scripts and via an Azure template deploys all the other resources.
Once Locust performed the tests and you gathered the results, you can delete the testing infrastructure, not incurring in any additional costs.

To give a perspective of a test run cost on this infrastructure, according to public prices of Azure Container Instances, a 1 hour long test with 4 slaves can cost about 0.24€.

Be conscious about storage

A reminder: the deployment script copies everything from the local /locust folder to the Azure Files, so your Locust will have the scripts and files/payloads for test mounted on the Azure Container Instances
As soon as my first test tried to swarm an Azure Function by POSTing images I noticed it did not perform as I expected.
It took me a while to understand the impact of my Locust python code on the test itself: I was reading the image from the mounted Azure File Share at each Locust task execution, therefore #1 I was wasting my testing resources (slaves) in repetitive task while not pushing enough requests, missing the whole point of a load test, and #2 I did not take into account that Azure Files has its own scalability target.
This was my code:

from locust import HttpUser, TaskSet, task, between

class APICalls(TaskSet):    
    @task()
    def analyzeimage(self):
        image = open('/locust/sample_face.png', 'rb').read()
        self.client.post("Analyze", files={"shot": image}, name="/analyzeimage")

class APIUser(HttpUser):
    tasks = [APICalls]
    wait_time = between(0.05, 0.1) # seconds

I changed the python code by loading the payload into global variable at the definition of the Locust user, therefore the image is read from Azure File Share once for each of the few hundreds of Locust I hatched, instead of several thousands per minute.

from locust import HttpUser, TaskSet, task, between, events

global image

class APICalls(TaskSet):    
    @task()
    def analyzeimage(self):
        global image
        self.client.post("Analyze", files={"shot": image}, name="/analyzeimage")

class APIUser(HttpUser):
    tasks = [APICalls]
    wait_time = between(0.05, 0.1) # seconds

    def on_start(self):
        global image
        image = open('/locust/sample_face.png', 'rb').read()

After this much-needed change the tests, free of unnecessary constraints on the slaves behavior, performed as expected: I am always amazed by the power of Azure Functions!

There surely are other more efficient options to define the tests: please refer to the Locust documentation for details.

How to record Locust test

While my search for a load testing framework was based on a specific need (API load testing primarily) it is common to define a test suite by recording the browsing of a complex resource such as a web site, therefore avoiding the need to manually write the tests.
Namely JMeter is the champion of this use case: if you are interested about JMeter please check the great work by Paolo Salvatori on JMeter load testing on Azure.
It is possible to accomplish the same goal with Locust too: by using MitM, an open source interactive proxy project, the browsing of a website is recorded and Locust.Replay lets you export captured flows to Locust script format.

The dev experience is complete with Firefox browser, as it lets you have a separate proxy from the system-wide configuration, therefore not impacting the rest of the apps and services on your local dev machine.
This is how the dev flow looks like:

I love how immediate is Locust in defining simple test scenario and complex ones as well.
Start now to leverage Locust on Azure in your testing plan!