Disclaimer: I am neither an expert regarding the Robot Framework nor parallelization with Python. I simply found this possibility very use- (and learn-)ful and would like to share it whilst trying to explain how it works.
If you are looking for more sophistiacted ways to execute concurrent Robot tests you should take a look at the pabot library or Taurus.
Requirements:
- Knowledge of (and/or interest in) the Robot Framework
- Basic Python knowledge
What?
- ... how to use concurrent subprocesses in Python
- ... to run our Robot test multiple times in parallel
- ... with different parameters without writing multiple tests (for example to allow testing with different test users)
Why?
👾 simple way performing stress tests against a system
🐢 speed up your testing
🐿️ fun
Excursion: Robot Framework
From robotframework.org:
Robot Framework is a generic open source automation framework for acceptance testing, acceptance test driven development (ATDD), and robotic process automation (RPA).
Its Keyword based syntax is easy to learn and makes it possible to write human readable tests very fast.
I like to use it together with the Selenium Library which provides a rich set of keywords for testing web pages and applications.
In cases where those are not sufficient Robot Framework allows you to extend its functionality by:
- Using additional libraries
- Implementing your own keywords in Java or Python
- Executing Javascript via Selenium Keywords 🎉
- ...
See the official documentation for more information and posibilites.
Let's start ⌨️
Parameterizing Robot tests
We start with a single Robot test file called suite.robot
containing our test(s). This file contains a global variable for the login name of our test user:
${USER} test-user
...
Login Test
Go To ${HOMEPAGE}
Wait Until Element Is Visible username
Input Text username ${USER}
Input Password password ${PASSWORD}
...
Via command line options you are able to override the default value for this variable when running the test.
For example
robot --variable USER:another-user suite.robot
runs the test for the user another-user
(instead of test-user
).
Subprocesses with Python
To make use of the above described options via Python we can use subprocess.Popen
(doc). Above example would be translated into:
import subprocess
args = ['robot', '--variable', 'USER:another@user.org', 'suite.robot']
subprocess.Popen(args)
If you try this, you will see that the output of our Robot test gets logged to the console. This is not very helpful if you want to run multiple tests at the same time.
We can disable this behavior by redirecting the output to /dev/null
like this:
import os
FNULL = open(os.devnull, 'w')
subprocess.Popen(args, stdout=FNULL)
Another argument for Popen
we will use is cwd
. It allows us to specify the working directory for the created process.
This is useful since robot
will create the log files of our test in the current working directory. If we would run all our tests within the same directory those log files would collide.
In our final script we make sure to create a separate folder per user, containing only the logs of the test for this user:
os.mkdir(userPath)
subprocess.Popen(cmdParts, cwd=userPath, stdout=FNULL)
Put it together
Now we are able to put together a script expecting following files as input:
-
suite.robot
containing our Robot test(s) -
user_list.txt
containing list of usernames (one per line)
and doing the following:
- Create a base folder containing our user folders
- Load
user_list.txt
- For each user:
- a subfolder is created
- a subprocess is created running our test with the users parameter(s) within its subfolder
- Wait until all subprocesses finished
Full script:
Possible improvements:
- Make it more dynamic: Create input file containing arbitrary variable parameters per test (not only the username)
- Live view of each test in the console by redirecting
stdout
more sophisticated - Thread pool management (for example to restrict maximum number of concurrent subprocesses)
- Aggregated report
Thank you for reading :)
Top comments (3)
hey will you check out this library im building for robotframework?
github.com/teaglebuilt/robotframew...
Hi, currently I am not using AWS, but looks like a great library for all those who do use it!
A very informative article, thank you for sharing. :)