The Context: Test Parameterization
At a previous job, we built home gateway middleware, which had to be tested on a plethora of devices. I advised to start using pytest for its succinctness and extensibility.
Now a typical use case was to upload new firmware to a device, boot it, and run a test suite against it. The device's IP address and the firmware binary were variables.
Thanks to pytest's hook architecture, a test suite could be extended with a number of command line options. Thanks to pytest fixtures, it is possible to inject information into a test.
If only we could combine those two nice features to retrieve command line options from within your test....
So the idea grew to create a library to declare options, and make them available as test fixtures. I named the library 'fixtopt' because optfixt is so much harder to pronounce :). It's out in the open, because I think it is useful for more than just that one case.
Action: let's code
Let's start with an arbitrarily simple example piece of code we can test - and work our way up to the command line.
def format_message(body, recipient):
return "Hi there, {recipient},\n{body}"
Simple Test Case
def test_a_person_is_greeted_properly():
message = "Dear Lisa, there is a hole in my bucket. Regards, John"
recipient = "Lisa"
assert format_message(message, recipient)\
.startswith("Dear {recipient}")
... of course it's trivial. But let's look at what we can do with the test input variables message and recipient.
Extracting Test Fixtures
Pytest lets you register functions as fixtures. When executing a test function, it will take the test function's argument names, and call the test function with the results of the corresponding fixture functions.
@pytest.fixture
def message():
return "Dear Lisa, there is a hole in my bucket. Regards, John"
@pytest.fixture
def recipient():
return "Lisa"
def test_a_person_is_greeted_properly(message, recipient):
assert format_message(message, recipient)\
.startswith("Dear {recipient}")
Command Line Options
Now, we would like these fixtures to become injected by the pytest command line, like so:
pytest . --message "Nothing else to say" --recipient Lisa
And that's where this fixtopt-xtofl library comes in: add a file conftest.py in your test directory, and declare the options:
# conftest.py
from fixtopt import Option, register
def pytest_options(parser):
register(globals(), parser, (
Option(
name="message",
default="Dear Lisa, there is a hole in my bucket. Regards, John",
help="the message to send"),
Option(
name="recipient",
default="Olav",
help="the one to adress"),
))
And there you go! pytest picks it up, adds it to its own help text and you can just use the options as you like!
/path/to/testdir> pytest --help
...
custom options:
--message=MESSAGE the message to send
--recipient=RECIPIENT
the recipient
...
Running the test (with a broken implementation):
/path/to/testdir> pytest . --message "whatever" --recipient "Lisaz"
==== test session starts ====
platform linux -- Python 3.6.9, pytest-6.0.1, py-1.9.0, pluggy-0.13.1
rootdir: /path/to/testdir
collected 1 item
test_messageformat.py F [100%]
==== FAILURES ====
____ test_a_person_is_greeted_properly ____
message = 'whatever', recipient = 'Lisaz'
def test_a_person_is_greeted_properly(message, recipient):
> assert format_message(message, recipient)\
.startswith("Dear {recipient}")
E AssertionError: assert False
E + where False = <built-in method startswith of str object at 0x7f9a80faa670>('Dear {recipient}')
E + where <built-in method startswith of str object at 0x7f9a80faa670> = 'Dear, {recipient},\n{body}'.startswith
E + where 'Dear, {recipient},\n{body}' = format_message('whatever', 'Lisaz')
test_messageformat.py:5: AssertionError
==== short test summary info ====
FAILED test_messageformat.py::test_a_person_is_greeted_properly - AssertionError: assert False
==== 1 failed in 0.07s ====
Agreed, the resulting failure details are a bit painful to the eyes, but this has nothing to do with the command line arguments or the fixtures.
Conclusion
Well... this is all. A small and simple library to add command line parameters to your tests. It's not feature-complete yet, but it gets the job done. Do take a look, try it out, file an issue, add a pull request. It can only get better.
Top comments (0)