DEV Community

Cover image for Experimentations on Bazel: Python (3), linter & pytest
David Bernard
David Bernard

Posted on

Experimentations on Bazel: Python (3), linter & pytest

#bazel #python #pytest #pylint

In the previous articles,we saw how to launch linter for python with a custom rule, and in the one before how to launch pytest test. In this article, we'll see how to combine both usage in more friendly way (IMHO).

Since the previous article, I did more experimentations (on my side), and I was limited by my current knowledge about how to implement rules (and managed dependency). But with bazel's macro, we could fix (or reduce) some issues:

  • duplication of pytest's boilerplate every BUILD.bazel and *test.py
  • duplication between pytest and potential linter
  • too many custom code to please bazel

Create the macro

Create the empty tools/pytest/BUILD.bazel , just to initiate the package.

Create the macro tools/pytest/defs.bzl, it is like the existing py_test defined in exp_python/webapp/BUILD.bazel but with the call inside a function and some part configurable.

"""experience wrap py_test"""

load("@rules_python//python:defs.bzl", "py_test")
load("@my_python_deps//:requirements.bzl", "requirement")

def pytest_test(name, srcs, deps = [], args = [], **kwargs):
    """
        Call pytest
    """
    py_test(
        name = name,
        srcs = [
            "//tools/pytest:pytest_wrapper.py",
        ] + srcs,
        main = "//tools/pytest:pytest_wrapper.py",
        args = [
            "--capture=no",
        ] + args + ["$(location :%s)" % x for x in srcs],
        python_version = "PY3",
        srcs_version = "PY3",
        deps = deps + [
            requirement("pytest"),
        ],
        **kwargs
    )
Enter fullscreen mode Exit fullscreen mode
  • load to import definition of symbols
  • pytest_test is the macro, in fact a function that call py_test
  • //tools/pytest:pytest_wrapper.py is the wrapper to call pytest (see below)
  • name, srcs, deps, args are the argument of that will customize
  • **kwargs allow to forward some args from the caller side for the part that are not overwitten by the macro
  • ["$(location :%s)" % x for x in srcs] is to append at the end of the call the list of source file to process, to have the path of those file (srcs is a list of string at this point), we call $(location ...) to have the path in the execution context of the py_test and and the prefix : is to convert the file name into a label. It's hacky but it works

Then create the wrapper that will call pytest tools/pytest/pytest_wrapper.py

import sys
import pytest

# if using 'bazel test ...'
if __name__ == "__main__":
    sys.exit(pytest.main(sys.argv[1:]))
Enter fullscreen mode Exit fullscreen mode

Replace the current call to py_test by a call to hour macro in exp_python/webapp/BUILD.bazel.

load("//tools/pytest:defs.bzl", "pytest_test")
...

# py_test(
#     name = "test",
#     srcs = [
#         "test.py",
#     ],
#     # main = "test.py",
#     args = [
#         "--capture=no",
#     ],
#     python_version = "PY3",
#     srcs_version = "PY3",
#     deps = [
#         ":webapp",
#         requirement("requests"),
#         requirement("fastapi"),
#         requirement("pytest"),
#     ],
# )

pytest_test(
    name = "test",
    srcs = ["test.py"],
    deps = [
        ":webapp",
        requirement("requests"),
        # requirement("fastapi"),
    ],
)
Enter fullscreen mode Exit fullscreen mode

We keep commented the previous code for comparison, it's shorter and we move into a central shared place code that could be shared with other python package. Give a try...

❯ bazel test //exp_python/webapp:test
ERROR: /home/david/src/github.com/davidB/sandbox_bazel/exp_python/webapp/BUILD.bazel:32:12: no such target '//tools/pytest:pytest_wrapper.py': target 'pytest_wrapper.py' not declared in package 'tools/pytest'; however, a source file of this name exists.  (Perhaps add 'exports_files(["pytest_wrapper.py"])' to tools/pytest/BUILD?) defined by /home/david/src/github.com/davidB/sandbox_bazel/tools/pytest/BUILD.bazel and referenced by '//exp_python/webapp:test'
ERROR: /home/david/src/github.com/davidB/sandbox_bazel/exp_python/webapp/BUILD.bazel:32:12: no such target '//tools/pytest:pytest_wrapper.py': target 'pytest_wrapper.py' not declared in package 'tools/pytest'; however, a source file of this name exists.  (Perhaps add 'exports_files(["pytest_wrapper.py"])' to tools/pytest/BUILD?) defined by /home/david/src/github.com/davidB/sandbox_bazel/tools/pytest/BUILD.bazel and referenced by '//exp_python/webapp:test'
ERROR: Analysis of target '//exp_python/webapp:test' failed; build aborted: Analysis failed
INFO: Elapsed time: 0.704s
INFO: 0 processes.
FAILED: Build did NOT complete successfully (18 packages loaded, 19 targets configured)
FAILED: Build did NOT complete successfully (18 packages loaded, 19 targets configured)
Enter fullscreen mode Exit fullscreen mode

Ok, we need to fix some stuff, because it's a macro we need to expose pytest_wrapper.py to caller of the macro. So just follow the recommendation and change the empty tools/pytest/BUILD.bazel into

exports_files(["pytest_wrapper.py"])
Enter fullscreen mode Exit fullscreen mode

Retry...

❯ bazel test //exp_python/webapp:test
INFO: Analyzed target //exp_python/webapp:test (26 packages loaded, 15198 targets configured).
INFO: Found 1 test target...
Target //exp_python/webapp:test up-to-date:
  bazel-bin/exp_python/webapp/test
INFO: Elapsed time: 2.242s, Critical Path: 0.99s
INFO: 5 processes: 3 internal, 2 linux-sandbox.
INFO: Build completed successfully, 5 total actions
//exp_python/webapp:test                                                 PASSED in 0.7s

Executed 1 out of 1 test: 1 test passes.
There were tests whose specified size is too big. Use the --test_verbose_timeout_warnings command line option to see which ones these aINFO: Build completed successfully, 5 total actions
Enter fullscreen mode Exit fullscreen mode

First victory.

We can now remove the boilerblate to call pytest at the end of exp_python/webapp/test.py because it is now provided by the shared tools/pytest/pytest_wrapper.py.

from fastapi.testclient import TestClient
from exp_python.webapp.main import app

client = TestClient(app)


def test_read_main():
    response = client.get("/status")
    assert response.status_code == 200
    assert response.json() == {"status": "UP", "version": "0.1.0"}
Enter fullscreen mode Exit fullscreen mode

You can mofidy the test to force the fail, and check if it'll fail as expected.

Use Pytest to launch linter

Pytest have lot of plugin/extension to launch tool like black, pylint, flake8, mypy,... So instead of create one linter call we can configure pytest run all linter.

Add linter into third_party/requirements.txt

...
#test
requests==2.25.1
pytest==6.1.2

#tools
black==20.8b1
pytest-black==0.3.12
pytest-mypy==0.8.1
pytest-pylint==0.18.0
pylint==2.8.2
Enter fullscreen mode Exit fullscreen mode

Configure tools/pytest/defs.bzl to also launch linter and configure the call to pytest (it's like a script and a configuration)

"""experience wrap py_test"""

load("@rules_python//python:defs.bzl", "py_test")
load("@my_python_deps//:requirements.bzl", "requirement")

def pytest_test(name, srcs, deps = [], args = [], **kwargs):
    """
        Call pytest
    """
    py_test(
        name = name,
        srcs = [
            "//tools/pytest:pytest_wrapper.py",
        ] + srcs,
        main = "//tools/pytest:pytest_wrapper.py",
        args = [
            "--capture=no",
            "--black",
            "--pylint",
            "--mypy",
        ] + args + ["$(location :%s)" % x for x in srcs],
        python_version = "PY3",
        srcs_version = "PY3",
        deps = deps + [
            requirement("pytest"),
            requirement("pytest-black"),
            requirement("pytest-pylint"),
            requirement("pytest-mypy"),
        ],
        **kwargs
    )
Enter fullscreen mode Exit fullscreen mode

Run it

❯ bazel test //exp_python/webapp:test
INFO: Analyzed target //exp_python/webapp:test (20 packages loaded, 2332 targets configured).
INFO: Found 1 test target...
FAIL: //exp_python/webapp:test (see /home/david/.cache/bazel/_bazel_david/76e87152cc51687aee6e05b5bdcf89aa/execroot/__main__/bazel-out/k8-fastbuild/testlogs/exp_python/webapp/test/test.log)
INFO: From Testing //exp_python/webapp:test:
==================== Test output for //exp_python/webapp:test:
============================= test session starts ==============================
platform linux -- Python 3.9.2, pytest-6.1.2, py-1.10.0, pluggy-0.13.1
rootdir: /home/david/.cache/bazel/_bazel_david/76e87152cc51687aee6e05b5bdcf89aa/sandbox/linux-sandbox/3/execroot/__main__/bazel-out/k8-fastbuild/bin/exp_python/webapp/test.runfiles/__main__
plugins: black-0.3.12, pylint-0.18.0, mypy-0.8.1
collected 5 items
--------------------------------------------------------------------------------
Linting files
....Unable to create directory /home/david/.pylint.d
Unable to create file /home/david/.pylint.d/exp_python.__init__1.stats: [Errno 2] No such file or directory: '/home/david/.pylint.d/exp_python.__init__1.stats'

--------------------------------------------------------------------------------

exp_python/webapp/test.py FFF..

=================================== FAILURES ===================================
______________________ [pylint] exp_python/webapp/test.py ______________________
C:  1, 0: Missing module docstring (missing-module-docstring)
C:  7, 0: Missing function or method docstring (missing-function-docstring)
__________________________ exp_python/webapp/test.py ___________________________
1: error: Cannot find implementation or library stub for module named 'fastapi.testclient'
1: note: See https://mypy.readthedocs.io/en/latest/running_mypy.html#missing-imports
_________________________________ test session _________________________________
mypy exited with status 1.
===================================== mypy =====================================
exp_python/webapp/main.py:1: error: Cannot find implementation or library stub for module named 'fastapi'
Found 2 errors in 2 files (checked 1 source file)
=========================== short test summary info ============================
FAILED exp_python/webapp/test.py::PYLINT
FAILED exp_python/webapp/test.py::mypy
FAILED exp_python/webapp/test.py::mypy-status
========================= 3 failed, 2 passed in 1.99s ==========================
================================================================================
Target //exp_python/webapp:test up-to-date:
  bazel-bin/exp_python/webapp/test
INFO: Elapsed time: 3.841s, Critical Path: 3.45s
INFO: 5 processes: 3 internal, 2 linux-sandbox.
INFO: Build completed, 1 test FAILED, 5 total actions
//exp_python/webapp:test                                                 FAILED in 3.0s
  /home/david/.cache/bazel/_bazel_david/76e87152cc51687aee6e05b5bdcf89aa/execroot/__main__/bazel-out/k8-fastbuild/testlogs/exp_python/webapp/test/test.log

INFO: Build completed, 1 test FAILED, 5 total actions
Enter fullscreen mode Exit fullscreen mode

Seems to work (or failed as expected, linter detect issue)

Now extend pytest_test to lint over every *.py of the package (and let pytest detects test). As we check every python file, we also need to add all their dependencies.

pytest_test(
    name = "test",
    srcs = glob(["*.py"]),
    deps = [
        ":run",
        ":webapp",
        requirement("requests"),
        requirement("fastapi"),
    ],
)
Enter fullscreen mode Exit fullscreen mode

To fix the linter issue (I comments mypy, not useful for the purpose of this article), provide a configuration file for pylint and disable a pylint rule localy into run.py (full code available into the repo, see bellow). So the final tools/pytest/defs.bzl looks like (with a configuration file for the demo, provide as data to py_test and as args)

"""Wrap pytest"""

load("@rules_python//python:defs.bzl", "py_test")
load("@my_python_deps//:requirements.bzl", "requirement")

def pytest_test(name, srcs, deps = [], args = [], data = [], **kwargs):
    """
        Call pytest
    """
    py_test(
        name = name,
        srcs = [
            "//tools/pytest:pytest_wrapper.py",
        ] + srcs,
        main = "//tools/pytest:pytest_wrapper.py",
        args = [
            "--capture=no",
            "--black",
            "--pylint",
            "--pylint-rcfile=$(location //tools/pytest:.pylintrc)",
            # "--mypy",
        ] + args + ["$(location :%s)" % x for x in srcs],
        python_version = "PY3",
        srcs_version = "PY3",
        deps = deps + [
            requirement("pytest"),
            requirement("pytest-black"),
            requirement("pytest-pylint"),
            # requirement("pytest-mypy"),
        ],
        data = [
            "//tools/pytest:.pylintrc",
        ] + data,
        **kwargs
    )
Enter fullscreen mode Exit fullscreen mode

And modify tools/pytest/BUILD.bazel to export configuration files.

exports_files([
    "pytest_wrapper.py",
    ".pylintrc",
])
Enter fullscreen mode Exit fullscreen mode

Clean up

Remove code and reference to pytest_check (that we introduce previously), it's no longer needed. Now we have

❯ tree -a -I '.vscode|bazel-*|.venv|.git'                                                                                      21:16:36
.
├── .bazelrc
├── .bazelversion
├── BUILD.bazel
├── exp_genrule
│   ├── a.txt
│   ├── b.txt
│   └── BUILD.bazel
├── exp_python
│   └── webapp
│       ├── BUILD.bazel
│       ├── main.py
│       ├── __pycache__
│       │   └── main.cpython-39.pyc
│       ├── run.py
│       └── test.py
├── .github
│   ├── dependabot.yml
│   └── workflows
│       └── ci.yml
├── .gitignore
├── .python-version
├── setup_localdev.sh
├── third_party
│   ├── BUILD.bazel
│   └── requirements.txt
├── tools
│   ├── pytest
│   │   ├── BUILD.bazel
│   │   ├── defs.bzl
│   │   ├── .pylintrc
│   │   └── pytest_wrapper.py
│   └── workspace_status.sh
└── WORKSPACE.bazel
Enter fullscreen mode Exit fullscreen mode

Limits of the solution

  • Dependencies of test, runtime, tools are all mixed (no isolation), but it's the common (crappy ?) way to work in python ecosystem.
  • The macro is harder to extract as a workspace, tools for reuse into an other project, but in the other side the code is small enough to be copied and customized for each need
  • No split in the output of bazel every linter and test will be show as 1 test into report of bazel.

At the end, currently it's the most pleasant solution I found to have pytest, linters run by bazel without too many (and duplicate) boilerplate into each package. A central configuration place, and pretty simple to extends if pytest as the extension (else we can tweak the pytest_wrapper.py to do complementary stuff by example).

To be continued

It's not the end, we'll continue to setup other stuff like depending of another package (a python lib), make a docker image,...

The sandbox_bazel is hosted on github (not with the same history, due to errors), use tag to have the expected view at end of article: article/6_python_3. I'll be happy to have your comments on this article, or to discuss on github repo.

Top comments (0)

Read next

cereal84 profile image

Hacktoberfest from a maintainer's point of view

Alessandro Pischedda -

mrcaption49 profile image

CRUD FUNCTIONALITY in Django

Pranav Bakare -

silvio_riveros_49b5505866 profile image

Migrar de SQLite a MySQL.

Silvio Riveros -

dchaley profile image

Performance trap: general libraries & helper objects

David Haley -