Day 13: CI for a Python project of the National Institute of Standards and Technology

Interesting coincidences.

Just a few days ago as I started to work on the Open Source Developer Course I started
to collect Open Source projects by governments and today the first project I found on PyDigger that I wanted to try to contribute to was a project of the NIST - the National Institute of Standards and Technology.

Side note: I find it funny that in this age of globalization when anyone can easily bump into any organization (barring language barrier), organizations still call themselves "National" without including an indication of which nation, in their name. At least on the web site of NIST you can see US Department of Commerce in their logo.

The process

Anyway, what did I do?

I noticed that the project has a folder called tests but that it also has a file called runtests.py.
I am not sure why do some projects have their own test-runner instead of just running pytest, but I seem to recall I saw this in other Django-related projects as well.

So I set started a Docker container that I use for better isolation of the foreign code I run from my computer and ran

python runtests.py

Not surprisingly it failed complaining about missing Django.

So I went ahead and installed all the requirements I could find:

pip install -r requirements.txt
pip install -r requirements.core.txt

Running again

python runtests.py

still failed. This time it was missing psycopg2 which is the Python package to connect to PostgreSQL.

So I tried to install it:

pip install psycopg2

It failed, but it pointed me at the installation documentation.
That's a very nice touch!

From that website I understood I need to install libpq. So I ran

apt-cache search libpq

(I use an Ubuntu 22.10 based Docker container for my experiments.)

In the results of this command I saw libpq-dev. This is the development package. Libraries usually need that so I installed it:

apt-get install -y libpq-dev

and ran the installation again:

pip install psycopg2

After that I ran the tests again:

python runtests.py

The output looked very much like an output from a test-run and it ended with

Ran 86 tests in 0.271s

It was not hard at all.

I created the GitHub Actions configuration file using a Docker container and configured python 3.11. I let the developers of the package decide which other versions of Python they might want to test with and if they might want to test on Windows and macOS as well.

I sent the Pull-Request

GitHub Actions

name: CI

on:
  push:
  pull_request:
  workflow_dispatch:
  schedule:
    - cron: '42 5 * * *'

jobs:
  test:
    strategy:
      fail-fast: false
      matrix:
        python-version: ["3.11"]

    runs-on: ubuntu-latest
    name: OS Python ${{matrix.python-version}}
    container: python:${{matrix.python-version}}

    steps:
    - name: Checkout
      uses: actions/checkout@v3

    - name: Install dependencies
      run: |
        apt-get install -y libpq-dev
        pip install -r requirements.txt
        pip install -r requirements.core.txt
        pip install psycopg2

    - name: Check Python version
      run: python -V

    - name: Run tests
      run: python runtests.py

Conclusion

I am already quite experienced setting up CI in general and GitHub Actions in particular, but I think even people who are new to this area could do similar task within a few hours. So I think these will be reasonable assignments in the Open Source Development Courses.