C.I. Continuous Integration

Adding CI

This week the enhancement at hand for my project was to add some basic continuous integration by running the tests I have written every time I push.

To achieve this goal all I had to do was create a .yml file in my new .github/workflows directory under the root of my project. Git hub knows to read the instructions specified in this .yml and create its Actions from the jobs and settings specified therein.

The first thing to specify is when the following will be run with the on: clause. In my case I want it to be run upon every push and every pull request to my GitHub repository.

on: [push, pull_request] does the trick for this.

Then we state what environment we want to run the workflow on, I chose the latest version of ubuntu that git hub can provide using the .yml tag ubuntu-latest.

Once the runner is established we can start listing the steps that we mean to take in our workflow.

I want to checkout my code which I can do through actions with the following allowing my workflow to use what I have written:
- uses: actions/checkout@v3

Then I setup the version of Python that I mean to use to run my tests in the workflow with another actions provided by github that will install Python and add it to the PATH.

- name: Set up Python ${{ matrix.python-version }}
          uses: actions/setup-python@v4
          with:
            python-version: 3.11

Here you can see I have some other .yml tags to show information relating to what is happening at this step and I specify which version of python I would like to use. For my purposes I wanted to start with what I was using locally most recently which happened to be 3.11.

Now almost everything I need is installed and configured, all I have left is the dependencies that I use to lint, test and format my code. All of my dependencies I have specified in a requirements text file that I checked out earlier, the next step will require me to run an installation of those dependencies so that further down the line I can use them in the workflow. So in the next set of commands I install any updates I can to the pip package installer and pip install my dependencies:

run: |
            python -m pip install --upgrade pip
            pip install -r requirements-dev.txt

Now I am really ready to do the operations I set out to do on my code. Not only can I test but I can also format and lint any code and even set a score at which the workflow should fail with a non-zero response should I want to... and I think I would like that.

Now that I think about the order I have done the next steps in I am thinking maybe formatting should be the first but in any case, the current workflow I have on my repository is as follows:

I Lint the code with PyLint and fail under a perfect 10/10 score:

- name: Lint with PyLint
          run: |
            pylint src --fail-under=10

I then format using Black:

- name: Check formatting with black
          run: | 
            black src/ tests/ --check --verbose

and finally I run the unit tests using pytest:

- name: Run unit tests
          run: |
            pytest

I commited the pyproject.toml configuration file I talked about last week under my Testing blog and at this stage of the workflow the pytest command is taking the configuration from that file and outputting a nice coverage matrix as well (coverage and pytest-cov are both in my dependencies text file and were installed with the other dependencies above).

---------- coverage: platform linux, python 3.11.0-final-0 -----------
Name                                 Stmts   Miss  Cover
--------------------------------------------------------
src/ssj.py                             128     62    52%
tests/test_ssj_convert_markdown.py      16      0   100%
tests/test_ssj_edit_template.py          7      0   100%
tests/test_ssj_line_by_line.py          42      0   100%
--------------------------------------------------------
TOTAL                                  193     62    68%


============================== 13 passed in 0.07s ==============================

My opinions on CI

In the current era of increasing automation I believe CI is absolutely integral to a smoothly running project. To be able to set a set of processes that insure the quality of code additions, to keep a unified format and maintain readability, it is as good of a solution as I have seen. I think it is currently the norm but will continue to see growth and adoption among more and more coders especially with how easy git is making it for freeeee.

Here is an example workflow from my class mate Rudy's recent PR to my repository where he added some testing to see that my CI was working: Successful actions run.

Which brings me to...

Collaborating: Adding tests to SauSaGe

My classmate Rudy is creating a similar site generator of his own except he is writing his in C++ and internally it is quite different, for example he makes use of a markdown converting dependency which is much more robust and sophisticated than my current implementation of markdown conversion.

Coming to terms with someone else's project and code and thinking about how to add to it is always a bit of a hurdle, it helps that Rudy is good at writing code and it's neat and easier to understand than most.

We spoke a bit about what I should test, he did not have any tests for markdown with markdown specific syntax yet since he trusted the markdown conversation dependency so I added a test to show that the bold, italics and code syntax worked.

Apart from that I wanted to test something else and so I took a small function in his text.cpp class that returned a files name with an .html extension and with those two small testing modifications I made a commit to his repository that had his own CI to go through, it went through all the tests just fine as I hoped it would since locally they had succeeded and he approved my PR once the CI run came back positive.

Naturally his CI is very different to mine, his C++ program is run in a windows environment and is first built into an .exe before being tested with his -t argument using Catch2. Writing tests for it was not too difficult though, the existing ones set a clear example for how the tests were separated into test cases and subsections and REQUIRE seemed to be similar to my ASSERT.

The following is my PR to his repo to test his CI: pull18