DEV Community

Cover image for Automate Clean Code and Linting in Python
Rahul Banerjee
Rahul Banerjee

Posted on • Originally published at realpythonproject.com

Automate Clean Code and Linting in Python

We will be looking at some packages to format, lint, test our code and later create a pre-commit hook to automate the process.

Before we talking about the packages we will use, let's look at the python files we will work with

We have a function called helpers.py

It has simple arithmetic functions. All the functions accept two parameters and perform an arithmetic operation on them.

We have another file called tester_helpers.py

This file simply tests the functions we defined earlier. It uses assert to do a simple equality check.

Now let's look at the packages we will be using.

Testing 🧪

Pytest 7.7k+ ⭐️

This package helps us run unit tests. One requirement to keep in mind is that your python file containing the unit tests should begin with 'test_'.

Only assert statements are supported. To install the package

pip install pytest
Enter fullscreen mode Exit fullscreen mode

To run the unit tests, type the following commands

pytest test_helpers.py
Enter fullscreen mode Exit fullscreen mode

If all your tests pass, you should see a similar output

test_helpers.py ....             [100%]

========= 4 passed in 0.01s ===========
Enter fullscreen mode Exit fullscreen mode

If you get an error related to multiple relative imports

astroid.exceptions.TooManyLevelsError:
Enter fullscreen mode Exit fullscreen mode

It is probably an issue with one of pytest's dependencies. You'll have to uninstall astroid and install it again. This ensures the altest astroid version is installed.

pip uninstall astroid
pip install astroid
Enter fullscreen mode Exit fullscreen mode

After this, we will have to uninstall pytest and install pytest

pip uninstall pytest
pip install pytest
Enter fullscreen mode Exit fullscreen mode

Formatting ✍️

YAPF 12k+ ⭐️

This was developed by google and supports in-place formatting. To install the package

pip install yapf
Enter fullscreen mode Exit fullscreen mode

To format your files, type the following

yapf --in-place *.py
Enter fullscreen mode Exit fullscreen mode

This will format all your top level python files, if you want to include folders as well you can use the following

yapf --in-place **/*.py
Enter fullscreen mode Exit fullscreen mode

However, this will also include our virtual environment folder. To ignore the venv folder, simply create a file .yapfignore and add venv to it.

Note: This command might take some time to run. Instead of '**' you could use the folder's specific names.

isort 4.1k+ ⭐️

This packages sorts your import statements to ensure they follow pep8 rules.

Imports should be grouped in the following order:

  • Standard library imports.
  • Related third party imports.
  • Local application/library specific imports.

isort re-orders import statements to ensure the above rule is followed.
To install the package

pip install isort
Enter fullscreen mode Exit fullscreen mode

To run isort

isort .
Enter fullscreen mode Exit fullscreen mode

autoflake 400+⭐️

It helps in getting rid of unused imports, variables and object keys.

To install the package

pip install autoflake
Enter fullscreen mode Exit fullscreen mode

To run autoflake

autoflake --in-place --remove-unused-variables --remove-all-unused-imports *.py
Enter fullscreen mode Exit fullscreen mode

Some other formatters

Linting 🔎

Pylint 3.5k+ ⭐️

pylint ensures your code is following pep8 rules and standards. It gives each python file a score out of 10 (It can given you a negative score as well)

To install the package

pip install pylint
Enter fullscreen mode Exit fullscreen mode

To run the linter

pylint --fail-under=7 *.py
Enter fullscreen mode Exit fullscreen mode

The argument --fail-under is the lower bound, if any file has a score below the lower bound, an error will be returned.

Pre-commit Hook 🪝

What Are Git Hooks?

Git hooks are basically scripts fired before an important action occurs, e.g., before a commit is made, before code is pushed to a repo after a commit is made, etc. You can learn more about Git Hooks and the different kinds of hooks over here.

We will be focussing on a pre-commit hook.A pre-commit hook is a hook that is run before you make a commit.

First let's install the package

pip install pre-commit
Enter fullscreen mode Exit fullscreen mode

Now we will generate a sample pre-commit hook YAML file, we will edit this later.

pre-commit sample-config
Enter fullscreen mode Exit fullscreen mode

Now let's add our hook

pre-commit install
Enter fullscreen mode Exit fullscreen mode

Now before every commit, the pre-commit hook defined in our YAML file will be executed.

Now let's update our YAML file.
Remove everything and only keep the following

repos:
    - repo: local
      hooks:
Enter fullscreen mode Exit fullscreen mode

We will add our plugins(packages) under hooks: in the YAML file. Below is the general syntax for the plugin

   - id: (unique id of hook)
     name: (name to be displayed in terminal)
     entry: (command to excute)
     language: system (for our case, always system) 
     always_run: true (if true, it will always run)
     pass_filenames: true (if true, hook will have access to the file name)
Enter fullscreen mode Exit fullscreen mode

Let's define a sample plugin for YAPF

   - id: YAPF 
     name: YAPF 🧹
     entry: zsh -c 'yapf --in-place *.py'
     language: system
     always_run: true
     pass_filenames: true
Enter fullscreen mode Exit fullscreen mode

If you are using bash or are on windows, replace the zsh in 'entry' with bash.

All the other plugins are pretty similar, below is the entire YAML file with all the plugins

Whenever you update your YAML file, you will have to add the file to the staging area using git add . or git add .pre-commit-config.yaml

Below is a successful commit

Pre-commit in action

Conclusion

Setting up a pre-commit hook will ensure your code follows pep8 standards and is properly formatted.
I hope you found the article useful. Add me on LinkedIn, Twitter

Discussion (0)