There are already hundreds of great hooks around. But what if we want one more? Well, reading official docs is the great place to start (and reference in the future).
What I want is to:
- show by example how particular idea can be developed
- describe pitfalls which surprised me the most (and how to overcome them)
- bring attention to such a wonderful tool that
pre-commit
is
The theme I chose for illustration is assignment expressions (PEP572). And down the line I will create three different hooks for that.
Preparations
For experiments with hooks already existing in the wild you would need only one git repository. And then you would configure it to install/use hooks from somewhere else.
But to write your own, there should be two separate repositories.
Let's begin.
If you didn't install pre-commit into system/environment - see documentation on how to do that.
Start with basic structure like this (two folders with git init)
[~/code/temp/try-pre-commit]
$ tree -a -L 2
.
├── code
│ └── .git
└── hooks
└── .git
- Create dummy hooks/.pre-commit-hooks.yaml
$ touch hooks/.pre-commit-hooks.yaml
- Create dummy code/experiments.txt
$ touch code/experiments.txt
- cd into code folder and initialize pre-commit (don't let this second install confuse you, it is different from installation into system/environment. It's scope is repository-wide, one git hook file, actually)
$ cd code
$ pre-commit install
pre-commit installed at .git/hooks/pre-commit
Here is how it should look now
[~/code/temp/try-pre-commit]
$ tree -a -L 2
.
├── code
│ ├── experiments.txt
│ └── .git
└── hooks
├── .git
└── .pre-commit-hooks.yaml
Example 1: grep
The first hook will be very basic, leveraging existing possibilities of the framework. I will call it "walrus-grep".
Update hooks/.pre-commit-hooks.yaml
- id: walrus-grep
name: walrus-grep
description: Warn about assignment expressions
entry: ":="
language: pygrep
pygrep is a cross-platform version of grep.
entry takes regexp of something that you don't want to commit. In my case I used ":="-pattern without actual regexp magic or escaping symbols.
Update code/experiments.txt
text file
a := 1
The most convenient way to experiment is with try-repo
command. It allows to test hooks without commiting .pre-commit-hooks.yaml every time, and without pinning to revision later on in .pre-commit-config.yaml.
We can either stage files each time before running checks or use try-repo
command with --all-files
flag.
[master][~/code/temp/try-pre-commit/code]
$ pre-commit try-repo ../hooks walrus-grep --verbose --all-files
This happened because hooks repo doesn't have a single commit - there is nothing HEAD could point to. Commit .pre-commit-hooks.yaml and repeat.
See "(no files to check)"? Often this would be true, checks could have pre-defined limits e.g. file extension, path, ignore options. But we have our experiments.txt file and do not expect any of those!
Gotcha: --all-files
flag is more like "all tracked, not only staged files". So either way you should add files to git before running checks.
At last! "Failed" is what we needed to prove that grep hook catches ":=" inside some random staged file. So here failure is a good thing.
But to think about it situation is far from ideal:
- first, we don't actually want to catch ":=" inside text files, only in python code
- second, what if ":=" happens inside comments, docstrings, strings?
For "python code only" part there is types: [python]
option which I will add to .pre-commit-hooks.yaml
- id: walrus-grep
...
types: [python]
We see "(no files to check)Skipped" again, but now it is expected.
To double-check
$ mv experiments.txt experiments.py
$ git add experiments.py
$ pre-commit try-repo ../hooks walrus-grep --verbose --all-files
and we have our beloved failure back.
As for the second concern, ":=" inside comments or docstrings, we would require something different.
Example 2: AST
The second hook will be analyzing syntax tree to find exact node type. I will call it "walrus-ast".
Disclaimer: This one will work only in Python 3.8 environment (beta1 is currently available for testing).
Update hooks/.pre-commit-hooks.yaml
...
- id: walrus-ast
name: walrus-ast
description: Warn about assignment expressions
entry: walrus-ast
language: python
types: [python]
and experiments.py
# .py file with `:=` (Python3.8 syntax)
(x := 123)
The hook itself will be at hooks/walrus_ast.py
import argparse
import ast
def check(filename):
with open(filename) as f:
contents = f.read()
try:
tree = ast.parse(contents)
except SyntaxError:
print('SyntaxError, continue')
return
for node in ast.walk(tree):
if isinstance(node, ast.NamedExpr):
return node.lineno
def main():
parser = argparse.ArgumentParser()
parser.add_argument('filenames', nargs='*')
args = parser.parse_args()
for filename in args.filenames:
lineno = check(filename)
if lineno:
print(f"{filename}:{lineno}")
return 1
return 0
if __name__ == '__main__':
exit(main())
What we can or can't do here? The rules per documentation are simple
The hook must exit nonzero on failure or modify files in the working directory
Things to note:
-
exit()
with some code ("0" is when everything OK) - will decide whether hook fails or not - checks like
if __name__ == '__main__':
and names likemain()
orcheck()
are not required (i.e. you can have useless file withexit(1)
as its only contents) - parsing filenames is needed if you want to be able to limit by filename. Not required but nice to have
Now run try-repo
with new hook name
$ pre-commit try-repo ../hooks walrus-ast --verbose --all-files
"Directory '.' is not installable". What happened?
It turns out that (in case of python scripts) we should describe how to install it as usual, with setup.py or pyproject.toml.
Create hooks/setup.py
from setuptools import setup
setup(
name="hooks",
py_modules=["walrus_ast",],
entry_points={
'console_scripts': [
"walrus-ast=walrus_ast:main",
],
},
)
Note: instead of py_modules
you can use something like packages=find_packages(".")
or anything else you are used to do in setup.py
Track both new files in git
$ git add setup.py
$ git add walrus_ast.py
Now working directory should look like this
[master][~/code/temp/try-pre-commit/hooks]
$ tree -a -L 1
.
├── .git
├── .pre-commit-hooks.yaml
├── setup.py
└── walrus_ast.py
Try running it again
$ pre-commit try-repo ../hooks walrus-ast --verbose --all-files
Reminder on how experiments.py looks:
# .py file with `:=` (Python3.8 syntax)
(x := 123)
The ":=" is indeed on the second line, and the one from comments is not reported. Thanks, AST!
Note: And if you forgot that this is Python3.8 only thing - you'll get (hopefully) some indication. In that case hook doesn't work, but shouldn't stop others from doing their job either.
Example 3: Less negativity, more opportunity!
It was not my intention to stop you from using assignment expressions, rather to play with new things and describe the process.
Let's approach the idea from the opposite direction and find a place where walrus can fit. Thanks to this tweet and very interesting library by Chase Stevens we can find such places.
Create hooks/walrus_opportunity.py
import astpath
def main():
search_path = "//Assign[targets/Name/@id = following-sibling::*[1][name(.) = 'If']/test/Name/@id]"
found = astpath.search('.', search_path, print_matches=True, after_context=1)
return len(found)
if __name__ == "__main__":
exit(main())
Here astpath uses xpath to find AST nodes.
Update setup.py
from setuptools import setup
setup(
name="hooks",
py_modules=["walrus_ast", "walrus_opportunity"],
entry_points={
'console_scripts': [
"walrus-ast=walrus_ast:main",
"walrus-opportunity=walrus_opportunity:main"
],
},
install_requires=[
# Dep. for `walrus-opportunity`
"astpath[xpath]",
],
)
Update hooks/.pre-commit-hooks.yaml
- id: walrus-opportunity
name: walrus-opportunity
description: Warn if you could have used ":=" somewhere
entry: walrus-opportunity
language: python
types: [python]
Update code/experiments.py
# .py file with `:=` (Python3.8 syntax)
(x := 123)
# missed chance to use `:=`
x = calculate()
if x:
print(f"found {x}")
and run it
$ pre-commit try-repo ../hooks walrus-opportunity --verbose --all-files
Wrapping up
If you decide that local development is done:
Push hooks folder to your-repo (e.g. on github), create code/.pre-commit-config.yaml
repos:
- repo: https://github.com/<your-repo>/pre-commit-hooks
rev: <commit-sha> or <tag>
hooks:
- id: walrus-grep
- id: walrus-ast
- id: walrus-opportunity
And now you should be able to use git from code repo as usual: change files, stage and commit them. Before each commit git will run a script from code/.git/hooks/pre-commit. And pre-commit framework should show report and allow commit if everything is OK, or show report and abort commit if some checks have returned non-zero or files were changed.
Addendum
More about AST visitors and working on the tree here.
Thanks to Anthony Sottile for such a wonderful framework.
Top comments (0)